CN117194111A - Data synchronization method and device, electronic equipment and storage medium - Google Patents
Data synchronization method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117194111A CN117194111A CN202311249105.6A CN202311249105A CN117194111A CN 117194111 A CN117194111 A CN 117194111A CN 202311249105 A CN202311249105 A CN 202311249105A CN 117194111 A CN117194111 A CN 117194111A
- Authority
- CN
- China
- Prior art keywords
- data
- time point
- database
- point
- standby
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000001360 synchronised effect Effects 0.000 claims abstract description 74
- 238000004519 manufacturing process Methods 0.000 claims abstract description 52
- 230000037452 priming Effects 0.000 claims abstract description 42
- 230000008569 process Effects 0.000 claims description 23
- 238000012544 monitoring process Methods 0.000 claims description 12
- 238000011084 recovery Methods 0.000 claims description 9
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 230000009471 action Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000010076 replication Effects 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data synchronization method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: stopping synchronizing data to the standby host database and the standby open database, and recording a stopping synchronization time point; performing data priming on the standby host database by utilizing data before a priming time point in the production host database; gradually synchronizing data from a first count time point in the production host database to the standby host database; recording a first target time point when synchronizing to the data of the second tracking time point and a second target time point when synchronizing to the data of the third tracking time point; the second tracking time point is the difference between the synchronization stopping time point and a second preset value; the third tracking time point is the sum of the synchronization stopping time point and a third preset value; synchronizing data from a first target time point to a second target time point in the standby host database to a standby open database; and restoring the real-time synchronous data of the open database to the standby.
Description
Technical Field
The present application relates to the field of data synchronization technologies, and in particular, to a data synchronization method and apparatus, an electronic device, and a storage medium.
Background
In order to ensure high availability of the system, a disaster recovery center is usually additionally arranged besides the production center, so that data in the production host database can be synchronized to a standby host database of the disaster recovery center in real time. In order to avoid overlarge storage pressure of the host database, corresponding open databases are arranged in the production center and the disaster recovery center and are used for synchronizing data in the host database, so that the host database can delete part of data and then acquire the data from the open databases when needed.
In the use process, the data of the production host database and the data of the standby host database have larger difference sometimes because of the problems of environment, hardware and the like. When the situation occurs, all data which is cut off to a certain time point in the production host database is synchronized into the standby host database by the operation of a worker, original data in the standby host database is covered, and the standby host database is recommenced and primed again. Correspondingly, the staff also uses the data in the standby host database to brush the corresponding standby open database again for priming.
However, due to the large data volume, the operation is easy to occur by the manual intervention mode, so that the data error is caused. In addition, the backup open database stores a mode that the backup host database needs to be used for priming the number of the re-brush of the disaster recovery open database, so that the data are deleted, and the service degradation of the disaster recovery center can be caused by the reduction of the data.
Disclosure of Invention
Based on the defects of the prior art, the application provides a data synchronization method and device, electronic equipment and a storage medium, so as to solve the problem that service degradation in disaster recovery occurs in the prior art.
In order to achieve the above object, the present application provides the following technical solutions:
the first aspect of the present application provides a data synchronization method, including:
stopping synchronizing data from the production host database to the standby host database, stopping synchronizing data from the standby host database to the standby open database, and recording a stopping synchronization time point;
performing data priming on the standby host database by utilizing data before a priming time point in the production host database; wherein the priming time point is before the stop synchronization time point;
gradually synchronizing data from a first tracking time point in the production host database to the standby host database; the first tracking time point is the difference value between the bottoming time point and a first preset value;
recording a first target time point when synchronizing to the data of the second count time point, and recording a second target time point when synchronizing to the data of the third count time point; the second count time point is a difference value between the synchronization stopping time point and a second preset value; the third count time point is the sum of the stop synchronization time point and a third preset value;
gradually synchronizing data from the first target time point in the standby host database to the standby open database;
and after synchronizing the data to the second target time point, restoring the real-time synchronization data from the standby host database to the standby open database.
Optionally, in the above data synchronization method, the step-by-step synchronizing the data from the first point in time of tracking in the production host database into the standby host database includes:
starting database synchronization software QREP, and capturing the log of the production host database through a log capturing process in the database synchronization software QREP;
and playing back the log of the production host database through an application process in the standby host database, extracting the data not earlier than the first count point in time, and storing the data into the standby host database.
Optionally, in the above data synchronization method, the recording the first target time point when synchronizing to the data of the second tracking time point, and the recording the second target time point when synchronizing to the data of the third tracking time point includes:
querying a time point when the data of each data packet is synchronized to the data of the second count point in time from an application monitoring table when the data of each data packet is synchronized to the data of the second count point in time;
determining the latest time point among time points when the data of each data packet is synchronized to the data of the second trace time point as the first target time point;
querying, from the application monitoring table, a point in time when the data of each of the data packets is synchronized to the data of the second point in time when the data of each of the data packets is synchronized to the data of the third point in time when the data of each of the data packets is synchronized to the data of the second point in time;
and determining the latest time point in time points when the data of the data packets are synchronized to the data of the third tracking time point as the second target time point.
Optionally, in the above data synchronization method, the step of synchronizing the data from the first target time point in the standby host database to the standby open database includes:
changing the subscription mode of the standby open database from a standard mode to an adaptive mode;
setting a log point of data subscription as the first target time point;
starting subscription of the standby host database, gradually acquiring data of the standby host database from the currently set log point through the self-adaptive mode, and storing the data into the standby open database;
and after synchronizing the data to the second target time point, recovering real-time synchronous data from the standby host database to a standby open database, including:
and after synchronizing to the data of the second target time point, restoring the subscription mode of the standby open database to a standard mode.
Optionally, in the above data synchronization method, the method further includes:
comparing whether the incremental data of the standby host database and the standby open database in the same time end are consistent or not;
and if the incremental data of the standby host database and the incremental data of the standby open database in the same time end are consistent, determining that the seamless connection of the data is successful.
A second aspect of the present application provides a data synchronizing device, including:
a stopping unit for stopping synchronizing data from the production host database to the standby host database and stopping synchronizing data from the standby host database to the standby open database, and recording a stopping synchronization time point;
a priming unit for priming the backup host database with data before a priming time point in the production host database; wherein the priming time point is before the stop synchronization time point;
the counting unit is used for gradually synchronizing the data from the first counting time point in the production host database to the standby host database; the first tracking time point is the difference value between the bottoming time point and a first preset value;
a recording unit configured to record a first target time point when data is synchronized to the second count time point, and record a second target time point when data is synchronized to the third count time point; the second count time point is a difference value between the synchronization stopping time point and a second preset value; the third count time point is the sum of the stop synchronization time point and a third preset value;
the synchronization unit is used for gradually synchronizing the data from the first target time point in the standby host database into the standby open database;
and the recovery unit is used for recovering the real-time synchronous data from the standby host database to the standby open database after synchronizing the data to the second target time point.
Optionally, in the above data synchronization device, the tracking unit includes:
the first starting unit is used for starting database synchronization software QREP and grabbing logs of the production host database through a log grabbing process in the database synchronization software QREP;
and the storage unit is used for playing back the log of the production host database through an application process in the standby host database, extracting the data which is not earlier than the first count point in time and storing the data into the standby host database.
Optionally, in the above data synchronization device, the recording unit includes:
a first inquiry unit configured to inquire, when data of each data packet has been synchronized to data of a second trace time point, a time point when the data of each data packet is synchronized to the data of the second trace time point from an application monitoring table;
a first recording unit configured to synchronize data of each of the data packets to a latest time point among time points when the data of the second trace time point is recorded, as the first target time point;
a second inquiry unit configured to inquire, when data of each of the data packets has been synchronized to data of a third trace point in time, a point in time when data of each of the data packets is synchronized to data of the second trace point in time from the application monitoring table;
a second recording unit configured to determine, as the second target time point, a latest time point among time points when data of the respective data packets are synchronized to data of the third trace time point.
Optionally, in the above data synchronization device, the synchronization unit includes:
a changing unit, configured to change a subscription mode of the standby open database from a standard mode to an adaptive mode;
a setting unit, configured to set a log point of a data subscription as the first target time point;
the second starting unit is used for starting subscription of the standby host database, gradually acquiring data of the standby host database from the currently set log point through the self-adaptive mode, and storing the data into the standby open database;
wherein the recovery unit includes:
and the recovery subunit is used for recovering the subscription mode of the standby open database to the standard mode after synchronizing the data to the second target time point.
Optionally, in the above data synchronization device, the method further includes:
the comparison unit is used for comparing whether the incremental data of the standby host database and the incremental data of the standby open database in the same time end are consistent;
and the determining unit is used for determining that the seamless connection of the data is successful when the incremental data of the standby host database and the incremental data of the standby open database in the same time end are compared to be consistent.
A third aspect of the present application provides an electronic device, comprising:
a memory and a processor;
wherein the memory is used for storing programs;
the processor is configured to execute the program, where the program is executed, and specifically configured to implement a data synchronization method according to any one of the foregoing methods.
A fourth aspect of the present application provides a computer storage medium storing a computer program for implementing a data synchronization method as claimed in any one of the preceding claims when executed.
The embodiment of the application provides a data synchronization method, which comprises the steps of firstly stopping synchronizing data from a production host database to a standby host database, stopping synchronizing data from the standby host database to a standby open database, and recording a stopping synchronization time point. And then, data priming is carried out on the standby host database by utilizing the data before the priming time point in the production host database. And then gradually synchronizing the data from the first tracking time point in the production host database into the standby host database so as to continue tracking after priming. The first tracking time point is a difference value between the priming time point and a first preset value, so that data omission is avoided. Then, a first target time point when data synchronized to the second tracking time point is recorded, and a second target time point when data synchronized to the third tracking time point is recorded. The second tracking time point is a difference value between the synchronization stopping time point and a second preset value. The third tracking time point is the sum of the stop synchronization time point and a third preset value, so that seamless connection of data synchronization of the subsequent standby open database is realized, and the standby open database is not subjected to recoiling tracking. And finally, gradually synchronizing the data from the first target time point in the standby host database into the standby open database, and recovering the real-time synchronization data from the standby host database to the standby open database after synchronizing the data to the second target time point. Therefore, the recoating number of the backup host database can be realized without manual intervention, the recoating number of the backup open database can be realized, the seamless connection of data synchronization can be realized, the consistency of the data is ensured, the accuracy is further effectively ensured, and the service level is not reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a data synchronization method according to an embodiment of the present application;
FIG. 2 is a flow chart of a method for stepwise synchronizing data into a standby host database according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for recording a first target time point and a second target time point according to an embodiment of the present application;
FIG. 4 is a flowchart of a method for synchronizing data into a standby open database according to an embodiment of the present application;
fig. 5 is a schematic diagram of a data synchronization device according to an embodiment of the present application;
fig. 6 is a schematic diagram of an architecture of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In the present application, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The embodiment of the application provides a data synchronization method, as shown in fig. 1, which specifically comprises the following steps:
s101, synchronizing data from the production host database to the standby host database and synchronizing data from the standby host database to the standby open database are stopped, and a stopping synchronization time point is recorded.
Wherein the production host database refers to a host database in a production center. Accordingly, the standby host database refers to the host database in the disaster recovery center. The standby open database is an open database in the disaster recovery center, and an oracle database is generally adopted.
It should be noted that, since the number of the backup host databases needs to be refreshed by using the data in the production host databases, and the process is equivalent to deleting all the data in the backup host databases, that is, clearing the data in the backup host databases, the process needs to stop synchronizing the data from the production host databases to the backup host databases, so as to avoid the situation that the data clearing cannot be realized all the time. Accordingly, there is no need to synchronize data from the standby host database to the standby open database, and thus synchronization of data from the standby host database to the standby open database is also stopped.
In order to facilitate the subsequent data synchronization, the synchronization can be accurately performed until the time of stopping, so that the time point of stopping the synchronization needs to be recorded.
Optionally, the data synchronization between the current host databases is mainly implemented by database synchronization software qrps, and the data synchronization between the host databases and the open databases is implemented by CDC subscription replication, so that the forward qrps synchronization replication and the CDC subscription replication of the disaster recovery center are specifically stopped.
S102, data priming is carried out on the standby host database by utilizing the data before the priming time point in the production host database.
The priming time point is the time point when the data in the production host database is accurately synchronized to the standby host database is cut off.
Because the data synchronization has a certain time difference and other problems, the data synchronization condition of the current synchronization stopping time point has uncertainty, so that the priming time point is before the synchronization stopping time point in order to ensure the accurate synchronization of the data.
And copying the data generated from the production host database up to the priming time point into the standby host database, and covering the data in the standby host database to realize the data priming of the standby host database.
S103, gradually synchronizing the data from the first tracking time point in the production host database into the standby host database.
The first tracking time point is the difference between the priming time point and a first preset value. The first preset value can be set according to the requirement, and can be set to be 10 minutes generally.
It should be noted that, since the data synchronized to the backup host database at the time of priming only ends up to the data at the priming time point, the subsequent data still needs to be synchronized to the backup database. In order to avoid missing data synchronization, the data synchronization from the production host database is recovered from the first point in time after the priming is completed, instead of from the priming point in time. Although partial data repetition occurs, synchronization of all data can be effectively ensured.
Optionally, in another embodiment of the present application, a specific implementation of step S103, as shown in fig. 2, includes the following steps:
s201, starting database synchronization software QREP, and capturing the log of the production host database through a log capturing process in the database synchronization software QREP.
It should be noted that, in the embodiment of the present application, the data synchronization between the host databases is implemented by the database synchronization software qrps, so that the corresponding tracking is started by starting the database synchronization software qrps.
Alternatively, a log grabbing process CAPTURE in the database synchronization software QREP may be started, and then the log serial number of the log grabbing process CAPTURE is assigned to, that is, the CAPTURE LSN value is assigned as the first count time, so that the user can grab the log from the first count time and extract data for synchronization.
S202, replaying the log of the production host database through an application process in the standby host database, extracting the data which is not earlier than the first count time point, and storing the data into the standby host database.
S104, recording a first target time point when the data is synchronized to the second tracking time point and recording a second target time point when the data is synchronized to the third tracking time point.
In addition, when the synchronization is stopped, the backup open database already stores the data before the synchronization stopping time point. The standby open database need not be data synchronized until the standby host database is also data that catches up with the synchronization point in time in the production host database.
When the standby host catches up with the data of the synchronization time point in the production host database, the time for storing the data is the catch-up time point, and the uncertainty of the data synchronization of the catch-up time point is overcome, so that in order to ensure that the follow-up standby open database catches up, a gap does not exist, and data omission is avoided, and therefore, the next time point before the data synchronized to the synchronization stopping time point and the next time point after the data synchronized to the synchronization stopping time point need to be recorded. The backup open database can be subsequently enabled to synchronize data for that period of time. The method can also generate partial data repetition, but can ensure that data dependence does not occur, and realize seamless connection of data synchronization.
Therefore, it is necessary to record the first target time point when the data is synchronized to the second count time point, i.e., the time of the current system when the data is synchronized to the second count time point, and record the second target time point when the data is synchronized to the third count time point, i.e., the time of the current system when the data is synchronized to the third count time point. The second tracking time point is the difference between the synchronization stopping time point and the second preset value. The third tracking time point is the sum of the stop synchronization time point and a third preset value. The second preset value and the third preset value are usually set to the same value, and in order to avoid excessive repeated data, the second preset value and the third preset value are usually set to 5 minutes.
Optionally, in another embodiment of the present application, a specific implementation of step S104, as shown in fig. 3, includes the following steps:
s301, when the data of each data packet is synchronized to the data of the second count point in time, the point in time when the data of each data packet is synchronized to the data of the second count point in time is queried from the application monitoring table.
It should be noted that qrps typically divide data into multiple data packets (Consistency Group, CG) for multi-threaded transmission when performing data synchronization.
Specifically, when the data of each data packet is synchronized and all the data are synchronized to the data of the second count point in TIME, that is, when the data of the second count point in TIME in the data packet is already synchronized to the standby host database, the point in TIME when the data of each data packet is synchronized to the data of the second count point in TIME, that is, the monitor_time at that TIME is queried.
S302, determining a latest time point among time points when data of each data packet is synchronized to data of the second trace time point as a first target time point.
Since the latest time point is the time point when all the data packets are synchronized to the second trace time point, the latest time point among time points when the data of the respective data packets are synchronized to the data of the second trace time point is determined as the first target time point.
S303, when the data of each data packet is synchronized to the data of the third count point in time, the point in time when the data of each data packet is synchronized to the data of the second count point in time is queried from the application monitoring table.
S304, determining the latest time point among the time points when the data of the respective data packets are synchronized to the data of the third trace time point as the second target time point.
S105, gradually synchronizing the data from the first target time point in the standby host database into the standby open database.
Since the standby host database can already be synchronized from the production host database to the data not available in the standby open database at this time, the open database begins to perform data synchronization at this time. In order to realize seamless connection of data synchronization, data omission is avoided. It is therefore necessary to synchronize data from a first target point in time to a second target point in time in the standby host database into the standby open database.
Alternatively, in another embodiment of the present application, a specific implementation of step S105, as shown in fig. 4, includes the following steps:
s401, changing the subscription mode of the standby open database from a standard mode to an adaptive mode.
It should be noted that, in the embodiment of the present application, the standby open database synchronizes data from the standby host database by means of CDC software subscription. And the CDC software synchronous data has three modes, one is a standard mode which is adapted under normal conditions, and the adding, deleting and deleting of the source end are needed to be correspondingly synchronized at the target end in the mode. The second is an audit mode in which the addition, deletion and deletion of the source are recorded, for example, the source inserts a record and the target inserts a record. The source end modifies a non-key record, and the target end inserts a modified record or two records before modification. And the self-adaptive mode is to map inconsistent data into the target terminal based on the data of the source terminal.
Therefore, in the embodiment of the application, in order to synchronize the data between the first target time point and the second target time point into the standby open database, the subscription mode is changed from the standard mode to the adaptive mode.
S402, setting a log point of data subscription as a first target time point.
In order to enable data subscription copying from the first target point in time, a log point of data subscription (setlog pos) needs to be set as the first target point in time.
Optionally, since the current software needs to perform data synchronization based on greenwich time, and the system time generally recorded is a locally used time, for example, beijing time, when the SETLOGGOS sets the synchronization point, the clock tool may be used to convert the first target time point into the greenwich time, so that a Log Record Sequence Number (LRSN) corresponding to the greenwich time may be obtained, and further data synchronization may be performed based on the LRSN.
S403, starting subscription of the standby host database, so as to gradually acquire data of the standby host database from the currently set log point through a self-adaption mode, and storing the data into a standby open database.
Accordingly, in the embodiment of the present application, the specific implementation manner of step S106 is as follows:
and after synchronizing the data to the second target time point, restoring the subscription mode of the standby open database to the standard mode.
S106, after the data is synchronized to the second target time point, the real-time synchronization data from the standby host database to the standby open database is restored.
Because the standby open database is also completed in seamless connection with the standby host data on the data at this time, the real-time synchronization of the data can be normally performed in the follow-up process, so that the real-time synchronization of the data from the standby host database to the standby open database can be restored at this time.
Optionally, in another embodiment of the present application, after performing step S106, further performing may be performed:
and comparing whether the incremental data of the standby host database and the standby open database in the same time end are consistent.
If the incremental data of the standby host database and the incremental data of the standby open database in the same time end are compared to be consistent, the seamless connection of the data is determined to be successful.
The embodiment of the application provides a data synchronization method, which comprises the steps of firstly stopping synchronizing data from a production host database to a standby host database, stopping synchronizing data from the standby host database to a standby open database, and recording a stopping synchronization time point. And then, data priming is carried out on the standby host database by utilizing the data before the priming time point in the production host database. And then gradually synchronizing the data from the first tracking time point in the production host database into the standby host database so as to continue tracking after priming. The first tracking time point is a difference value between the priming time point and a first preset value, so that data omission is avoided. Then, a first target time point when data synchronized to the second tracking time point is recorded, and a second target time point when data synchronized to the third tracking time point is recorded. The second tracking time point is a difference value between the synchronization stopping time point and a second preset value. The third tracking time point is the sum of the stop synchronization time point and a third preset value, so that seamless connection of data synchronization of the subsequent standby open database is realized, and the standby open database is not subjected to recoiling tracking. And finally, gradually synchronizing the data from the first target time point in the standby host database into the standby open database, and recovering the real-time synchronization data from the standby host database to the standby open database after synchronizing the data to the second target time point. Therefore, the recoating number of the backup host database can be realized without manual intervention, the recoating number of the backup open database can be realized, the seamless connection of data synchronization can be realized, the consistency of the data is ensured, the accuracy is further effectively ensured, and the service level is not reduced.
Another embodiment of the present application provides a data synchronization device, as shown in fig. 5, including:
a stopping unit 501 for stopping synchronizing data from the production host database to the backup host database and stopping synchronizing data from the backup host database to the backup open database, and recording a stopping synchronization time point.
A priming unit 502, configured to perform data priming on the standby host database by using data before a priming time point in the production host database.
Wherein the priming time point is before the stopping synchronization time point.
The tracking unit 503 is configured to synchronize data from the first tracking time point in the production host database to the standby host database step by step.
The first tracking time point is the difference between the priming time point and a first preset value.
A recording unit 504 for recording a first target time point when data is synchronized to the second count time point, and recording a second target time point when data is synchronized to the third count time point.
The second tracking time point is a difference value between the synchronization stopping time point and a second preset value. The third tracking time point is the sum of the stop synchronization time point and a third preset value.
And the synchronization unit 505 is configured to gradually synchronize data from the first target time point in the standby host database to the standby open database.
And a recovery unit 506, configured to recover real-time synchronization data from the standby host database to the standby open database after synchronizing the data to the second target time point.
Optionally, in the data synchronization device provided in another embodiment of the present application, the tracking unit includes:
the first starting unit is used for starting database synchronization software QREP and grabbing logs of the production host database through a log grabbing process in the database synchronization software QREP.
And the storage unit is used for playing back the log of the production host database through an application process in the standby host database, extracting the data not earlier than the first count time point and storing the data into the standby host database.
Optionally, in the data synchronization device provided in another embodiment of the present application, the recording unit includes:
and a first inquiry unit configured to inquire, from the application monitoring table, a point in time when the data of each data packet is synchronized to the data of the second point in time.
A first recording unit configured to determine, as a first target time point, a latest time point among time points when data of the respective data packets are synchronized to data of the second catch-up time point.
And a second inquiry unit configured to inquire from the application monitoring table a point in time when the data of each data packet is synchronized to the data of the second point in time when the data of each data packet is synchronized to the data of the third point in time.
A second recording unit configured to determine, as a second target time point, a latest time point among time points when data of the respective data packets are synchronized to data of the third trace time point.
Optionally, in the data synchronization device provided in another embodiment of the present application, the synchronization unit includes:
and the changing unit is used for changing the subscription mode of the standby open database from the standard mode to the adaptive mode.
And the setting unit is used for setting the log point of the data subscription as a first target time point.
And the second starting unit is used for starting subscription of the standby host database so as to gradually acquire data of the standby host database from the currently set log point through the self-adaptive mode and store the data into the standby open database.
The recovery unit in the embodiment of the application comprises:
and the recovery subunit is used for recovering the subscription mode of the standby open database to the standard mode after synchronizing the data to the second target time point.
Optionally, in the data synchronization device provided in another embodiment of the present application, the data synchronization device further includes:
the comparison unit is used for comparing whether the incremental data of the standby host database and the standby open database in the same time end are consistent.
And the determining unit is used for determining that the seamless connection of the data is successful when the incremental data of the standby host database and the incremental data of the standby open database in the same time end are compared to be consistent.
It should be noted that, for the specific working process of each unit provided in the above embodiment of the present application, reference may be made correspondingly to the implementation process of the corresponding step in the above method embodiment, which is not repeated herein.
Another embodiment of the present application provides an electronic device, as shown in fig. 6, including:
a memory 601 and a processor 602.
Wherein the memory 601 is used for storing programs.
The processor 602 is configured to execute a program stored in the memory 601, and the program is specifically configured to implement the data synchronization method provided in any one of the embodiments described above when executed.
Another embodiment of the present application provides a computer storage medium storing a computer program for implementing the data synchronization method according to any one of the above, when the computer program is executed.
Computer storage media, including both non-transitory and non-transitory, removable and non-removable media, may be implemented in any method or technology for storage of information. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, read only compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by the computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of data synchronization, comprising:
stopping synchronizing data from the production host database to the standby host database, stopping synchronizing data from the standby host database to the standby open database, and recording a stopping synchronization time point;
performing data priming on the standby host database by utilizing data before a priming time point in the production host database; wherein the priming time point is before the stop synchronization time point;
gradually synchronizing data from a first tracking time point in the production host database to the standby host database; the first tracking time point is the difference value between the bottoming time point and a first preset value;
recording a first target time point when synchronizing to the data of the second count time point, and recording a second target time point when synchronizing to the data of the third count time point; the second count time point is a difference value between the synchronization stopping time point and a second preset value; the third count time point is the sum of the stop synchronization time point and a third preset value;
gradually synchronizing data from the first target time point in the standby host database to the standby open database;
and after synchronizing the data to the second target time point, restoring the real-time synchronization data from the standby host database to the standby open database.
2. The method of claim 1, wherein the step of comparing the data from the first point in time to the production host database, step-by-step synchronization into the standby host database, comprising:
starting database synchronization software QREP, and capturing the log of the production host database through a log capturing process in the database synchronization software QREP;
and playing back the log of the production host database through an application process in the standby host database, extracting the data not earlier than the first count point in time, and storing the data into the standby host database.
3. The method of claim 1, wherein recording a first target point in time when synchronizing to the data of the second point in time and recording a second target point in time when synchronizing to the data of the third point in time comprises:
querying a time point when the data of each data packet is synchronized to the data of the second count point in time from an application monitoring table when the data of each data packet is synchronized to the data of the second count point in time;
determining the latest time point among time points when the data of each data packet is synchronized to the data of the second trace time point as the first target time point;
querying, from the application monitoring table, a point in time when the data of each of the data packets is synchronized to the data of the second point in time when the data of each of the data packets is synchronized to the data of the third point in time when the data of each of the data packets is synchronized to the data of the second point in time;
and determining the latest time point in time points when the data of the data packets are synchronized to the data of the third tracking time point as the second target time point.
4. The method of claim 1, wherein synchronizing data from the first target point in time in the standby host database step-by-step into the standby open database comprises:
changing the subscription mode of the standby open database from a standard mode to an adaptive mode;
setting a log point of data subscription as the first target time point;
starting subscription of the standby host database, gradually acquiring data of the standby host database from the currently set log point through the self-adaptive mode, and storing the data into the standby open database;
and after synchronizing the data to the second target time point, recovering real-time synchronous data from the standby host database to a standby open database, including:
and after synchronizing to the data of the second target time point, restoring the subscription mode of the standby open database to a standard mode.
5. The method as recited in claim 1, further comprising:
comparing whether the incremental data of the standby host database and the standby open database in the same time end are consistent or not;
and if the incremental data of the standby host database and the incremental data of the standby open database in the same time end are consistent, determining that the seamless connection of the data is successful.
6. A data synchronization device, comprising:
a stopping unit for stopping synchronizing data from the production host database to the standby host database and stopping synchronizing data from the standby host database to the standby open database, and recording a stopping synchronization time point;
a priming unit for priming the backup host database with data before a priming time point in the production host database; wherein the priming time point is before the stop synchronization time point;
the counting unit is used for gradually synchronizing the data from the first counting time point in the production host database to the standby host database; the first tracking time point is the difference value between the bottoming time point and a first preset value;
a recording unit configured to record a first target time point when data is synchronized to the second count time point, and record a second target time point when data is synchronized to the third count time point; the second count time point is a difference value between the synchronization stopping time point and a second preset value; the third count time point is the sum of the stop synchronization time point and a third preset value;
the synchronization unit is used for gradually synchronizing the data from the first target time point in the standby host database into the standby open database;
and the recovery unit is used for recovering the real-time synchronous data from the standby host database to the standby open database after synchronizing the data to the second target time point.
7. The apparatus of claim 6, wherein the tracking unit comprises:
the first starting unit is used for starting database synchronization software QREP and grabbing logs of the production host database through a log grabbing process in the database synchronization software QREP;
and the storage unit is used for playing back the log of the production host database through an application process in the standby host database, extracting the data which is not earlier than the first count point in time and storing the data into the standby host database.
8. The apparatus according to claim 6, wherein the recording unit includes:
a first inquiry unit configured to inquire, when data of each data packet has been synchronized to data of a second trace time point, a time point when the data of each data packet is synchronized to the data of the second trace time point from an application monitoring table;
a first recording unit configured to synchronize data of each of the data packets to a latest time point among time points when the data of the second trace time point is recorded, as the first target time point;
a second inquiry unit configured to inquire, when data of each of the data packets has been synchronized to data of a third trace point in time, a point in time when data of each of the data packets is synchronized to data of the second trace point in time from the application monitoring table;
a second recording unit configured to determine, as the second target time point, a latest time point among time points when data of the respective data packets are synchronized to data of the third trace time point.
9. An electronic device, comprising:
a memory and a processor;
wherein the memory is used for storing programs;
the processor is configured to execute the program, in particular when the program is executed, for implementing the data synchronization method according to any one of claims 1 to 5.
10. A computer storage medium storing a computer program which, when executed, is adapted to carry out the data synchronization method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311249105.6A CN117194111A (en) | 2023-09-26 | 2023-09-26 | Data synchronization method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311249105.6A CN117194111A (en) | 2023-09-26 | 2023-09-26 | Data synchronization method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117194111A true CN117194111A (en) | 2023-12-08 |
Family
ID=88992346
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311249105.6A Pending CN117194111A (en) | 2023-09-26 | 2023-09-26 | Data synchronization method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117194111A (en) |
-
2023
- 2023-09-26 CN CN202311249105.6A patent/CN117194111A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108241555B (en) | Backup and recovery method and device of distributed database and server | |
| US11256715B2 (en) | Data backup method and apparatus | |
| US20220327026A1 (en) | Partial database restoration | |
| CN111506251B (en) | Data processing method and device, SMR storage system and storage medium | |
| CN107515874B (en) | Method and equipment for synchronizing incremental data in distributed non-relational database | |
| US20230041089A1 (en) | State management methods, methods for switching between master application server and backup application server, and electronic devices | |
| US10204016B1 (en) | Incrementally backing up file system hard links based on change logs | |
| CN106933843B (en) | Database heartbeat detection method and device | |
| CN109298978B (en) | Recovery method and system for database cluster of specified position | |
| CN113360322A (en) | Method and equipment for recovering data based on backup system | |
| CN110287251B (en) | MongoDB-HBase distributed high fault-tolerant data real-time synchronization method | |
| CN112579820B (en) | Time hopping video data processing method, device, medium and electronic equipment | |
| CN117194111A (en) | Data synchronization method and device, electronic equipment and storage medium | |
| US9003018B2 (en) | System and method for data set synchronization and replication | |
| WO2017067397A1 (en) | Data recovery method and device | |
| CN110928945B (en) | Data processing method and device for database and data processing system | |
| US8918364B1 (en) | Online mirror state transitioning in databases | |
| CN114282048B (en) | Playing record warehousing method and device, storage medium and electronic equipment | |
| CN106354830B (en) | Method and device for data synchronization between database cluster nodes | |
| CN110427423A (en) | A kind of method, equipment and readable medium for avoiding database session from interrupting | |
| CN111382024B (en) | Database master-slave copy delay monitoring method, device and system | |
| CN111984472B (en) | Data snapshot method, device and related equipment | |
| CN107153699B (en) | Method and device for dynamically expanding cluster server | |
| CN111221801A (en) | Database migration method, system and related device | |
| CN106375354B (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |