CN111723066B - Database switching method and database switching system based on log analysis synchronization - Google Patents
Database switching method and database switching system based on log analysis synchronization Download PDFInfo
- Publication number
- CN111723066B CN111723066B CN202010382970.8A CN202010382970A CN111723066B CN 111723066 B CN111723066 B CN 111723066B CN 202010382970 A CN202010382970 A CN 202010382970A CN 111723066 B CN111723066 B CN 111723066B
- Authority
- CN
- China
- Prior art keywords
- database
- target
- source
- switching
- data synchronization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides a database switching method and a database switching system based on log analysis synchronization, wherein the database switching method comprises the following steps: the source data synchronization system inserts a switching mark in a switching mark table of a source database; capturing and analyzing an operation log of a source database; judging whether the inserted switching mark exists in the operation log; if yes, stopping the log capturing service, and sending a master-slave switching command to a target-end data synchronization system; after receiving the main and standby switching commands, the target-end data synchronization system synchronizes the current asynchronous transaction to a target-end database; and switching the source end database into a standby database and switching the target end database into a main database. In the invention, a mode of inserting a row of switching marks in the switching mark table is adopted, and the source-end data synchronization system can clearly know that the operation is the end of the log stream after capturing the operation so as to perform switching preparation work, thereby ensuring that all unsynchronized transactions are synchronized to the target end.
Description
Technical Field
The invention belongs to the technical field of data synchronization, and particularly relates to a database switching method and a database switching system based on log analysis synchronization.
Background
In a database real-time synchronization system (supporting heterogeneous) based on a log analysis architecture, a source data synchronization system is responsible for capturing an operation log of a source database, analyzing and restoring the log to obtain corresponding database operations, and then sending the corresponding database operations to a target data synchronization system, wherein the target data synchronization system is responsible for executing the operations in a target database, so that a data synchronization process is completed.
Under the data synchronization environment, an application scene exists, wherein a source database is mainly responsible for writing business in a production system, a target database is used for sharing reading business in the production system, so that the pressure of the production system on the source database is relieved, and the source database is called a master database and the target database is called a standby database in the application scene. When the source database needs to be maintained but the service cannot be interrupted for a long time, the current data synchronization environment needs to be switched, the source database is switched to the standby database, and the target database is switched to the main database, so that the service system can continue running the written service on the original target database, and the influence of operation and maintenance on the service system is reduced.
However, the data synchronization environment constructed by using the log parsing method is asynchronous data synchronization, and is different from the environment in which the database uses the REDO log to synchronize in real time, when the primary and the secondary in the asynchronous data synchronization environment are switched, the switching can be performed after all the services on the source database are synchronized to the target database, so how to complete the switching between the primary and the secondary in the asynchronous synchronization of the dual-active database is a technical problem to be solved in the industry.
Disclosure of Invention
In order to meet the above-mentioned defects or improvement requirements of the prior art, the present invention provides a database switching method and a database switching system based on log analysis synchronization, and adopts a mode of inserting a row of switching marks in a switching mark table, the operation represents the termination of all the previous production data logs in the log stream, and the source data synchronization system can clearly know that the operation is the end of the log stream after capturing the operation, and then performs switching preparation work. By adopting the scheme of the invention, when the main and standby are switched based on an asynchronous data synchronization environment, all the services on the source end database can be ensured to be synchronized to the target end database, and the consistency of the transaction is ensured.
In order to achieve the above object, according to one aspect of the present invention, there is provided a database switching method based on log parsing synchronization, which constructs a bidirectional synchronization link between a source database and a target database, initializes the source database to a master database and initializes the target database to a backup database, wherein the master database is used for writing data, and the backup database is used for reading data;
wherein, the source end database and the target end database are respectively provided with a switching mark table;
the database switching method comprises the following steps:
the source data synchronization system inserts a switching mark in a switching mark table of the source database;
capturing and analyzing an operation log of the source database;
judging whether the inserted switching mark exists in the operation log or not;
if yes, stopping the log capturing service, and sending a master-slave switching command to a target-end data synchronization system;
after receiving the master-slave switching command, the target-side data synchronization system synchronizes the current asynchronous transaction to the target-side database;
and switching the source end database into a standby database and switching the target end database into a main database.
Preferably, the source data synchronization system further includes, before inserting a switch flag in a switch flag table of the source database:
starting a synchronous link from the source end database to the target end database;
and closing a synchronous link from the target database to the source database.
Preferably, the source data synchronization system further includes, before inserting a switch flag in a switch flag table of the source database:
when the source database is required to be changed into a standby database, the source data synchronization system acquires target connection established by an application on the source database and blocks the target connection.
Preferably, after blocking the target connection, the database switching method further includes:
and recovering the write permission of the production user corresponding to the application on the source database, and configuring the read permission for the production user.
Preferably, after receiving the active-standby switch command, the target-side data synchronization system synchronizes the current unsynchronized transaction to the target-side database, and then further includes:
the target end data synchronization system acquires target connection established by an application on the target end database, and blocks the target connection;
and configuring write permission for a production user corresponding to the application on the target database.
Preferably, after configuring the write permission for the production user corresponding to the application on the target database, the method further includes:
the target-side data synchronization system stops log receiving service;
and sending a feedback instruction to the source-side data synchronization system, wherein the feedback instruction is used for indicating that the switching preparation work is finished.
Preferably, switching the source database to a standby database and switching the target database to a primary database includes:
the source data synchronization system starts a log receiving service to switch the source database into a standby database;
the source end data synchronization system sends a feedback message of switching completion to the target end data synchronization system;
after receiving the feedback message, the target-side data synchronization system starts a log capturing service to switch the target-side database into a main database.
Preferably, the database switching method further includes:
acquiring a current maximum log sequence number LSN0 in the target-end database;
after the target end database is switched to a main database, acquiring an operation log larger than the LSN0;
synchronizing the operation log larger than the LSN0 to the source database.
Preferably, the determining whether the inserted switching flag exists in the operation log includes:
judging whether the operation object aimed by the operation log is the switching mark table or not;
if yes, judging whether the inserted switching mark exists in the operation log;
if not, continuing to acquire the next operation log.
To achieve the above object, according to another aspect of the present invention, there is provided a database switching system comprising: the system comprises a source end database, a source end data synchronization system, a target end database and a target end data synchronization system, wherein the source end database is connected with the source end data synchronization system, the target end database is connected with the target end data synchronization system, and the source end data synchronization system is connected with the target end data synchronization system;
the method comprises the steps of constructing a bidirectional synchronous link between a source end database and a target end database, initializing the source end database into a main database and initializing the target end database into a standby database, wherein the main database is used for writing data, and the standby database is used for reading data; a switching mark table is established in the source end database and the target end database;
the source data synchronization system is used for inserting a switching mark in a switching mark table of the source database;
the source data synchronization system is used for capturing and analyzing an operation log of the source database;
the source-side data synchronization system is used for judging whether the inserted switching mark exists in the operation log or not; if yes, stopping the log capturing service, and sending a master-slave switching command to a target-end data synchronization system;
and the target-end data synchronization system is used for synchronizing the current asynchronous transaction to the target-end database after receiving the main-standby switching command so as to switch the source-end database into a standby database and switch the target-end database into a main database.
In general, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects: the invention provides a database switching method and a database switching system based on log analysis synchronization, wherein the database switching method comprises the following steps: the source data synchronization system inserts a switching mark in a switching mark table of the source database; capturing and analyzing an operation log of the source database; judging whether the inserted switching mark exists in the operation log or not; if yes, stopping the log capturing service, and sending a master-slave switching command to a target-end data synchronization system; after receiving the master-slave switching command, the target-side data synchronization system synchronizes the current asynchronous transaction to the target-side database; and switching the source end database into a standby database and switching the target end database into a main database.
In the invention, because the architecture based on log analysis synchronization is an asynchronous synchronization architecture, and the data synchronization has uncertain delay, the log capture analysis still cannot be stopped immediately under the condition that the source database does not generate any log, and the current log can be switched after the synchronization is completed, and the aim can be effectively achieved by using the switching mark table. A line of switching mark is inserted in the switching mark table, the operation represents the termination of all the previous production data logs in the log stream, and the source data synchronization system can clearly know that the operation is the end of the log stream after capturing the operation, and then performs switching preparation work. By adopting the scheme of the invention, when the main and standby are switched based on an asynchronous data synchronization environment, all the services on the source end database can be ensured to be synchronized to the target end database, and the consistency of the transaction is ensured.
Drawings
Fig. 1 is a schematic flow chart of a database switching method based on log parsing synchronization according to an embodiment of the present invention;
FIG. 2 is a flowchart of another database switching method based on log resolution synchronization according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a database switching system according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In the description of the present invention, the terms "inner", "outer", "longitudinal", "transverse", "upper", "lower", "top", "bottom", etc. refer to an orientation or positional relationship based on that shown in the drawings, merely for convenience of describing the present invention and do not require that the present invention must be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.
In addition, the technical features of the embodiments of the present invention described below may be combined with each other as long as they do not collide with each other.
Example 1:
in the practical application scene, in order to realize the fast switching of the data synchronization main and standby, the bidirectional synchronization of the main library and the standby library can be built, but partial hardware resources are consumed when the standby library synchronizes to the main library, so that the whole synchronization performance is affected. Therefore, the bidirectional synchronous environment of the main library and the standby library is established in advance, but only the main-to-standby data synchronous service is started according to the requirement, so that the influence of bidirectional synchronization on the whole synchronous performance is reduced.
Before the database switching method of the present embodiment is executed, an environment on which the database switching method depends needs to be constructed. Firstly, a bidirectional synchronous link between a source end database and a target end database is constructed, specifically, a bidirectional data synchronous system from the source end database to the target end database is respectively built, and only one data synchronous link is started according to the current synchronous requirement. For example, in an initial state, the source database is initialized to a main database, and the target database is initialized to a standby database, wherein the main database is used for writing data, and the standby database is used for reading data. And a switching mark table is pre-established in the source end database and the target end database.
In the practical application scenario, the source end database is provided with an active end data synchronization system, the target end database is provided with a target end data synchronization system, the source end database synchronization system reads REDO logs from the source end database, and the target end database synchronization system is responsible for applying the synchronization operation sent by the source end to the target end database.
Next, referring to fig. 1, the present embodiment provides a database switching method based on log parsing synchronization, where the database switching method includes the following steps:
step 101: the source data synchronization system inserts a switching mark in a switching mark table of the source database.
In this embodiment, a switch flag table is created in the source database, and when the active/standby database needs to be switched, the source data synchronization system inserts a switch flag in the switch flag table of the source database, and completes the submission.
The switching mark is used for informing the source-side data synchronization system to stop log capturing service after capturing the switching mark, and then informing the target-side data synchronization system to perform switching operation after completing the existing transaction.
In this embodiment, to avoid the problem of additional hardware resource consumption caused by bidirectional synchronization, one of the synchronization links needs to be started before step 101 as follows: starting a synchronous link from the source end database to the target end database; and closing a synchronous link from the target database to the source database. Specifically, the source-side data synchronization system starts a log capturing service and closes a log receiving service; and the target-end data synchronization system closes the log capturing service and opens the log receiving service.
In the practical application scenario, in order to avoid modifying the data of the database, after inserting the switch flag, it needs to be ensured that the data of the database is not changed, and then the following procedure is further included before step 101: when the source database is required to be changed into a standby database, the source data synchronization system acquires target connection established by an application on the source database and blocks the target connection; and recovering the write permission of the production user corresponding to the application on the source database, and configuring the read permission for the production user. When the user rights are changed, the rights of the synchronous users are kept unchanged, wherein the synchronous users refer to users logging in the data synchronous system.
In this embodiment, the account number of the synchronous user is preset, and the production user and the synchronous user can be distinguished by the user name, so that the authority of the production user is changed.
In this embodiment, after the write permission of the production user is recovered, the operation of inserting the switching mark is performed, that is, after the switching mark is inserted, the production user cannot modify the data of the source database, so that the occurrence of inconsistent data of the source database and the target database can be avoided.
Step 102: and capturing and analyzing the operation log of the source database.
The source data synchronization system captures and analyzes an operation log of the source database.
Step 103: and judging whether the inserted switching mark exists in the operation log.
The source-side data synchronization system analyzes the operation log according to the log sequence, acquires an operation object corresponding to the operation log, judges whether the inserted switching mark exists in the operation log when the operation object corresponding to the operation log is a switching mark table, if so, executes step 104, and if not, continues to acquire the operation log, and executes step 103.
Step 104: if yes, stopping the log capturing service, and sending a master-slave switching command to the target-end data synchronization system.
When the inserted switching mark exists in the operation log, the source-end data synchronization system stops log capturing service and sends a main-standby switching command to the target-end data synchronization system so as to inform the target-end data synchronization system of the preparation work of main-standby switching.
Step 105: and after receiving the master-slave switching command, the target-end data synchronization system synchronizes the current asynchronous transaction to the target-end database.
Under the actual application scene, when the database is switched, the source data synchronization system ensures that all production logs on the source database are completely captured and delivered to the target data synchronization system, and then the target data synchronization system can switch after the warehousing work of the existing active transaction is completed.
Therefore, in this embodiment, after receiving the active/standby switch command, the target data synchronization system synchronizes the current unsynchronized transaction to the target database, so as to ensure that all existing logs synchronize the target.
Then, the target-end data synchronization system acquires target connection established by an application on the target-end database, and blocks the target connection; and configuring write permission for a production user corresponding to the application on the target database. When the user rights are changed, the rights of the synchronous users are kept unchanged, wherein the synchronous users refer to users logging in the data synchronous system.
In this embodiment, the account number of the synchronous user is preset, and the production user and the synchronous user can be distinguished by the user name, so that the authority of the production user is changed.
Further, after configuring write permission for the production user corresponding to the application on the target database, the target data synchronization system stops log receiving service; and sending a feedback instruction to the source-side data synchronization system, wherein the feedback instruction is used for indicating that the switching preparation work is finished.
Step 106: and switching the source end database into a standby database and switching the target end database into a main database.
In this embodiment, after all the current unsynchronized transactions are synchronized to the target database, the source data synchronization system starts a log receiving service to switch the source database to a standby database; the source end data synchronization system sends a feedback message of switching completion to the target end data synchronization system; after receiving the feedback message, the target-side data synchronization system starts a log capturing service to switch the target-side database into a main database.
In addition, after the target-side database is switched to the main database, in order to facilitate the target-side data synchronization system to accurately locate the initial log analysis position, in this embodiment, the database switching method further includes: acquiring a current maximum log sequence number LSN0 in the target-end database; after the target end database is switched to a main database, acquiring an operation log larger than the LSN0; synchronizing the operation log larger than the LSN0 to the source database.
In the practical application scenario, when the active/standby switch needs to be performed again, the switching method is the same as the foregoing description, except that the execution object is changed.
For a more intuitive database switching method, refer to fig. 2, where fig. 2 shows a specific flow during database switching, and a brief description is made below based on fig. 2.
When the primary and backup databases need to be switched, firstly, the source data synchronization system recovers the write permission of a production user corresponding to the application on the source database, configures the read permission for the production user, then inserts a switching mark in a switching mark table, captures an operation log of the source database, judges whether the switching mark exists in the operation log, stops log capturing service if the switching mark exists, notifies a target end to prepare for primary and backup switching, and continuously captures the operation log of the source database if the switching mark does not exist.
After receiving an instruction for preparing the master-slave switching, the target data synchronization system finishes all the transaction warehouse-in, then acquires the largest log serial number LSN of the target database, recovers the read authority of the production user corresponding to the application on the target database, grants the write authority of the production user corresponding to the application on the target database, and stops the log receiving service. After the preparation is completed, the source is notified that the handover preparation is ready.
After the source data synchronization system receives the information fed back by the target, the log receiving service is started, so that the source database is switched to the standby database, and the target is informed to prepare for cutting. After receiving the master cutting message fed back by the source end, the target end starts a log capturing service, captures an operation log with a log serial number greater than the largest log serial number LSN, and sends the log to the source end, so that the target end database is switched to the master database.
In the invention, in the whole primary and standby database switching flow, the mode of recovering the user authority of the source database is adopted, so that the production system can be effectively prevented from continuously modifying the data on the source database after the database switching is completed.
Because the architecture based on log analysis synchronization is an asynchronous synchronization architecture, the data synchronization has uncertain delay, so the log capture analysis still cannot be stopped immediately under the condition that the source database does not generate any log, the existing log can be switched after the synchronization is completed, and the aim can be effectively achieved by using the switching mark table. After recovering the relevant user authority of the source database, a line of switching mark is inserted into the switching mark list, the operation represents the termination of all the previous production data logs in the log stream, and the source data synchronization system can clearly know that the operation is the end of the log stream after capturing the operation, and then performs switching preparation work.
Secondly, the target-end data synchronization system needs to ensure that all the previously received transactions have completed warehousing before switching, then when switching to a main database, the log capturing service needs an initial log analysis position, and the log capturing service can be realized by acquiring the largest log serial number LSN in the current target-end database, and the logs smaller than the LSN belong to the logs generated by the previous source-end synchronization operation without capturing.
Example 2:
to facilitate understanding of the database switching method in the foregoing embodiment 1, examples are as follows:
in the prior art, a database A and a database B are used, wherein in an initial state, the database A is a main database, and the database B is a standby database.
The implementation process of the data cross-switching method is as follows:
(1) A switch flag table T is created on database a and database B.
(2) And constructing a bidirectional synchronization system based on log capturing on the database A and the database B, and starting only a synchronization link from the database A to the database B, wherein the data synchronization system needs to use exclusive synchronization users on the database A and the database B.
(3) When the database A is required to be switched to a standby database and the database B is required to be switched to a main database, the synchronous system at the side of the database A kills connection established by the application on the database A, recovers the write permission of the application corresponding to the production user and reconfigures the read permission for the production user.
(4) The synchronization system at the side of the database A inserts a row of switching marks in the switching mark table T on the database A and submits the switching marks.
(5) After the synchronous system at the database A captures the switching mark inserted in the step (4), stopping the log capturing service, and sending a master-slave switching command to the synchronous system at the database B to inform the synchronous system at the database B to prepare master-slave switching work.
(6) And the synchronous system at the side of the database B completes the warehousing of all the unsynchronized transactions.
(7) The synchronization system at the database B side obtains the maximum log sequence number LSN of the database B (wherein, in ORACLE, english corresponding to the log sequence number is abbreviated as SCN).
(8) The synchronous system at the side of the database B grants the write permission of the production user corresponding to the application on the database B.
(9) The synchronization system on the database B side stops the log receiving service and notifies the synchronization system on the database a side that the preparation work has been completed.
(10) The synchronization system at the database A side starts a log receiving service, and changes the role conversion from log capturing analysis to log receiving and warehousing.
(11) And the synchronization system at the side of the database B starts a log capturing service by using the obtained maximum log serial number LSN to complete role conversion from log receiving and warehousing to log capturing analysis.
(12) And finishing the primary and standby switching.
Example 3:
referring to fig. 3, corresponding to the foregoing embodiments 1 and 2, the present embodiment further provides a database switching system, where the database switching system includes: the system comprises a source end database, a source end data synchronization system, a target end database and a target end data synchronization system, wherein the source end database is connected with the source end data synchronization system, the target end database is connected with the target end data synchronization system, and the source end data synchronization system is connected with the target end data synchronization system.
The method comprises the steps of constructing a bidirectional synchronous link between a source end database and a target end database, initializing the source end database into a main database and initializing the target end database into a standby database, wherein the main database is used for writing data, and the standby database is used for reading data; a switching mark table is established in the source end database and the target end database;
the source data synchronization system is used for inserting a switching mark in a switching mark table of the source database;
the source data synchronization system is used for capturing and analyzing an operation log of the source database;
the source-side data synchronization system is used for judging whether the inserted switching mark exists in the operation log or not; if yes, stopping the log capturing service, and sending a master-slave switching command to a target-end data synchronization system;
and the target-end data synchronization system is used for synchronizing the current asynchronous transaction to the target-end database after receiving the main-standby switching command so as to switch the source-end database into a standby database and switch the target-end database into a main database.
Under the actual application scene, initializing the source end database as a main database, and after the target end database is a standby database, the source end data synchronization system is used for starting a log capturing service and closing a log receiving service; the target-end data synchronization system is used for closing the log capturing service and opening the log receiving service, so that only the synchronization link from the main library to the standby library is opened.
When the primary and standby switching is needed, the source data synchronization system is used for acquiring target connection established by the application on the source database and blocking the target connection; and the method is also used for recovering the write permission of the production user corresponding to the application on the source database and configuring the read permission for the production user. When the user rights are changed, the rights of the synchronous users are kept unchanged, wherein the synchronous users refer to users logging in the data synchronous system.
After the write permission of the production user is recovered, the source data synchronization system executes the operation of inserting the switching mark, that is, after the switching mark is inserted, the production user cannot modify the data of the source database, so that the condition that the data of the source database and the data of the target database are inconsistent can be avoided.
After the switching mark is captured in the operation log, the source-side data synchronization system is used for stopping log capturing service and sending a master-slave switching command to the target-side data synchronization system.
After receiving the active-standby switching command, the target-side data synchronization system is used for synchronizing a current asynchronous transaction to the target-side database, and then the target-side data synchronization system is used for acquiring target connection established by an application on the target-side database and blocking the target connection; and configuring write permission for a production user corresponding to the application on the target database. When the user rights are changed, the rights of the synchronous users are kept unchanged, wherein the synchronous users refer to users logging in the data synchronous system.
Further, after configuring write permission for the production user corresponding to the application on the target database, the target data synchronization system is used for stopping log receiving service; and sending a feedback instruction to the source-side data synchronization system, wherein the feedback instruction is used for indicating that the switching preparation work is finished.
After the target end database finishes the primary-backup switching preparation work, the source end data synchronization system is used for starting a log receiving service so as to switch the source end database into a backup database; the source end data synchronization system is also used for sending a feedback message of switching completion to the target end data synchronization system. And the target-end data synchronization system is used for starting a log capturing service after receiving the feedback message so as to switch the target-end database into a main database.
In addition, after the target end database is switched to the main database, in order to facilitate the target end data synchronization system to accurately locate the initial log analysis position, in this embodiment, the target end data synchronization system is further configured to obtain a current maximum log serial number LSN0 in the target end database; after the target end database is switched to a main database, acquiring an operation log larger than the LSN0; synchronizing the operation log larger than the LSN0 to the source database.
In the invention, because the architecture based on log analysis synchronization is an asynchronous synchronization architecture, and the data synchronization has uncertain delay, the log capture analysis still cannot be stopped immediately under the condition that the source database does not generate any log, and the current log can be switched after the synchronization is completed, and the aim can be effectively achieved by using the switching mark table. A line of switching mark is inserted in the switching mark table, the operation represents the termination of all the previous production data logs in the log stream, and the source data synchronization system can clearly know that the operation is the end of the log stream after capturing the operation, and then performs switching preparation work. By adopting the scheme of the invention, when the main and standby are switched based on an asynchronous data synchronization environment, all the services on the source end database can be ensured to be synchronized to the target end database, and the consistency of the transaction is ensured.
It will be readily appreciated by those skilled in the art that the foregoing description is merely a preferred embodiment of the invention and is not intended to limit the invention, but any modifications, equivalents, improvements or alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (7)
1. The database switching method based on log analysis synchronization is characterized by constructing a bidirectional synchronous link between a source database and a target database, initializing the source database into a main database and initializing the target database into a standby database, wherein the main database is used for writing data, and the standby database is used for reading data;
wherein, the source end database and the target end database are respectively provided with a switching mark table;
the database switching method comprises the following steps:
when the source database is required to be changed into a standby database, the source data synchronization system acquires target connection established by an application on the source database and blocks the target connection; recovering the write permission of a production user corresponding to the application on the source database, and configuring the read permission for the production user;
the source data synchronization system inserts a switching mark in a switching mark table of the source database;
capturing and analyzing an operation log of the source database;
when an operation object corresponding to an operation log is a switching mark table, judging whether the inserted switching mark exists in the operation log or not;
if yes, stopping the log capturing service, and sending a master-slave switching command to a target-end data synchronization system;
after receiving the master-slave switching command, the target-side data synchronization system synchronizes the current asynchronous transaction to the target-side database; the target end data synchronization system acquires target connection established by an application on the target end database, and blocks the target connection; configuring write permission for a production user corresponding to an application on the target database;
and switching the source end database into a standby database and switching the target end database into a main database.
2. The database switching method according to claim 1, wherein the source data synchronization system further comprises, before inserting a switch flag in a switch flag table of the source database:
starting a synchronous link from the source end database to the target end database;
and closing a synchronous link from the target database to the source database.
3. The database switching method according to claim 1, further comprising, after configuring write permissions for production users corresponding to applications on the target database:
the target-side data synchronization system stops log receiving service;
and sending a feedback instruction to the source-side data synchronization system, wherein the feedback instruction is used for indicating that the switching preparation work is finished.
4. The database switching method according to claim 1, wherein switching the source side database to a standby database and switching the target side database to a primary database comprises:
the source data synchronization system starts a log receiving service to switch the source database into a standby database;
the source end data synchronization system sends a feedback message of switching completion to the target end data synchronization system;
after receiving the feedback message, the target-side data synchronization system starts a log capturing service to switch the target-side database into a main database.
5. The database switching method according to claim 1, wherein the database switching method further comprises:
acquiring a current maximum log sequence number LSN0 in the target-end database;
after the target end database is switched to a main database, acquiring an operation log larger than the LSN0;
synchronizing the operation log larger than the LSN0 to the source database.
6. The database switching method according to claim 1, wherein the determining whether the inserted switching flag exists in the operation log includes:
judging whether the operation object aimed by the operation log is the switching mark table or not;
if yes, judging whether the inserted switching mark exists in the operation log;
if not, continuing to acquire the next operation log.
7. A database switching system, the database switching system comprising: the system comprises a source end database, a source end data synchronization system, a target end database and a target end data synchronization system, wherein the source end database is connected with the source end data synchronization system, the target end database is connected with the target end data synchronization system, and the source end data synchronization system is connected with the target end data synchronization system;
the method comprises the steps of constructing a bidirectional synchronous link between a source end database and a target end database, initializing the source end database into a main database and initializing the target end database into a standby database, wherein the main database is used for writing data, and the standby database is used for reading data; a switching mark table is established in the source end database and the target end database;
when the source database is required to be changed into a standby database, the source data synchronization system acquires target connection established by an application on the source database and blocks the target connection; recovering the write permission of a production user corresponding to the application on the source database, and configuring the read permission for the production user;
the source data synchronization system is used for inserting a switching mark in a switching mark table of the source database;
the source data synchronization system is used for capturing and analyzing an operation log of the source database;
when the operation object corresponding to the operation log is a switching mark table, the source end data synchronization system is used for judging whether the inserted switching mark exists in the operation log; if yes, stopping the log capturing service, and sending a master-slave switching command to a target-end data synchronization system;
the target-end data synchronization system is used for synchronizing the current asynchronous transaction to the target-end database after receiving the active-standby switching command; the target end data synchronization system acquires target connection established by an application on the target end database, and blocks the target connection; configuring write permission for a production user corresponding to an application on the target database; and switching the source end database to a standby database and switching the target end database to a main database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010382970.8A CN111723066B (en) | 2020-05-08 | 2020-05-08 | Database switching method and database switching system based on log analysis synchronization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010382970.8A CN111723066B (en) | 2020-05-08 | 2020-05-08 | Database switching method and database switching system based on log analysis synchronization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111723066A CN111723066A (en) | 2020-09-29 |
| CN111723066B true CN111723066B (en) | 2023-06-13 |
Family
ID=72564392
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010382970.8A Active CN111723066B (en) | 2020-05-08 | 2020-05-08 | Database switching method and database switching system based on log analysis synchronization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111723066B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112749045B (en) * | 2021-01-11 | 2023-05-26 | 深圳市伊欧乐科技有限公司 | Database cluster switching method, device, storage medium and apparatus |
| CN113407641A (en) * | 2021-07-23 | 2021-09-17 | 中共四川省委组织部 | Data synchronization method based on domestic database |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8996461B1 (en) * | 2011-03-28 | 2015-03-31 | Emc Corporation | Method and apparatus for replicating the punch command |
| CN104639366A (en) * | 2014-12-31 | 2015-05-20 | 北京奇虎科技有限公司 | DNS (domain name server) disaster tolerance system islanding response automatic switching method and device |
| CN104866395A (en) * | 2014-02-24 | 2015-08-26 | 中国移动通信集团福建有限公司 | Data backup method, data backup system, main-backup switching method and main-backup switching system |
| CN108897641A (en) * | 2018-06-21 | 2018-11-27 | 武汉达梦数据库有限公司 | A kind of log analysis under the active and standby environment of database services real-time synchronization system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101706795B (en) * | 2009-11-30 | 2012-05-09 | 上海世范软件技术有限公司 | Method for synchronizing data of database in active/standby server |
| CN104182295B (en) * | 2013-05-27 | 2018-11-02 | 阿里巴巴集团控股有限公司 | A kind of data back up method and device |
| CN106802892B (en) * | 2015-11-26 | 2021-12-28 | 阿里巴巴集团控股有限公司 | Method and equipment for checking consistency of main and standby data |
| CN107196987A (en) * | 2016-03-15 | 2017-09-22 | 中国移动通信集团河南有限公司 | The switching method and server of a kind of pair of centring system |
| CN108984569A (en) * | 2017-06-05 | 2018-12-11 | 中兴通讯股份有限公司 | Database switching method, system and computer readable storage medium |
| CN108228812B (en) * | 2017-12-29 | 2021-06-15 | 北京奇虎科技有限公司 | Self-adaptive main node switching method and device |
-
2020
- 2020-05-08 CN CN202010382970.8A patent/CN111723066B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8996461B1 (en) * | 2011-03-28 | 2015-03-31 | Emc Corporation | Method and apparatus for replicating the punch command |
| CN104866395A (en) * | 2014-02-24 | 2015-08-26 | 中国移动通信集团福建有限公司 | Data backup method, data backup system, main-backup switching method and main-backup switching system |
| CN104639366A (en) * | 2014-12-31 | 2015-05-20 | 北京奇虎科技有限公司 | DNS (domain name server) disaster tolerance system islanding response automatic switching method and device |
| CN108897641A (en) * | 2018-06-21 | 2018-11-27 | 武汉达梦数据库有限公司 | A kind of log analysis under the active and standby environment of database services real-time synchronization system |
Non-Patent Citations (1)
| Title |
|---|
| 海量数据库迁移与升级;楼宏良;殷云飞;胡建;;计算机时代(第06期);第 38-41页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111723066A (en) | 2020-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111723066B (en) | Database switching method and database switching system based on log analysis synchronization | |
| US5428771A (en) | Transparent transaction coordination between distributed networks having different communication protocols | |
| CN110347746B (en) | Heterogeneous database synchronous data consistency checking method and device | |
| US10895996B2 (en) | Data synchronization method, system, and apparatus using a work log for synchronizing data greater than a threshold value | |
| KR101265388B1 (en) | High Availability Data Base Management System and Method for Managing Database Using High Availability Data Base Management System | |
| CN102968486B (en) | A kind of highly reliable file synchronisation method based on change journal | |
| CN110209726A (en) | Distributed experiment & measurement system system, method of data synchronization and storage medium | |
| CN108897641B (en) | Log analysis service real-time synchronization system under database master-slave environment | |
| CN106776121B (en) | Data disaster recovery device, system and method | |
| CN108984569A (en) | Database switching method, system and computer readable storage medium | |
| WO2013131448A1 (en) | Method and system for data synchronization and data access apparatus | |
| CN102855314B (en) | Real-time database backup system and method based on FPGA (field programmable gate array) | |
| CN106503257A (en) | Distributed transaction server method and system based on binlog compensation mechanism | |
| CN110334077A (en) | The method and device of database cross computer room migration | |
| CN100385407C (en) | Allopatric data image method of network information system | |
| CN111274065A (en) | Disaster recovery method and device based on software | |
| CN113535665B (en) | Method and device for synchronizing log files between main database and standby database | |
| CN112783694B (en) | Long-distance disaster recovery method for high-availability Redis | |
| CN104376088A (en) | Distributed synchronization method of cloud database and database system | |
| WO2021082925A1 (en) | Transaction processing method and apparatus | |
| CN111506668B (en) | Robot cluster intelligent data synchronization method and system | |
| CN111831490B (en) | Method and system for synchronizing memories between redundant main and standby nodes | |
| CN113407638A (en) | Method for realizing real-time relational database data synchronization | |
| CN112667440A (en) | Long-distance disaster recovery method for high-availability MySQL | |
| CN116303791A (en) | Data synchronization method and device based on acceleration system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Applicant after: Wuhan dream database Co.,Ltd. Address before: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Applicant before: WUHAN DAMENG DATABASE Co.,Ltd. |
|
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Chen Qi Inventor after: Sun Feng Inventor after: Yu Yuanlan Inventor before: Fu Quan Inventor before: Chen Qi Inventor before: Sun Feng Inventor before: Yu Yuanlan |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |