CN114116885A - Database synchronization method, system, device and medium - Google Patents
Database synchronization method, system, device and medium Download PDFInfo
- Publication number
- CN114116885A CN114116885A CN202111206625.XA CN202111206625A CN114116885A CN 114116885 A CN114116885 A CN 114116885A CN 202111206625 A CN202111206625 A CN 202111206625A CN 114116885 A CN114116885 A CN 114116885A
- Authority
- CN
- China
- Prior art keywords
- node
- synchronization
- queue
- log
- standby
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000004044 response Effects 0.000 claims abstract description 41
- 230000002159 abnormal effect Effects 0.000 claims abstract description 15
- 238000001514 detection method Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 8
- 230000001360 synchronised effect Effects 0.000 claims description 7
- 230000008439 repair process Effects 0.000 abstract description 7
- 230000001960 triggered effect Effects 0.000 abstract description 4
- 230000005856 abnormality Effects 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000010076 replication Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Hardware Redundancy (AREA)
Abstract
The invention discloses a database synchronization method, which comprises the following steps: in response to receiving a data processing request, recording an operation statement carried in the request into a queue; sending the operation statement to a main node to enable the main node to execute the operation statement; and in response to detecting that the synchronization between the main node and the standby node is abnormal, performing data synchronization on the standby node by using the operation statements in the queue. The invention also discloses a system, a computer device and a readable storage medium. According to the scheme provided by the invention, the SQL information carried in the request is stored in the queue to record the SQL operation of the application, and when the synchronization abnormality between the main node and the standby node is detected, the repair process is triggered according to the SQL operation in the queue, so that the integrity of the data is effectively maintained.
Description
Technical Field
The invention relates to the field of databases, in particular to a database synchronization method, a database synchronization system, a database synchronization device and a storage medium.
Background
MySQL is used as a relational database, has the advantages of low cost, active open source communities, strong stability, high query speed and the like, and is widely applied to systems such as web sites, database servers and the like. The high availability of a system is an important index for measuring the reliability of the system, and the database is usually an important component in the system, and the high availability is particularly important. Generally, disaster recovery of a database is realized by forming a high-availability environment by a plurality of database nodes, and in order to solve data synchronization among the nodes, the prior art includes an asynchronous Replication technology, a semi-synchronous Replication technology and a Group synchronization technology (mysql Group Replication), which are realized based on mysql binary logs and relay logs, wherein a binlog records modification operations on the database during the database operation process, and data can be synchronized at a standby node by copying and replaying the binlog to the standby node. The log synchronization among a plurality of database nodes adopts asynchronous operation, so that the synchronization process can be damaged in some cases, and data loss is caused. This can have serious consequences in scenarios with high data consistency requirements.
If the MySQL standby node network fails and synchronization fails, the main node generates a lot of logs if a large amount of database operations are performed, and if synchronization is performed when the standby node is started again, a large amount of io consumption is caused. In order to solve the problem, in the prior art, a synchronization process is monitored, and if a count value of accumulated count exceeds a preset count value threshold, a binary log of data operation of a host database is not stored, but data of a master database is directly exported and then imported into a slave database, so that data synchronization is completed. This invention reduces the io pressure caused by the synchronization of a large number of binlogs.
However, in the synchronization process, if the master library encounters an emergency such as equipment failure, power failure, restart, etc., the master library cannot provide service and store data. The backup library will provide service and store data, and at the same time, the backup library may lose part of the data at the instant that the main library is down due to the delay in the synchronization process.
Meanwhile, MySQL is a common storage database, and the high availability is an important index. Commonly the highly available solution for MySQL is used as: the MySQL performs master-slave synchronization or Group synchronization (MySQL Group Replication) to perform full synchronization of a master database and a slave database, keepalive provides vip, an application program uses the vip to access MySQL service, and when a certain MySQL service cannot normally operate, the keepalive switches the vip to a normally operating MySQL node, so that the MySQL service accessed by the application node is normal, and high availability of the MySQL service is ensured. The MySQL synchronization process can generate and use: binary logs and relay logs. When the master library a has data written or modified, the master library a writes the change of the database into a local binary log. And the IO thread of the standby library b reads the records of the related operations from the binary log of the main library a, then writes the operations into the relay log of the local computer, and then the SQL thread of the standby library b replays the operation records in the relay log and writes the operation records into the local database to finish the synchronous operation. However, when an emergency such as a device failure, a power failure, a restart, or the like occurs, the main library a may have data modification written into a binary file of the host, but the standby library b does not acquire the log, and in a high availability environment, the main node of MySQL may be switched, which may cause a part of data that is not synchronized from the main library a to be lost after the standby library b becomes the main node, thereby generating the following possible errors:
1. data loss exists, so that information loss is caused;
2. when the main library a becomes the main node again, the lost data is operated, and the spare library b does not exist, so that synchronization errors are caused;
3. GTID serial numbers in the Binlog of the standby library b are lower than serial numbers in the main library a, so that synchronization failure is caused.
Disclosure of Invention
In view of the above, in order to overcome at least one aspect of the above problems, an embodiment of the present invention provides a database synchronization method, including:
in response to receiving a data processing request, recording an operation statement carried in the request into a queue;
sending the operation statement to a main node to enable the main node to execute the operation statement;
and in response to detecting that the synchronization between the main node and the standby node is abnormal, performing data synchronization on the standby node by using the operation statements in the queue.
In some embodiments, the performing data synchronization on the standby node by using the operation statements in the queue further includes:
comparing the operation statement in the queue with a first log in the standby node;
and responding to the first log of the standby node that the operation corresponding to the operation statements recorded in the queue is lost, executing the operation corresponding to the operation statements at the standby node, and recording the operation into the first log.
In some embodiments, further comprising:
acquiring a first log of the main node;
judging whether a first log of the main node has a record of the operation corresponding to the operation statements;
and in response to the existence, deleting the records of the operations corresponding to the operation statements in the first log of the main node.
In some embodiments, further comprising:
and synchronizing the records of the operations corresponding to the operation statements in the first log of the standby node into the first log of the main node.
In some embodiments, further comprising:
in response to the absence of the operation statements, synchronizing records of the operations corresponding to the operation statements in a first log of the standby node to a second log of the main node;
the master node replays from the record of the operation in the second log.
In some embodiments, in response to detecting a synchronization exception between the primary node and a standby node, performing data synchronization on the standby node using the operation statements in the queue, further comprising:
in response to a trigger detection task, checking the synchronization state between the main node and the slave node through a preset command and acquiring the last detected synchronization state;
and responding to the fact that the last detected synchronization state is normal and the currently detected synchronization state is abnormal, and performing data synchronization on the standby node by using the operation statements in the queue.
In some embodiments, further comprising:
and in response to the currently detected synchronization state being normal, clearing the operation statements in the queue.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a database synchronization system, including:
the receiving module is configured to respond to a received data processing request and record an operation statement carried in the request into a queue;
a sending module configured to send the operation statement to a master node to cause the master node to execute the operation statement;
and the detection synchronization module is configured to respond to the detection of abnormal synchronization between the main node and the standby node and perform data synchronization on the standby node by using the operation statements in the queue.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer apparatus, including:
at least one processor; and
a memory storing a computer program operable on the processor, wherein the processor executes the program to perform any of the steps of the database synchronization method described above.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program which, when executed by a processor, performs the steps of any of the database synchronization methods described above.
The invention has one of the following beneficial technical effects: according to the scheme provided by the invention, the SQL information carried in the request is stored in the queue to record the SQL operation of the application, and when the synchronization abnormality between the main node and the standby node is detected, the repair process is triggered according to the SQL operation in the queue, so that the integrity of the data is effectively maintained.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a database synchronization method according to an embodiment of the present invention;
FIG. 2 is a block flow diagram of a database synchronization method provided by an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a database synchronization system according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a computer device provided in an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
According to an aspect of the present invention, an embodiment of the present invention proposes a database synchronization method, as shown in fig. 1, which may include the steps of:
s1, responding to the received data processing request, recording the operation statement carried in the request into a queue;
s2, sending the operation statement to a main node to enable the main node to execute the operation statement;
s3, responding to the detected abnormal synchronization between the main node and the standby node, and performing data synchronization on the standby node by using the operation statements in the queue.
According to the scheme provided by the invention, the SQL information carried in the request is stored in the queue to record the SQL operation of the application, and when the synchronization abnormality between the main node and the standby node is detected, the repair process is triggered according to the SQL operation in the queue, so that the integrity of the data is effectively maintained.
In some embodiments, in step S1, in response to receiving the data processing request, the operation statement carried in the request is recorded in a queue, specifically, as shown in fig. 2, when the upper layer application interacts with MySQL, if there is a write and update (insert, update) operation, the request may be intercepted by the controller, and then the executed statement and the IP information of the MySQL host node are placed in an SQL information storage queue of the controller, and the operation is sent to the MySQL host node for execution.
In some embodiments, the performing data synchronization on the standby node by using the operation statements in the queue further includes:
comparing the operation statement in the queue with a first log in the standby node;
and responding to the first log of the standby node that the operation corresponding to the operation statements recorded in the queue is lost, executing the operation corresponding to the operation statements at the standby node, and recording the operation into the first log.
Specifically, when detecting that the synchronization state between the main node and the standby node is abnormal, that is, the main node cannot provide service normally, the controller reads the SQL statements recorded in the storage queue, and then compares the SQL statements with the SQL records executed in the binlog (first log) of the standby node. If the SQL operation record executed by the main node is missing from the binlog of the standby node, the standby node loses the data. The controller persists the record in a preset position, the record is the operation executed by the main node and not synchronized by the standby node. And finally, synchronously executing in the standby node according to the missing records, and keeping the data complete.
In some embodiments, further comprising:
acquiring a first log of the main node;
judging whether a first log of the main node has a record of the operation corresponding to the operation statements;
and in response to the existence, deleting the records of the operations corresponding to the operation statements in the first log of the main node.
Specifically, when the master node is restarted, the controller reads the SQL record persisted to the preset position, and searches the SQL record in the binlog (first log) of the master node, and if the same record exists in the binlog of the master node, deletes the corresponding record in the binlog of the master node to avoid a conflict.
It should be noted that, in MySQL, the same data record corresponds to the same ID, and if the backup node performs corresponding operations according to the persistent record, the generated record is different from the ID of the record of the master node, so when the master node is restarted, the SQL record at the preset position can be persisted, and the corresponding record in the binlog of the master node is deleted.
In some embodiments, further comprising:
and synchronizing the records of the operations corresponding to the operation statements in the first log of the standby node into the first log of the main node.
Specifically, in order to ensure data synchronization between the main node and the standby node, after the main node deletes the corresponding record in the binlog, the records of the operations corresponding to the operation statements in the first log of the standby node may be synchronized into the first log of the main node.
In some embodiments, further comprising:
in response to the absence of the operation statements, synchronizing records of the operations corresponding to the operation statements in a first log of the standby node to a second log of the main node;
the master node replays from the record of the operation in the second log.
Specifically, if there is no record of operations corresponding to the operation statements in the first log in the master node, that is, the controller has not sent the operations in the queue to the master node, and the master node has failed, at this time, records of operations corresponding to the operation statements in the first log of the standby node may be synchronized to a second log (for example, a relay log) of the master node, and the master node may replay the records of operations according to the records of operations in the second log, so as to implement data synchronization between the master node and the slave node.
In some embodiments, in response to detecting a synchronization exception between the primary node and a standby node, performing data synchronization on the standby node using the operation statements in the queue, further comprising:
in response to a trigger detection task, checking the synchronization state between the main node and the slave node through a preset command and acquiring the last detected synchronization state;
and responding to the fact that the last detected synchronization state is normal and the currently detected synchronization state is abnormal, and performing data synchronization on the standby node by using the operation statements in the queue.
In some embodiments, further comprising:
and in response to the currently detected synchronization state being normal, clearing the operation statements in the queue.
Specifically, a periodic task may be set in the controller, and after the task is triggered, the data synchronization state between the main node and the standby node is acquired, and if the state is normal, the operation statement in the queue is cleared. And if the operation statements in the queue are incorrect and the last time that the synchronization state is detected to be normal, performing data synchronization on the standby node by using the operation statements in the queue. Therefore, when the state abnormity of the main node and the standby node or the synchronization abnormity between the main node and the standby node is detected for the first time, the MySQL synchronization error is represented, and the controller can traverse the SQL operation recorded in the storage queue and process the SQL operation.
According to the technical scheme provided by the invention, the controller is added between the upper-layer application and the MySQL service. When the upper application interacts with the MySQL, if write-in and update (insert and update) operations exist, the controller intercepts the request, then puts the executed statement and the IP information of the MySQL main node into an SQL information storage queue of the controller, and simultaneously sends the operations to the MySQL main node for execution. And if the node fault occurs, the timing task in the controller acquires the historical operation stored in the SQL information storage queue to repair the data. Therefore, the problem of data loss caused by accidents of equipment is effectively avoided. Meanwhile, the consistency of GTIDs in the binlog is effectively maintained, and the problem of starting synchronization failure caused by inconsistent GTIDs of the main library and the standby library when synchronization is restored is avoided.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a database synchronization system 400, as shown in fig. 3, including:
a receiving module 401, configured to respond to a received data processing request, and record an operation statement carried in the request into a queue;
a sending module 402 configured to send the operation statement to a master node to cause the master node to execute the operation statement;
a synchronization detection module 403 configured to perform data synchronization on the standby node by using the operation statements in the queue in response to detecting the synchronization exception between the primary node and the standby node
In some embodiments, the detection synchronization module 403 is further configured to:
comparing the operation statement in the queue with a first log in the standby node;
and responding to the first log of the standby node that the operation corresponding to the operation statements recorded in the queue is lost, executing the operation corresponding to the operation statements at the standby node, and recording the operation into the first log.
In some embodiments, further comprising:
acquiring a first log of the main node;
judging whether a first log of the main node has a record of the operation corresponding to the operation statements;
and in response to the existence, deleting the records of the operations corresponding to the operation statements in the first log of the main node.
In some embodiments, further comprising:
and synchronizing the records of the operations corresponding to the operation statements in the first log of the standby node into the first log of the main node.
In some embodiments, further comprising:
in response to the absence of the operation statements, synchronizing records of the operations corresponding to the operation statements in a first log of the standby node to a second log of the main node;
the master node replays from the record of the operation in the second log.
In some embodiments, the detection synchronization module 403 is further configured to:
in response to a trigger detection task, checking the synchronization state between the main node and the slave node through a preset command and acquiring the last detected synchronization state;
and responding to the fact that the last detected synchronization state is normal and the currently detected synchronization state is abnormal, and performing data synchronization on the standby node by using the operation statements in the queue.
In some embodiments, further comprising:
and in response to the currently detected synchronization state being normal, clearing the operation statements in the queue.
According to the technical scheme provided by the invention, the controller is added between the upper-layer application and the MySQL service. When the upper application interacts with the MySQL, if write-in and update (insert and update) operations exist, the controller intercepts the request, then puts the executed statement and the IP information of the MySQL main node into an SQL information storage queue of the controller, and simultaneously sends the operations to the MySQL main node for execution. And if the node fault occurs, the timing task in the controller acquires the historical operation stored in the SQL information storage queue to repair the data. Therefore, the problem of data loss caused by accidents of equipment is effectively avoided. Meanwhile, the consistency of GTIDs in the binlog is effectively maintained, and the problem of starting synchronization failure caused by inconsistent GTIDs of the main library and the standby library when synchronization is restored is avoided.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 4, an embodiment of the present invention further provides a computer apparatus 501, including:
at least one processor 520; and
a memory 510, the memory 510 storing a computer program 511 executable on the processor, the processor 520 executing the program to perform the steps of:
in response to receiving a data processing request, recording an operation statement carried in the request into a queue;
sending the operation statement to a main node to enable the main node to execute the operation statement;
and in response to detecting that the synchronization between the main node and the standby node is abnormal, performing data synchronization on the standby node by using the operation statements in the queue.
In some embodiments, the performing data synchronization on the standby node by using the operation statements in the queue further includes:
comparing the operation statement in the queue with a first log in the standby node;
and responding to the first log of the standby node that the operation corresponding to the operation statements recorded in the queue is lost, executing the operation corresponding to the operation statements at the standby node, and recording the operation into the first log.
In some embodiments, further comprising:
acquiring a first log of the main node;
judging whether a first log of the main node has a record of the operation corresponding to the operation statements;
and in response to the existence, deleting the records of the operations corresponding to the operation statements in the first log of the main node.
In some embodiments, further comprising:
and synchronizing the records of the operations corresponding to the operation statements in the first log of the standby node into the first log of the main node.
In some embodiments, further comprising:
in response to the absence of the operation statements, synchronizing records of the operations corresponding to the operation statements in a first log of the standby node to a second log of the main node;
the master node replays from the record of the operation in the second log.
In some embodiments, in response to detecting a synchronization exception between the primary node and a standby node, performing data synchronization on the standby node using the operation statements in the queue, further comprising:
in response to a trigger detection task, checking the synchronization state between the main node and the slave node through a preset command and acquiring the last detected synchronization state;
and responding to the fact that the last detected synchronization state is normal and the currently detected synchronization state is abnormal, and performing data synchronization on the standby node by using the operation statements in the queue.
In some embodiments, further comprising:
and in response to the currently detected synchronization state being normal, clearing the operation statements in the queue.
According to the technical scheme provided by the invention, the controller is added between the upper-layer application and the MySQL service. When the upper application interacts with the MySQL, if write-in and update (insert and update) operations exist, the controller intercepts the request, then puts the executed statement and the IP information of the MySQL main node into an SQL information storage queue of the controller, and simultaneously sends the operations to the MySQL main node for execution. And if the node fault occurs, the timing task in the controller acquires the historical operation stored in the SQL information storage queue to repair the data. Therefore, the problem of data loss caused by accidents of equipment is effectively avoided. Meanwhile, the consistency of GTIDs in the binlog is effectively maintained, and the problem of starting synchronization failure caused by inconsistent GTIDs of the main library and the standby library when synchronization is restored is avoided.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 5, an embodiment of the present invention further provides a computer-readable storage medium 601, where the computer-readable storage medium 601 stores computer program instructions 610, and the computer program instructions 610, when executed by a processor, perform the following steps:
in response to receiving a data processing request, recording an operation statement carried in the request into a queue;
sending the operation statement to a main node to enable the main node to execute the operation statement;
and in response to detecting that the synchronization between the main node and the standby node is abnormal, performing data synchronization on the standby node by using the operation statements in the queue.
In some embodiments, the performing data synchronization on the standby node by using the operation statements in the queue further includes:
comparing the operation statement in the queue with a first log in the standby node;
and responding to the first log of the standby node that the operation corresponding to the operation statements recorded in the queue is lost, executing the operation corresponding to the operation statements at the standby node, and recording the operation into the first log.
In some embodiments, further comprising:
acquiring a first log of the main node;
judging whether a first log of the main node has a record of the operation corresponding to the operation statements;
and in response to the existence, deleting the records of the operations corresponding to the operation statements in the first log of the main node.
In some embodiments, further comprising:
and synchronizing the records of the operations corresponding to the operation statements in the first log of the standby node into the first log of the main node.
In some embodiments, further comprising:
in response to the absence of the operation statements, synchronizing records of the operations corresponding to the operation statements in a first log of the standby node to a second log of the main node;
the master node replays from the record of the operation in the second log.
In some embodiments, in response to detecting a synchronization exception between the primary node and a standby node, performing data synchronization on the standby node using the operation statements in the queue, further comprising:
in response to a trigger detection task, checking the synchronization state between the main node and the slave node through a preset command and acquiring the last detected synchronization state;
and responding to the fact that the last detected synchronization state is normal and the currently detected synchronization state is abnormal, and performing data synchronization on the standby node by using the operation statements in the queue.
In some embodiments, further comprising:
and in response to the currently detected synchronization state being normal, clearing the operation statements in the queue.
According to the technical scheme provided by the invention, the controller is added between the upper-layer application and the MySQL service. When the upper application interacts with the MySQL, if write-in and update (insert and update) operations exist, the controller intercepts the request, then puts the executed statement and the IP information of the MySQL main node into an SQL information storage queue of the controller, and simultaneously sends the operations to the MySQL main node for execution. And if the node fault occurs, the timing task in the controller acquires the historical operation stored in the SQL information storage queue to repair the data. Therefore, the problem of data loss caused by accidents of equipment is effectively avoided. Meanwhile, the consistency of GTIDs in the binlog is effectively maintained, and the problem of starting synchronization failure caused by inconsistent GTIDs of the main library and the standby library when synchronization is restored is avoided.
Finally, it should be noted that, as will be understood by those skilled in the art, all or part of the processes of the methods of the above embodiments may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above.
Further, it should be appreciated that the computer-readable storage media (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A database synchronization method, comprising the steps of:
in response to receiving a data processing request, recording an operation statement carried in the request into a queue;
sending the operation statement to a main node to enable the main node to execute the operation statement;
and in response to detecting that the synchronization between the main node and the standby node is abnormal, performing data synchronization on the standby node by using the operation statements in the queue.
2. The method of claim 1, wherein the standby node is data synchronized using the operation statements in the queue, further comprising:
comparing the operation statement in the queue with a first log in the standby node;
and responding to the first log of the standby node that the operation corresponding to the operation statements recorded in the queue is lost, executing the operation corresponding to the operation statements at the standby node, and recording the operation into the first log.
3. The method of claim 2, further comprising:
acquiring a first log of the main node;
judging whether a first log of the main node has a record of the operation corresponding to the operation statements;
and in response to the existence, deleting the records of the operations corresponding to the operation statements in the first log of the main node.
4. The method of claim 3, further comprising:
and synchronizing the records of the operations corresponding to the operation statements in the first log of the standby node into the first log of the main node.
5. The method of claim 3, further comprising:
in response to the absence of the operation statements, synchronizing records of the operations corresponding to the operation statements in a first log of the standby node to a second log of the main node;
the master node replays from the record of the operation in the second log.
6. The method of claim 1, wherein in response to detecting a synchronization anomaly between the master node and a standby node, data synchronizing the standby node with an operation statement in the queue, further comprising:
in response to a trigger detection task, checking the synchronization state between the main node and the slave node through a preset command and acquiring the last detected synchronization state;
and responding to the fact that the last detected synchronization state is normal and the currently detected synchronization state is abnormal, and performing data synchronization on the standby node by using the operation statements in the queue.
7. The method of claim 6, further comprising:
and in response to the currently detected synchronization state being normal, clearing the operation statements in the queue.
8. A database synchronization system, comprising:
the receiving module is configured to respond to a received data processing request and record an operation statement carried in the request into a queue;
a sending module configured to send the operation statement to a master node to cause the master node to execute the operation statement;
and the detection synchronization module is configured to respond to the detection of abnormal synchronization between the main node and the standby node and perform data synchronization on the standby node by using the operation statements in the queue.
9. A computer device, comprising:
at least one processor; and
memory storing a computer program operable on the processor, wherein the processor executes the program to perform the steps of the method according to any of claims 1-7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111206625.XA CN114116885A (en) | 2021-10-17 | 2021-10-17 | Database synchronization method, system, device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111206625.XA CN114116885A (en) | 2021-10-17 | 2021-10-17 | Database synchronization method, system, device and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114116885A true CN114116885A (en) | 2022-03-01 |
Family
ID=80375822
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111206625.XA Pending CN114116885A (en) | 2021-10-17 | 2021-10-17 | Database synchronization method, system, device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114116885A (en) |
-
2021
- 2021-10-17 CN CN202111206625.XA patent/CN114116885A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4415610B2 (en) | System switching method, replica creation method, and disk device | |
| WO2017177941A1 (en) | Active/standby database switching method and apparatus | |
| JP4624829B2 (en) | Data backup system and method | |
| EP2790112B1 (en) | Method and system for data synchronization and data access apparatus | |
| US9189348B2 (en) | High availability database management system and database management method using same | |
| CN106933843B (en) | Database heartbeat detection method and device | |
| US11892922B2 (en) | State management methods, methods for switching between master application server and backup application server, and electronic devices | |
| WO2021226905A1 (en) | Data storage method and system, and storage medium | |
| WO2022088861A1 (en) | Database fault handling method and apparatus | |
| US10282256B1 (en) | System and method to enable deduplication engine to sustain operational continuity | |
| US9330153B2 (en) | System, method, and computer readable medium that coordinates between devices using exchange of log files | |
| CN111158955B (en) | High-availability system based on volume replication and multi-server data synchronization method | |
| CN108243031B (en) | Method and device for realizing dual-computer hot standby | |
| CN113535665B (en) | Method and device for synchronizing log files between main database and standby database | |
| US10860411B2 (en) | Automatically detecting time-of-fault bugs in cloud systems | |
| JP2006185108A (en) | Management computer for managing data of storage system, and data management method | |
| CN114116885A (en) | Database synchronization method, system, device and medium | |
| CN116361078A (en) | Data synchronization method, device, system and medium | |
| US20210240351A1 (en) | Remote copy system and remote copy management method | |
| JP2004272318A (en) | System changeover system, processing method therefor, and processing program therefor | |
| CN108897645B (en) | Database cluster disaster tolerance method and system based on standby heartbeat disk | |
| CN111460035A (en) | Database system and disaster tolerance method thereof | |
| CN111176886A (en) | Database mode switching method and device and electronic equipment | |
| JP7371547B2 (en) | Node, mirroring type cluster system, restore detection method, and program | |
| CN116821092A (en) | Data migration method, device and storage medium based on MasterHA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |