CN106802892B - Method and equipment for checking consistency of main and standby data - Google Patents
Method and equipment for checking consistency of main and standby data Download PDFInfo
- Publication number
- CN106802892B CN106802892B CN201510835059.7A CN201510835059A CN106802892B CN 106802892 B CN106802892 B CN 106802892B CN 201510835059 A CN201510835059 A CN 201510835059A CN 106802892 B CN106802892 B CN 106802892B
- Authority
- CN
- China
- Prior art keywords
- log
- database server
- mark
- update
- packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/273—Asynchronous replication or reconciliation
Abstract
The device specifically comprises a main database server and a standby database server, wherein the main database server writes mark information into a current log packet to be sent, generates a mark log packet and sends the mark log packet to the standby database server, and after receiving the mark log packet, the standby database server determines the consistency of the main and standby data according to the log update state of the main database server indicated by the mark information in the mark log packet. Compared with the prior art, the marking information is transmitted through the current log packet to be transmitted, so the log packet can be transmitted to the standby database server in a very short time, and the standby database server can always acquire the log updating state of the main database server in the shortest time, so that the accuracy of judging the consistency of the main and standby data is improved, and whether the data is lost after the main and standby are switched is accurately judged.
Description
Technical Field
The present application relates to the field of computers, and in particular, to a method and device for checking consistency of primary and secondary data.
Background
In order to achieve high reliability, the database system generally adopts a master-slave or master-slave architecture, which includes a master database server and one or more slave database servers. In the database system of the main and standby architecture, the main database server completes data backup by transmitting logs to the standby database server, and because the operation of the main database server on the data is recorded in the logs, the standby database server can recover the data according to the logs when the main database is down.
For performance, the primary and backup database servers generally adopt an asynchronous mode, that is, the primary database server does not have to wait for the corresponding transaction commit operation log to be sent to the backup database server before completing the commit when the transaction is committed. The asynchronous mode may cause a situation that after the transaction is submitted and the user is notified, the corresponding log is not transmitted to the standby database server. If the main database server is down and switched to the standby database server at the moment, the corresponding data cannot be recovered through the log, and then the data of the transactions are lost.
In order to prevent data loss after the main-standby switching, whether all logs of submitted transactions exist in the standby database server needs to be checked during the switching, and then whether the switching is performed is determined. However, because the main database server is down, the last generated log number cannot be obtained to compare with the log of the current standby database server, so that whether all logs exist in the standby database server or not cannot be judged, that is, whether data is lost or not after the standby database server is switched over.
In one prior art scheme, the primary database server writes the current time into a heartbeat table at regular intervals (e.g., 10 seconds), and the recorded time stamps of the heartbeat table are continuously synchronized to the secondary database server. When the master database server and the slave database server are switched, the last time recorded by the heartbeat table is inquired from the slave database server, and the fact that how long the data of the master database server are not synchronized to the slave database server can be judged. If the last time of the database server heartbeat table is 09: 00: 10, the current time is 9: 00: 35, it can be estimated that up to 25 seconds of data is not synchronized (and possibly that the 25 second master database server does not have data updates). If this time value is within a tolerance range (e.g., 1 minute), a handover is performed. However, this method cannot accurately compare whether the data in the primary database server and the backup database server are consistent, and therefore, it cannot accurately determine whether there is data loss after the primary/backup switching.
Content of application
An object of the present application is to provide a method and a device for checking consistency of primary and secondary data, so as to solve the problem that it is impossible to accurately determine whether data is lost after primary and secondary switches in the prior art.
In order to achieve the above object, the present application provides a method for checking consistency of primary and secondary data at a primary database server, where the method includes:
writing mark information into a current log packet to be sent to generate a mark log packet, wherein the mark information represents the log updating state of a main database server;
and sending the marked log packet to a standby database server so that the standby database server determines the consistency of the main and standby data according to the log updating state of the main database server represented by the marked information in the marked log packet.
The application provides a method for master-slave data consistency check at a slave database server, which comprises the following steps:
receiving a marking log packet sent by a main database server, wherein the marking log packet comprises marking information;
and determining the consistency of the main and standby data according to the log updating state of the main database server represented by the mark information in the mark log packet.
Based on another aspect of the present application, there is also provided a primary database server for primary and secondary data consistency check, including:
the system comprises a mark writing device, a mark updating device and a mark updating device, wherein the mark writing device is used for writing mark information into a current log packet to be sent and generating a mark log packet, and the mark information represents the log updating state of a main database server;
and the sending device is used for sending the marked log packet to the standby database server so that the standby database server determines the consistency of the main and standby data according to the log updating state of the main database server represented by the marked information in the marked log packet.
The application provides a backup database server for checking consistency of primary and backup data, the backup database server includes:
the receiving device is used for receiving a marked log packet sent by main data, wherein the marked log packet contains marked information;
and the checking device is used for determining the consistency of the main and standby data according to the log updating state of the main database server represented by the mark information in the mark log packet.
Compared with the prior art, in the technical scheme of the application, when the master database server sends the log packet each time, the master database server inserts the mark information which represents the current log updating state of the master database server into the current log packet to be sent, generates a mark log packet containing the mark information, and sends the mark log packet to the slave database server. Because the mark information is transmitted through the current log packet to be sent, the log packet can be sent to the standby database server in a very short time, and therefore the log updating state of the main database server is almost synchronized to the standby database server through the log packet in real time, and the standby database server can always acquire the log updating state of the main database server in the shortest time, so that the accuracy of judging the consistency of the main and standby data is improved, and whether the data is lost after the main and standby are switched is accurately judged.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
fig. 1 is a database system for checking consistency of primary and secondary data according to an embodiment of the present disclosure;
FIG. 2 is a flowchart illustrating primary and secondary database servers performing primary and secondary data consistency checks;
FIG. 3 is a flowchart of a preferred manner of performing a primary/secondary data consistency check by the primary and secondary database servers;
fig. 4 is a schematic structural diagram of a primary database server for checking consistency of primary and secondary data according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a backup database server for checking consistency of primary and backup data according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a preferred primary database server for checking consistency of primary and secondary data according to an embodiment of the present disclosure;
the same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present application is described in further detail below with reference to the attached figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
Fig. 1 shows a database system for checking consistency of primary and backup data, which is a primary-backup architecture, for example, and includes a primary database server 110 and a backup database server 120 connected thereto via a network. According to actual application requirements, the database system can also be a main-standby framework comprising a plurality of standby database servers, wherein a main-standby data consistency check mode is performed between any one standby database server and the main database server, and the main-standby data consistency check mode is similar to the main-standby framework in the embodiment.
It will be understood by those skilled in the art that the primary database server 110 and the secondary database server 120 in the database system may include, but are not limited to, user equipment, network equipment, or equipment formed by integrating user equipment and network equipment through a network. The user equipment comprises but is not limited to a personal computer, a touch terminal and the like; including but not limited to implementations such as a network host, a single network server, multiple sets of network servers, or a cloud-computing-based collection of computers. Here, the Cloud is made up of a large number of hosts or web servers based on Cloud Computing (Cloud Computing), which is a type of distributed Computing, one virtual computer consisting of a collection of loosely coupled computers.
Fig. 2 is a flowchart illustrating a process of performing a primary/secondary data consistency check between the primary database server 110 and the secondary database server 120. In the primary database server 110, the method for checking the consistency of the primary and secondary data includes:
step S201, the master database server writes the mark information into the current log packet to be sent, and generates a mark log packet.
Step S202, the main database server sends the marked log packet to the standby database server, so that the standby database server determines the consistency of the main and standby data according to the log updating state of the main database server, which is represented by the marked information in the marked log packet.
The log packet to be sent is a log packet to be sent by the main database server to the standby database server at the current moment, namely, the log packet is sent to the standby database server within a very short time, so that the log updating state of the main database server is almost synchronized to the standby database server in real time through the log packet, and the standby database server can always acquire the log updating state of the main database server within the shortest time, so that the accuracy of judging the consistency of the main and standby data is improved, and whether data is lost after the main and standby are switched is accurately judged.
The mark information represents the log updating state of the main database server, and the log updating state can be set in two situations as the scheme aims to check the consistency of the main and standby data: 1. an updating or updated log; 2. the log is not updated. The log update status indicates whether a new log is generated or is being generated in the master database server compared to the last generated log in the log packet when the log packet to be transmitted is to be transmitted. In case 1, it indicates that a new log is generated or is being generated, there is a data update in the primary database server, and the operation of the data update does not mark the log record in the log packet, which may cause the data in the primary database server to be inconsistent with the data in the secondary database server. Accordingly, in case 2, it means that no new log is generated, no data is updated in the master database server, and the last generated log by the master database server is ready to be transmitted in the marked log packet, so that the master database server can replicate the data of the master database server at the time of transmitting the marked log packet according to the log in the marked log packet received by the master database server, so that the data of the two are consistent.
Corresponding to the log update state of the updating log or the updated log, the corresponding mark information can be an update mark; and corresponding to the log update state of the non-updated log, the corresponding mark information may be a non-updated mark.
Specifically, in step S201, the master database server writes flag information into a log packet to be currently sent, including: if the log updating state of the main database server is the updating or updated log, writing an updating mark into the current log packet to be sent; and if the log updating state of the main database server is the non-updated log, writing a non-updated mark into the current log packet to be sent.
In practical applications, the update flag and the non-update flag may take different forms based on coding schemes or other considerations. For example, the update flag may be a log number, where the log number of the log recently generated by the master database server or the log number of the log currently being generated is used as the update flag, and correspondingly, the log number of a log recently generated in the log packet to be sent or a preset special log number (such as 0 or 00) is used as the non-update flag.
Also for example, it may be in the form of a bit, i.e., one bit (1 or 0) represents two cases. Specifically, a bit of 1 indicates that the log update status is an updating log or an updated log, and a bit of 0 indicates that the log update status is an un-updated log. Of course, the opposite value may be used to indicate the corresponding log update status.
Further, it is also possible to write empty flag information in one log update state and write arbitrary flag information in another log update state. Specifically, when the log update state of the log is not updated, empty flag information is written (i.e., no flag is written); and when the log is updated or the log update state of the log is updated, the mark information (which can be a bit, a log number and the like) with any content is written. Of course, the corresponding log update status may also be represented in the reverse form.
It will be understood by those skilled in the art that the specific forms of the update flag and the non-update flag are merely examples, and other existing or future forms of update flag and non-update flag may be applied to the present invention, and are included within the scope of the present invention and are incorporated by reference herein.
Correspondingly, at the backup database server 120, the method for checking the consistency of the backup data includes:
step S203, the standby database server receives a mark log packet sent by the main database server, wherein the mark log packet contains mark information.
Step S204, the standby database server determines the consistency of the main and standby data according to the log update state of the main database server represented by the mark information in the mark log packet.
The mark information written into the mark log packet by the main database server may be an update mark or an un-update mark according to different log update states of the main database server. Therefore, step S204 specifically includes: acquiring a log update state of the main database server as an updating log or an updated log according to an update mark in the mark log packet, and determining that the main and standby data are inconsistent according to the log update state; or acquiring the log update state of the main database server as an un-updated log according to the un-updated mark in the mark log packet, and determining the consistency of the main and standby data according to the log update state.
Taking the form of the log number as an example, if the log numbers of all logs included in the marked log packet are 101-105. When the log packet is to be transmitted, the master database server is generating a new log 106, and therefore the flag information written in the log packet is the update flag indicated by the log number 106. After the backup database server receives the marked log packet containing the log number 106, because the log number of the last generated log in the log packet is 105, which is different from the log number 106 of the update mark, the backup database server can acquire the update state of the log of the primary database server as the updating or updated log, and the log in the backup database server is less than the log of the primary database server. If the master-slave switching is performed at this time, the master-slave database server may not be able to recover all data in the master database server according to the logs owned by the master-slave database server, and thus it is determined that the master-slave data is inconsistent, which may result in data loss.
In another case, if the master database server does not generate a new log when the log packet is to be transmitted, that is, the last log generated by the master database server is the last log generated in the log packet (log with log number 105), the flag information written in the log packet is the update flag represented by the log number 105. After the backup database server receives the marked log packet, because the log number of the last generated log in the log packet is 105, which is the same as the log number 105 of the update mark, the backup database server can acquire the log update state of the primary database server as an un-updated log, and can determine that all logs in the primary database server are synchronized to the backup database server.
In practical application, the last log among all logs generated by one transaction is a transaction commit operation log. When the database server performs transaction processing, only the committed transaction will actually update the data, thereby possibly causing data loss. Therefore, as a preferred embodiment, only the transaction COMMIT operation log (e.g. the log corresponding to the COMMIT statement in the database system) in the primary database server may be considered, that is, the update state of the log may only consider the update state of the transaction COMMIT operation log, so that the result of the primary and secondary data consistency check is more accurate.
At this time, in step S201, the writing, by the master database server, of the mark information into the current log packet to be sent specifically includes: if the update state of the transaction submitting operation log of the main database server is the transaction submitting operation log which is being updated or is updated, writing an update mark into a current log packet to be sent; and if the update state of the transaction submitting operation log of the main database server is the non-updated transaction submitting operation log, writing a non-updated mark into the current log packet to be sent.
Correspondingly, in step S204, determining the consistency of the primary and secondary data according to the log update state of the primary database server indicated by the tag information in the tag log packet specifically includes: acquiring a log update state of the main database server as an updating log or an updated log according to an update mark in the mark log packet, and determining that the main and standby data are inconsistent according to the log update state; or acquiring the log update state of the main database server as an un-updated log according to the un-updated mark in the mark log packet, and determining the consistency of the main and standby data according to the log update state.
Taking the form of the bit as an example, if when a certain log packet is to be sent, the master database server is performing a commit operation of a transaction and simultaneously generating a corresponding transaction commit operation log, at this time, the master database server will write a bit with a value of 1 as an update flag into the log packet to generate a flag log packet. After receiving the marked log packet, the standby database server can learn that the log update state of the main database server is the transaction commit operation log which is being updated or has been updated according to the value of the bit, and the transaction commit operation log which is generated recently in the main database server is not synchronized to the standby database server. If the master-slave switching is performed at this time, the master-slave database server cannot recover all data in the master database server according to the logs owned by the master-slave database server, so that the master-slave data are determined to be inconsistent, and data loss is caused.
In practical applications, the time from writing the tag information to sending the tag log packet to the backup database server is very short, but there is a low possibility that a problem occurs in the time. For example, the main database server commits the transaction a and is generating a transaction commit operation log of the transaction a, at this time, the flag information written into the currently transmitted flag log packet is an update flag, and then the main database server completes the commit operation of the transaction a and generates a corresponding transaction commit operation log. If the marked log packets are not successfully sent due to the downtime of the main database server, the standby database server may make an erroneous judgment due to the fact that the marks in the marked log packets received last time are not updated.
In order to solve the foregoing problem, an embodiment of the present invention further provides a preferred method for checking consistency of primary and secondary data, where as shown in fig. 3, at a primary database server 110, after sending the marked log packet to a secondary database server, the method further includes:
step S205, if the marked log packet includes an update mark, after it is confirmed that the marked log packet is received by the backup database server, the transaction commit operation corresponding to the transaction commit operation log being updated is completed.
The manner of the master database server confirming that the backup database server receives the marked log packet may be determined according to the requirements of the application scenario, for example, a reliable transmission protocol is used to ensure that the backup database server sends a corresponding confirmation character (e.g., ACK of TCP protocol) after receiving the marked log packet, or the backup database server separately returns a feedback message to inform the master database server that the marked log packet has been received.
After the marked log packet is received by the standby database server, the transaction submitting operation corresponding to the transaction submitting operation log which is being updated is completed, so that the state change caused by the transaction submitting can be notified to the standby database server, the standby database server is prevented from generating wrong judgment due to the condition, and the accuracy of the primary and standby data consistency check is further improved.
If the transaction is committed every time, the transaction needs to wait for the completion of the sending of the corresponding marked log packet, which may affect the transaction processing speed. Therefore, the step S205 may be optimized, specifically: and if the marked log packet contains an update mark and a previous marked log packet sent to the standby database server contains an un-updated mark, after the standby database server is confirmed to receive the marked log packet sent this time, the transaction submitting operation corresponding to the transaction submitting operation log which is being updated is completed.
If the previous marked log packet sent by the main database server to the standby database server contains the update mark, the standby database server can already determine that the main and standby data are inconsistent when receiving the previous marked log packet, and if the marked log packet sent this time also contains the update mark, the determined main and standby data states are still inconsistent after the standby database server receives the updated marked log packet. Therefore, whether the standby database server receives the marked log packet or not does not influence the judgment result of the standby database server on the consistency of the current main and standby data.
Therefore, only when the currently sent marked log packet contains an update mark and the previously marked log packet contains an un-updated mark, waiting is carried out, and after the database server is confirmed to receive the currently sent marked log packet, the transaction submitting operation corresponding to the transaction submitting operation log which is being updated is completed. When the mark log packet sent this time contains the update mark and the previous mark log packet also contains the update mark, waiting is not needed, so that the speed of transaction processing is ensured.
Fig. 4 shows a master database server 110 for master-slave data consistency check according to an embodiment of the present application, which includes a mark writing device 111 and a sending device 112. Specifically, the mark writing device 111 is configured to write mark information into a log packet to be currently sent, and generate a mark log packet, where the mark information indicates a log update state of a master database server; the sending device 112 is configured to send the marked log packet to the standby database server, so that the standby database server determines consistency of the main and standby data according to the log update state of the main database server, which is indicated by the mark information in the marked log packet.
The log packet to be sent is a log packet to be sent by the main database server to the standby database server at the current moment, namely, the log packet is sent to the standby database server within a very short time, so that the log updating state of the main database server is almost synchronized to the standby database server in real time through the log packet, and the standby database server can always acquire the log updating state of the main database server within the shortest time, so that the accuracy of judging the consistency of the main and standby data is improved, and whether data is lost after the main and standby are switched is accurately judged.
The mark information represents the log updating state of the main database server, and the log updating state can be set in two situations as the scheme aims to check the consistency of the main and standby data: 1. an updating or updated log; 2. the log is not updated. The log update status indicates whether a new log is generated or is being generated in the master database server compared to the last generated log in the log packet when the log packet to be transmitted is to be transmitted. In case 1, it indicates that a new log is generated or is being generated, there is a data update in the primary database server, and the operation of the data update does not mark the log record in the log packet, which may cause the data in the primary database server to be inconsistent with the data in the secondary database server. Accordingly, in case 2, it means that no new log is generated, no data is updated in the master database server, and the last generated log by the master database server is ready to be transmitted in the marked log packet, so that the master database server can replicate the data of the master database server at the time of transmitting the marked log packet according to the log in the marked log packet received by the master database server, so that the data of the two are consistent.
Corresponding to the log update state of the updating log or the updated log, the corresponding mark information can be an update mark; and corresponding to the log update state of the non-updated log, the corresponding mark information may be a non-updated mark.
Specifically, the mark writing device 111 is configured to write an update mark into a current log packet to be sent when the log update status of the master database server is that the log is being updated; or when the log update state of the main database server is the non-update log, writing a non-update mark into the current log packet to be sent to generate a marked log packet.
In practical applications, the update flag and the non-update flag may take different forms based on coding schemes or other considerations. For example, the update flag may be a log number, where the log number of the log recently generated by the master database server or the log number of the log currently being generated is used as the update flag, and correspondingly, the log number of a log recently generated in the log packet to be sent or a preset special log number (such as 0 or 00) is used as the non-update flag.
Also for example, it may be in the form of a bit, i.e., one bit (1 or 0) represents two cases. Specifically, a bit of 1 indicates that the log update status is an updating log or an updated log, and a bit of 0 indicates that the log update status is an un-updated log. Of course, the opposite value may be used to indicate the corresponding log update status.
Further, it is also possible to write empty flag information in one log update state and write arbitrary flag information in another log update state. Specifically, when the log update state of the log is not updated, empty flag information is written (i.e., no flag is written); and when the log is updated or the log update state of the log is updated, the mark information (which can be a bit, a log number and the like) with any content is written. Of course, the corresponding log update status may also be represented in the reverse form.
It will be understood by those skilled in the art that the specific forms of the update flag and the non-update flag are merely examples, and other existing or future forms of update flag and non-update flag may be applied to the present invention, and are included within the scope of the present invention and are incorporated by reference herein.
Correspondingly, an embodiment of the present application further provides a standby database server 120 for checking consistency of the standby data, where a structure of the standby database server 120 is shown in fig. 5, and includes a receiving device 121 and a checking device 122. Specifically, the receiving device 121 is configured to receive a marked log packet sent by main data, where the marked log packet includes mark information; the checking device 122 is configured to determine consistency of the primary and secondary data according to a log update state of the primary database server indicated by the tag information in the tag log packet.
The mark information written into the mark log packet by the main database server may be an update mark or an un-update mark according to different log update states of the main database server. Therefore, the verifying unit 122 is specifically configured to: acquiring a log update state of the main database server as an updating log or an updated log according to an update mark in the mark log packet, and determining that the main and standby data are inconsistent according to the log update state; or acquiring the log update state of the main database server as an un-updated log according to the un-updated mark in the mark log packet, and determining the consistency of the main and standby data according to the log update state.
Taking the form of the log number as an example, if the log numbers of all logs included in the marked log packet are 101-105. When the log packet is to be transmitted, the master database server is generating a new log 106, and therefore the flag information written in the log packet is the update flag indicated by the log number 106. After the backup database server receives the marked log packet containing the log number 106, because the log number of the last generated log in the log packet is 105, which is different from the log number 106 of the update mark, the backup database server can acquire the update state of the log of the primary database server as the updating or updated log, and the log in the backup database server is less than the log of the primary database server. If the master-slave switching is performed at this time, the master-slave database server may not be able to recover all data in the master database server according to the logs owned by the master-slave database server, and thus it is determined that the master-slave data is inconsistent, which may result in data loss.
In another case, if the master database server does not generate a new log when the log packet is to be transmitted, that is, the last log generated by the master database server is the last log generated in the log packet (log with log number 105), the flag information written in the log packet is the update flag represented by the log number 105. After the backup database server receives the marked log packet, because the log number of the last generated log in the log packet is 105, which is the same as the log number 105 of the update mark, the backup database server can acquire the log update state of the primary database server as an un-updated log, and can determine that all logs in the primary database server are synchronized to the backup database server.
In practical application, the last log among all logs generated by one transaction is a transaction commit operation log. When the database server performs transaction processing, only the committed transaction will actually update the data, thereby possibly causing data loss. Therefore, as a preferred embodiment, only the transaction COMMIT operation log (e.g. the log corresponding to the COMMIT statement in the database system) in the primary database server may be considered, that is, the update state of the log may only consider the update state of the transaction COMMIT operation log, so that the result of the primary and secondary data consistency check is more accurate.
At this time, the mark writing device 111 in the master database server 110 is specifically configured to: when the update state of the transaction submitting operation log of the main database server is the updating transaction submitting operation log, writing an update mark into the current log packet to be sent, or when the update state of the transaction submitting operation log of the main database server is the non-updating transaction submitting operation log, writing a non-update mark into the current log packet to be sent, and generating a marked log packet.
Correspondingly, the verifying unit 122 in the database server 120 is specifically configured to: acquiring the update state of the transaction submitting operation log of the main database server as an updating or updated transaction submitting operation log according to the update mark in the mark log packet, and determining that the main and standby data are inconsistent according to the update state of the transaction submitting operation log; or acquiring the update state of the transaction submitting operation log of the main database server as an un-updated transaction submitting operation log according to the un-updated mark in the mark log packet, and determining the consistency of the main and standby data according to the update state of the transaction submitting operation log.
Taking the form of the bit as an example, if when a certain log packet is to be sent, the master database server is performing a commit operation of a transaction and simultaneously generating a corresponding transaction commit operation log, at this time, the master database server will write a bit with a value of 1 as an update flag into the log packet to generate a flag log packet. After receiving the marked log packet, the standby database server can learn that the log update state of the main database server is the transaction commit operation log which is being updated or has been updated according to the value of the bit, and the transaction commit operation log which is generated recently in the main database server is not synchronized to the standby database server. If the master-slave switching is performed at this time, the master-slave database server cannot recover all data in the master database server according to the logs owned by the master-slave database server, so that the master-slave data are determined to be inconsistent, and data loss is caused.
In practical applications, the time from writing the tag information to sending the tag log packet to the backup database server is very short, but there is a low possibility that a problem occurs in the time. For example, the main database server commits the transaction a and is generating a transaction commit operation log of the transaction a, at this time, the flag information written into the currently transmitted flag log packet is an update flag, and then the main database server completes the commit operation of the transaction a and generates a corresponding transaction commit operation log. If the marked log packets are not successfully sent due to the downtime of the main database server, the standby database server may make an erroneous judgment due to the fact that the marks in the marked log packets received last time are not updated.
In order to solve the above problem, an embodiment of the present application further provides a preferred master database server for checking consistency of master and slave data, as shown in fig. 6, in addition to the mark writing device 111 and the sending device 112 shown in fig. 4, a transaction committing device 113 is further included. Specifically, the transaction committing device 113 is configured to include an update flag in the flag log packet, and after confirming that the database server receives the flag log packet, complete the transaction committing operation corresponding to the transaction committing operation log being updated.
The manner of the master database server confirming that the backup database server receives the marked log packet may be determined according to the requirements of the application scenario, for example, a reliable transmission protocol is used to ensure that the backup database server sends a corresponding confirmation character (e.g., ACK of TCP protocol) after receiving the marked log packet, or the backup database server separately returns a feedback message to inform the master database server that the marked log packet has been received.
After the marked log packet is received by the standby database server, the transaction submitting operation corresponding to the transaction submitting operation log which is being updated is completed, so that the state change caused by the transaction submitting can be notified to the standby database server, the standby database server is prevented from generating wrong judgment due to the condition, and the accuracy of the primary and standby data consistency check is further improved.
If the transaction is committed every time, the transaction needs to wait for the completion of the sending of the corresponding marked log packet, which may affect the transaction processing speed. Therefore, the transaction committing device 113 may be optimized, and specifically, the transaction committing device 113 is configured to include an update flag in the marked log packet, include an un-updated flag in a previous marked log packet sent to the standby database server, and complete the transaction committing operation corresponding to the transaction committing operation log being updated after confirming that the standby database server receives the marked log packet sent this time.
If the previous marked log packet sent by the main database server to the standby database server contains the update mark, the standby database server can already determine that the main and standby data are inconsistent when receiving the previous marked log packet, and if the marked log packet sent this time also contains the update mark, the determined main and standby data states are still inconsistent after the standby database server receives the updated marked log packet. Therefore, whether the standby database server receives the marked log packet or not does not influence the judgment result of the standby database server on the consistency of the current main and standby data.
Therefore, only when the currently sent marked log packet contains an update mark and the previously marked log packet contains an un-updated mark, waiting is carried out, and after the database server is confirmed to receive the currently sent marked log packet, the transaction submitting operation corresponding to the transaction submitting operation log which is being updated is completed. When the mark log packet sent this time contains the update mark and the previous mark log packet also contains the update mark, waiting is not needed, so that the speed of transaction processing is ensured.
It should be noted that the present application may be implemented in software and/or a combination of software and hardware, for example, implemented using Application Specific Integrated Circuits (ASICs), general purpose computers or any other similar hardware devices. In one embodiment, the software programs of the present application may be executed by a processor to implement the steps or functions described above. Likewise, the software programs (including associated data structures) of the present application may be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Additionally, some of the steps or functions of the present application may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present application may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or techniques in accordance with the present application through the operation of the computer. Program instructions which invoke the methods of the present application may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the present application comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or a solution according to the aforementioned embodiments of the present application.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (16)
1. A method for master-slave data consistency check at a master database server side is disclosed, wherein the method comprises the following steps:
writing mark information into a current log packet to be sent to generate a mark log packet, wherein the mark information represents a log updating state of a main database server, and the log updating state represents whether a new log is generated or is being generated in the main database server compared with a log finally generated in the log packet when the log packet to be sent is to be sent;
and sending the marked log packet to a standby database server so that the standby database server determines the consistency of the main and standby data according to the log updating state of the main database server represented by the marked information in the marked log packet.
2. The method of claim 1, wherein the flag information is an update flag or an un-update flag;
writing mark information into a current log packet to be sent, including:
if the log updating state of the main database server is the updating or updated log, writing an updating mark into the current log packet to be sent;
and if the log updating state of the main database server is the non-updated log, writing a non-updated mark into the current log packet to be sent.
3. The method of claim 1, wherein the flag information is an update flag or an un-update flag, and the log update status is an update status of a transaction commit operation log;
writing mark information into a current log packet to be sent, including:
if the update state of the transaction submitting operation log of the main database server is the transaction submitting operation log which is being updated or is updated, writing an update mark into a current log packet to be sent;
and if the update state of the transaction submitting operation log of the main database server is the non-updated transaction submitting operation log, writing a non-updated mark into the current log packet to be sent.
4. The method of claim 3, wherein after sending the marked log packet to a standby database server, further comprising:
and if the marked log packet contains the updating mark, after the database server is confirmed to receive the marked log packet, completing the transaction submitting operation corresponding to the transaction submitting operation log which is being updated.
5. The method according to claim 4, wherein if the marked log packet includes an update mark, after confirming that the marked log packet is received by the database server, completing the transaction commit operation corresponding to the transaction commit operation log being updated, includes:
and if the marked log packet contains an update mark and a previous marked log packet sent to the standby database server contains an un-updated mark, after the standby database server is confirmed to receive the marked log packet sent this time, the transaction submitting operation corresponding to the transaction submitting operation log which is being updated is completed.
6. A method for primary and standby data consistency check at a standby database server side is disclosed, wherein the method comprises the following steps:
receiving a marking log packet sent by a main database server, wherein the marking log packet comprises marking information;
and determining the consistency of the main and standby data according to the log updating state of the main database server represented by the mark information in the mark log packet, wherein the log updating state represents whether a new log is generated or is being generated in the main database server compared with the last generated log in the log packet when the log packet to be transmitted is to be transmitted.
7. The method of claim 6, wherein the flag information is an update flag or an un-update flag;
determining the consistency of the main and standby data according to the log update state of the main database server represented by the mark information in the mark log packet, including:
acquiring a log update state of the main database server as an updating log or an updated log according to an update mark in the mark log packet, and determining that the main and standby data are inconsistent according to the log update state; or
And acquiring the log update state of the main database server as an un-updated log according to the un-updated mark in the mark log packet, and determining the consistency of the main and standby data according to the log update state.
8. The method of claim 6, wherein the flag information is an update flag or an un-update flag, and the log update status is an update status of a transaction commit operation log;
determining the consistency of the main and standby data according to the log update state of the main database server represented by the mark information in the mark log packet, including:
acquiring the update state of the transaction submitting operation log of the main database server as an updating or updated transaction submitting operation log according to the update mark in the mark log packet, and determining that the main and standby data are inconsistent according to the update state of the transaction submitting operation log; or
And acquiring the update state of the transaction submitting operation log of the main database server as an un-updated transaction submitting operation log according to the un-updated mark in the mark log packet, and determining the consistency of the main and standby data according to the update state of the transaction submitting operation log.
9. A primary database server for primary-backup data consistency checking, wherein the primary database server comprises:
a mark writing device, configured to write mark information into a log packet to be sent currently, and generate a mark log packet, where the mark information indicates a log update state of a master database server, and the log update state indicates whether a new log is generated in the master database server when the log packet to be sent is to be sent;
and the sending device is used for sending the marked log packet to the standby database server so that the standby database server determines the consistency of the main and standby data according to the log updating state of the main database server represented by the marked information in the marked log packet.
10. The master database server according to claim 9, wherein the flag information is an update flag or an un-update flag;
the mark writing device is used for writing an update mark into a current log packet to be sent when the log update state of the main database server is that the log is being updated; or when the log update state of the main database server is the non-update log, writing a non-update mark into the current log packet to be sent to generate a marked log packet.
11. The master database server according to claim 9, wherein the flag information is an update flag or an un-update flag, and the log update state is an update state of a transaction commit operation log;
the mark writing device is used for writing an update mark into the current log packet to be sent when the update state of the transaction submitting operation log of the main database server is the transaction submitting operation log which is being updated, or writing an un-updated mark into the current log packet to be sent when the update state of the transaction submitting operation log of the main database server is the un-updated transaction submitting operation log, so as to generate a mark log packet.
12. The master database server according to claim 11, wherein the master database server further comprises:
and the transaction submitting device is used for finishing the transaction submitting operation corresponding to the transaction submitting operation log which is being updated after the marked log packet contains the updating mark and the database server is confirmed to receive the marked log packet.
13. The primary database server according to claim 12, wherein the transaction committing means is configured to include an update flag in the flag log packet, include an un-updated flag in a previous flag log packet sent to the secondary database server, and complete the transaction committing operation corresponding to the transaction committing operation log being updated after confirming that the secondary database server receives the flag log packet sent this time.
14. A backup database server for primary and backup data consistency check, wherein the backup database server comprises:
the receiving device is used for receiving a marking log packet sent by a main database server, wherein the marking log packet comprises marking information;
and the checking device is used for determining the consistency of the main and standby data according to the log updating state of the main database server represented by the mark information in the mark log packet, wherein the log updating state represents whether a new log is generated in the main database server when the log packet to be sent is to be sent.
15. The database server according to claim 14, wherein the flag information is an update flag or an un-update flag;
the checking device is used for acquiring a log updating state of the main database server as an updating log or an updated log according to an updating mark in the mark log packet, and determining that the main and standby data are inconsistent according to the log updating state; or acquiring the log update state of the main database server as an un-updated log according to the un-updated mark in the mark log packet, and determining the consistency of the main and standby data according to the log update state.
16. The database server according to claim 14, wherein the flag information is an update flag or an un-update flag, and the log update status is an update status of a transaction commit operation log;
the checking device is used for acquiring the update state of the transaction submitting operation log of the main database server as an updating or updated transaction submitting operation log according to the update mark in the mark log packet, and determining that the main and standby data are inconsistent according to the update state of the transaction submitting operation log; or acquiring the update state of the transaction submitting operation log of the main database server as an un-updated transaction submitting operation log according to the un-updated mark in the mark log packet, and determining the consistency of the main and standby data according to the update state of the transaction submitting operation log.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510835059.7A CN106802892B (en) | 2015-11-26 | 2015-11-26 | Method and equipment for checking consistency of main and standby data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510835059.7A CN106802892B (en) | 2015-11-26 | 2015-11-26 | Method and equipment for checking consistency of main and standby data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106802892A CN106802892A (en) | 2017-06-06 |
| CN106802892B true CN106802892B (en) | 2021-12-28 |
Family
ID=58977371
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510835059.7A Active CN106802892B (en) | 2015-11-26 | 2015-11-26 | Method and equipment for checking consistency of main and standby data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106802892B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109672712A (en) * | 2017-10-17 | 2019-04-23 | 中兴通讯股份有限公司 | Method of data synchronization, device, super controller, domain controller and storage medium |
| CN109857523B (en) * | 2017-11-30 | 2023-05-09 | 阿里巴巴集团控股有限公司 | Method and device for realizing high availability of database |
| CN110109934B (en) * | 2019-05-08 | 2021-07-30 | 上海达梦数据库有限公司 | Database management method, device, server and storage medium |
| CN110209554B (en) * | 2019-06-14 | 2023-08-11 | 上海中通吉网络技术有限公司 | Database log distribution method, device and equipment |
| CN110471909B (en) * | 2019-08-26 | 2022-03-08 | 上海达梦数据库有限公司 | Database management method, device, server and storage medium |
| CN110727550B (en) * | 2019-12-18 | 2020-06-12 | 中国银联股份有限公司 | Data replication processing method, data replication processing device, disaster recovery system, disaster recovery equipment and storage medium |
| CN111723066B (en) * | 2020-05-08 | 2023-06-13 | 武汉达梦数据库股份有限公司 | Database switching method and database switching system based on log analysis synchronization |
| CN113076343B (en) * | 2021-04-30 | 2024-04-05 | 北京京东振世信息技术有限公司 | Data query method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102982087A (en) * | 2012-11-01 | 2013-03-20 | 华为技术有限公司 | Method and device and data bank system for detecting data bank uniformity |
| CN103020242A (en) * | 2012-12-19 | 2013-04-03 | 中国人民解放军第二炮兵装备研究院第四研究所 | Data synchronization method based on oracle database |
| CN103560906A (en) * | 2013-10-22 | 2014-02-05 | 珠海多玩信息技术有限公司 | Data replication method and device |
| CN104021132A (en) * | 2013-12-08 | 2014-09-03 | 郑州正信科技发展股份有限公司 | Method and system for verification of consistency of backup data of host database and backup database |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5555371A (en) * | 1992-12-17 | 1996-09-10 | International Business Machines Corporation | Data backup copying with delayed directory updating and reduced numbers of DASD accesses at a back up site using a log structured array data storage |
| CN102752372A (en) * | 2012-06-18 | 2012-10-24 | 天津神舟通用数据技术有限公司 | File based database synchronization method |
| CN103685350B (en) * | 2012-09-04 | 2016-12-21 | 阿里巴巴集团控股有限公司 | The synchronous method of storage system and relevant equipment |
| CN102891849B (en) * | 2012-09-25 | 2015-07-22 | 北京星网锐捷网络技术有限公司 | Service data synchronization method, data recovery method, data recovery device and network device |
| CN103312549B (en) * | 2013-06-26 | 2016-08-24 | 华为技术有限公司 | A kind of office management method and device and system |
-
2015
- 2015-11-26 CN CN201510835059.7A patent/CN106802892B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102982087A (en) * | 2012-11-01 | 2013-03-20 | 华为技术有限公司 | Method and device and data bank system for detecting data bank uniformity |
| CN103020242A (en) * | 2012-12-19 | 2013-04-03 | 中国人民解放军第二炮兵装备研究院第四研究所 | Data synchronization method based on oracle database |
| CN103560906A (en) * | 2013-10-22 | 2014-02-05 | 珠海多玩信息技术有限公司 | Data replication method and device |
| CN104021132A (en) * | 2013-12-08 | 2014-09-03 | 郑州正信科技发展股份有限公司 | Method and system for verification of consistency of backup data of host database and backup database |
Non-Patent Citations (2)
| Title |
|---|
| "The research of synchronization and consistency of data in mobile environment";JiuLing Feng等;《2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems》;20131114;第869-874页 * |
| "一种基于记录标记的数据库同步算法";李立亚;《计算机与数字工程》;20150630;第946-952页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106802892A (en) | 2017-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106802892B (en) | Method and equipment for checking consistency of main and standby data | |
| US20190079839A1 (en) | Locality based quorums | |
| US8463746B2 (en) | Method and system for replicating data | |
| US8689047B2 (en) | Virtual disk replication using log files | |
| CN101706802B (en) | Method, device and sever for writing, modifying and restoring data | |
| US8762337B2 (en) | Storage replication systems and methods | |
| JP6987340B2 (en) | Database data change request processing method and equipment | |
| US9229970B2 (en) | Methods to minimize communication in a cluster database system | |
| US20140108345A1 (en) | Exchanging locations of an out of synchronization indicator and a change recording indicator via pointers | |
| US10324905B1 (en) | Proactive state change acceptability verification in journal-based storage systems | |
| CN106126374A (en) | Method for writing data, method for reading data and device | |
| CN110121694B (en) | Log management method, server and database system | |
| US11593216B2 (en) | Method, device, and computer program product for managing file system | |
| CN107402848A (en) | A kind of implementation method of snapshot data uniformity | |
| US20180121531A1 (en) | Data Updating Method, Device, and Related System | |
| CN108241676A (en) | Realize the method and apparatus that data synchronize | |
| US11693844B2 (en) | Processing delete requests based on change feed of updates | |
| US10620872B2 (en) | Replicating data in a data storage system | |
| CN110121712B (en) | Log management method, server and database system | |
| CN103729269A (en) | Cloud architecture-based network test data double-cache method | |
| CN113535477B (en) | Method and equipment for data disaster recovery | |
| CN111274258A (en) | Block chain data uplink method | |
| CN111522883A (en) | Backup method, device and equipment of object data and storage medium | |
| US11513717B2 (en) | Online data rehydration | |
| US7962710B1 (en) | Techniques for creating checkpoints |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |