WO2015024377A1

WO2015024377A1 - Data synchronization method, apparatus and device, and computer storage medium

Info

Publication number: WO2015024377A1
Application number: PCT/CN2014/073628
Authority: WO
Inventors: 王奎; 张广军; 宋劼明
Original assignee: 中兴通讯股份有限公司
Priority date: 2013-08-21
Filing date: 2014-03-18
Publication date: 2015-02-26
Also published as: CN104426923A

Abstract

Disclosed are a data synchronization method, apparatus and device, and a computer storage medium. The method comprises: after a first device carries out data synchronization on a second device, the first device receiving check information sent by the second device, wherein the check information is used for determining whether the data synchronization succeeds or not; the first device judging whether the data synchronization succeeds or not at least according to the check information; and in the case where the data synchronization fails, the first device carrying out data synchronization on the second device. The present invention solves the problem of making errors in synchronizing data between devices, and improves the reliability of data synchronization.

Description

TECHNICAL FIELD The present invention relates to the field of communications, and in particular to a data synchronization method, apparatus, device, and computer storage medium. BACKGROUND OF THE INVENTION Synchronization technology of an existing in-memory database uses a change table and a change record synchronization method in a master-slave mode and a multi-module (when a plurality of modules are distributed as reference data to other modules), when a table and a record occur. When changing, these changes are packaged and sent to the standby or other modules. These synchronization mechanisms only trigger data synchronization when the data changes. The judgment mechanism of the data change is whether the table records are modified. If there is no data modification, Data synchronization will be triggered. Due to the complex network environment, the data between the active and standby modules and the multi-module are cross-board messages. The message is inevitably lost or abnormal at the receiving end during the synchronization process. Especially after the synchronization is completed, the memory in the area where the recording is stored at the receiving end is stepped on. In this case, this situation is generally difficult to find. At this time, if the No. 1 module data is not modified, it will not trigger data synchronization. In this case, the data of the standby machine and other modules will be wrong. If an active/standby switchover occurs in this situation, an unknown disaster occurs in the system. In view of the problem of synchronization data error between devices in the related art, an effective solution has not been proposed yet. SUMMARY OF THE INVENTION Embodiments of the present invention provide a data synchronization method, apparatus, device, and computer storage medium to solve at least the problem of synchronization data errors between devices. According to an aspect of the present invention, a data synchronization method is provided, including: after the first device performs data synchronization with the second device, the first device receives the verification information sent by the second device, The verification information is used to determine whether the data synchronization is successful; the first device determines, according to the verification information, whether the data synchronization is successful; in the case that the data synchronization fails, the A device performs data synchronization with the second device. Preferably, the data synchronization method is started if the first device and/or the device resource that is idle locally of the second device exceeds a threshold; or the startup command is received, and the data synchronization method is started. Preferably, the device comprises at least one of the following: a processor resource, a memory resource, a bandwidth resource, and 10 resources. Preferably, in the case that the verification information includes a data version number, determining, by the first device, whether the data synchronization is successful according to the verification information comprises: the first device determining, from the second device Whether the data version number is the same as the data version number of the first device. If they are the same, it is determined that the data synchronization is successful. If not, the data synchronization failure is determined. Preferably, in the case that the data is synchronized with the database data, the verification information includes: a checksum calculated by the second device to calculate some or all of the records in the predetermined table in the database, the first Determining, by the device, whether the data synchronization is successful according to the verification information, the first device calculating a record corresponding to the locally stored table corresponding to the predetermined table to obtain a checksum of the first device; The first device determines whether the checksum of the second device is the same as the checksum of the first device. If they are the same, it is determined that the data synchronization is successful. If not, the data synchronization is determined to be unsuccessful. According to another aspect of the present invention, a data synchronization device is provided, which is located in a first device, and includes: a receiving module, configured to receive the second after the first device performs data synchronization to the second device The verification information sent by the device, where the verification information is used to determine whether the data synchronization is successful; the determining module is configured to determine, according to the verification information, whether the data synchronization is successful; the synchronization module is set to be In the case that the data synchronization fails, data synchronization is performed to the second device. Preferably, the data synchronization device further includes: a startup module, configured to start the data synchronization device if the device resource that is idle locally by the first device and/or the second device exceeds a threshold; or, set to Upon receiving the start command, the data synchronization device is activated. Preferably, the device resource includes at least one of the following: a processor resource, a memory resource, a bandwidth resource, and 10 resources. Preferably, the determining module is further configured to: if the verification information includes a data version number, determine whether the data version number from the second device is the same as the data version number of the first device, if the same , it is determined that the data synchronization is successful, if not the same, it is determined that the data synchronization fails. Preferably, in the case that the data is synchronized with the database data, the verification information includes: a checksum calculated by the second device to calculate some or all of the records in the predetermined table in the database, and the determining module And determining, by calculating a corresponding record in the table corresponding to the predetermined table that is locally stored, obtaining a checksum of the first device, determining a checksum of the second device, and a calibration of the first device If the checksums are the same, if they are the same, it is determined that the data synchronization is successful. If they are not the same, it is determined that the data synchronization fails. According to the embodiment of the present invention, after the data synchronization of the first device to the second device is completed, the first device receives the verification information sent by the second device, where the verification information is used to determine whether the data synchronization is successful; Determining whether the data synchronization is successful according to the verification information. In the case that the data synchronization fails, the first device performs data synchronization with the second device, which solves the problem of synchronization data error between the devices, and improves the reliability of data synchronization. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in FIG. In the drawings: FIG. 1 is a flow chart of a data synchronization method according to an embodiment of the present invention; FIG. 2 is a structural diagram of a data synchronization apparatus according to an embodiment of the present invention; FIG. 3 is an in-memory database data according to a preferred embodiment of the present invention. FIG. 4 is a flow chart of data checking and synchronization according to a preferred embodiment of the present invention; FIG. 5 is a topological structural view of an in-memory database synchronous data verifying function module according to a preferred embodiment of the present invention; An in-memory database synchronization data verification interaction diagram in accordance with a preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict. The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. In this embodiment, a data synchronization method is provided. FIG. 1 is a flowchart of a data synchronization method according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps: Step S102, in a first device After the second device performs data synchronization, the first device receives the verification information sent by the second device, where the verification information is used to determine whether the data synchronization is successful. Step S104: The first device determines the data according to at least the verification information. Whether the synchronization is successful; Step S106: In the case that the data synchronization fails, the first device performs data synchronization with the second device. Through the above steps, a mechanism for determining according to the verification information is added, so that the first device can determine whether the data synchronization is successful, thereby providing a possibility for finding synchronization data errors, and solving the problem of synchronization data errors between devices. Improves the reliability of data synchronization between devices. Preferably, in the foregoing method, in the embodiment, in a case where the device resource that is idle locally of the first device and/or the second device exceeds a threshold, the data synchronization method may be started; or After receiving the start command, start the data synchronization method. For example, the resources involved in the execution process include CPU, memory, network port bandwidth, and disk 10. The user can configure the idle thresholds of the four types of resources. When the four types of resource idle rates exceed the four thresholds, the system is considered idle. . The four resources can be required to idle more than the threshold at the same time, or the single resource can be idle to exceed the threshold to start the above method. For example, when the CPU resource is idle more than 50%, this data checking mechanism can be started. As another preferred manner, it is also possible to provide a user to manually trigger or configure a check point in time to initiate this check mechanism. In this embodiment, the verification information between the first device and the second device may be various. For example, if the verification information includes the data version number, the first device determines, according to the verification information, whether the data synchronization is successful. The first device determines whether the data version number from the second device is the same as the data version number of the first device. If they are the same, it is determined that the data synchronization is successful. If not, the data synchronization failure is determined. The verification information may further include: a checksum calculated by the second device on a part or all of the records in the predetermined table in the database, and determining, by the first device, whether the data synchronization is successful according to the verification information, the first device pair The locally stored record corresponding to the predetermined table is calculated to obtain a checksum of the first device; the first device determines whether the checksum of the second device is the same as the checksum of the first device, if the same , to determine the data synchronization success, if not the same, then determine the data synchronization failed. Preferably, these predetermined tables may be all the tables involved in the data synchronization, or some important tables involved in the data synchronization. As another preferred embodiment, after detecting the data synchronization failure, it is also possible to detect which part of the data synchronization has failed, and then only synchronize the failed data again. In this embodiment, a data synchronization device is also provided, and the menu processing device is used to implement the foregoing method. It has been described in the above embodiments and preferred embodiments, and details are not described herein again. It should be noted that the name of the module in the following device does not constitute a practical limitation of the module. For example, the receiving module may be expressed as: “set to receive the data after the first device performs data synchronization with the second device. The module of the verification information sent by the second device, the following modules may be implemented in the processor, for example, the receiving module may be expressed as "a processor, configured to perform data synchronization on the first device to the second device. After receiving the verification information sent by the second device, or "a processor, including a receiving module" or the like. 2 is a block diagram of a data synchronizing apparatus according to an embodiment of the present invention. As shown in FIG. 2, the apparatus 20 includes: a receiving module 22, a judging module 24, and a synchronizing module 26. The apparatus will be described below. The receiving module 22 is configured to receive, after the first device performs data synchronization with the second device, the verification information sent by the second device, where the verification information is used to determine whether the data synchronization is successful; the determining module 24 is configured to be at least Determining whether the data synchronization is successful according to the verification information; the synchronization module 26 is configured to perform data synchronization to the second device in the case that the data synchronization fails. Preferably, in this embodiment, the data synchronization device further includes: a startup module, configured to start the data synchronization device if the device resource that is idle locally of the first device and/or the second device exceeds a threshold; or, set to Upon receiving the start command, the data synchronization device is started. Preferably, the device resource includes at least one of the following: a processor resource, a memory resource, a bandwidth resource, and an input/output 10 resource. Preferably, the determining module 24 is further configured to: if the verification information includes the data version number, determine whether the data version number from the second device is the same as the data version number of the first device, and if the same, determine the data. The synchronization is successful. If it is not the same, it determines that the data synchronization failed. Preferably, in the case that the data is synchronized with the database data, the verification information includes: a checksum calculated by the second device to calculate some or all of the records in the predetermined table in the database, and the determining module is further configured to Calculating a checksum of the first device in the locally stored record corresponding to the predetermined table, and determining whether the checksum of the second device is the same as the checksum of the first device, if the same , to determine the data synchronization success, if not the same, then determine the data synchronization failed. In the embodiment of the present invention, a data synchronization device is further provided, including the data synchronization device in the above implementation. In the embodiment of the present invention, a computer storage medium is further provided, and the computer storage medium may store an execution instruction for executing the method in the foregoing embodiment. The invention will now be described in connection with preferred embodiments. 3 is a structural diagram of in-memory database data check and synchronization according to a preferred embodiment of the present invention. As shown in FIG. 3, the method includes: a startup policy management module 32, a data inspection module 34, and a data distribution module 36. Description. The startup policy management module 32 is responsible for ensuring that the inspection mechanism is started during the system idle time, mainly including the configuration of the CPU, the memory, the network port bandwidth, and the disk 10 idle threshold, the statistics of the four types of resource occupancy, and the execution of the human task trigger ( The function of the above startup module is implemented). The data checking module 34 is responsible for managing the registration of the table to be checked, the checksum calculation of the data table record, the sending and receiving of the checksum data, and the check and check between the master and the backup, and the inter-module (implementing the function of the above-mentioned judging module 24) The data distribution module 36 is responsible for distributing the records of the single table to the designated receiving end, the receiving end is responsible for receiving the data and updating to the memory table, and the sending end is responsible for transmitting the packed data from the table to the receiving end (the synchronization module 26 is implemented). Features). 4 is a flow chart of data checking and synchronization according to a preferred embodiment of the present invention. As shown in FIG. 4, the data receiving end is represented by the Rx end, and the data transmitting end is represented by Tx, and the steps are as follows: Step S402, Tx at power-on The terminal and the Rx end respectively register all the tables that need to be periodically verified to the verification management module. Step S404, the Tx terminal periodically performs the judgment of the startup policy, and notifies the Rx terminal to make the determination of the startup policy, when the policy is met or the user manually triggers When starting, sending a message to the Rx end, starting the data checking mechanism; Step S406, the Rx end sends the database data version number of the local machine to the Tx end; Step S408, the Tx end receives the data version number sent by the Rx end, and The data version number of the local device is compared. If the version numbers are inconsistent, the data normal synchronization process is triggered to perform data synchronization. If the version numbers are the same, the message is sent to the Rx end to start the data check. Step S410, the Tx end traverses all the registered data. a checksum calculation is performed on the records of each table, and stored in the table record check data area; in step S412, the Rx end receives the data sent from the Tx end. Check the message, start traversing all the registry, perform checksum calculation on each table record, and send the table and checksum information to the Tx end; Step S414, the Tx end receives the check sent by the Rx end And the information and the table record check area data comparison, if the check is consistently ended, if the inconsistency, the table is packaged one by one to the Rx end; Step S416, the Rx end receives the table record data and checksum information sent by the Tx end , perform data verification, if they are consistent, update to the memory table, and notify the Tx end to continue the transmission of the next table; if not, notify the Tx end to resend the table; In step S418, after the Tx end sends all the inconsistent table records, the Rx end is notified to end the check, and the current check process is ended. 5 is a topological structural diagram of an in-memory database synchronization data verification function module according to a preferred embodiment of the present invention. As shown in FIG. 5, startup policy management is used to ensure that the data inspection mechanism is started when system resources are idle; It is set to check whether the data data of the data sender and the data receiver records the memory data. For the inconsistent data, it is distributed through the data distribution module to ensure the correctness of the data. FIG. 6 is a diagram showing an in-memory database synchronization data verification interaction diagram according to a preferred embodiment of the present invention. As shown in FIG. 6, the method includes the following steps: Step S602: Registering a table to be inspected, a configuration table whose main registration changes little, such a table If the user triggers the startup check mechanism or the current system resource meets the idle policy, the check mechanism is started. Step S606, the receiver requests the verification data version number, if the version number is inconsistent. It indicates that the data has changed, triggering the regular synchronization process for data synchronization; if the data version numbers are the same, it indicates that the data of the receiving end and the transmitting end should be consistent, and the table record verification is performed; Step S608, if the data is verified If the agreement is the same, the current check is ended. If the check is inconsistent, the inconsistent table record is sent to the receiving end, and the check list data of the table is sent for the receiving end check when the table record is sent. Step S610: After receiving the record, the receiving end performs data verification, and if the verification fails, the transmitting end is notified to resend, and when the verification is passed, the record is updated into the table memory. The preferred embodiment provides a mechanism for periodically performing checksum checking using memory blocks between the active and standby modules to ensure the correctness of the data. The checksum mechanism is used to check the database memory when the system resources are relatively idle: Since the in-memory database responds to data access quickly, all the records are stored in the memory, and the data is stored in a continuous memory data area. , the records of each table are also stored continuously. The records of each table are checked and checked. In the case that the data synchronization between the active and standby modules is completed, the standby machine and other modules periodically send the checksum data to the main side of module No. 1, and the main side judges the receiving end. Checksum is consistent with the checksum of the local table record. If they are inconsistent, the trigger data is synchronized to the specified receiver. The preferred embodiment is a protection mechanism for in-memory database synchronization, because the resource idle check policy and the user manually trigger the mechanism to ensure the correctness and reliability of the in-memory database data without affecting the service and rationally utilizing the system resources. . Obviously, those skilled in the art should understand that the above modules or steps of the embodiments of the present invention can be implemented by a general computing device, which can be concentrated on a single computing device or distributed in multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device, such that they may be stored in the storage device by the computing device, or they may be separately fabricated into individual integrated circuit modules, or Multiple of these modules or steps are fabricated as a single integrated circuit module. Thus, embodiments of the invention are not limited to any specific combination of hardware and software. The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention. Industrial Applicability: The present invention relates to the field of data communication, and provides a data synchronization method and device, which solves the problem of synchronization data error between devices and improves the reliability of data synchronization.

Claims

Claim

A data synchronization method, comprising: after the first device performs data synchronization with the second device, the first device receives the verification information sent by the second device, where the verification information is used for Determining whether the data synchronization is successful; the first device determining, according to the verification information, whether the data synchronization is successful; in the case that the data synchronization fails, the first device performs the second device The data is the same.

2. The method according to claim 1, wherein

Initiating the data synchronization method if the first device and/or the second device's locally idle device resource exceeds a threshold; or

The data synchronization method is initiated by receiving a startup command.

The method according to claim 2, wherein the device comprises at least one of the following: a processor resource, a memory resource, a bandwidth resource, and 10 resources.

The method according to any one of claims 1 to 3, wherein, in a case where the verification information includes a data version number, the first device determines, according to the verification information, whether the data synchronization is Success includes:

The first device determines whether the data version number from the second device is the same as the data version number of the first device. If they are the same, it determines that the data synchronization is successful. If not, it determines that the data synchronization fails.

The method according to any one of claims 1 to 3, wherein, in the case that the data is synchronized with the database data, the verification information comprises: the second device in the predetermined table in the database And determining, by the first device, whether the data synchronization is successful according to the verification information, where the first device records the locally stored table corresponding to the predetermined table. Calculating a checksum of the first device by using a corresponding record; determining, by the first device, whether a checksum of the second device is the same as a checksum of the first device, and if the same, determining data synchronization Success, if not the same, then determine that the data synchronization failed.

A data synchronization device, comprising: a receiving module, configured to: after the first device performs data synchronization to the second device, receive verification information sent by the second device, where the verification information is used Determining whether the data synchronization is successful;

The determining module is configured to determine whether the data synchronization is successful according to at least the verification information; and the synchronization module is configured to perform data synchronization to the second device if the data synchronization fails.

The device according to claim 6, further comprising: a startup module, configured to start the data if a device resource that is locally idle by the first device and/or the second device exceeds a threshold The synchronization device; or, configured to receive the startup command, to activate the data synchronization device.

The device according to claim 7, wherein the device resource comprises at least one of the following: a processor resource, a memory resource, a bandwidth resource, and an input/output 10 resource.

The device according to any one of claims 6 to 8, wherein the determining module is further configured to determine a data version from the second device if the verification information includes a data version number Whether the number is the same as the data version number of the first device. If they are the same, it is determined that the data synchronization is successful. If not, it is determined that the data synchronization fails.

The apparatus according to any one of claims 6 to 8, wherein, in the case that the data is synchronized with the database data, the verification information comprises: the second device in the predetermined table in the database The checksum is calculated by the calculation, and the determining module is further configured to calculate, according to the locally stored record corresponding to the predetermined table, a checksum of the first device, and determine the Whether the checksum of the second device is the same as the checksum of the first device. If they are the same, it is determined that the data synchronization is successful. If not, the data synchronization is determined to be unsuccessful.

A data synchronization device comprising the apparatus of any one of claims 6 to 10.

A computer storage medium storing an execution instruction for performing the method of any one of claims 1 to 5.