CN112231403B

CN112231403B - Consistency verification method, device, equipment and storage medium for data synchronization

Info

Publication number: CN112231403B
Application number: CN202011103505.2A
Authority: CN
Inventors: 王凯龙
Original assignee: Beijing Kingbase Information Technologies Co Ltd
Current assignee: Beijing Kingbase Information Technologies Co Ltd
Priority date: 2020-10-15
Filing date: 2020-10-15
Publication date: 2024-01-30
Anticipated expiration: 2040-10-15
Also published as: CN112231403A

Abstract

The embodiment of the disclosure discloses a data synchronization consistency verification method, device, equipment and storage medium. The method comprises the following steps: acquiring data information of each table to be checked, generating a plurality of check tasks based on the data information of each table, and recording the task state of each check task as an unexecuted state; for any check task, if the task state of the check task is a non-executing state, modifying the task state of the check task to be an executing state, processing the check task, and modifying the task state of the check task to be an executing completion state under the condition that the check task is normally checked; if the checking abnormality is detected, the checking tasks which are not checked are continuously checked based on the task states of the checking tasks. The consistency checking process is divided into a plurality of checking tasks, and the task state of each checking task is dynamically modified, so that the checking is continued based on each task state after the fault is recovered, repeated checking of table data is avoided, the checking time is saved, and the checking efficiency is improved.

Description

Consistency verification method, device, equipment and storage medium for data synchronization

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for checking consistency of data synchronization.

Background

The process of real-time synchronization of data of a database using a data synchronization tool comprises three phases: the first stage is to carry out the initialization loading of stock data to obtain the basic point of data synchronization; the second stage uses the synchronization basic point established by the initial data loading as a reference to perform incremental data synchronization; and in the third stage, the source data and the target data of the data synchronization are regularly compared and checked. And in the third stage, all synchronous table data of the source terminal and the target terminal are required to be pulled simultaneously for verification so as to judge whether the data of the source terminal and the target terminal are consistent.

In the process of consistency check, faults such as unstable network, abnormal restarting of a database, abnormal operating system and the like can be encountered, so that the consistency check is interrupted. At present, the processing mode for the situation mainly carries out consistency check on all table data again after fault recovery.

However, since a large number of tables are required for data verification in one data verification process and the data amount of the tables is large, the time for performing one full-data consistency verification is long. Then, according to the above consistency verification scheme, after the verification is aborted and resumed, the completed data table needs to be repeatedly verified, which not only occupies more resources, but also needs to consume a lot of time to affect the database synchronization efficiency.

Disclosure of Invention

In order to solve the technical problems described above or at least partially solve the technical problems described above, the present disclosure provides a method, an apparatus, a device, and a storage medium for checking consistency of data synchronization.

In a first aspect, the present disclosure provides a method for checking consistency of data synchronization, including:

acquiring data information of each table to be verified, generating a plurality of verification tasks based on the data information of each table, and recording the task state of each verification task as an unexecuted state;

for any check task, if the task state of the check task is the unexecuted state, modifying the task state of the check task to be an executing state, processing the check task, and modifying the task state of the check task to be an executing completion state under the condition that the check task is checked normally;

if the checking abnormality is detected, continuously checking each checking task which is not checked based on the task state of each checking task.

In some embodiments, one of the verification tasks includes one of the table data information.

In some embodiments, if the verification exception is detected, continuing to verify each of the verification tasks that is not performed based on a task state of each of the verification tasks includes:

For any check task, if the task state of the check task is modified to be the executing state, and then the check task is detected to be not normally checked, the task state of the check task is modified from the executing state to the non-executing state;

and returning to execute any check task, if the task state of the check task is the unexecuted state, modifying the task state of the check task into the executing state, processing the check task, and modifying the task state of the check task into the executing completion state under the condition that the check task is checked normally.

if a restarting signal after verification interrupt is detected, searching for a task state as a verification task in the executing state, and modifying the task state of the searched verification task from the executing state to the non-executing state;

In some embodiments, after modifying the task state of the verification task to the in-process state if the task state of the verification task is the unexecuted state, the method further comprises:

removing the verification task from the task pool;

accordingly, after modifying the task state of the verification task from the executing state to the non-executing state, the method further includes:

and re-adding the verification task into a task pool.

In some embodiments, after the detecting of the verification exception, the method further comprises:

recording abnormal information of verification abnormality; and/or the number of the groups of groups,

and clearing the checking intermediate result corresponding to the checking task.

In some embodiments, the method further comprises:

after the table data information to be verified is obtained, determining each table data information as a verification total task, and recording the task state of the verification total task as the unexecuted state; wherein, the checking total task and each checking task have an association relation;

before modifying the task state of the checking task into the executing state if the task state of the checking task is the non-executing state for any checking task, modifying the task state of the checking total task from the non-executing state into the executing state;

And when the task state of each checking task is the execution completion state, modifying the task state of the checking total task from the execution state to the execution completion state.

In a second aspect, the present disclosure provides a consistency check device for data synchronization, the device comprising:

the verification task generation module is used for acquiring data information of each table to be verified, generating a plurality of verification tasks based on the data information of each table, and recording the task state of each verification task as an unexecuted state;

the task state modifying module is used for modifying the task state of the checking task into an executing state if the task state of the checking task is the unexecuted state aiming at any checking task, processing the checking task, and modifying the task state of the checking task into an executing completion state under the condition that the checking task is checked normally;

and the verification exception handling module is used for continuing to verify each verification task which is not verified based on the task state of each verification task if the verification exception is detected.

In some embodiments, the verification exception handling module is specifically configured to:

In some embodiments, the apparatus further comprises a task management module to:

after modifying the task state of the verification task into an executing state if the task state of the verification task is the unexecuted state, removing the verification task from a task pool;

and after the task state of the check task is changed from the executing state to the non-executing state, the check task is added into a task pool again.

In some embodiments, the check exception handling module is further to:

after detecting the checking abnormality, recording abnormality information of the checking abnormality; and/or the number of the groups of groups,

and after detecting the verification abnormality, clearing a verification intermediate result corresponding to the verification task.

In some embodiments, the apparatus further comprises a check total task processing module for:

In a third aspect, the present disclosure provides an electronic device comprising:

a processor and a storage device;

the processor is configured to perform the steps of the method described in any embodiment of the disclosure by invoking a program or instructions stored in the storage device.

In a fourth aspect, the present disclosure provides a computer-readable storage medium storing a program or instructions that cause a computer to perform the steps of the method described in any of the embodiments of the present disclosure.

According to the technical scheme provided by the embodiment of the disclosure, each table data information to be verified is obtained, a plurality of verification tasks are generated based on each table data information, and the task state of each verification task is recorded as an unexecuted state; for any check task, if the task state of the check task is a non-executing state, modifying the task state of the check task to be an executing state, processing the check task, and modifying the task state of the check task to be an executing completion state under the condition that the check task is normally checked; if the checking abnormality is detected, the checking tasks which are not checked are continuously checked based on the task states of the checking tasks. The method and the device have the advantages that the primary data consistency check process is divided into a plurality of check tasks with smaller execution granularity, the task state of each check task is dynamically modified in the execution process of the check task, so that after the check process is interrupted and resumed, check of the uncore table data can be continuously executed based on the task state of each check task, the problem that all table data check needs to be carried out again when the data consistency check is interrupted is solved, the repeated check process of some table data is reduced, therefore, the operation resources required by the consistency check are reduced, the check time is saved, and the efficiency of the data consistency check is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.

FIG. 1 is a flow chart of a method for consistency verification of data synchronization provided by an embodiment of the present disclosure;

FIG. 2 is a flow chart of another method for consistency verification of data synchronization provided by an embodiment of the present disclosure;

FIG. 3 is a flow chart of yet another method for consistency verification of data synchronization provided by an embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of a data synchronization consistency check device according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further detailed description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.

The consistency verification method for data synchronization provided by the embodiment of the disclosure is mainly suitable for a scene of verifying consistency between data at two ends when data synchronization is performed between different devices, such as a data consistency verification process in a database synchronization scene. The data synchronization consistency check method provided by the embodiment of the disclosure may be performed by a data synchronization consistency check device, which may be implemented in software and/or hardware, and the device may be integrated in an electronic device having a table data operation function, for example, a tablet computer, a notebook computer, a desktop computer, a server, or the like. In some embodiments, the electronic device may be a device corresponding to a source end of the production data, a device corresponding to a target end of the synchronization data, or a third party device different from the source end corresponding device and the target end corresponding device.

Fig. 1 is a flowchart of a data synchronization consistency check method according to an embodiment of the present disclosure. Referring to fig. 1, the method specifically includes:

s110, acquiring data information of each table to be verified, generating a plurality of verification tasks based on the data information of each table, and recording the task state of each verification task as an unexecuted state.

The table data information is information for representing data to be verified, and may be table data pulled from a source end and a target end, or may be descriptive information of the table data, so as to request corresponding table data from the source end and the target end. The verification tasks are units of execution of data consistency checks, each verification operation corresponding to execution of one verification task. The check task contains the table data information but its number is less than the total number of table data information. In some embodiments, a verification task includes a table of data information. Although one check task may contain a plurality of table data information, in order to improve the efficiency of data consistency check to a greater extent, one table data information is determined as one check task in the present embodiment. The task state is used for representing the execution progress of a task, and may be, for example, an unexecuted state representing that the task is not executed, such as "unexecuted", "to be executed", etc., an executing state representing that the task is being executed, such as "executing", "checking", etc., or an executing completion state representing that the task has been normally executed and completed, such as "executing completion", "checking completion", etc.

In view of the need to re-check all table data after each failure recovery in the related art, in the embodiment of the present disclosure, a process of checking data consistency is divided into a plurality of check tasks with smaller execution granularity, and a task state is dynamically set for each check task according to an execution progress of the check task. Therefore, when the checking process is interrupted due to faults, one checking task which is being checked is interrupted, and after the follow-up faults are recovered, checking tasks which are not checked and completed can be screened according to the task states of the checking tasks to continue checking, and table data which is checked and completed does not need to be checked repeatedly.

Based on the above conception, when in implementation, all table data information needing to be subjected to consistency check is acquired first. Then, all the table data information is split according to the preset task quantity (such as one table data information) corresponding to one check task, so as to generate a plurality of check tasks. And, an initial task state, i.e., a non-executed state, is set for each check task. In some embodiments, the check total tasks corresponding to all table data information and the task information (such as task descriptive information, task states, etc.) of each check task generated above may be persisted for subsequent easier querying and modification.

S120, aiming at any check task, if the task state of the check task is a non-executing state, modifying the task state of the check task into an executing state, processing the check task, and modifying the task state of the check task into an executing completion state under the condition that the check task is normally checked.

For one of the check tasks, if the task state is queried to be the unexecuted state, processing of the check task can be started. Firstly, modifying the task state of the check task from an unexecuted state to an executing state; then, executing the verification task, such as comparing whether the table data corresponding to the source end and the target end in the verification task are consistent; whether the comparison result is consistent or inconsistent, the verification task is normally verified. At this time, the task state of the checking task is changed from the executing state to the executing completion state, which indicates that the checking task has been checked. According to this process, each verification task is traversed. The verification tasks may be executed in a cyclic manner in series or in parallel.

And S130, if the verification abnormality is detected, continuously verifying each verification task which is not verified based on the task state of each verification task.

If a check abnormality is detected in the process of processing each check task, for example, the check task caused by network abnormality, database connection failure, etc. cannot be continuously executed, or the consistency check program of data synchronization caused by operating system failure, hardware failure, program memory overflow, etc. is restarted after being interrupted, etc., after the failure is recovered, the check tasks in the executing state and the non-executing state are continuously processed based on the task state of each check task until all the check tasks are processed.

According to the technical scheme, the table data information to be verified is obtained, a plurality of verification tasks are generated based on the table data information, and the task state of each verification task is recorded as an unexecuted state; for any check task, if the task state of the check task is a non-executing state, modifying the task state of the check task to be an executing state, processing the check task, and modifying the task state of the check task to be an executing completion state under the condition that the check task is normally checked; if the checking abnormality is detected, the checking tasks which are not checked are continuously checked based on the task states of the checking tasks. The method and the device have the advantages that the primary data consistency check process is divided into a plurality of check tasks with smaller execution granularity, the task state of each check task is dynamically modified in the execution process of the check task, so that after the check process is interrupted and resumed, check of the uncore table data can be continuously executed based on the task state of each check task, the problem that all table data check needs to be carried out again when the data consistency check is interrupted is solved, the repeated check process of some table data is reduced, therefore, the operation resources required by the consistency check are reduced, the check time is saved, and the efficiency of the data consistency check is improved.

In some embodiments, after detecting the verification exception, the method further comprises: recording abnormal information of verification abnormality; and/or clearing the checking intermediate result corresponding to the checking task. In this embodiment, if a verification abnormality is detected, it is necessary to record abnormality information of the verification abnormality in a log. In addition, when detecting the verification abnormality, the verification task currently being processed cannot complete the normal verification, and needs to be re-verified after the fault is recovered, so various verification intermediate results which are already generated by the verification task need to be removed.

In some embodiments, the method for checking consistency of data synchronization further comprises: after obtaining the data information of each table to be checked, determining the data information of each table as a checking total task, and recording the task state of the checking total task as an unexecuted state; wherein, the verification total task and each verification task have an association relation; if the task state of the checking task is the unexecuted state aiming at any checking task, modifying the task state of the checking task into the executing state before modifying the task state of the checking task into the executing state, and modifying the task state of the checking total task from the unexecuted state into the executing state; when the task state of each checking task is the execution completion state, the task state of the checking total task is changed from the execution state to the execution completion state. The present embodiment is directed to a scenario in which a plurality of data consistency check processes exist at the same time. In this scenario, in order to ensure smooth execution of each verification process, all table data information included in each data consistency verification process is abstracted into one verification total task, and then the verification total task and each generated verification task have an association relationship. Before the checking of the total task is not performed, an initial task state, i.e. a non-performed state, is set for it. When starting to traverse each verification task, the task state of the verification total task needs to be modified from the unexecuted state to the executing state, which indicates that the verification total task is being processed. After all the verification tasks are processed, the task state of the verification total task needs to be changed from the executing state to the executing completion state, which indicates that all the verification total tasks are processed. Therefore, through dynamic modification of the task states of the verification total task and the verification tasks related to the verification total task, the verification total task and the verification tasks which need to be continuously processed can be accurately positioned after the verification process is interrupted and recovered, and the efficiency of data consistency verification is further improved.

Fig. 2 is a flowchart of another method for checking consistency of data synchronization according to an embodiment of the present disclosure. It further optimizes "if a verification abnormality is detected, based on the task state of each verification task, each verification task that is not executed is continuously verified". On this basis, the relevant steps of checking task management in the task pool can be added. The consistency check method of the data synchronization aims at a consistency check interruption scene caused by faults which do not relate to a data consistency check program, such as network abnormality, database abnormality and the like. Wherein the explanation of the same or corresponding terms as those of the above embodiments is not repeated herein. Referring to fig. 2, the method includes:

s210, acquiring data information of each table to be checked, determining the data information of each table as a checking total task, and recording the task state of the checking total task as an unexecuted state.

S220, generating a plurality of verification tasks based on the data information of each table, and recording the task state of each verification task as an unexecuted state.

S210 and S220 complete the initialization process of the verification total task and each verification task corresponding to the data consistency verification process, and obtain the task identification, the task type identification and the task state of the verification total task and each verification task shown in the table 1. The verification total task (ID 1) includes three verification tasks (ID 2, 3, 4 respectively) and is in an unexecuted state.

TABLE 1 task information Table (initial State)

Task Identification (ID)	Task type identification (PID)	Task state (STATUS)
			1	0 (check total task)	Not being executed
2	1 (check task)	Not being executed
			3	1	Not being executed
4	1	Not being executed

S230, modifying the task state of the checking total task from the unexecuted state to the executing state.

And determining a checking total task (PID is 0) according to the task type identifier, and modifying the task state into an executing state.

S240, aiming at any check task, if the task state of the check task is a non-executing state, modifying the task state of the check task into an executing state, removing the check task from the task pool, processing the check task, and modifying the task state of the check task into an executing completion state under the condition that the check task is normally checked.

For a certain verification task, such as a verification task with an ID of 2, the task state is modified to be the executing state. The verification task is removed from the task pool because the verification task has begun to be processed. The task pool stores unprocessed verification tasks. Then, the check task is processed, and the task state of the check task with ID 2 in table 1 is modified to the execution completion state after the processing is completed.

S250, aiming at any check task, if the task state of the check task is modified to be the executing state and then the check task is detected to be not checked normally, the task state of the check task is modified from the executing state to the non-executing state.

For a certain verification task, for example, a verification task with an ID of 2, after the task state is modified to be the executing state, if the verification task is detected to be not normally verified, the task state is re-modified to be the non-executing state.

S260, recording abnormal information of the verification abnormality, clearing the verification intermediate result corresponding to the verification task, and adding the verification task into the task pool again.

The above-mentioned abnormal checking task needs to be checked again after the fault is recovered, so after the task state is modified to the non-executing state, it also needs to be added again into the task pool.

S270, returning to S240.

If there are more check tasks in the task pool, S240-S270 are circularly executed to ensure that all check tasks are processed.

According to the technical scheme, through removal and addition of the verification tasks in the task pool, each verification task can be efficiently managed, and the integrity of data consistency verification is further ensured. By aiming at any check task, if the task state of the check task is modified to be the executing state, the check task is detected to be not normally checked, the task state of the check task is modified from the executing state to the non-executing state, and the check task is returned to be the executing state aiming at any check task, if the task state of the check task is not executed, the task state of the check task is modified to be the executing state, the check task is processed, and under the condition that the check task is normally checked, the task state of the check task is modified to be the executing completion state. When the data consistency check program can normally run but the data consistency check is interrupted, the task state of the check task with abnormal check is automatically modified into an unexecuted state, so that the check task can be continuously checked, the completed check task is not required to be repeatedly checked, and the fault tolerance and the check efficiency of the data consistency check are improved.

Fig. 3 is a flowchart of yet another consistency check method for data synchronization provided by an embodiment of the present disclosure. It further optimizes "if a verification abnormality is detected, based on the task state of each verification task, each verification task that is not executed is continuously verified". On this basis, the relevant steps of checking task management in the task pool can be added. The data synchronization consistency check method aims at a consistency check interrupt scene caused by faults of an operating system, hardware faults, program memory overflow and the like, which cause that a data consistency check program cannot run. Wherein the explanation of the same or corresponding terms as those of the above embodiments is not repeated herein. Referring to fig. 3, the method includes:

s310, acquiring data information of each table to be checked, determining the data information of each table as a checking total task, and recording the task state of the checking total task as an unexecuted state.

S320, generating a plurality of verification tasks based on the data information of each table, and recording the task state of each verification task as an unexecuted state.

S330, modifying the task state of the checking total task from the unexecuted state to the executing state.

S340, aiming at any check task, if the task state of the check task is a non-executing state, modifying the task state of the check task into an executing state, removing the check task from the task pool, processing the check task, and modifying the task state of the check task into an executing completion state under the condition that the check task is normally checked.

S350, if the restarting signal after the verification interrupt is detected, searching the task state as a verification task in the executing state, and modifying the task state of the searched verification task from the executing state to an unexecuted state.

If the data consistency check program is restarted due to a failure during one process of checking the total task, the task information in the intermediate state, for example, table 2, can be obtained through the processes of S310 to S340 described above. As can be seen from the task state of the checking total task in table 2, the checking total task is not processed, wherein the checking total task includes a checking task in an executing state (ID is 3), which indicates that the checking task is being processed before restarting, and includes a checking task in an unexecuted state (ID is 4).

TABLE 2 task information Table (intermediate State)

Task Identification (ID)	Task type identification (PID)	Task state (STATUS)
			1	0 (check total task)	In execution
2	1 (check task)	Completion of
			3	1	In execution
4	1	Not being executed

In the specific implementation, after detecting the restart signal, the checking total task (ID is 1) of the executing state is searched from the task information table. Then, a check task (ID 3) associated with the check total task and having a task state of being executed is searched. And then, modifying the task state of the searched check task from the executing state to the non-executing state.

S360, recording abnormal information of the verification abnormality, clearing the verification intermediate result corresponding to the searched verification task, and re-adding the verification task into the task pool.

S370, returning to S340.

According to the technical scheme, if the restarting signal after verification is interrupted is detected, the task state is searched for as the verification task in the executing state, and the task state of the searched verification task is modified from the executing state to the non-executing state; and returning to execute any check task, if the task state of the check task is a non-executed state, modifying the task state of the check task to be an executing state, processing the check task, and modifying the task state of the check task to be an executing completion state under the condition that the check task is normally checked. Under the condition that the data consistency check program cannot normally run due to faults, the data consistency check program is interrupted, automatically searching the check task in the executing state and modifying the task state after restarting the program, and automatically searching the check task in the non-executing state and processing the check task, so that the check task which is not completely checked can be accurately and comprehensively obtained, the data consistency check can be continuously carried out, the completed check task is not required to be repeatedly checked, and the efficiency of the data consistency check is further improved.

Fig. 4 is a schematic structural diagram of a data synchronization consistency check device according to an embodiment of the present disclosure. Referring to fig. 4, the apparatus specifically includes:

the verification task generating module 410 is configured to obtain data information of each table to be verified, generate a plurality of verification tasks based on the data information of each table, and record a task state of each verification task as an unexecuted state;

the task state modifying module 420 is configured to modify, for any check task, if the task state of the check task is an unexecuted state, the task state of the check task into an executing state, process the check task, and modify, under a condition that the check task is normally checked, the task state of the check task into an executing completion state;

and the checking exception handling module 430 is configured to, if checking exception is detected, continue checking each checking task that is not checked based on the task state of each checking task.

In some embodiments, a verification task includes a table of data information.

In some embodiments, the check exception handling module 430 is specifically configured to:

for any check task, if the task state of the check task is modified to be the executing state, and the check task is detected to be not checked normally, the task state of the check task is modified from the executing state to the non-executing state;

And returning to execute any check task, if the task state of the check task is a non-executed state, modifying the task state of the check task to be an executing state, processing the check task, and modifying the task state of the check task to be an executing completion state under the condition that the check task is normally checked.

if the restarting signal after the verification interrupt is detected, searching the task state as a verification task in the executing state, and modifying the task state of the searched verification task from the executing state to an unexecuted state;

if the task state of the check task is the unexecuted state, modifying the task state of the check task to the executing state, and then removing the check task from the task pool;

After the task state of the verification task is modified from the executing state to the non-executing state, the verification task is rejoined into the task pool.

In some embodiments, the check exception handling module 430 is further configured to:

after detecting the verification abnormality, recording abnormality information of the verification abnormality; and/or the number of the groups of groups,

and after detecting the verification abnormality, clearing the verification intermediate result corresponding to the verification task.

after obtaining the data information of each table to be checked, determining the data information of each table as a checking total task, and recording the task state of the checking total task as an unexecuted state; wherein, the verification total task and each verification task have an association relation;

if the task state of the checking task is the unexecuted state aiming at any checking task, modifying the task state of the checking task into the executing state before modifying the task state of the checking task into the executing state, and modifying the task state of the checking total task from the unexecuted state into the executing state;

when the task state of each checking task is the execution completion state, the task state of the checking total task is changed from the execution state to the execution completion state.

According to the data synchronization consistency verification device, the primary data consistency verification process is divided into the plurality of verification tasks with smaller execution granularity, the task state of each verification task is dynamically modified in the execution process of the verification task, so that after the verification process is interrupted and resumed, verification of the uncore table data can be continuously executed based on the task state of each verification task, the problem that all table data verification needs to be carried out again when the data consistency verification is interrupted is solved, the repeated verification process of some table data is reduced, therefore, operation resources required by the consistency verification are reduced, verification time is saved, and the efficiency of the data consistency verification is improved.

The consistency verification device for data synchronization provided by the embodiment of the disclosure can execute the consistency verification method for data synchronization provided by any embodiment of the disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

It should be noted that, in the embodiment of the data synchronization consistency check device, each included module is only divided according to the functional logic, but not limited to the above division, so long as the corresponding function can be implemented; in addition, the specific names of the functional modules are also only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present disclosure.

Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. Referring to fig. 5, an electronic device 500 provided in an embodiment of the present disclosure includes: a processor 520 and a memory 510; processor 520 is operable to perform the steps of the data synchronization consistency check method provided by the embodiments of the present disclosure by invoking a program or instruction stored in memory 510:

acquiring data information of each table to be checked, generating a plurality of check tasks based on the data information of each table, and recording the task state of each check task as an unexecuted state;

For any check task, if the task state of the check task is a non-executing state, modifying the task state of the check task to be an executing state, processing the check task, and modifying the task state of the check task to be an executing completion state under the condition that the check task is normally checked;

if the checking abnormality is detected, the checking tasks which are not checked are continuously checked based on the task states of the checking tasks.

Of course, those skilled in the art will appreciate that the processor 520 may also implement the technical solution of the method for checking consistency of data synchronization provided in any embodiment of the present disclosure.

The electronic device 500 shown in fig. 5 is merely an example and should not be construed to limit the functionality and scope of use of embodiments of the present disclosure in any way.

As shown in fig. 5, the electronic device 500 is embodied in the form of a general purpose computing device. The components of electronic device 500 may include, but are not limited to: one or more processors 520, a memory 510, and a bus 550 that connects the different system components (including the memory 510 and the processor 520).

Bus 550 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Electronic device 500 typically includes many types of computer system readable media. Such media can be any available media that is accessible by electronic device 500 and includes both volatile and nonvolatile media, removable and non-removable media.

Memory 510 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 511 and/or cache memory 512. Electronic device 500 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 513 may be used to read from or write to non-removable, non-volatile magnetic media (not shown in FIG. 5, commonly referred to as a "hard disk drive"). Although not shown in fig. 5, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 550 via one or more data medium interfaces. Memory 510 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of the various embodiments of the disclosure.

A program/utility 514 having a set (at least one) of program modules 515 may be stored, for example, in memory 510, such program modules 515 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program module 515 generally performs the functions and/or methods described in any of the embodiments of the present disclosure.

The electronic device 500 may also communicate with one or more external devices 560 (e.g., keyboard, pointing device, display 570, etc.), one or more devices that enable a user to interact with the electronic device 500, and/or any devices (e.g., network card, modem, etc.) that enable the electronic device 500 to communicate with one or more other computing devices. Such communication may occur through an input/output interface (I/O interface) 530. Also, the electronic device 500 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through the network adapter 540. As shown in fig. 5, the network adapter 540 communicates with other modules of the electronic device 500 via a bus 550. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 500, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.

The present disclosure also provides a computer-readable storage medium storing a program or instructions that cause a computer to perform the steps of the consistency check method of data synchronization provided by the embodiments of the present disclosure:

Of course, the computer readable storage medium provided by the embodiments of the present disclosure, the program or the instructions stored in the computer readable storage medium are not limited to the above method operations, but may also perform the related operations in the data synchronization consistency check method provided by any embodiment of the present disclosure.

The computer storage media of the embodiments of the present disclosure may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

It is noted that the terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of the present application. As used in the specification and the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. The term "and/or" includes any and all combinations of one or more of the associated listed items. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method or apparatus comprising such elements.

The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for checking consistency of data synchronization, comprising:

executing each check task in parallel, and modifying the task state of the check task into an executing state if the task state of any check task is the unexecuted state, and processing the check task, and modifying the task state of the check task into an executing completion state under the condition that the check task is normally checked;

if detecting the verification abnormality, continuing to verify each verification task which is not verified based on the task state of each verification task, including:

if detecting that the verification is abnormal, searching a verification task with the task state being the executing state, and clearing a verification intermediate result corresponding to the verification task;

modifying the task state of the searched verification task from the executing state to the non-executing state;

2. The method of claim 1, wherein one of said verification tasks includes one of said table data information.

3. The method of claim 1, wherein detecting a verification exception comprises:

and detecting that the check task is not checked normally.

4. The method of claim 1, wherein detecting a verification exception comprises:

a restart signal after a verification interrupt is detected.

5. The method according to claim 3 or 4, wherein after modifying the task state of the verification task to an in-process state if the task state of the verification task is the unexecuted state, the method further comprises:

removing the verification task from the task pool;

after modifying the task state of the verification task from the executing state to the unexecuted state, the method further comprises:

and re-adding the verification task into a task pool.

6. The method of claim 1, wherein after the detecting of the verification exception, the method further comprises:

and recording the abnormal information of the verification abnormality.

7. The method according to claim 1, wherein the method further comprises:

8. A consistency check device for data synchronization, comprising:

the task state modifying module is used for executing each checking task in parallel, and for any checking task, if the task state of the checking task is the unexecuted state, modifying the task state of the checking task into an executing state, processing the checking task, and under the condition that the checking task is checked normally, modifying the task state of the checking task into an executing completion state;

The verification exception handling module is used for continuing to verify each verification task which is not verified based on the task state of each verification task if the verification exception is detected;

the verification exception handling module is specifically configured to:

9. An electronic device, the electronic device comprising:

a processor and a memory;

the processor is adapted to perform the steps of the method according to any of claims 1 to 7 by invoking a program or instruction stored in the memory.

10. A computer readable storage medium storing a program or instructions for causing a computer to perform the steps of the method according to any one of claims 1 to 7.