CN110427422A - Data consistency verification method, equipment and storage medium when data synchronous abnormality - Google Patents

Data consistency verification method, equipment and storage medium when data synchronous abnormality Download PDF

Info

Publication number
CN110427422A
CN110427422A CN201910436428.3A CN201910436428A CN110427422A CN 110427422 A CN110427422 A CN 110427422A CN 201910436428 A CN201910436428 A CN 201910436428A CN 110427422 A CN110427422 A CN 110427422A
Authority
CN
China
Prior art keywords
affairs
error
data
supplementary table
transaction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910436428.3A
Other languages
Chinese (zh)
Other versions
CN110427422B (en
Inventor
付铨
孙峰
余院兰
赵家威
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Dream Database Co ltd
Original Assignee
Wuhan Dameng Database Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Dameng Database Co Ltd filed Critical Wuhan Dameng Database Co Ltd
Priority to CN201910436428.3A priority Critical patent/CN110427422B/en
Publication of CN110427422A publication Critical patent/CN110427422A/en
Application granted granted Critical
Publication of CN110427422B publication Critical patent/CN110427422B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2365Ensuring data consistency and integrity
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Hardware Redundancy (AREA)

Abstract

The invention discloses data consistency verification method, equipment and storage mediums when a kind of data synchronous abnormality, which comprises the affairs ID of synchronized transaction is inserted into target client database in supplementary table first;Then when receive report an error notify when, judge current synchronized transaction be in which kind of operation;When for affairs change operation, judgement, which reports an error, notifies whether belong to preset type of error, and rollback has carried out when belonging to simultaneously operating and in the affairs ID insertion supplementary table of the synchronized transaction and transactional synchronization operation will be re-started again, when for transaction commit operation, it inquires in supplementary table and whether includes the affairs ID of the synchronized transaction, and operation will be synchronized in the affairs ID insertion supplementary table of the affairs and again when not including.The present invention first registers transaction tag in synchronizing process in supplementary table when synchronized transaction starts, if malfunctioned in synchronizing process, by being handled according to current operation to do different judgements, to guarantee the consistency of synchrodata.

Description

Data consistency verification method, equipment and storage medium when data synchronous abnormality
Technical field
The present invention relates to database synchronization technical fields, and in particular to consistency verification of data when a kind of data synchronous abnormality Method, equipment and storage medium.
Background technique
In database data real-time synchronization technique, wherein based on database journal analysis real time data synchronization technology because It has support heterogeneous operating system and database platform, it is invasive to source Database Systems it is smaller, data synchronizing relay is low The features such as, and be widely studied and apply.Journal format and log content between heterogeneous database system differ greatly, because The log content of this data synchronisation source client database can not be applied directly in target database.Data based on log analysis Real-time synchronization technique, General Principle are to capture the Incremental Log content of source database, are then sent to target side by network, The transaction operation SQL statement of the reverse generation technique recovery resource client database of SQL is used in target side, then passes through general database The interfaces such as access interface such as ODBC, OCI execute the SQL statement after restoring, and realize the real time data synchronization at source and target end.
Further, the database data real-time synchronization device constructed by using above-mentioned real time data synchronization technology is main Including following comprising modules, in synchronization system source, synchronizing software log read module reads source database journal text in real time Part captures the affairs Incremental Log content on source database;In synchronizing software log analysis module real time parsing Incremental Log Hold, extracts the affairs ID in log, operates table id, the action type of affairs such as INSERT, UPDATE, DELETE, transaction operation Timestamp, log LSN value, operation data etc., then assemble them into the message of specific format, are stored in transmit queue;Together Software log sending module is walked, sends the message in transmit queue to by TCP/IP network the target side of synchronization system.
In synchronization system target side, synchronizing software log receiving module receives the message that source is sent, and parses in message Hold, the operation for belonging to an affairs is cached according to affairs ID, when receiving the COMMIT message of affairs, is then transferred to Log execution module synchronizes execution;Synchronizing software log execution module constructs SQL statement, according to transaction operation content Target database is synchronous to be executed.
In above-mentioned database data real-time synchronization technique, target side data synchronization service is in the enterprising line number of target client database When according to storage, there are Network Abnormals between target side synchronous service and target client database or target client database to break down and work as The case where machine, is likely to will cause repeating for synchronized transaction, influences the consistency of synchrodata.
Thus the prior art needs to improve.
Summary of the invention
Place in view of above-mentioned deficiencies of the prior art, the purpose of the present invention is to provide data when a kind of data synchronous abnormality Consistency desired result method, equipment and storage medium, when solving to occur abnormal in current data synchronization process caused by it is synchronous The inconsistent problem of data.
In order to achieve the above object, this invention takes following technical schemes:
Data consistency verification method when a kind of data synchronous abnormality, includes the following steps:
The log information packet that source is sent is received, and the affairs ID of the synchronized transaction in log information packet is inserted into target side Laggard behaviour business simultaneously operating in the supplementary table created in database;
When receive target client database report an error notify when, judge current synchronized transaction be in which kind of operation, wherein The operation includes the affairs change operation and transaction commit operation successively executed;
When current operation is affairs change operation, judgement, which reports an error, notifies whether belong to preset type of error, and is reporting Mistake notifies the simultaneously operating that rollback has carried out when belonging to preset type of error and is again inserted into the affairs ID of the synchronized transaction In supplementary table and re-start transactional synchronization operation;
When current operation is transaction commit operation, target client database is reconnected, and inquire in supplementary table and whether wrap Affairs ID containing the synchronized transaction, and the affairs ID of the synchronized transaction is inserted into supplementary table when not including and re-starts thing Business simultaneously operating.
Preferably, when the data synchronous abnormality in data consistency verification method, the day for receiving source and sending Will message package, and the affairs ID of the synchronized transaction in log information packet is inserted into the supplementary table created in target client database Laggard behaviour is engaged in the step of simultaneously operating
Parsing current sync office log information packet and obtain the affairs ID of the synchronized transaction in log information packet;
Insertion operation is executed in the supplementary table created in target client database so that the affairs ID is inserted into supplementary table In.
Preferably, when the data synchronous abnormality in data consistency verification method, it is described current operation be affairs When change operation, judgement, which reports an error, notifies whether belong to the preset type that reports an error, and belongs to preset type of error in the notice that reports an error When the simultaneously operating that has carried out of rollback and again will be in the affairs ID insertion supplementary table of the synchronized transaction and to re-start affairs same Walking the step of operating includes:
When current operation is affairs change operation, according to the class for the error code identification mistake that target client database returns Type;
Judge whether the type of error belongs to preset type of error;
When the type of error is preset type of error, disconnect what connection and rollback with target client database had carried out Simultaneously operating, and again will be in the affairs ID of synchronized transaction insertion supplementary table and to re-start affairs same after the completion of rollback Step operation.
Preferably, described in the type of error when data synchronous abnormality in data consistency verification method When for preset type of error, the simultaneously operating that connection and rollback with target client database have carried out is disconnected, and complete in rollback It again will be in the affairs ID insertion supplementary table of the synchronized transaction and the step of re-starting transactional synchronization operation after further include:
When the type of error is not preset type of error, preset synchronized transaction error handling processing process is executed.
Preferably, when the data synchronous abnormality in data consistency verification method, the preset type of error packet Include Network Abnormal mistake and database system-level errors.
Preferably, when the data synchronous abnormality in data consistency verification method, it is described current operation be affairs When submitting operation, reconnect target client database, and inquire in supplementary table whether include the synchronized transaction affairs ID, and Not comprising when will the synchronized transaction affairs ID insertion supplementary table in and re-start transactional synchronization operation the step of include:
When current operation is transaction commit operation, the connection with target client database and again linking objective end number are disconnected According to library;
Supplementary table in invocation target client database, and inquire in the supplementary table and start with the presence or absence of the synchronized transaction The affairs ID of Shi Dengji;
When affairs ID being not present in the supplementary table, synchronized transaction ID is inserted into the auxiliary in target client database In table and restart to carry out transactional synchronization operation.
Preferably, when the data synchronous abnormality in data consistency verification method, it is described in the supplementary table not There are when affairs ID, in supplementary table that synchronized transaction ID is inserted into target client database and will restart to carry out affairs same The step of step operation further include:
There are when affairs ID in the supplementary table, next synchronized transaction is executed.
Preferably, when the data synchronous abnormality in data consistency verification method, the affairs change operation includes Insertion operation updates operation and delete operation.
Consistency verification of data equipment when a kind of data synchronous abnormality comprising: processor, memory and communication bus;
The computer-readable program that can be executed by the processor is stored on the memory;
The communication bus realizes the connection communication between processor and memory;
Data one when the processor realizes data synchronous abnormality as described above when executing the computer-readable program Step in cause property method of calibration.
A kind of computer readable storage medium, the computer-readable recording medium storage have one or more program, One or more of programs can be executed by one or more processor, when realizing data synchronous abnormality as described above Step in data consistency verification method.
When data synchronous abnormality provided by the invention in data consistency verification method, equipment and storage medium, the side Method includes: the log information packet for receiving source first and sending, and the affairs ID of the synchronized transaction in log information packet is inserted into mesh Laggard behaviour business simultaneously operating in the supplementary table created in mark client database;Then logical when receiving reporting an error for target client database When knowing, judge which kind of operation is current synchronized transaction be in, wherein the operation include the affairs change operation that successively executes and Transaction commit operation;When current operation is affairs change operation, judgement, which reports an error, notifies whether belong to preset type of error, and The simultaneously operating that has carried out of rollback and again by the affairs ID of the synchronized transaction when the notice that reports an error belongs to preset type of error In insertion supplementary table and transactional synchronization operation is re-started, when current operation is transaction commit operation, reconnect target side Database, and inquire in supplementary table whether include the synchronized transaction affairs ID, and when not including by the thing of the synchronized transaction In business ID insertion supplementary table and re-start transactional synchronization operation.The present invention first exists when synchronized transaction starts in synchronizing process Transaction tag is registered in supplementary table, if malfunctioned in synchronizing process, by judging that transaction tag is according to current operation It is no registered, to judge whether synchronously completed, further to guarantee the consistency of synchrodata.
Detailed description of the invention
The flow chart of data consistency verification method when Fig. 1 is data synchronous abnormality provided by the invention.
When Fig. 2 is data synchronous abnormality provided by the invention in data consistency verification method, the stream of the step S100 Cheng Tu.
When Fig. 3 is data synchronous abnormality provided by the invention in data consistency verification method, the stream of the step S300 Cheng Tu.
When Fig. 4 is data synchronous abnormality provided by the invention in data consistency verification method, the stream of the step S400 Cheng Tu.
The running environment signal of the preferred embodiment of consistency verification of data program when Fig. 5 is data synchronous abnormality of the present invention Figure.
The function of the system preferred embodiment of consistency verification of data program when Fig. 6 is installation data synchronous abnormality of the present invention Module map.
Specific embodiment
In view of synchrodata inconsistent problem is easy to appear when being abnormal in data synchronization process in the prior art, originally Data consistency verification method, equipment and the storage medium when being designed to provide a kind of data synchronous abnormality of invention, can guarantee Occur being not in synchrodata inconsistent problem when exception in data synchronization process.
To make the purpose of the present invention, technical solution and effect clearer, clear and definite, right as follows in conjunction with drawings and embodiments The present invention is further described.It should be appreciated that described herein, specific examples are only used to explain the present invention, is not used to Limit the present invention.
Referring to Fig. 1, data consistency verification method when data synchronous abnormality provided by the invention, includes the following steps:
S100, the log information packet that source is sent is received, and the affairs ID of the synchronized transaction in log information packet is inserted into Laggard behaviour business simultaneously operating in the supplementary table created in target client database.
Specifically, source will need to carry out the synchronous table of data and send the past in the form of log information packet, therefore source The generation for also needing to carry out log information packet before carrying out the transmission of log information packet, due in heterogeneous database system environment In, Incremental Log information caused by source database can not directly apply to target client database, therefore need by source The log information of client database is parsed, and the message package of specific format inside synchronous service is then generated, and is stored in and is sent team Log sending module is waited to be sent in column, when it is implemented, the log trapping module of source data synchronization service is based on day Will capture technique captures the Incremental Log data of source Database Systems in real time, Incremental Log data is then passed to log Analysis module is parsed, later the log analyzing module of source data synchronization service, Incremental Log is resolved to internal specific Message packet format is stored in transmit queue.Preferably, the log information packet include affairs ID, transaction operation type and Operation table name, column information and data value etc..Target side is when progress data are synchronous, the synchronized transaction received according to affairs ID To classify.Later, the log sending module of source data synchronization service is based on thread synchronization mechanism, will be in internal transmit queue Log information packet be sent in target side data synchronization service system by TCP/IP network.Preferably, in order to guarantee data The reliability and data integrity of transmission, source log sending module take complete acknowledgement messaging mechanism, and log sends mould Block only just thinks that data are transmitted after obtaining target side confirmation message, otherwise will transmit data, target side again automatically After receiving log information packet, sequence number is carried out to each log information packet while needing to return to confirmation message.
Furthermore, it is understood that target side data synchronization service is interacted by the driving and target client database of database, Therefore synchronous service and database can not be on same machines.After arranged apart, it is necessary to consider the net between them The possibility that lattice break down;In addition, in synchronizing process, target client database is possible to because BUG occurs when machine or other Reason causes database to work as machine, and in these cases, synchronous service requires to consider when executing synchronous in target client database These possible failures prevent repeating or executing less for synchronized transaction, therefore the present invention is first in target client database Create a supplementary table, which is used to store the transaction tag of synchronized transaction, and in the present embodiment, the transaction tag is Affairs ID first registers the affairs ID of current sync affairs when synchronized transaction starts, then in synchronizing process in supplementary table When error, different judgement processing is done, according to current simultaneously operating to guarantee the consistency of synchrodata;By the thing It is engaged in after the completion of ID registration, that is, starts to synchronize affairs, a specific synchronized transaction is as follows:
BEGIN
INSERT INTO T VALUES(1);
COMMIT;
END;
The structure of one specific supplementary table is as shown in the table:
Field name Field type Explanation of field
TRXID NUMBER The affairs ID executed in registration data synchronizing process
Referring to Fig. 2, it is the flow chart of the step S100, include the following steps:
S101, parsing current sync office log information packet and obtain the thing of the synchronized transaction in log information packet Be engaged in ID;
Insertion operation is executed in S102, the supplementary table created in target client database to assist so that the affairs ID to be inserted into In table.
For above-mentioned concrete operations affairs, before executing synchronized transaction operation, one is first carried out on supplementary table INSERT operation, affairs ID is inserted into supplementary table, specific example is as follows:
INSERT INTO DMHS_TRXID(TRXID)VALUES(12345);
S200, when receive target client database report an error notify when, judge current synchronized transaction be in which kind of operation, Wherein the operation includes the affairs change operation and transaction commit operation successively executed.
In the present embodiment, be not received by target client database report an error notify when, executed the synchronization thing by process Business, then executes next synchronized transaction again, is receiving (occurring in synchronizing process when reporting an error and notifying for target client database Mistake), need to judge which kind of operation is current synchronized transaction be in, it is however generally that, when synchronizing affairs, need first to carry out Affairs change operation carries out data conversion, then executes transaction commit operation after converting and conveys the data after conversion Into target client database, therefore the present invention need to specifically give the processing method after error by judging action type, in difference Operational phase, processing method have difference, the affairs change operation stage, data conversion also do not complete, therefore need to come again into Row data conversion, in the transaction commit operation stage, data conversion is completed but may need to verify affairs without being put in storage successfully ID guarantees the consistency of synchrodata, it is preferred that the affairs change operation includes insertion operation (INSERT operation), updates (UPDATE operation) and delete operation (DELETE operation) are operated, data insertion, update or deletion can be carried out when synchronous.
Further, since there are many type of mistake, in order to distinguish to mistake, the present invention sets error code (error Code) type of mistake is indicated, in case of mistake in synchronizing process, will be judged by its internal original setting, The display mode reality mistake of error code is identified and passes through, system is identified by error code, is quickly found out the type of mistake.
S300, current operation be affairs change operation when, judgement report an error notice whether belong to preset type of error, and The simultaneously operating that has carried out of rollback and again by the affairs ID of the synchronized transaction when the notice that reports an error belongs to preset type of error In insertion supplementary table and re-start transactional synchronization operation.
In the present embodiment, the type of mistake is identified according to the notice that reports an error (being error code in the present embodiment) first, if Type of error belongs to preset type, then re-execute the steps S100, and the specific flow chart of the step S300 is as shown in Figure 3.
Referring to Fig. 3, it is the flow chart of the step S300, include the following steps:
S301, current operation be affairs change operation when, according to target client database return error code identify mistake Type;
S302, judge whether the type of error belongs to preset type of error;
S303, when the type of error is preset type of error, disconnect with the connection of target client database and rollback The simultaneously operating of progress, and the affairs ID of the synchronized transaction is inserted into supplementary table again after the completion of rollback and re-starts thing Business simultaneously operating.
In the present embodiment, preset type of error includes Network Abnormal mistake and database system-level errors, is being carried out It is presented in a manner of error code when reporting an error, if there is such mistake, indicates carrying out insertion operation, update operation or deleting behaviour When making, data, which synchronize, not to be completed correctly, at this time in order to avoid abnormal data imports target database, needs first to disconnect and mesh The connection for marking database, then synchronizes operation, the present invention realizes the weight of simultaneously operating by the way of rollback again New synchronous, rollback (Rollback) refers to program or data processing mistake, and program or data are restored to last correct shape The behavior of state can be such that data synchronize and be restored to correct state by rollback mode, guarantee synchrodata consistency to reach Purpose.
Further, the step S303 further include:
When the type of error is not preset type of error, preset synchronized transaction error handling processing process is executed.
Specifically, due to being also possible that other mistakes in synchronizing process, therefore the present invention is to the mistake of non-default property Provided with synchronized transaction error handling processing process, this error handling processing process can be set according to actual needs, and the present invention is to specific Affairs error handling processing process without limitation, such as when there is non-default mistake, is directly cancelled the synchronization of Current transaction, is carried out down The simultaneously operating etc. of one affairs is not in dead state so as to guarantee system in synchronized transaction, and influences to synchronize Process.
For the synchronized transaction in above-mentioned specific embodiment, synchronized transaction operates INSERT in target client database It when (1) VALUES INTO T, reports an error if synchronized, needs to judge the error code that target client database returns to identify mistake Type need by the way of disconnecting if it is the mistake of Network Abnormal or data base set irrespective of size come before rollback Simultaneously operating, then affairs of the re-synchronization error.
S400, when current operation is transaction commit operation, reconnect target client database, and inquire in supplementary table and be The no affairs ID comprising the synchronized transaction, and when not including by the affairs ID of synchronized transaction insertion supplementary table and again into Act business simultaneously operating.
Specifically, network event occurs when waiting operation to be committed to return when submitting operation to the execution of target client database Barrier causes to execute error, and the target client database of distal end is possible to have been completed the synchronization for submitting operation but without normal direction target side Service returns, and in this case, target side synchronous service needs to reconnect target client database, inquires and steps in supplementary table Whether the affairs ID of note successfully submits come affairs when confirming last time failure, submits behaviour in addition, executing to target client database When making, when waiting operation to be committed to return, database works as machine extremely causes to execute error, and database is when machine, it is also possible to should Operation is submitted to complete but there are no being in time for returning to target side synchronous service, in this case, the synchronous clothes of target side Business need to reconnect target client database, inquired in supplementary table the affairs ID of registration come affairs when confirming last time failure whether It successfully submits, therefore synchronous thing when failure can be confirmed by whether inquiring in supplementary table comprising affairs ID in the present invention Whether business submitted success, re-start transactional synchronization, when unsuccessful to guarantee the consistency of synchrodata.
Referring to Fig. 4, it is the flow chart of the step S400, include the following steps:
S401, when current operation is transaction commit operation, disconnect and the connection of target client database and reconnect mesh Mark client database;
Supplementary table in S402, invocation target client database, and inquire in the supplementary table with the presence or absence of the synchronous thing The affairs ID that business is registered when starting;
S403, in the supplementary table be not present the affairs ID when, will synchronized transaction ID be inserted into target client database in Supplementary table in and restart carry out transactional synchronization operation.
The present invention confirms whether synchrodata has successfully been submitted by way of inquiry transaction ID, using supplementary table and When table to be synchronized is in the same affairs, ACID characteristic (atomicity (Atomicity), consistency of affairs are utilized (Consistency), isolation (Isolation), persistence (Durability)) guarantee in grid or database failure In the case where synchrodata consistency.
Preferably, the step S403 further include:
There are when affairs ID in the supplementary table, next synchronized transaction is executed.
In other words, the present invention in judging the supplementary table there are when affairs ID, can be confirmed synchronization succeeded, The synchronization for going to next affairs is synchronized to terminate Current transaction, to guarantee the consistency of synchrodata.
For the synchronized transaction in above-mentioned specific embodiment, transaction commit operation is executed in target client database When, it reports an error if executed, needs to disconnect and reconnect target client database, then inquire in supplementary table and whether wrap Contain affairs, executes inquiry SELECT TRXID FROM DMHS_TRXID WHERE TRXID=12345;If finding affairs The result set that ID is 12345, that illustrates the affairs synchronized success, terminates synchronizing for Current transaction and goes to next affairs It is synchronous;Otherwise re-synchronization affairs.
As shown in figure 5, being based on data consistency verification method when above-mentioned data synchronous abnormality, the present invention is further correspondingly provided Consistency verification of data equipment when a kind of data synchronous abnormality, consistency verification of data equipment can be with when the data synchronous abnormality It is that mobile terminal, desktop PC, notebook, palm PC and server etc. calculate equipment.Number when the data synchronous abnormality It include processor 10, memory 20 and display 30 according to consistency desired result equipment.Data when Fig. 5 illustrates only data synchronous abnormality The members of consistency desired result equipment can substitute it should be understood that being not required for implementing all components shown Implement more or less component.
Consistency verification of data equipment when the memory 20 can be the data synchronous abnormality in some embodiments Internal storage unit, such as when data synchronous abnormality consistency verification of data equipment hard disk or memory.The memory 20 It is also possible to the External memory equipment of consistency verification of data equipment when the data synchronous abnormality, example in further embodiments The plug-in type hard disk being equipped in consistency verification of data equipment when such as the data synchronous abnormality, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash Card) etc..
Further, consistency verification of data equipment was interior when the memory 20 can also both include data synchronous abnormality Portion's storage unit also includes External memory equipment.The number when memory 20 is installed on the data synchronous abnormality for storing According to the application software and Various types of data of consistency desired result equipment, such as consistency verification of data when the installation data synchronous abnormality The program code etc. of equipment.The memory 20 can be also used for temporarily storing the data that has exported or will export. In one embodiment, consistency verification of data program 40 when being stored with data synchronous abnormality on memory 20, the data synchronous different Consistency verification of data program 40 can be performed by processor 10, to realize that the data of each embodiment of the application are synchronous when often Data consistency verification method when abnormal.
The processor 10 can be in some embodiments a central processing unit (Central Processing Unit, CPU), microprocessor or other data processing chips, for running the program code stored in the memory 20 or processing number According to, such as data consistency verification method etc. when executing the data synchronous abnormality.
The display 30 can be light-emitting diode display, liquid crystal display, touch-control liquid crystal display in some embodiments And OLED (Organic Light-Emitting Diode, Organic Light Emitting Diode) touches device etc..The display 30 is used The information of consistency verification of data equipment and for showing visual user circle when being shown in the data synchronous abnormality Face.The component 10-30 of consistency verification of data equipment is in communication with each other by system bus when the data synchronous abnormality.
In one embodiment, the consistency verification of data when processor 10 executes data synchronous abnormality in the memory 20 It is performed the steps of when program 40
The log information packet that source is sent is received, and the affairs ID of the synchronized transaction in log information packet is inserted into target side Laggard behaviour business simultaneously operating in the supplementary table created in database;
When receive target client database report an error notify when, judge current synchronized transaction be in which kind of operation, wherein The operation includes the affairs change operation and transaction commit operation successively executed;
When current operation is affairs change operation, judgement, which reports an error, notifies whether belong to preset type of error, and is reporting Mistake notifies the simultaneously operating that rollback has carried out when belonging to preset type of error and is again inserted into the affairs ID of the synchronized transaction In supplementary table and re-start transactional synchronization operation;
When current operation is transaction commit operation, target client database is reconnected, and inquire in supplementary table and whether wrap Affairs ID containing the synchronized transaction, and the affairs ID of the synchronized transaction is inserted into supplementary table when not including and re-starts thing Business simultaneously operating.
The affairs change operation includes insertion operation, updates operation and delete operation.
Further, the log information packet for receiving source and sending, and by the thing of the synchronized transaction in log information packet The step of laggard behaviour business simultaneously operating, includes: in the supplementary table created in business ID insertion target client database
Parsing current sync office log information packet and obtain the affairs ID of the synchronized transaction in log information packet;
Insertion operation is executed in the supplementary table created in target client database so that the affairs ID is inserted into supplementary table In.
Further, described when current operation is affairs change operation, judgement, which reports an error, notifies whether belong to preset report Wrong type, and the simultaneously operating that has carried out of rollback and again by the synchronized transaction when the notice that reports an error belongs to preset type of error Affairs ID insertion supplementary table in and re-start transactional synchronization operation the step of include:
When current operation is affairs change operation, according to the class for the error code identification mistake that target client database returns Type;
Judge whether the type of error belongs to preset type of error;
When the type of error is preset type of error, disconnect what connection and rollback with target client database had carried out Simultaneously operating, and again will be in the affairs ID of synchronized transaction insertion supplementary table and to re-start affairs same after the completion of rollback Step operation.
Further, described when the type of error is preset type of error, disconnect the connection with target client database And the simultaneously operating that rollback has carried out, and will lay equal stress in the affairs ID insertion supplementary table of the synchronized transaction again after the completion of rollback New the step of carrying out transactional synchronization operation further include:
When the type of error is not preset type of error, preset synchronized transaction error handling processing process is executed.
Further, the preset type of error includes Network Abnormal mistake and database system-level errors.
Further, described when current operation is transaction commit operation, target client database is reconnected, and inquire auxiliary Help in table whether include the synchronized transaction affairs ID, and will be in the affairs ID insertion supplementary table of the synchronized transaction when not including And the step of re-starting transactional synchronization operation, includes:
When current operation is transaction commit operation, the connection with target client database and again linking objective end number are disconnected According to library;
Supplementary table in invocation target client database, and inquire in the supplementary table and start with the presence or absence of the synchronized transaction The affairs ID of Shi Dengji;
When affairs ID being not present in the supplementary table, synchronized transaction ID is inserted into the auxiliary in target client database In table and restart to carry out transactional synchronization operation.
Further, it is described in the supplementary table be not present the affairs ID when, by synchronized transaction ID be inserted into target side In supplementary table in database and restart the step of carrying out transactional synchronization operation further include:
There are when affairs ID in the supplementary table, next synchronized transaction is executed.
Referring to Fig. 6, the system of consistency verification of data program is preferably real when it is installation data synchronous abnormality of the present invention Apply the functional block diagram of example.In the present embodiment, the system of consistency verification of data program can be with when installation data synchronous abnormality Be divided into one or more modules, one or more of modules are stored in the memory 20, and by one or Multiple processors (the present embodiment is the processor 10) are performed, to complete the present invention.For example, installation data is same in Fig. 6 The system of consistency verification of data program can be divided into transaction tag insertion module 21, operation judges module when step is abnormal 22, change operation error handling module 23 and submission operating mistake processing module 24.The so-called module of the present invention is to have referred to At the series of computation machine program instruction section of specific function, data one when than program more suitable for describing the data synchronous abnormality Implementation procedure of the cause property checking routine in the data synchronous abnormality in consistency verification of data equipment.Being described below will be specific Introduce the function of the module 21-24.
Transaction tag is inserted into module 21, for receiving the log information packet of source transmission, and will be same in log information packet Laggard behaviour business simultaneously operating in the supplementary table created in the affairs ID insertion target client database of step affairs;
Operation judges module 22, for when receive target client database report an error notify when, judge current synchronization thing Business is in which kind of operation, wherein the operation includes the affairs change operation and transaction commit operation successively executed;
Change operation error handling module 23, for when current operation is affairs change operation, the judgement notice that reports an error is It is no to belong to preset type of error, and the simultaneously operating that rollback has carried out when the notice that reports an error belongs to preset type of error is laid equal stress on Newly the affairs ID of the synchronized transaction is inserted into supplementary table and re-starts transactional synchronization operation;
Operating mistake processing module 24 is submitted, for reconnecting target side when current operation is transaction commit operation Database, and inquire in supplementary table whether include the synchronized transaction affairs ID, and when not including by the thing of the synchronized transaction In business ID insertion supplementary table and re-start transactional synchronization operation.
Wherein, the affairs change operation includes insertion operation, updates operation and delete operation.
Further, transaction tag insertion module 21 specifically includes:
Transaction tag acquiring unit, for parse current sync office log information packet and obtain log information packet In synchronized transaction affairs ID;
Transaction tag is inserted into unit, and insertion operation is executed in the supplementary table for creating in target client database with by institute It states in affairs ID insertion supplementary table.
Change operation error handling module 23 specifically includes:
Garbled-reception unit, the mistake for being returned according to target client database when current operation is affairs change operation The type of error code identification mistake;
Type of error judging unit, for judging whether the type of error belongs to preset type of error;
First error handling unit, for disconnecting and target end data when the type of error is preset type of error The simultaneously operating that the connection and rollback in library have carried out, and the affairs ID of the synchronized transaction is inserted into auxiliary again after the completion of rollback In table and re-start transactional synchronization operation.
Preferably, first error handling unit is also used to hold when the type of error is not preset type of error The preset synchronized transaction error handling processing process of row.
Preferably, the preset type of error includes Network Abnormal mistake and database system-level errors.
The submission operating mistake processing module 24 specifically includes:
Switching units are connected, for disconnecting the connection with target client database when current operation is transaction commit operation And linking objective client database again;
Query unit for the supplementary table in invocation target client database, and is inquired in the supplementary table with the presence or absence of institute State the affairs ID registered when synchronized transaction starts;
The synchronized transaction ID when affairs ID being not present in the supplementary table, is inserted into mesh by the second error handling unit In supplementary table in mark client database and restart to carry out transactional synchronization operation.
Preferably, second error handling unit is also used in the supplementary table there are when affairs ID, under execution One synchronized transaction.
In conclusion data consistency verification method, equipment and storage medium when data synchronous abnormality provided by the invention In, which comprises the log information packet that sends of reception source first, and by the affairs of the synchronized transaction in log information packet Laggard behaviour business simultaneously operating in the supplementary table created in ID insertion target client database;Then when receiving target client database Report an error when notifying, judge which kind of operation is current synchronized transaction be in, wherein the operation includes that the affairs successively executed become More operation and transaction commit operation;When current operation is affairs change operation, judgement, which reports an error, notifies whether belong to preset mistake Accidentally type, and the simultaneously operating that has carried out of rollback and again by the synchronized transaction when the notice that reports an error belongs to preset type of error Affairs ID insertion supplementary table in and re-start transactional synchronization operation, current operation be transaction commit operation when, connect again Connect target client database, and inquire in supplementary table whether include the synchronized transaction affairs ID, and when not including by the synchronization In the affairs ID insertion supplementary table of affairs and re-start transactional synchronization operation.The present invention in synchronizing process, open by synchronized transaction Transaction tag is first registered when the beginning in supplementary table, if malfunctioned in synchronizing process, by judging thing according to current operation Whether business label is registered, to judge whether synchronously completed, further to guarantee the consistency of synchrodata.
Certainly, those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, It is that related hardware (such as processor, controller etc.) can be instructed to complete by computer program, the program can store In a computer-readable storage medium, which may include the process such as above-mentioned each method embodiment when being executed.Its Described in storage medium can be for memory, magnetic disk, CD etc..
It should be understood that the application of the present invention is not limited to the above for those of ordinary skills can With improvement or transformation based on the above description, all these modifications and variations all should belong to the guarantor of appended claims of the present invention Protect range.

Claims (10)

1. data consistency verification method when a kind of data synchronous abnormality, which comprises the steps of:
The log information packet that source is sent is received, and the affairs ID of the synchronized transaction in log information packet is inserted into target end data Laggard behaviour business simultaneously operating in the supplementary table created in library;
When receive target client database report an error notify when, judge which kind of operation is current synchronized transaction be in, wherein described Operation includes the affairs change operation and transaction commit operation successively executed;
When current operation is affairs change operation, judgement, which reports an error, notifies whether belong to preset type of error, and logical reporting an error Know the simultaneously operating that rollback has carried out when belonging to preset type of error and again by the affairs ID insertion auxiliary of the synchronized transaction In table and re-start transactional synchronization operation;
When current operation is transaction commit operation, target client database is reconnected, and whether is inquired in supplementary table comprising being somebody's turn to do The affairs ID of synchronized transaction, and will be in the affairs ID of synchronized transaction insertion supplementary table and to re-start affairs same when not including Step operation.
2. data consistency verification method when data synchronous abnormality according to claim 1, which is characterized in that the reception The log information packet that source is sent, and the affairs ID of the synchronized transaction in log information packet is inserted into target client database and is created Supplementary table in laggard behaviour business simultaneously operating the step of include:
Parsing current sync office log information packet and obtain the affairs ID of the synchronized transaction in log information packet;
Insertion operation is executed in the supplementary table created in target client database so that the affairs ID to be inserted into supplementary table.
3. data consistency verification method when data synchronous abnormality according to claim 2, which is characterized in that described to work as When preceding operation is affairs change operation, whether the judgement notice that reports an error belongs to the preset type that reports an error, and belongs in the notice that reports an error pre- If type of error when the simultaneously operating that has carried out of rollback and will lay equal stress in the affairs ID insertion supplementary table of the synchronized transaction again New the step of carrying out transactional synchronization operation includes:
When current operation is affairs change operation, according to the type for the error code identification mistake that target client database returns;
Judge whether the type of error belongs to preset type of error;
When the type of error is preset type of error, disconnection has carried out synchronous with the connection and rollback of target client database Operation, and the affairs ID of the synchronized transaction is inserted into supplementary table again after the completion of rollback and re-starts transactional synchronization behaviour Make.
4. data consistency verification method when data synchronous abnormality according to claim 3, which is characterized in that described at this When type of error is preset type of error, the simultaneously operating that connection and rollback with target client database have carried out is disconnected, and It again will be in the affairs ID insertion supplementary table of the synchronized transaction and the step of re-starting transactional synchronization operation after the completion of rollback Further include:
When the type of error is not preset type of error, preset synchronized transaction error handling processing process is executed.
5. data consistency verification method when data synchronous abnormality according to claim 4, which is characterized in that described default Type of error include Network Abnormal mistake and database system-level errors.
6. data consistency verification method when data synchronous abnormality according to claim 3, which is characterized in that described to work as When preceding operation is transaction commit operation, target client database is reconnected, and whether is inquired in supplementary table comprising the synchronized transaction Affairs ID, and in the affairs ID of synchronized transaction insertion supplementary table and transactional synchronization operation will be re-started when not including Step includes:
When current operation is transaction commit operation, the connection with target client database and again linking objective end data are disconnected Library;
Supplementary table in invocation target client database, and inquire and stepped on when starting in the supplementary table with the presence or absence of the synchronized transaction The affairs ID of note;
When affairs ID being not present in the supplementary table, synchronized transaction ID is inserted into the supplementary table in target client database And restart to carry out transactional synchronization operation.
7. data consistency verification method when data synchronous abnormality according to claim 6, which is characterized in that described in institute It states in supplementary table there is no when affairs ID, will be opened in the supplementary table in synchronized transaction ID insertion target client database and again The step of beginning to carry out transactional synchronization operation further include:
There are when affairs ID in the supplementary table, next synchronized transaction is executed.
8. data consistency verification method when data synchronous abnormality according to claim 1, which is characterized in that the affairs Change operation includes insertion operation, updates operation and delete operation.
9. consistency verification of data equipment when a kind of data synchronous abnormality characterized by comprising processor, memory and logical Believe bus;
The computer-readable program that can be executed by the processor is stored on the memory;
The communication bus realizes the connection communication between processor and memory;
The processor realizes that the data as described in claim 1-8 any one are synchronous when executing the computer-readable program Step when abnormal in data consistency verification method.
10. a kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage have one or Multiple programs, one or more of programs can be executed by one or more processor, to realize that claim 1-8 such as appoints Step when data synchronous abnormality described in meaning one in data consistency verification method.
CN201910436428.3A 2019-05-23 2019-05-23 Data consistency checking method and equipment during data synchronization abnormity and storage medium Active CN110427422B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910436428.3A CN110427422B (en) 2019-05-23 2019-05-23 Data consistency checking method and equipment during data synchronization abnormity and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910436428.3A CN110427422B (en) 2019-05-23 2019-05-23 Data consistency checking method and equipment during data synchronization abnormity and storage medium

Publications (2)

Publication Number Publication Date
CN110427422A true CN110427422A (en) 2019-11-08
CN110427422B CN110427422B (en) 2020-10-30

Family

ID=68407549

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910436428.3A Active CN110427422B (en) 2019-05-23 2019-05-23 Data consistency checking method and equipment during data synchronization abnormity and storage medium

Country Status (1)

Country Link
CN (1) CN110427422B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111221800A (en) * 2019-12-19 2020-06-02 中国平安人寿保险股份有限公司 Database migration method and device, electronic equipment and storage medium
CN111367705A (en) * 2020-03-31 2020-07-03 中国工商银行股份有限公司 SQL error reporting processing method and device
CN111737331A (en) * 2020-06-22 2020-10-02 中国银行股份有限公司 Transaction consistency processing method and system for database and object storage
CN112307118A (en) * 2020-09-30 2021-02-02 武汉达梦数据库有限公司 Method for guaranteeing data consistency based on log analysis synchronization and synchronization system
CN113282393A (en) * 2021-06-04 2021-08-20 银盛支付服务股份有限公司 Method for solving task scheduling facing to multiple portrait label jobs
CN113761056A (en) * 2021-04-07 2021-12-07 西安京迅递供应链科技有限公司 Distributed service processing method, device and system

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105338093A (en) * 2015-11-16 2016-02-17 中国建设银行股份有限公司 Data synchronizing method and system
US20160147859A1 (en) * 2014-11-25 2016-05-26 Juchang Lee Transactional and Parallel Log Replay for Asynchronous Table Replication
US9436752B2 (en) * 2003-10-02 2016-09-06 Progress Software Corporation High availability via data services
CN106484807A (en) * 2016-09-23 2017-03-08 北京许继电气有限公司 Build the method and system of autonomous controlled data storehouse dual-active or many aggregated structures living
CN106775959A (en) * 2016-12-06 2017-05-31 上海亿账通互联网科技有限公司 Distributed transaction processing method and system
CN109241185A (en) * 2018-08-27 2019-01-18 武汉达梦数据库有限公司 A kind of method and data synchronization unit that data are synchronous
CN109634970A (en) * 2018-10-19 2019-04-16 平安科技(深圳)有限公司 Table method of data synchronization, equipment, storage medium and device
CN109657007A (en) * 2018-12-17 2019-04-19 武汉达梦数据库有限公司 Database in phase based on asynchronous submission executes method and apparatus parallel

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9436752B2 (en) * 2003-10-02 2016-09-06 Progress Software Corporation High availability via data services
US20160147859A1 (en) * 2014-11-25 2016-05-26 Juchang Lee Transactional and Parallel Log Replay for Asynchronous Table Replication
CN105338093A (en) * 2015-11-16 2016-02-17 中国建设银行股份有限公司 Data synchronizing method and system
CN106484807A (en) * 2016-09-23 2017-03-08 北京许继电气有限公司 Build the method and system of autonomous controlled data storehouse dual-active or many aggregated structures living
CN106775959A (en) * 2016-12-06 2017-05-31 上海亿账通互联网科技有限公司 Distributed transaction processing method and system
CN109241185A (en) * 2018-08-27 2019-01-18 武汉达梦数据库有限公司 A kind of method and data synchronization unit that data are synchronous
CN109634970A (en) * 2018-10-19 2019-04-16 平安科技(深圳)有限公司 Table method of data synchronization, equipment, storage medium and device
CN109657007A (en) * 2018-12-17 2019-04-19 武汉达梦数据库有限公司 Database in phase based on asynchronous submission executes method and apparatus parallel

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
SUBHASH BHALLA ET AL.: ""Synchronization and Recovery in an Embedded Database System for Read-Only Transactions"", 《INTERNATIONAL CONFERENCE ON EMBEDDED AND UBIQUITOUS COMPUTING》 *
刘娟娟 等: ""数据库同步技术的研究与实现"", 《软件工程》 *
吴雅娟 等: ""基于操作日志的完井数据同步模型"", 《计算机系统应用》 *
钟容 等: ""基于事务日志的数据同步模型研究"", 《科技创新与应用》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111221800A (en) * 2019-12-19 2020-06-02 中国平安人寿保险股份有限公司 Database migration method and device, electronic equipment and storage medium
CN111367705A (en) * 2020-03-31 2020-07-03 中国工商银行股份有限公司 SQL error reporting processing method and device
CN111367705B (en) * 2020-03-31 2023-08-04 中国工商银行股份有限公司 SQL error reporting processing method and device
CN111737331A (en) * 2020-06-22 2020-10-02 中国银行股份有限公司 Transaction consistency processing method and system for database and object storage
CN111737331B (en) * 2020-06-22 2023-08-18 中国银行股份有限公司 Transaction consistency processing method and system for database and object storage
CN112307118A (en) * 2020-09-30 2021-02-02 武汉达梦数据库有限公司 Method for guaranteeing data consistency based on log analysis synchronization and synchronization system
CN112307118B (en) * 2020-09-30 2024-03-22 武汉达梦数据库股份有限公司 Method for guaranteeing data consistency based on log analysis synchronization and synchronization system
CN113761056A (en) * 2021-04-07 2021-12-07 西安京迅递供应链科技有限公司 Distributed service processing method, device and system
CN113282393A (en) * 2021-06-04 2021-08-20 银盛支付服务股份有限公司 Method for solving task scheduling facing to multiple portrait label jobs

Also Published As

Publication number Publication date
CN110427422B (en) 2020-10-30

Similar Documents

Publication Publication Date Title
CN110427422A (en) Data consistency verification method, equipment and storage medium when data synchronous abnormality
CN109542639B (en) Processing method and processing device for guaranteeing consistency of microservice calling data
CN110347746B (en) Heterogeneous database synchronous data consistency checking method and device
US9286368B2 (en) Linking framework for information technology management
US7251747B1 (en) Method and system for transferring data using a volatile data transfer mechanism such as a pipe
CN102609281B (en) Distributed software patch update method and system
CN104809202A (en) Database synchronization method and device
CN110222119B (en) Data conversion synchronization method, equipment and storage medium for heterogeneous database
US20120303761A1 (en) Breakpoint continuous transmission method
CN104809201A (en) Database synchronization method and device
CN111400104B (en) Data synchronization method and device, electronic equipment and storage medium
CN111026735B (en) Data transmission method, device, equipment and medium
EP2696297B1 (en) System and method for generating information file based on parallel processing
CN114612101A (en) Reliable inter-link route cross-link method and system for connection
CN113468143A (en) Data migration method, system, computing device and storage medium
CN112800060A (en) Data processing method and device, computer readable storage medium and electronic equipment
US20080178182A1 (en) Work state returning apparatus, work state returning method, and computer product
CN112261114A (en) Data backup system and method
CN113835786B (en) Data docking system, method and computer readable storage medium
WO2019196227A1 (en) Platform integration method and apparatus, and computer device and storage medium
CN115185787A (en) Method and device for processing transaction log
CN111090648B (en) Relational database data synchronization conflict resolution method
CN117763052B (en) Data synchronization method and system for charging multi-center memory database
CN116108110B (en) Kafka message consumption compensation method, system and medium
CN103312728B (en) Method of data synchronization between a kind of storage system, system and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CP01 Change in the name or title of a patent holder

Address after: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province

Patentee after: Wuhan dream database Co.,Ltd.

Address before: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province

Patentee before: WUHAN DAMENG DATABASE Co.,Ltd.

CP01 Change in the name or title of a patent holder
CB03 Change of inventor or designer information
CB03 Change of inventor or designer information

Inventor after: Sun Feng

Inventor after: Yu Yuanlan

Inventor after: Zhao Jiawei

Inventor before: Fu Quan

Inventor before: Sun Feng

Inventor before: Yu Yuanlan

Inventor before: Zhao Jiawei

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20220914

Address after: 430073 16-19 / F, building C3, future science and technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province

Patentee after: Wuhan dream database Co.,Ltd.

Patentee after: HUAZHONG University OF SCIENCE AND TECHNOLOGY

Address before: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province

Patentee before: Wuhan dream database Co.,Ltd.

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230725

Address after: 16-19/F, Building C3, Future Science and Technology Building, No. 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province, 430206

Patentee after: Wuhan dream database Co.,Ltd.

Address before: 430073 16-19 / F, building C3, future science and technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province

Patentee before: Wuhan dream database Co.,Ltd.

Patentee before: HUAZHONG University OF SCIENCE AND TECHNOLOGY