CN113032176A - Distributed transaction double-compensation method and device based on daily account checking - Google Patents
Distributed transaction double-compensation method and device based on daily account checking Download PDFInfo
- Publication number
- CN113032176A CN113032176A CN202110310873.2A CN202110310873A CN113032176A CN 113032176 A CN113032176 A CN 113032176A CN 202110310873 A CN202110310873 A CN 202110310873A CN 113032176 A CN113032176 A CN 113032176A
- Authority
- CN
- China
- Prior art keywords
- transaction
- compensation
- real
- database
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000007246 mechanism Effects 0.000 claims abstract description 77
- 238000012545 processing Methods 0.000 claims abstract description 67
- 230000002159 abnormal effect Effects 0.000 claims abstract description 25
- 230000005856 abnormality Effects 0.000 claims abstract description 3
- 238000003860 storage Methods 0.000 claims description 18
- 230000004048 modification Effects 0.000 claims description 7
- 238000012986 modification Methods 0.000 claims description 7
- 230000009977 dual effect Effects 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims 1
- 230000009545 invasion Effects 0.000 abstract description 16
- 230000008569 process Effects 0.000 description 18
- 238000011161 development Methods 0.000 description 16
- 230000009471 action Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 8
- 230000001360 synchronised effect Effects 0.000 description 7
- 230000007547 defect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002441 reversible effect Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
- G06F11/0709—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0793—Remedial or corrective actions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a distributed transaction double-compensation method and device based on daily terminal reconciliation. The method comprises the following steps: carrying out normal processing on each transaction according to the business processing flow, and not considering the abnormal condition of the transaction; after the transaction is normally processed, submitting a database corresponding to the transaction in real time; if the preset transaction is processed and an abnormality is encountered, a real-time compensation mechanism is adopted to perform real-time compensation on the database corresponding to each transaction before the preset transaction; and if the global transaction consistency principle is not met after the real-time compensation, performing daily and final account checking compensation. The consistency of a plurality of databases is ensured through a double compensation mechanism of real-time compensation and daily final account checking compensation. The balance of transaction real-time and consistency is realized. The scheme has the advantages of high efficiency, real-time transaction performance and low invasion to service codes.
Description
Technical Field
The application relates to the field of distributed transactions, in particular to a distributed transaction double-compensation method and device based on daily terminal reconciliation, a computer readable storage medium and a processor.
Background
When a business system uses a plurality of databases at the same time, the plurality of databases need to be coordinated to ensure the global transaction consistency of the databases, namely, the distributed database transaction. The traditional solution of distributed transaction has various modes such as 2PC two-stage commit, TCC acknowledgement compensation, MQ message and the like, but the traditional solution of distributed transaction has the defects of low transaction real-time performance, high service code intrusion, high development cost and the like.
Specifically, the conventional distributed transaction solution has the following problems:
2PC two-stage compensation mechanism: the mechanism operates in two steps: preparation and submission. The method is a synchronous blocking mechanism, and can be finally submitted only after all nodes in the distributed mode are confirmed, so that the time consumption is long, and the performance is low. And a fault tolerance mechanism is lacked, and a self-correction mechanism is lacked when only partial nodes commit and transactions are inconsistent.
TCC acknowledgment compensation mechanism: the mechanism operates in three general aspects: try reserves resources, Confirm submission and Cancel release of resources by Cancel. The fault tolerance step of the mechanism has higher invasion on business development codes. For example, there may be errors in both the second step of acknowledging the commit and the third step of de-releasing the resource. If these errors are not handled, the transaction coherency breaks. If these errors are handled, the business system is required to develop a large amount of error handling logic to ensure transactional consistency, the intrusion into the business system code is high, and there may also be defects in the error handling code that result in transactional inconsistency.
MQ message mechanism: the MQ message mechanism is used for asynchronously degrading the transaction and transferring the transaction to a third-party MQ middleware for completion. The mechanism has the defects that database transactions depend on third-party MQ middleware excessively, MQ is an asynchronous mechanism, and a queue queuing mechanism is adopted, so that when the traffic is high, the message backlog exists in an MQ queue, the mechanism is not suitable for a high-real-time service system and is not suitable for use, and the transaction instantaneity is poor.
Disclosure of Invention
The application mainly aims to provide a distributed transaction double compensation method and device based on daily-end reconciliation, a computer-readable storage medium and a processor, so as to solve the problems of low transaction instantaneity, high service code intrusiveness, high development cost and the like of a distributed transaction solution in the prior art.
In order to achieve the above object, according to an aspect of the present application, there is provided a distributed transaction double compensation method based on end-of-day reconciliation, including: carrying out normal processing on each transaction according to the business processing flow, and not considering the abnormal condition of the transaction; submitting a database corresponding to the transaction in real time after the transaction is normally processed; if an exception occurs during processing of a predetermined transaction, a real-time compensation mechanism is adopted to perform real-time compensation on the database corresponding to each transaction before the predetermined transaction; and if the global transaction consistency principle is not met after the real-time compensation, performing daily and final account checking compensation.
Further, compensating the database corresponding to each transaction before the predetermined transaction in real time includes: and correcting the database corresponding to each transaction before the predetermined transaction into a state before submission.
Further, if the global transaction consistency principle is not satisfied after the real-time compensation, performing a daily and final reconciliation compensation, including: under the condition that the database is failed to be corrected, checking all data in the database at the end of the day based on a day-end reconciliation consistency guarantee mechanism; if partial database is inconsistent, modifying the abnormal data in the database to ensure the global transactions of all the databases to be consistent.
Further, checking all the data in the database at the end of the day includes: and checking all data in the database at the end of the day based on a stroke-by-stroke checking principle.
Further, the real-time compensation mechanism is a common compensation mechanism.
According to another aspect of the present application, there is provided a distributed transaction double compensation apparatus based on daily end-of-day reconciliation, including: the processing unit is used for carrying out normal processing on each transaction according to the business processing flow, and the abnormal condition of the transaction is not considered; the submitting unit is used for submitting the database corresponding to the transaction in real time after the transaction is normally processed; the first compensation unit is used for compensating the database corresponding to each transaction before the predetermined transaction in real time by adopting a real-time compensation mechanism if the predetermined transaction is processed and an abnormality is encountered; and the second compensation unit is used for performing daily and final account checking compensation if the global transaction consistency principle is not met after the real-time compensation.
Further, the first compensation unit is further configured to modify the database corresponding to each transaction before the predetermined transaction into a state before commit.
Further, the second compensation unit comprises a checking module and a modification module, wherein the checking module is used for checking all data in the database at the end of a day based on a day-to-end reconciliation consistency guarantee mechanism under the condition that the database is unsuccessfully corrected; the modification module is used for modifying the abnormal data in the database if part of the database is inconsistent so as to ensure that all the global transactions of the database are consistent.
According to still another aspect of the present application, a computer-readable storage medium is provided, where the computer-readable storage medium includes a stored program, where the program, when executed, controls a device in which the computer-readable storage medium is located to perform any one of the methods for dual compensation of distributed transactions based on daily account checking.
According to still another aspect of the present application, a processor is provided, where the processor is configured to execute a program, where the program executes any one of the methods for daily-based distributed transaction double compensation.
By applying the technical scheme, each transaction is normally processed according to the business processing flow, the database corresponding to the transaction is submitted in real time after normal processing, if the abnormal condition is met in the normal processing process, the database corresponding to each transaction before the scheduled transaction is processed is compensated in real time, if the global transaction consistency principle is not met after the real-time compensation, the daily and final account checking compensation is carried out, and the consistency of the databases is ensured through a double compensation mechanism of the real-time compensation and the daily and final account checking compensation. The balance of transaction real-time and consistency is realized. The method has the advantages of high efficiency, transaction instantaneity and low service code invasion, and can normally process each transaction according to the service processing flow, and avoid the invasion of the service code without considering the abnormal condition of the transaction. The scheme avoids the time-consuming and low-efficiency actions of synchronous coordination submitted by a 2PC two-stage system, and ensures high efficiency; the real-time compensation action in the double compensation mechanism is to compensate in time, and ensure the cross-database (namely cross-database) distributed transaction to be reflected in real time to the maximum extent. The problem of insufficient real-time performance of the existing MQ-based transaction mechanism is solved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:
FIG. 1 shows a flowchart of a distributed transaction double compensation method based on end-of-day reconciliation according to an embodiment of the application;
FIG. 2 shows a schematic diagram of a distributed transaction double compensation arrangement based on end-of-day reconciliation according to an embodiment of the application;
fig. 3 shows a flowchart of a specific end-of-day reconciliation distributed transaction double compensation method according to an embodiment of the application.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" another element, it can be directly on the other element or intervening elements may also be present. Also, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element or "connected" to the other element through a third element.
For convenience of description, some terms or expressions referred to in the embodiments of the present application are explained below:
and (4) checking account at end of day: in the financial field of banks, insurance and the like, batch detailed data is checked one by one from the business and technical aspects of each system and each module at the end of each day (usually at night or in the morning), even if certain errors or inconsistency exist in the daytime, a summary reconciliation, a detailed reconciliation, an error identification and processing mechanism exist at the end of each day, and the data among the systems is adjusted to be consistent through difference processing specifications and rules. If special abnormal conditions are met, manual intervention processing is carried out to ensure that all system data can be checked to be consistent in the end of the day, and the final consistency in the end of the day is guaranteed.
As introduced in the background art, the distributed transaction solution in the prior art has the disadvantages of low transaction instantaneity, high intrusiveness for service codes, high development cost, and the like, and is intended to solve the problems of the distributed transaction solution that the disadvantages of low transaction instantaneity, high intrusiveness for service codes, high development cost, and the like exist. The embodiment of the application provides a distributed transaction double compensation method and device based on daily account checking, a computer readable storage medium and a processor.
According to the embodiment of the application, a distributed transaction double compensation method based on daily account checking is provided.
Fig. 1 is a flowchart of a distributed transaction double compensation method based on end-of-day reconciliation according to an embodiment of the application. As shown in fig. 1, the method comprises the steps of:
step S101, performing normal processing on each transaction according to a business processing flow, and not considering the abnormal condition of the transaction;
step S102, after the transaction is normally processed, submitting a database corresponding to the transaction in real time;
step S103, if an exception occurs during processing of a predetermined transaction, a real-time compensation mechanism is adopted to perform real-time compensation on the database corresponding to each transaction before the predetermined transaction;
and step S104, if the global transaction consistency principle is not met after the real-time compensation, performing daily and final account checking compensation.
Specifically, for example, if an exception occurs during processing of the third transaction, the database corresponding to the first transaction and the database corresponding to the second transaction are compensated in real time.
Specifically, the daily final reconciliation compensation refers to correcting the database which does not meet the global transaction consistency principle based on the daily final reconciliation principle.
According to the scheme, each transaction is normally processed according to a business processing flow, the database corresponding to the transaction is submitted in real time after normal processing, if the abnormal condition is met in the normal processing process, the database corresponding to each transaction before the scheduled transaction is subjected to real-time compensation, if the global transaction consistency principle is not met after the real-time compensation, the daily final account checking compensation is carried out, and the consistency of the databases is ensured through a double compensation mechanism of the real-time compensation and the daily final account checking compensation. The balance of transaction real-time and consistency is realized. The method has the advantages of high efficiency, transaction instantaneity and low service code invasion, and can normally process each transaction according to the service processing flow, and avoid the invasion of the service code without considering the abnormal condition of the transaction. The scheme avoids the time-consuming and low-efficiency actions of synchronous coordination submitted by a 2PC two-stage system, and ensures high efficiency; the real-time compensation action in the double compensation mechanism is to compensate in time, and ensure the cross-database (namely cross-database) distributed transaction to be reflected in real time to the maximum extent. The problem of insufficient real-time performance of the existing MQ-based transaction mechanism is solved.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
In an embodiment of the application, the compensating the database corresponding to each transaction before the predetermined transaction in real time includes: and correcting the database corresponding to each transaction before the predetermined transaction into a state before submission.
In an embodiment of the present application, if the global transaction consistency principle is not satisfied after the real-time compensation, the daily and final reconciliation compensation includes: under the condition that the database is failed to be corrected, checking all data in the database at the end of the day based on a day-end reconciliation consistency guarantee mechanism; if partial database is inconsistent, modifying the abnormal data in the database to ensure the global transaction of all the databases to be consistent.
In an embodiment of the present application, the checking of all the data in the database at the end of the day includes: and checking all the data in the database at the end of the day based on a stroke-by-stroke checking principle.
In an embodiment of the present application, the real-time compensation mechanism is a common compensation mechanism. The common compensation mechanism means that the real-time compensation mechanism in the scheme is a common mechanism and is suitable for different distributed transactions. And a large amount of control and processing codes related to distributed transactions do not need to be developed in a business system. The mechanism of the scheme generally only needs one line of code, and a common compensation mechanism is called. The mode avoids the invasion of a large amount of codes of a TCC mechanism to a business system, reduces the probability of development errors and has low invasion to the business system. Because a large amount of distributed transaction processing codes are not needed in the service system, the development difficulty and the development cost are greatly reduced. The real-time compensation mechanism is a public service and only needs to be developed once, so that the problem that development cost and error probability are increased due to the fact that a TCC (transmission control function) mechanism is dispersed in a plurality of service functions and needs to be developed by a plurality of people is solved. High performance, high real-time performance, low intrusion and low development cost are realized.
The embodiment of the present application further provides a distributed transaction double compensation device based on daily final reconciliation, and it should be noted that the distributed transaction double compensation device based on daily final reconciliation according to the embodiment of the present application can be used for executing the distributed transaction double compensation method based on daily final reconciliation provided by the embodiment of the present application. The distributed transaction double compensation device based on the daily-end reconciliation provided by the embodiment of the application is described below.
Fig. 2 is a schematic diagram of a distributed transaction double compensation device based on a daily end-to-end reconciliation according to an embodiment of the application. As shown in fig. 2, the apparatus includes:
the processing unit 10 is configured to perform normal processing on each transaction according to the service processing flow, without considering the abnormal condition of the transaction;
a submitting unit 20, configured to submit the database corresponding to the transaction in real time after the transaction is normally processed;
a first compensation unit 30, configured to, if an exception occurs during processing of a predetermined transaction, perform real-time compensation on the database corresponding to each transaction before the predetermined transaction by using a real-time compensation mechanism;
and the second compensation unit 40 is configured to perform daily and final reconciliation compensation if the global transaction consistency principle is not satisfied after the real-time compensation.
Specifically, for example, if an exception occurs during processing of the third transaction, the database corresponding to the first transaction and the database corresponding to the second transaction are compensated in real time.
Specifically, the daily final reconciliation compensation refers to correcting the database which does not meet the global transaction consistency principle based on the daily final reconciliation principle.
In the scheme, the processing unit normally processes each transaction according to a business processing flow, the submitting unit submits the database corresponding to the transaction in real time after normal processing, if the database corresponding to each transaction before the scheduled transaction is processed is subjected to real-time compensation in the normal processing process under the abnormal condition, if the global transaction consistency principle is not met after the real-time compensation, the second compensating unit performs daily and final account checking compensation, and the consistency of the databases is ensured through a double compensation mechanism of the real-time compensation and the daily and final account checking compensation. The balance of transaction real-time and consistency is realized. The method has the advantages of high efficiency, transaction instantaneity and low service code invasion, and can normally process each transaction according to the service processing flow, and avoid the invasion of the service code without considering the abnormal condition of the transaction. The scheme avoids the time-consuming and low-efficiency actions of synchronous coordination submitted by a 2PC two-stage system, and ensures high efficiency; the real-time compensation action in the double compensation mechanism is to compensate in time, and ensure the cross-database (namely cross-database) distributed transaction to be reflected in real time to the maximum extent. The problem of insufficient real-time performance of the existing MQ-based transaction mechanism is solved.
In an embodiment of the application, the first compensation unit is further configured to modify the database corresponding to each of the transactions before the predetermined transaction into a state before commit.
In an embodiment of the present application, the second compensation unit includes a checking module and a modifying module, and the checking module is configured to check all data in the database at the end of the day based on a day-to-day account checking consistency guarantee mechanism when the database is failed to be corrected; the modification module is used for modifying the abnormal data in the database if part of the database is inconsistent so as to ensure that all the global transactions of the database are consistent.
In an embodiment of the application, the checking module is further configured to check all data in the database at the end of the day based on a rule of checking one by one.
The distributed transaction double compensation device based on the daily terminal reconciliation comprises a processor and a memory, wherein the processing unit, the submitting unit, the first compensation unit, the second compensation unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the problems of low transaction real-time performance, high service code invasion, high development cost and the like in the distributed transaction solution are solved by adjusting kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The embodiment of the invention provides a computer-readable storage medium, which comprises a stored program, wherein when the program runs, a device where the computer-readable storage medium is located is controlled to execute the distributed transaction double compensation method based on the daily terminal reconciliation.
The embodiment of the invention provides a processor, which is used for running a program, wherein the distributed transaction double compensation method based on daily terminal reconciliation is executed when the program runs.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein when the processor executes the program, at least the following steps are realized:
step S101, performing normal processing on each transaction according to a business processing flow, and not considering the abnormal condition of the transaction;
step S102, after the transaction is normally processed, submitting a database corresponding to the transaction in real time;
step S103, if an exception occurs during processing of a predetermined transaction, a real-time compensation mechanism is adopted to perform real-time compensation on the database corresponding to each transaction before the predetermined transaction;
and step S104, if the global transaction consistency principle is not met after the real-time compensation, performing daily and final account checking compensation.
The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides a computer program product adapted to perform a program of initializing at least the following method steps when executed on a data processing device:
step S101, performing normal processing on each transaction according to a business processing flow, and not considering the abnormal condition of the transaction;
step S102, after the transaction is normally processed, submitting a database corresponding to the transaction in real time;
step S103, if an exception occurs during processing of a predetermined transaction, a real-time compensation mechanism is adopted to perform real-time compensation on the database corresponding to each transaction before the predetermined transaction;
and step S104, if the global transaction consistency principle is not met after the real-time compensation, performing daily and final account checking compensation.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
Examples
The embodiment relates to a specific distributed transaction double compensation method based on daily terminal reconciliation, and a flow chart of the method is shown in fig. 3. The method mainly comprises three steps:
the method comprises the following steps: and normally processing, namely, according to an optimistic principle, the business logic gradually performs normal processing without excessively considering transaction exception, so that the code is prevented from being invaded. In the actual environment, most services can be processed successfully, so that most services only need to be processed normally without compensation action, and the overall performance of the service system is improved.
Step two: and (4) compensating in real time, and if an exception is encountered in the normal processing process, rolling back across the whole database. For example, in fig. 3, the whole business process flow involves 4 transactions and 4 databases. If transactions 1 and 2 have completed, completion has been committed in database A and database B. But when transaction 3 is processed, an exception is encountered requiring a rollback of the whole (i.e. the data in database a and database B needs to be corrected to the state before commit). At this time, the service system only needs to send out the real-time compensation notification. And after receiving the compensation notification, the public real-time compensation mechanism performs reverse database compensation operation (such as data reverse operation) on the data in the database A and the database B according to the relevant information, so as to recover the submitted key data to the state before submission. Real-time compensation attempts as successful compensation as possible. Usually real-time compensation is successful in most cases, i.e. the transaction takes effect in real-time.
Step three: the daily final account checking compensation and the real-time compensation can be successful under most conditions, but a small amount of occasional abnormity or errors still exist and fail. Therefore, by means of a daily final reconciliation consistency guarantee mechanism, the daily final reconciliation compensation can check the data in all databases at the end of the day and check the data one by one, and if the transactions are not consistent, the final consistency compensation mechanism is carried out according to the relevant information, so that the transactions are consistent across the databases by modifying the data. Therefore, the bottom-of-pocket mechanism of the final consistency of the transaction is realized through daily account checking compensation.
According to the scheme, the balance of real-time performance and consistency of the transaction is realized through a double compensation mechanism of real-time compensation and daily final account checking compensation.
The double compensation mechanism of the scheme has the characteristics of four aspects:
1. the performance is high. The time-consuming and inefficient actions of synchronous coordination of two-stage submission of 2PC are avoided.
2. The transaction is near real-time. The real-time compensation mechanism in the double compensation mechanism is used for compensating in time, and the cross-library distributed transaction can be reflected in real time to the maximum extent. The problem that the existing MQ-based transaction mechanism is insufficient in real-time is solved.
3. The intrusiveness to the business system is low. And a large amount of control and processing codes related to distributed transactions are not required to be developed in the service system. The mechanism of the scheme generally only needs one line of codes, and a common compensation mechanism (the real-time compensation mechanism in the scheme is the common compensation mechanism) is called. The mode avoids the invasion of a large amount of codes of a TCC mechanism to a business system, and simultaneously reduces the probability of development errors.
4. Development costs. Because a large amount of distributed transaction processing codes are not needed in the service system, the development difficulty and the development cost are greatly reduced. The real-time compensation mechanism is a public service and only needs to be developed once, so that the problem that the TCC mechanism is dispersed in a plurality of service functions and the development cost and the error probability are increased due to the fact that a plurality of persons need to develop the TCC mechanism is avoided.
From the above description, it can be seen that the above-described embodiments of the present application achieve the following technical effects:
1) according to the distributed transaction double-compensation method based on the daily final account checking, each transaction is normally processed according to a business processing flow, the database corresponding to the transaction is submitted in real time after the transaction is normally processed, if the abnormal condition is met in the normal processing process, the database corresponding to each transaction before the scheduled transaction is subjected to real-time compensation, if the global transaction consistency principle is not met after the real-time compensation, the daily final account checking compensation is carried out, and the consistency of the databases is ensured through a double-compensation mechanism of the real-time compensation and the daily final account checking compensation. The balance of transaction real-time and consistency is realized. The method has the advantages of high efficiency, transaction instantaneity and low service code invasion, and can normally process each transaction according to the service processing flow, and avoid the invasion of the service code without considering the abnormal condition of the transaction. The scheme avoids the time-consuming and low-efficiency actions of synchronous coordination submitted by a 2PC two-stage system, and ensures high efficiency; the real-time compensation action in the double compensation mechanism is to compensate in time, and ensure the cross-database (namely cross-database) distributed transaction to be reflected in real time to the maximum extent. The problem of insufficient real-time performance of the existing MQ-based transaction mechanism is solved.
2) The utility model provides a distributed affairs double compensation arrangement based on daily end reconciliation, processing unit carries out normal processing to each affair according to the business processing flow, the unit of submitting is after normal processing, submit the database that the affair corresponds in real time, if at normal processing's in-process, meet under the unusual condition, first compensation unit just carries out real-time compensation to the database that each affair before the predetermined affair of just handling corresponds, still satisfy global affair uniformity principle after the real-time compensation, second compensation unit then carries out daily end reconciliation compensation, guarantee the uniformity of a plurality of databases through the double compensation mechanism of real-time compensation and daily end reconciliation compensation. The balance of transaction real-time and consistency is realized. The method has the advantages of high efficiency, transaction instantaneity and low service code invasion, and can normally process each transaction according to the service processing flow, and avoid the invasion of the service code without considering the abnormal condition of the transaction. The scheme avoids the time-consuming and low-efficiency actions of synchronous coordination submitted by a 2PC two-stage system, and ensures high efficiency; the real-time compensation action in the double compensation mechanism is to compensate in time, and ensure the cross-database (namely cross-database) distributed transaction to be reflected in real time to the maximum extent. The problem of insufficient real-time performance of the existing MQ-based transaction mechanism is solved.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A distributed transaction double compensation method based on daily account checking is characterized by comprising the following steps:
carrying out normal processing on each transaction according to the business processing flow, and not considering the abnormal condition of the transaction;
submitting a database corresponding to the transaction in real time after the transaction is normally processed;
if an exception occurs during processing of a predetermined transaction, a real-time compensation mechanism is adopted to perform real-time compensation on the database corresponding to each transaction before the predetermined transaction;
and if the global transaction consistency principle is not met after the real-time compensation, performing daily and final account checking compensation.
2. The method of claim 1, wherein compensating the database corresponding to each of the transactions prior to the predetermined transaction in real time comprises:
and correcting the database corresponding to each transaction before the predetermined transaction into a state before submission.
3. The method of claim 2, wherein if the global transaction consistency principle is not satisfied after the real-time compensation, performing an end-of-day reconciliation compensation comprises:
under the condition that the database is failed to be corrected, checking all data in the database at the end of the day based on a day-end reconciliation consistency guarantee mechanism;
if partial database is inconsistent, modifying the abnormal data in the database to ensure the global transactions of all the databases to be consistent.
4. The method of claim 3, wherein checking all of the data in the database at the end of the day comprises:
and checking all data in the database at the end of the day based on a stroke-by-stroke checking principle.
5. The method according to any of claims 1 to 4, wherein the real-time compensation mechanism is a common compensation mechanism.
6. A distributed transaction double compensation device based on daily end reconciliation is characterized by comprising:
the processing unit is used for carrying out normal processing on each transaction according to the business processing flow, and the abnormal condition of the transaction is not considered;
the submitting unit is used for submitting the database corresponding to the transaction in real time after the transaction is normally processed;
the first compensation unit is used for compensating the database corresponding to each transaction before the predetermined transaction in real time by adopting a real-time compensation mechanism if the predetermined transaction is processed and an abnormality is encountered;
and the second compensation unit is used for performing daily and final account checking compensation if the global transaction consistency principle is not met after the real-time compensation.
7. The apparatus according to claim 6, wherein the first compensation unit is further configured to modify the database corresponding to each of the transactions before the predetermined transaction to a state before commit.
8. The apparatus of claim 6, wherein the second compensation unit comprises a verification module and a modification module,
the checking module is used for checking all data in the database at the end of the day based on a day-end account checking consistency guarantee mechanism under the condition that the database is unsuccessfully corrected;
the modification module is used for modifying the abnormal data in the database if part of the database is inconsistent so as to ensure that all the global transactions of the database are consistent.
9. A computer-readable storage medium, comprising a stored program, wherein when the program runs, the computer-readable storage medium controls a device to execute the method for dual compensation of distributed transactions based on daily end-tie account according to any one of claims 1 to 5.
10. A processor, configured to execute a program, wherein the program executes the method for dual compensation of distributed transactions based on daily end-tie account according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110310873.2A CN113032176A (en) | 2021-03-23 | 2021-03-23 | Distributed transaction double-compensation method and device based on daily account checking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110310873.2A CN113032176A (en) | 2021-03-23 | 2021-03-23 | Distributed transaction double-compensation method and device based on daily account checking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113032176A true CN113032176A (en) | 2021-06-25 |
Family
ID=76473696
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110310873.2A Pending CN113032176A (en) | 2021-03-23 | 2021-03-23 | Distributed transaction double-compensation method and device based on daily account checking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113032176A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114510495A (en) * | 2022-04-21 | 2022-05-17 | 北京安华金和科技有限公司 | Database service data consistency processing method and system |
| CN116737465A (en) * | 2023-08-11 | 2023-09-12 | 梅州客商银行股份有限公司 | Test method for checking account and processing errors of bank payment system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105279693A (en) * | 2015-11-13 | 2016-01-27 | 中国建设银行股份有限公司 | Method for maintaining interbank transaction accounting consistency and system thereof |
| CN108459919A (en) * | 2018-03-29 | 2018-08-28 | 中信百信银行股份有限公司 | A kind of distributed transaction processing method and device |
| CN109120706A (en) * | 2018-08-28 | 2019-01-01 | 中国邮政储蓄银行股份有限公司 | Business scheduling method and system |
| CN111277639A (en) * | 2020-01-16 | 2020-06-12 | 中国建设银行股份有限公司 | Method and device for maintaining data consistency |
| CN111813791A (en) * | 2020-06-17 | 2020-10-23 | 上海悦易网络信息技术有限公司 | Method and equipment for distributed transaction compensation |
-
2021
- 2021-03-23 CN CN202110310873.2A patent/CN113032176A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105279693A (en) * | 2015-11-13 | 2016-01-27 | 中国建设银行股份有限公司 | Method for maintaining interbank transaction accounting consistency and system thereof |
| CN108459919A (en) * | 2018-03-29 | 2018-08-28 | 中信百信银行股份有限公司 | A kind of distributed transaction processing method and device |
| CN109120706A (en) * | 2018-08-28 | 2019-01-01 | 中国邮政储蓄银行股份有限公司 | Business scheduling method and system |
| CN111277639A (en) * | 2020-01-16 | 2020-06-12 | 中国建设银行股份有限公司 | Method and device for maintaining data consistency |
| CN111813791A (en) * | 2020-06-17 | 2020-10-23 | 上海悦易网络信息技术有限公司 | Method and equipment for distributed transaction compensation |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114510495A (en) * | 2022-04-21 | 2022-05-17 | 北京安华金和科技有限公司 | Database service data consistency processing method and system |
| CN114510495B (en) * | 2022-04-21 | 2022-07-08 | 北京安华金和科技有限公司 | Database service data consistency processing method and system |
| CN116737465A (en) * | 2023-08-11 | 2023-09-12 | 梅州客商银行股份有限公司 | Test method for checking account and processing errors of bank payment system |
| CN116737465B (en) * | 2023-08-11 | 2024-04-02 | 梅州客商银行股份有限公司 | Test method for checking account and processing errors of bank payment system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111666162B (en) | Distributed message transmission method, device, computer equipment and storage medium | |
| US11321303B2 (en) | Conflict resolution for multi-master distributed databases | |
| US9805104B2 (en) | Recovery point objective enforcement | |
| US8135987B2 (en) | Collection ordering for replicated state machines | |
| US8301600B1 (en) | Failover recovery in a distributed data store | |
| CN107016016B (en) | Data processing method and device | |
| CN113032176A (en) | Distributed transaction double-compensation method and device based on daily account checking | |
| CN109347669B (en) | Fault processing method and device for batch operation between bank systems | |
| CN106503974B (en) | Transaction data processing method and device | |
| CN112286661B (en) | Task scheduling method and device, storage medium and terminal | |
| US20230110826A1 (en) | Log execution method and apparatus, computer device and storage medium | |
| US11921745B2 (en) | Preventing data loss in event driven continuous availability systems | |
| CN112181723A (en) | Financial disaster recovery method and device, storage medium and electronic equipment | |
| WO2020258653A1 (en) | Cross-node data processing method and apparatus | |
| US20210218827A1 (en) | Methods, devices and systems for non-disruptive upgrades to a replicated state machine in a distributed computing environment | |
| CN107194712B (en) | Method and device for recording change information of shared account and method and system for supplementing account of internal account | |
| US9218386B2 (en) | Dual locking mechanism for a domain | |
| CN108650294B (en) | Method and device for transmitting transaction information between technical systems | |
| CN115208834A (en) | Service flow limiting method based on database storage process design | |
| CN112181686A (en) | Data processing method, device and system, electronic equipment and storage medium | |
| CN114116676A (en) | Data migration method and device, electronic equipment and computer readable storage medium | |
| CN111666132A (en) | Distributed transaction implementation method, device, computer system and readable storage medium | |
| CN115965480A (en) | Distributed account checking method, device and system | |
| CN116383893A (en) | Block chain data statistics method and system | |
| CN116599827A (en) | Data synchronization method, equipment and storage medium for dual-active disaster recovery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |