CN112783612A

CN112783612A - Distributed transaction processing system of sharing platform

Info

Publication number: CN112783612A
Application number: CN201911075779.2A
Authority: CN
Inventors: 张才俊; 张静; 马永波; 袁葆; 吴佐平; 欧阳红; 高士杰; 申蕾; 王晨飞; 王雨; 杨维
Original assignee: State Grid Corp of China SGCC; State Grid Information and Telecommunication Co Ltd; Beijing China Power Information Technology Co Ltd
Current assignee: State Grid Corp of China SGCC; State Grid Information and Telecommunication Co Ltd; Beijing China Power Information Technology Co Ltd
Priority date: 2019-11-06
Filing date: 2019-11-06
Publication date: 2021-05-11

Abstract

The invention discloses a distributed transaction processing system of a sharing platform, which comprises: MQ sender, MQ server and MQ subscriber, wherein: the MQ sender sends a message to the MQ server; the MQ server performs persistence processing on the message, and sends a first confirmation message to the MQ sender after the persistence is completed; after receiving the first confirmation message, the MQ sender executes a local transaction and sends a second confirmation message to the MQ server according to an execution result of the local transaction; and the MQ service terminal determines whether to send the first confirmation message to the MQ subscriber according to the second confirmation message. In the processing system, the transaction is processed by adopting the MQ transaction message function, resources do not need to be locked, and the overall throughput rate and performance of the database are improved.

Description

Distributed transaction processing system of sharing platform

Technical Field

The invention relates to the technical field of electric power marketing, in particular to a distributed transaction processing system of a shared platform.

Background

In an application scenario of a power marketing shared service center, there are many complex scenarios in which a database is operated across service calls, written across the database, and MQ messages are written at the same time. For example, after a new transaction is added in the transaction center, the order center service is called to add a piece of order data to the data on both sides of the service, so as to ensure consistency, for example, when a sub-database sub-table is used, the tables of different databases are operated at the same time, and the consistency of the transaction is ensured.

The traditional distributed transaction solution is to use a two-phase commit protocol. Stage one: begin sending commit information to all resources involved in the transaction. At this point, the resources involved in the transaction have the last chance to abort the transaction. If any resource determines to abnormally end the transaction, the whole transaction is cancelled, and the resource is not updated. Otherwise, the transaction will execute normally unless a catastrophic failure occurs. To prevent catastrophic failures, all resource updates are written to the log. These logs are persistent, so they survive the trouble and all resources can be updated again after a failure. And a second stage: this will only occur as soon as the phase has not ended abnormally. At this point, all resource managers that can be located and individually controlled will begin to perform the true data update. Resources need to be locked in the transaction submission process, and the overall throughput rate and performance of the database are affected.

Disclosure of Invention

In view of this, the present invention provides a distributed transaction processing system with a shared platform, which is used to solve the problem in the prior art that a two-phase commit protocol is adopted in the process of distributed transaction processing, so that resources need to be locked in the process of transaction commit, and the overall throughput and performance of a database are affected, and the specific scheme is as follows:

a shared platform distributed transaction processing system comprising: MQ sender, MQ server and MQ subscriber, wherein:

the MQ sender sends a message to the MQ server;

the MQ server performs persistence processing on the message, and sends a first confirmation message to the MQ sender after the persistence is completed;

after receiving the first confirmation message, the MQ sender executes a local transaction and sends a second confirmation message to the MQ server according to an execution result of the local transaction;

and the MQ service terminal determines whether to send the first confirmation message to the MQ subscriber according to the second confirmation message.

Optionally, in the system described above, the performing, by the MQ server, persistent processing on the message includes:

receiving the message;

and storing the message in a storage device of the MQ service end.

In the above system, optionally, the first acknowledgment message is a half message.

Optionally, in the above system, the determining, by the MQ server, whether to send the first acknowledgment message to the MQ subscriber according to the second acknowledgment message includes:

identifying a state of the second acknowledgement information;

if the state is a preset first state, the first confirmation message is sent to the MQ subscriber;

and if the state is a preset second state, deleting the first confirmation information.

The above system, optionally, further includes:

and if the state is a preset first state, marking the first confirmation message as deliverable.

The above system, optionally, further includes:

if the processing system is abnormal, judging whether the second confirmation message is sent to the MQ server side;

and if so, the MQ service end determines whether to send the first confirmation message to the MQ subscriber according to the second confirmation message.

The above system, optionally, further includes:

if not, after a preset time interval, the MQ service end initiates a message retrieval request to the second confirmation message.

The above system, optionally, further includes:

the MQ sender receives the message review request;

verifying the execution result of the local transaction to obtain a third confirmation message;

and the MQ sender sends the third confirmation message to the MQ server.

The above system, optionally, further includes:

and the MQ service terminal determines whether to send the first confirmation message to the MQ subscriber according to the third confirmation message.

In the above system, optionally, the third confirmation information includes: a preset first state or a preset second state.

Compared with the prior art, the invention has the following advantages:

Of course, it is not necessary for any product in which the invention is practiced to achieve all of the above-described advantages at the same time.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a block diagram of a distributed transaction processing system with a shared platform according to an embodiment of the present disclosure;

fig. 2 is an execution diagram of a distributed transaction processing system with a shared platform according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The invention discloses a sharing platform distributed transaction processing system which is applied to the processing process of distributed things in the power marketing process, wherein a sharing platform refers to the mature application experience of the platform service concept in the reference of the internet, and designs and provides a power marketing sharing service center concept. A transaction refers to a set of operations that are combined into a logical unit of work, and although errors may occur in the system, the transaction will control and maintain the consistency and integrity of each operation in the transaction. For example, in a banking application where funds are transferred from one account to another, one account credits an amount to one database table while another account debits the same amount to another database table. Because a computer may fail due to a power outage, network outage, etc., it is possible that a row in one table is updated, but not another table. If the database supports transactions, the database operations may be grouped into one transaction to prevent inconsistencies in the database due to these events. If a failure occurs at some point in the transaction, all updates may be rolled back to the state before the transaction started. If no failure has occurred, the update is completed by committing the transaction in a completion state. Distributed transaction means that with the popularization of micro service architecture, a large service system is often composed of a plurality of subsystems, and the subsystems have independent databases. Often a business process needs to be performed by multiple subsystems together, and these operations may need to be performed in one transaction. In microservice systems, these business scenarios are ubiquitous. At this time, it is necessary to implement transaction support across databases by some means on top of the databases, which is a distributed transaction.

In the embodiment of the invention, a plurality of complex scenes of cross-service calling operation of the database, cross-database writing and MQ message writing exist in the application scene of the power marketing sharing service center. For example, after a new transaction is added in the transaction center, the order center service is called to add a piece of order data to the data on both sides of the service, so as to ensure consistency, for example, when a sub-database sub-table is used, the tables of different databases are operated at the same time, and the consistency of the transaction is ensured. The above scenario, especially when transactions are mixed together, makes the processing of transactions more complicated. In view of the above problems, the present invention provides a shared platform distributed transaction processing system, which guarantees the final consistency of distributed transactions for MQ message queues. The MQ message queue is a communication method of an application program to the application program, and the application program is transmitted between the application programs through messages. The processing system based on the MQ message queue comprises: the structure block diagram of the process is shown in fig. 1. Wherein:

the MQ sender is connected with the MQ server and is connected with local transactions;

the MQ server is connected with the subscriber.

The execution flow of the processing system is shown in fig. 2, and includes the steps of:

s101, the MQ sender sends a message to the MQ server;

in the embodiment of the invention, MQ provides a distributed transaction function similar to X/Open XA, the final consistency of distributed things can be achieved through MQ transaction messages, and the problem of transaction consistency when asynchronous calls among shared platform services are decoupled is solved. The MQ sender sends a message to the MQ server.

S102, the MQ server performs persistence processing on the message, and sends a first confirmation message to the MQ sender after the persistence is completed;

in the embodiment of the invention, after the MQ server receives the message, the message is subjected to persistence processing, wherein the persistence processing is necessarily stored on a physical medium and cannot be stored in a memory. Persistence (Persistence), which is the saving of data (e.g., objects in memory) to a storage device (e.g., disk) that can be persisted. The main application of persistence is to store objects in memory in a relational database, which may be a disk file, an XML data file, or the like.

For example, sessions Cookies ViewState Application database, Cookies are divided into two types, the temporary Cookie browser is closed and then is finished, and permanent Cookies are written in the client side, so that the method is a persistent mode; sessions are divided into three types, neither InProc nor StateServer is a persistence mechanism, and only writes to a database, but the sessions for writing to the database are not necessarily used in practical application; the Application IIS may not be available, nor is it a persistent way,

and after the persistence is finished, sending a first acknowledgement message to the MQ, wherein the preferred first acknowledgement message is an acknowledgement character ACK (acknowledgement character), and the ACK receiving station sends a transmission type control character of a sending station. Indicating that the transmitted data is received without errors.

Further, the first acknowledgment message is now a half message, which is a special state of the message, and has been saved to the MQ server, but the MQ subscriber cannot perceive the first acknowledgment message.

S103, after receiving the first confirmation message, the MQ sender executes a local transaction and sends a second confirmation message to the MQ server according to an execution result of the local transaction;

in the embodiment of the invention, after the MQ sender receives the first confirmation message, a local transaction is executed, and when the execution of the local transaction is finished, a second confirmation message is sent to the MQ server according to the execution result of the local transaction, wherein the second confirmation message comprises a preset first state or a preset second state.

S104, the MQ service terminal determines whether to send the first confirmation message to the MQ subscriber according to the second confirmation message.

In the embodiment of the invention, when the second confirmation message is received by the MQ server, the state of the second confirmation message is identified, if the state is a preset first state, the first confirmation message is marked as deliverable, and the first confirmation message is sent to the MQ subscriber; and if the state is a preset second state, deleting the first confirmation message, so that the MQ subscriber is not subjected to the first confirmation message. Preferably, the preset first state Commit state and the preset second state Rollback state are set to the first state Commit state and the second state Commit state respectively

In the embodiment of the invention, if the scene of the distributed transaction realized by the message service is abnormal, a forward compensation mode is generally adopted, namely, the transaction link is continuously executed forward by a mode of continuous retry or manual intervention of the message without rolling back in sequence when the traditional transaction mode is abnormal, so that the transaction rolling back is avoided. The abnormal condition may be a network failure or an application restart.

In the embodiment of the invention, the forward compensation process is as follows: and judging whether the second confirmation message is sent to the MQ service end, if so, continuing to execute according to S104, and if not, after a preset time interval, initiating a message retrieval request by the MQ service end for the second confirmation message. Acquiring an ID value included in the second confirmation information, determining an MQ sender receiving the message recall request according to the ID value, and sending the message recall request to the MQ sender, wherein the preset time length may be set according to experience or specific conditions. When the MQ sender receives the message review request, the execution result is checked in the corresponding local transaction, a third confirmation message is determined according to the execution result, and the MQ sender sends the third confirmation message to the MQ server.

In the embodiment of the invention, after the MQ server receives the third confirmation message, the state contained in the third confirmation message is obtained, and if the state is the preset first state, the first confirmation message is sent to the MQ subscriber; and if the state is a preset second state, deleting the first confirmation message, and the MQ subscriber does not receive the first confirmation message.

In the embodiment of the present invention, the processing system is illustrated, and it is assumed that a transfers 100 money to B, and they are not on the same service.

The target is as follows: namely, A minus 100 pieces of money and B plus 100 pieces of money.

There may be four practical cases:

1) namely, account A is subtracted by 100 (success), account B is added by 100 (success)

2) Account A minus 100 (failure) and account B plus 100 (failure)

3) That is, account A minus 100 (success) and account B plus 100 (failure)

4) That is, account A minus 100 (failure) and account B plus 100 (success)

Here, the 1) and 2) cases are capable of guaranteeing the consistency of the transactions, but the 3) and 4) cases are incapable of guaranteeing the consistency of the transactions.

Based on fig. 2, the specific execution flow is as follows:

1. the A service (MQ sender) firstly sends a half message to the Brock terminal (MQ server), and the message carries the information about +100 elements about the B service (MQ subscriber).

2. When the A service (MQ sender) knows that the message is successfully sent, then step 3 execution of the local transaction is started.

3. Local transaction execution (there are three cases 1, successful execution.2, failure execution.3, network, etc. cause no response)

4.1), if the local transaction is successful, the Product sends Commit like the Brock server so that the B-service (MQ subscriber) can consume the message.

4.2), if the local transaction fails, the Product sends Rollback like the Brock end (MQ server), and the half message is directly deleted.

4.3), if failure or success is not returned late because of network, etc., then the callback interface of the rockmq is executed to perform a transaction lookback.

From the above flow, it can be known that the B service can only consume the message if the a service performs the local transaction successfully.

There are two reasons why the transmission (half message) is made: whether the Brock end (MQ service end) is normal or not can be confirmed firstly, and if the half messages are all sent in failure, the Brock end (MQ service end) hangs up. The transaction can be checked back through the half message, and if the half message is sent successfully and is not confirmed for the second time, the transaction state is checked back.

The reason why the message review is performed is that: when the local transaction is executed, if the result (commit or rolback) of the executed transaction is not returned all the time due to a sudden network or the like, and the unknown is finally returned, the backhunting is performed. After the local transaction is successfully executed, the service is hung when the Commit is returned for Message secondary confirmation, and when the service is restarted, the service is also called a Half Message at the cache end (MQ service end).

It is particularly noted that if the local transaction is checked back, the execution condition of the current transaction must be checked first, and then whether the local transaction needs to be re-executed is checked.

With the above process, account a minus 100 (failure) and account B plus 100 (success) never occur because: if both A-services (MQ sender) local transactions fail, then the B-services (MQ subscribers) never perform any action because the message is not passed to the B-services (MQ subscribers) at the root.

Then a case of a minus 100 (success) for the a account, a plus 100 (failure) for the B account is possible because the a service (MQ sender) is only responsible for ensuring that the message can be sent to B when the execution of my message is successful, as far as the final execution result a after the message is received by the B service (MQ subscriber) is not concerned.

If B finally fails, it can almost be concluded that it is the code that has a problem and therefore an exception is caused, because the consumer rockmq has a retry mechanism, which will generally be successful if not a code problem, if retried several times. If the code causes repeated retry failure, the exception is recorded, and the transaction can reach final consistency after manual processing and manual bottom-binding processing.

In the embodiment of the invention, the transaction asynchronization mode is carried out through the message, the synchronous execution success or failure of the two database transactions before and after the transaction is ensured, the consistency of the transaction is kept, and meanwhile, the long-time resource locking of the data in the traditional two-stage transaction submitting mode is avoided, so the overall throughput rate and performance of the database greatly exceed the traditional distributed transaction mode. If the scene of the distributed transaction realized by the message service is abnormal, a forward compensation mode is generally adopted, that is, the transaction link is continuously executed forward by means of continuous retry or manual intervention of the message, and the transaction rollback is avoided, unlike the conventional transaction mode in which the rollback is performed in sequence when the abnormality occurs.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the units may be implemented in the same software and/or hardware or in a plurality of software and/or hardware when implementing the invention.

From the above description of the embodiments, it is clear to those skilled in the art that the present invention can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

The above detailed description is made on a shared platform distributed transaction processing system provided by the present invention, and a specific example is applied in the present document to explain the principle and the implementation of the present invention, and the above description of the embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A shared platform distributed transaction processing system, comprising: MQ sender, MQ server and MQ subscriber, wherein:

the MQ sender sends a message to the MQ server;

2. The system of claim 1, wherein the MQ server persists the message, comprising:

receiving the message;

and storing the message in a storage device of the MQ service end.

3. The system of claim 1, wherein the first acknowledgement message is a half message.

4. The system as claimed in claim 1, wherein the MQ server determining whether to send the first acknowledgment message to the MQ subscriber based on the second acknowledgment message comprises:

identifying a state of the second acknowledgement information;

5. The system of claim 4, further comprising:

6. The system of claim 1, further comprising:

7. The system of claim 6, further comprising:

8. The system of claim 7, further comprising:

the MQ sender receives the message review request;

and the MQ sender sends the third confirmation message to the MQ server.

9. The system of claim 8, further comprising:

10. The system of claim 9, wherein the third confirmation information comprises: a preset first state or a preset second state.