CN112463810A - Data processing method, device, equipment and storage medium based on distributed transaction - Google Patents
Data processing method, device, equipment and storage medium based on distributed transaction Download PDFInfo
- Publication number
- CN112463810A CN112463810A CN202011461774.6A CN202011461774A CN112463810A CN 112463810 A CN112463810 A CN 112463810A CN 202011461774 A CN202011461774 A CN 202011461774A CN 112463810 A CN112463810 A CN 112463810A
- Authority
- CN
- China
- Prior art keywords
- transaction
- information
- interface
- data processing
- remote
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 238000000034 method Methods 0.000 claims abstract description 84
- 238000012545 processing Methods 0.000 claims abstract description 45
- 230000001360 synchronised effect Effects 0.000 claims abstract description 26
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000002372 labelling Methods 0.000 claims description 7
- 238000012805 post-processing Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000002955 isolation Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 12
- 230000000903 blocking effect Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 8
- 230000008878 coupling Effects 0.000 description 4
- 238000010168 coupling process Methods 0.000 description 4
- 238000005859 coupling reaction Methods 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The application provides a data processing method, a device, equipment and a storage medium based on distributed transactions, and relates to the technical field of data processing. The method comprises the following steps: acquiring transaction information sent by a client; analyzing and acquiring corresponding interface marking information according to the transaction information, wherein the interface marking information is used for indicating a transaction entrance; acquiring corresponding agent marking information according to the interface marking information; and acquiring remote information to be synchronized according to the agent marking information. Compared with the prior art, the problem of synchronous blocking in the execution process of the distributed solution is avoided.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method, apparatus, device, and storage medium based on distributed transactions.
Background
The single application architecture mixing all services together plays a role in the initial stage of informatization construction, along with the fact that the service scale is larger and larger, the logic is more and more complex, the deployment cost and the change risk of the single application architecture are higher and higher, and the demand change cannot be responded quickly. Therefore, it is a trend to build application systems featuring microservice architectures.
However, when a micro-service architecture is selected to solve the rapid response requirement change, the application is split into a large number of mutually independent service units according to the service module, and multiple remote calls among services are involved when one user request is completed, so that how to ensure the data consistency of multi-data-source write operation generated by remote procedure call among services becomes an urgent problem to be solved. Existing distributed transaction solutions are typically two-phase and three-phase commit methods.
However, in the distributed solution in the prior art, in the whole execution process, all the participants in all the stages are transaction-blocking type, and other participants accessing the common resource will be in a blocking waiting state, which seriously reduces the processing performance of the system, and all the participants have the defect of synchronous blocking.
Disclosure of Invention
An object of the present application is to provide a data processing method, apparatus, device and storage medium based on distributed transaction, so as to solve the problem of synchronization blocking in distributed transaction processing in the prior art.
In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:
in a first aspect, an embodiment of the present application provides a data processing method based on distributed transactions, where the method includes:
acquiring transaction information sent by a client;
analyzing and acquiring corresponding interface marking information according to the transaction information, wherein the interface marking information is used for indicating a transaction entrance;
acquiring corresponding agent marking information according to the interface marking information;
and acquiring remote information to be synchronized according to the agent marking information.
Optionally, the interface annotation information includes one or more of the following: propagation characteristics of the transaction, isolation level of the transaction, read-only properties of the transaction;
the agent annotation information comprises one or more of the following items: a provider identification of the service, a user identification of the service.
Optionally, before the obtaining the transaction information sent by the client, the method further includes:
obtaining annotation operation aiming at a remote call interface, and writing the agent annotation information in the remote call interface;
and responding to the method editing operation of the transaction call, and writing the interface marking information in the method program of the transaction call.
Optionally, the acquiring the transaction information sent by the client includes:
receiving the incomplete transaction information sent by the client, and locally storing the incomplete transaction information.
Optionally, the remote information to be synchronized includes a remote called method: interface full-class name, method name, parameter type, parameter name and method return type;
after the remote information to be synchronized is obtained according to the agent annotation information, the method further includes:
generating a globally unique main transaction identifier and a globally unique slave transaction identifier according to the remote information to be synchronized and the agent marking information, and storing the incomplete transaction information to a thread context environment;
executing a post-processing flow corresponding to the uncompleted transaction according to the thread context environment, and updating the state of the uncompleted transaction information into transaction completion information after the execution is completed;
and writing the transaction completion information into a remote transaction log record and a local transaction log record.
Optionally, the method further comprises:
and polling the remote transaction log record and the local transaction log record according to a preset period to finish data synchronization.
Optionally, the receiving the incomplete transaction information sent by the client, and after locally storing the incomplete transaction information, further includes:
acquiring transaction completion information which is transmitted by a client and is based on the same transaction with the incomplete transaction information;
and verifying the validity of the transaction completion information according to the incomplete transaction information.
In a second aspect, another embodiment of the present application provides a data processing apparatus based on distributed transactions, including: an acquisition module and an analysis module, wherein:
the acquisition module is used for acquiring the transaction information sent by the client;
the analysis module is used for analyzing and acquiring corresponding interface marking information according to the transaction information, wherein the interface marking information is used for indicating a transaction entrance;
the acquiring module is specifically configured to acquire corresponding agent labeling information according to the interface labeling information; and acquiring remote information to be synchronized according to the agent marking information.
Optionally, the apparatus further comprises: a write module, wherein:
the obtaining module is specifically configured to obtain an annotation operation for a remote call interface, and write the proxy annotation information in the remote call interface;
and the writing module is used for responding to the method editing operation called by the transaction and writing the interface marking information in the method program called by the transaction.
Optionally, the remote information to be synchronized includes a remote called method: interface full-class name, method name, parameter type, parameter name and method return type; the device further comprises: and the storage module is used for receiving the incomplete transaction information sent by the client and locally storing the incomplete transaction information.
Optionally, the apparatus further comprises: a generation module and an update module, wherein:
the generating module is used for generating a globally unique main transaction identifier and a globally unique slave transaction identifier according to the remote information to be synchronized and the agent marking information, and storing the incomplete transaction information to a thread context environment;
the updating module is used for executing the post-processing flow corresponding to the uncompleted transaction according to the thread context environment, and updating the state of the uncompleted transaction information into transaction completion information after the execution is completed;
the writing module is specifically configured to write the transaction completion information into a remote transaction log record and a local transaction log record.
Optionally, the apparatus further comprises: and the synchronization module is used for polling the remote transaction log record and the local transaction log record according to a preset period to finish data synchronization.
Optionally, the apparatus further comprises: a verification module, wherein:
the obtaining module is specifically configured to obtain transaction completion information that is transmitted by the client and based on the same transaction as the incomplete transaction information.
And the verification module is used for verifying the validity of the transaction completion information according to the incomplete transaction information.
In a third aspect, another embodiment of the present application provides a data processing device based on distributed transactions, including: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the distributed transaction based data processing apparatus is operating, the processor executing the machine-readable instructions to perform the steps of the method according to any one of the first aspect.
In a fourth aspect, another embodiment of the present application provides a storage medium having a computer program stored thereon, where the computer program is executed by a processor to perform the steps of the method according to any one of the above first aspects.
The beneficial effect of this application is: by adopting the data processing method based on the distributed transaction, after the transaction information sent by the client is obtained, the corresponding interface marking information is obtained according to the analysis of the transaction information in a mode of marking the interface marking information in advance, the corresponding agent marking information is obtained according to the interface marking information, and the remote information to be synchronized is obtained according to the agent marking information.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a schematic flowchart of a data processing method based on distributed transactions according to an embodiment of the present application;
FIG. 2 is a flow diagram illustrating a data processing framework building process for distributed transactions according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a data processing method based on distributed transactions according to another embodiment of the present application;
fig. 4 is a schematic flowchart of a data processing method based on distributed transactions according to another embodiment of the present application;
fig. 5 is a schematic structural diagram of a distributed transaction based data processing apparatus according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a distributed transaction based data processing apparatus according to another embodiment of the present application;
fig. 7 is a schematic structural diagram of a data processing device based on distributed transactions according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments.
The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
Additionally, the flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In an embodiment of the present application, the data processing method based on distributed transaction can be applied to a distributed transaction processing scenario of an application program including a client and an application, where interface label information corresponding to the client is proxy label information of a called party, and interface label information corresponding to the application is label information of the called party, and is used to indicate a transaction entry, and such a setting manner enables the client and the application to use the method provided by the present application directly by labeling an interface, and each step in the method provided by the present application is asynchronous processing, thereby avoiding the problems of synchronous blocking and data inconsistency; it should be understood that the foregoing scenarios are only exemplary names, and the method provided by the present application may also be applied to a distributed transaction processing scenario on a web page or an applet that includes a client and an application, where a specific application scenario may be flexibly adjusted according to a user requirement, and is not limited to the foregoing embodiments.
The following explains a data processing method based on distributed transactions provided in the embodiments of the present application with reference to a plurality of specific application examples. Fig. 1 is a flowchart of a data processing method based on a distributed transaction according to an embodiment of the present application, and is applied to a preset distributed transaction manager, where the preset distributed transaction manager may be constructed based on a spring framework, and the preset distributed transaction manager may include: the system comprises a distributed transaction message receiver, a distributed transaction message executor, a distributed transaction execution result processor and a distributed transaction automatic compensation mechanism functional module. As shown in fig. 1, the method includes:
s101: and acquiring the transaction information sent by the client.
Illustratively, in one embodiment of the present application, for example, the incomplete transaction information (prefix message) sent by the client may be received and stored locally.
The transaction information of the client may be, for example, initiated by a user of the client after logging in an application program on the client, for example, after the user logs in a shopping application program on the client, if the user clicks on a shopping application program for purchase, the initiated transaction information is the purchase transaction information.
S102: and analyzing and acquiring corresponding interface marking information according to the transaction information.
The interface marking information is used for indicating the transaction entry.
Optionally, the interface annotation information may include one or more of the following: propagation characteristics of the transaction, isolation level of the transaction, read-only properties of the transaction; in one embodiment of the present application, annotation interface information may be annotated, for example, by way of a global transaction GlonableTransactual annotation.
S103: and acquiring corresponding agent marking information according to the interface marking information.
The method comprises the steps of intercepting an interface with label information through an interceptor in a spring framework, then obtaining corresponding agent label information according to the interface label information, and then informing a local transaction execution result device to start a corresponding processing process of the distributed transaction.
Optionally, the agent annotation information comprises one or more of: a provider identifier of the service, a user identifier of the service; in an embodiment of the present application, the agent annotation information may be annotated by using a distributed transaction AsyncFeignClient annotation, for example.
The interface marking mode can directly mark the transaction inlet information on the corresponding interface in a mode of marking annotation, and obtain the corresponding agent marking information according to the transaction inlet information, so that the distributed processing can be distributed without compiling codes, the flow is simplified, and the building efficiency is improved.
For example, in an embodiment of the present application, the agent annotation information and the interface annotation information are both written in advance, and the writing manner may be, for example, before S102, obtaining an annotation operation for the remote call interface, and writing the agent annotation information on the remote call interface; and writing interface marking information in the method program called by the transaction in response to the method editing operation called by the transaction.
In an embodiment of the present application, each application program may construct a data processing framework of a distributed transaction corresponding to the application program according to a preset distributed manager, and fig. 2 is a flowchart illustrating a data processing framework construction process of a distributed transaction provided in an embodiment of the present application, and as shown in fig. 2, the construction steps are as follows:
1010: the distributed transaction manager is started.
The distributed transaction manager comprises functions of a distributed transaction message receiver, a distributed transaction message executor, a distributed transaction execution result processor and a distributed transaction automatic compensation mechanism.
1020: and identifying distributed transaction agent marking information on the called remote interface by relying on the class library of the distributed transaction client.
Illustratively, in one embodiment of the present application, the remote interface may be called, for example, through Feign in a spring framework, where the distributed transaction client class library includes functions of recording a local transaction log, a distributed transaction declarative client, a local transaction execution result processor, and a distributed transaction application identifier.
1030: a method of writing a distributed transaction call.
For example, in some possible embodiments, the calling method may call a remote calling method for writing to an operating local database.
1040: the distributed transaction application annotations are marked on methods of the distributed transaction call.
S104: and acquiring remote information to be synchronized according to the agent marking information.
Namely, the business processing flow of the method provided by the application is as follows: after the transaction information of the client is obtained, indicating a distributed transaction inlet through interface marking information so as to realize a loose coupling processing mechanism of the distributed transaction, introducing distributed transaction core processing into the distributed transaction inlet, namely obtaining corresponding agent marking information according to the interface marking information, wherein the service flow of the core processing is to determine an interface marked with AsyncFeign client annotation under an appointed path through analyzing the interface marking information, obtain the agent marking information of the interface, generate an agent for all methods in the interface, and the agent realizes the acquisition of the interface full-class name, the method name, the parameter type, the parameter name and the method return type of the called remote method.
By adopting the data processing method based on the distributed transaction, after the transaction information sent by the client is obtained, the corresponding interface marking information is obtained according to the analysis of the transaction information in a mode of marking the interface marking information in advance, the corresponding agent marking information is obtained according to the interface marking information, and the remote information to be synchronized is obtained according to the agent marking information.
Optionally, on the basis of the foregoing embodiments, an embodiment of the present application may further provide a data processing method based on a distributed transaction, and an implementation process of data processing based on a distributed transaction in the foregoing method is described as follows with reference to the accompanying drawings. Fig. 3 is a schematic flowchart of a data processing method based on distributed transactions according to another embodiment of the present application, where the remote information to be synchronized includes a remote called method: interface full-class name, method name, parameter type, parameter name and method return type; as shown in fig. 3, after S104, the method may further include:
s105: and generating a globally unique main transaction identifier and a globally unique auxiliary transaction identifier according to the remote information to be synchronized and the agent marking information, and storing the incomplete transaction information to the thread context environment.
For example, in an embodiment of the present application, for example, names of service providers of the application and service consumption of the client in the annotation may be obtained, a globally unique master transaction identifier (id) and a slave transaction identifier may be generated, a transaction prepare message corresponding to incomplete transaction information may be constructed, and the message may be stored in the thread context environment, where the message is in a client unconfirmed state.
S106: and executing a post-processing flow corresponding to the incomplete transaction according to the thread context environment, and updating the state of the incomplete transaction information after the execution is finished to be transaction completion information (commit message).
Illustratively, in an embodiment of the present application, the information of the incomplete transaction in the thread context is first sent to the message transaction manager, the annotated propagation characteristics of the transaction, the isolation level of the transaction, and the read-only property of the transaction are then obtained, the properties of the transaction are set in a programmatic manner, and a synchronous post-processing flow of the transaction is registered, where the processing flow includes updating the prepare message state to be confirmed by the consumer, writing the update result into the database, so that the local transaction and the recorded transaction log are in the same transaction, and after the transaction is successfully executed, sending a transaction complete commit message to the transaction manager.
For example, the data structures of the preamble message and the commit message may each include: a master transaction ID, a slave transaction ID, a service provider name, a service consumer name, an interface full class name, a method name, a parameter type, a parameter name, a method return type, a method call parameter value, a message state, and a creation time; the message state of prefix is the state that the consumption end does not confirm, namely transaction information is not finished, and the message state of commit is the state that the consumption end confirms, namely transaction information is finished.
S107: and writing the transaction completion information into the remote transaction log record and the local transaction log record.
For example, in an embodiment of the present application, the data structure of each log record in the local transaction log record may include: the transaction log comprises a unique transaction log identifier, a master transaction ID, a slave transaction ID, a service provider name, a service consumer name, an interface full-class name, a method name, a parameter type, a parameter name, a method return type, a method call parameter value and creation time; the data structure of each log record in the remote transaction log records may include: transaction log unique identification, master transaction ID, slave transaction ID, creation time.
The writing of the transaction completion information into the remote transaction log record and the local transaction log record means that a transaction log is recorded in the process of executing the local transaction log record, and the two processes are in the process of executing the same transaction, so that the execution results of the same transaction recorded by the remote transaction log record and the local transaction log record are consistent, and the remote transaction log record and the local transaction log record are either successful at the same time or failed at the same time.
The distributed transaction execution result processor registers a post method after the completion of transaction execution based on a spring transaction synchronization manager, and the post method writes an execution state into a remote transaction log record according to the execution state of a message executor on incomplete transaction information, if the execution is successful, the state of the remote transaction log is that a server side is submitted, and if the execution is failed, the state of the remote transaction log is that a service consumption side (client side) is submitted, so that a commit message is sent to the distributed transaction manager, and the condition that the log record in the local transaction log record is consistent with the execution result of sending the commit message, and the commit message is either successful at the same time or failed at the same time is ensured.
For example, in an embodiment of the present application, the remote transaction log record and the local transaction log record may be polled according to a preset period, so as to complete data synchronization. For example, the execution states of the logs of the remote transaction log record and the local drive-in log record can be polled at regular time, when the execution state of the log is detected to be failed, the local transaction information is inquired according to the identification information of the transaction in the log, and the execution result of the remote method calling processing abnormal transaction is reinitiated. The number of times of re-initiation can be, for example, a preset number, and after the number of times of re-initiation exceeds the preset number of times of re-initiation, the abnormal transaction is manually compensated according to the remote transaction log record and the local transaction log record, so that the problem that the manager has operating pressure due to re-initiation after continuous failure is prevented under the condition that the final consistency of data is ensured.
Optionally, on the basis of the foregoing embodiments, an embodiment of the present application may further provide a data processing method based on a distributed transaction, and an implementation process of data processing based on a distributed transaction in the foregoing method is described as follows with reference to the accompanying drawings. Fig. 4 is a schematic flowchart of a data processing method based on distributed transactions according to another embodiment of the present application, as shown in fig. 4, after S101, the method may further include:
s108: and acquiring the transaction completion information which is transmitted by the client and is based on the same transaction as the incomplete transaction information.
Illustratively, in one embodiment of the present application, the distributed transaction message receiver is responsible for receiving outstanding transaction messages (prepare messages) and transaction complete messages (commit messages) sent by clients.
S109: and verifying the validity of the transaction completion information according to the incomplete transaction information.
In order to prevent repeated commit of a commit message due to repeated consumption of the commit message and cause inconsistency of data, in one embodiment of the application, a received prepare message may be stored in a local memory, when the commit message is obtained, the prepare message may be used to perform validity check on the commit message, and at the same time, remote transaction log records are written, and according to the content of the commit message, a corresponding remote method call is executed based on a polling load balancing algorithm; under normal conditions, the prefix message is acquired first and then the commit message is acquired, in order to ensure that the messages arrive in sequence, the distributed transaction message receiver supports error correction of the arrival sequence of the two messages, and can correct the condition that the commit message arrives prior to the prefix message, so that the problem of data inconsistency caused by loss of effective messages due to verification is prevented.
By adopting the data processing method based on the distributed transaction, the distributed transaction manager corresponding to each application program can be obtained by pre-marking the interface and the agent interface of the client class library of each application program in a manner of presetting the distributed transaction manager, then the distributed transaction manager corresponding to the application program is used for realizing the distributed transaction in the application process of the application program, namely, the transaction information sent by the client is obtained in a manner of marking information on the interface in advance, the corresponding interface marking information is obtained according to the analysis of the transaction information, then the corresponding agent marking information is obtained according to the interface marking information, the remote information to be synchronized is obtained according to the agent marking information, and the transaction entry information can be directly marked on the corresponding interface in a manner of marking annotation, the distributed processing can be distributed without writing codes, and all steps in the whole processing process are asynchronous processing, so that the problem of synchronous blocking is avoided.
The following explains a data processing apparatus based on distributed transactions provided in the present application with reference to the drawings, where the data processing apparatus based on distributed transactions can execute any one of the data processing methods based on distributed transactions in fig. 1 to 4, and specific implementation and beneficial effects of the data processing apparatus based on distributed transactions refer to the above description, and are not described again below.
Fig. 5 is a schematic structural diagram of a data processing apparatus based on distributed transactions according to an embodiment of the present application, and as shown in fig. 5, the apparatus includes: an acquisition module 201 and a parsing module 202, wherein:
the obtaining module 201 is configured to obtain transaction information sent by a client.
And the analysis module 202 is configured to analyze and obtain corresponding interface label information according to the transaction information, where the interface label information is used to indicate a transaction entry.
An obtaining module 201, configured to obtain corresponding agent labeling information according to the interface labeling information; and acquiring remote information to be synchronized according to the agent marking information.
Fig. 6 is a schematic structural diagram of a data processing apparatus based on distributed transactions according to an embodiment of the present application, and as shown in fig. 6, the apparatus further includes: a write module 203, wherein:
the obtaining module 201 is specifically configured to obtain an annotation operation for the remote call interface, and write proxy annotation information in the remote call interface.
And the writing module 203 is used for responding to the method editing operation of the transaction call and writing the interface marking information in the method program of the transaction call.
As shown in fig. 6, the remote to-be-synchronized information includes a remote called method: interface full-class name, method name, parameter type, parameter name and method return type; the device also includes: the storage module 204 is configured to receive the incomplete transaction information sent by the client, and locally store the incomplete transaction information.
As shown in fig. 6, the apparatus further includes: a generation module 205 and an update module 206, wherein:
the generating module 205 is configured to generate a globally unique master transaction identifier and slave transaction identifier according to the remote information to be synchronized and the agent annotation information, and store the incomplete transaction information in the thread context environment.
The update module 206 is configured to execute a post-processing procedure corresponding to the incomplete transaction according to the thread context environment, and update the state of the incomplete transaction information to be transaction completion information after the execution is completed.
The writing module 203 is specifically configured to write the transaction completion information into the remote transaction log record and the local transaction log record.
As shown in fig. 6, the apparatus further includes: and the synchronization module 207 is configured to poll the remote transaction log record and the local transaction log record according to a preset period, so as to complete data synchronization.
As shown in fig. 6, the apparatus further includes: a verification module 208, wherein:
the obtaining module 201 is specifically configured to obtain transaction completion information that is transmitted by the client and based on the same transaction as the incomplete transaction information.
And the verifying module 208 is configured to verify the validity of the transaction completion information according to the incomplete transaction information.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 7 is a schematic structural diagram of a distributed transaction-based data processing device according to an embodiment of the present application, where the distributed transaction-based data processing device may be integrated in a terminal device or a chip of the terminal device.
The distributed transaction-based data processing apparatus includes: a processor 501, a storage medium 502, and a bus 503.
The processor 501 is used for storing a program, and the processor 501 calls the program stored in the storage medium 502 to execute the method embodiment corresponding to fig. 1-4. The specific implementation and technical effects are similar, and are not described herein again.
Optionally, the present application also provides a program product, such as a storage medium, on which a computer program is stored, including a program, which, when executed by a processor, performs embodiments corresponding to the above-described method.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to perform some steps of the methods according to the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Claims (10)
1. A method for data processing based on distributed transactions, the method comprising:
acquiring transaction information sent by a client;
analyzing and acquiring corresponding interface marking information according to the transaction information, wherein the interface marking information is used for indicating a transaction entrance;
acquiring corresponding agent marking information according to the interface marking information;
and acquiring remote information to be synchronized according to the agent marking information.
2. The data processing method of claim 1, wherein the interface annotation information comprises one or more of: propagation characteristics of the transaction, isolation level of the transaction, read-only properties of the transaction;
the agent annotation information comprises one or more of the following items: a provider identification of the service, a user identification of the service.
3. The data processing method according to claim 2, wherein before the obtaining the transaction information sent by the client, the method further comprises:
obtaining annotation operation aiming at a remote call interface, and writing the agent annotation information in the remote call interface;
and responding to the method editing operation of the transaction call, and writing the interface marking information in the method program of the transaction call.
4. The data processing method according to any one of claims 1 to 3, wherein the obtaining transaction information sent by the client comprises:
receiving the incomplete transaction information sent by the client, and locally storing the incomplete transaction information.
5. The data processing method of claim 4, wherein the remote information to be synchronized comprises a remote called method: interface full-class name, method name, parameter type, parameter name and method return type;
after the remote information to be synchronized is obtained according to the agent annotation information, the method further includes:
generating a globally unique main transaction identifier and a globally unique slave transaction identifier according to the remote information to be synchronized and the agent marking information, and storing the incomplete transaction information to a thread context environment;
executing a post-processing flow corresponding to the uncompleted transaction according to the thread context environment, and updating the state of the uncompleted transaction information into transaction completion information after the execution is completed;
and writing the transaction completion information into a remote transaction log record and a local transaction log record.
6. The data processing method of claim 5, wherein the method further comprises:
and polling the remote transaction log record and the local transaction log record according to a preset period to finish data synchronization.
7. The data processing method according to claim 4, wherein after receiving the incomplete transaction information sent by the client and storing the incomplete transaction information locally, the method further comprises:
acquiring transaction completion information which is transmitted by a client and is based on the same transaction with the incomplete transaction information;
and verifying the validity of the transaction completion information according to the incomplete transaction information.
8. A distributed transaction based data processing apparatus, the apparatus comprising: an acquisition module and an analysis module, wherein:
the acquisition module is used for acquiring the transaction information sent by the client;
the analysis module is used for analyzing and acquiring corresponding interface marking information according to the transaction information, wherein the interface marking information is used for indicating a transaction entrance;
the acquiring module is specifically configured to acquire corresponding agent labeling information according to the interface labeling information; and acquiring remote information to be synchronized according to the agent marking information.
9. A distributed transaction based data processing apparatus, characterized in that the apparatus comprises: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the distributed transaction based data processing apparatus is operating, the processor executing the machine-readable instructions to perform the method of any of claims 1-7.
10. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, performs the method of any of the preceding claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011461774.6A CN112463810A (en) | 2020-12-08 | 2020-12-08 | Data processing method, device, equipment and storage medium based on distributed transaction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011461774.6A CN112463810A (en) | 2020-12-08 | 2020-12-08 | Data processing method, device, equipment and storage medium based on distributed transaction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112463810A true CN112463810A (en) | 2021-03-09 |
Family
ID=74803976
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011461774.6A Pending CN112463810A (en) | 2020-12-08 | 2020-12-08 | Data processing method, device, equipment and storage medium based on distributed transaction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112463810A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114579194A (en) * | 2022-03-08 | 2022-06-03 | 杭州每刻科技有限公司 | Spring remote call-based exception handling method and system |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102970373A (en) * | 2012-12-03 | 2013-03-13 | 亚信联创科技（中国）有限公司 | Data synchronization method, device and system |
| CN108845867A (en) * | 2018-07-16 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of distributed transaction management method, apparatus, system and storage medium |
| CN110008224A (en) * | 2019-03-08 | 2019-07-12 | 阿里巴巴集团控股有限公司 | A kind of db transaction processing method and processing device |
| US20200065176A1 (en) * | 2018-08-21 | 2020-02-27 | Red Hat, Inc. | Optimizing logging of decision outcomes in distributed transactions |
| CN110865889A (en) * | 2018-08-27 | 2020-03-06 | 北京高绎信息技术有限公司 | Method and apparatus for transferring events between components |
| CN111062684A (en) * | 2019-11-29 | 2020-04-24 | 普元信息技术股份有限公司 | System and method for realizing consistent processing of business data and process data under cloud process platform |
| CN111131394A (en) * | 2019-11-29 | 2020-05-08 | 紫光云(南京)数字技术有限公司 | Concurrent request processing method and device based on distributed lock and computer readable storage medium |
| CN111159252A (en) * | 2019-12-27 | 2020-05-15 | 腾讯科技（深圳）有限公司 | Transaction execution method and device, computer equipment and storage medium |
| CN111679892A (en) * | 2020-06-04 | 2020-09-18 | 中国建设银行股份有限公司 | Distributed transaction processing method, device, equipment and medium |
| CN111813791A (en) * | 2020-06-17 | 2020-10-23 | 上海悦易网络信息技术有限公司 | Method and equipment for distributed transaction compensation |
| CN111813583A (en) * | 2020-07-28 | 2020-10-23 | 厦门市易联众易惠科技有限公司 | Transaction management method, device, equipment and storage medium under micro-service architecture |
-
2020
- 2020-12-08 CN CN202011461774.6A patent/CN112463810A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102970373A (en) * | 2012-12-03 | 2013-03-13 | 亚信联创科技（中国）有限公司 | Data synchronization method, device and system |
| CN108845867A (en) * | 2018-07-16 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of distributed transaction management method, apparatus, system and storage medium |
| US20200065176A1 (en) * | 2018-08-21 | 2020-02-27 | Red Hat, Inc. | Optimizing logging of decision outcomes in distributed transactions |
| CN110865889A (en) * | 2018-08-27 | 2020-03-06 | 北京高绎信息技术有限公司 | Method and apparatus for transferring events between components |
| CN110008224A (en) * | 2019-03-08 | 2019-07-12 | 阿里巴巴集团控股有限公司 | A kind of db transaction processing method and processing device |
| CN111062684A (en) * | 2019-11-29 | 2020-04-24 | 普元信息技术股份有限公司 | System and method for realizing consistent processing of business data and process data under cloud process platform |
| CN111131394A (en) * | 2019-11-29 | 2020-05-08 | 紫光云(南京)数字技术有限公司 | Concurrent request processing method and device based on distributed lock and computer readable storage medium |
| CN111159252A (en) * | 2019-12-27 | 2020-05-15 | 腾讯科技（深圳）有限公司 | Transaction execution method and device, computer equipment and storage medium |
| CN111679892A (en) * | 2020-06-04 | 2020-09-18 | 中国建设银行股份有限公司 | Distributed transaction processing method, device, equipment and medium |
| CN111813791A (en) * | 2020-06-17 | 2020-10-23 | 上海悦易网络信息技术有限公司 | Method and equipment for distributed transaction compensation |
| CN111813583A (en) * | 2020-07-28 | 2020-10-23 | 厦门市易联众易惠科技有限公司 | Transaction management method, device, equipment and storage medium under micro-service architecture |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114579194A (en) * | 2022-03-08 | 2022-06-03 | 杭州每刻科技有限公司 | Spring remote call-based exception handling method and system |
| CN114579194B (en) * | 2022-03-08 | 2024-04-05 | 杭州每刻科技有限公司 | Exception handling method and system based on Spring remote call |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108509392B (en) | Multi-mechanism account checking method, system, computer equipment and storage medium | |
| CN112073269B (en) | Block chain network testing method, device, server and storage medium | |
| CN107241315B (en) | Access method and device of bank gateway interface and computer readable storage medium | |
| CN111355802B (en) | Information pushing method and device | |
| CN112446786A (en) | Abnormal transaction processing method and device, electronic equipment and readable storage medium | |
| CN114237994A (en) | Test method and system for distributed system, electronic device and storage medium | |
| CN112329001A (en) | Data distribution method, system, terminal and medium between internal network and external network | |
| CN112463810A (en) | Data processing method, device, equipment and storage medium based on distributed transaction | |
| CN111260433A (en) | Service order processing method and device | |
| CN112291321B (en) | Service processing method, device and system | |
| CN112445860B (en) | Method and device for processing distributed transaction | |
| CN112818016A (en) | API-based real-time and off-line data query method and system | |
| CN111897738A (en) | Automatic testing method and device based on atomic service | |
| CN115756769A (en) | Task execution method and device and electronic equipment | |
| CN113065927A (en) | Account checking method and device, electronic equipment and computer readable storage medium | |
| CN111488236A (en) | Order abnormity processing method, server, storage medium and processing device | |
| CN111506411A (en) | Multi-tenant service management method and system | |
| CN113965489B (en) | Link timeout detection method, device, computer equipment and storage medium | |
| WO2022199387A1 (en) | File processing method and system, and computer device and medium | |
| CN112732415B (en) | Transaction processing method, device and equipment based on resource exchange proxy system | |
| CN115102879B (en) | Test method of shared service platform, electronic equipment and storage medium | |
| CN116501618A (en) | Analog interface test method, device, computer equipment and readable storage medium | |
| CN117675185A (en) | Cryptographic protocol transaction management method, medium and device based on transaction message | |
| CN115391373A (en) | Data construction method, device, computer program product and storage medium | |
| CN114510422A (en) | Service execution method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |