CN110765178A - Distributed transaction processing method and device and computer storage medium - Google Patents
Distributed transaction processing method and device and computer storage medium Download PDFInfo
- Publication number
- CN110765178A CN110765178A CN201910991987.0A CN201910991987A CN110765178A CN 110765178 A CN110765178 A CN 110765178A CN 201910991987 A CN201910991987 A CN 201910991987A CN 110765178 A CN110765178 A CN 110765178A
- Authority
- CN
- China
- Prior art keywords
- transaction
- branch
- data
- snapshot data
- distributed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
Abstract
The disclosure relates to a distributed transaction processing method and apparatus, and a computer-readable storage medium. The processing method of the distributed transaction comprises the following steps: receiving a service request initiated by a user, wherein the service request comprises a plurality of write operation requests; creating a global transaction according to the service request, and allocating a first identifier for the global transaction; packaging each write operation request into a branch transaction, and allocating a second identifier to each branch transaction; for each branch transaction, inquiring original data before executing each branch transaction from a database corresponding to each branch transaction, and generating first snapshot data corresponding to the original data in a memory; for each branch transaction, executing each branch transaction on the first snapshot data in the memory, and generating second snapshot data; and in response to a request initiated by a user for submitting the global transaction, writing the second snapshot data of each branch transaction into the database corresponding to each branch transaction.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for processing distributed transactions, and a computer-readable storage medium.
Background
Database transactions need to satisfy the ACID property. The characteristic of A is Atomicity (Atomicity). Atomicity refers to a transaction being executed as a whole, either all or none. The C property is Consistency (Consistency). Coherency refers to the fact that a transaction should ensure that data transitions from one coherent state to another. The I property is Isolation (Isolation). Isolation refers to the concurrent execution of multiple transactions, the execution of one transaction should not affect the execution of other transactions. D is characterized by persistence (Durability). Persistence refers to committed transactional modification data being persisted.
In a single data node, transactions are limited to access control to a single database resource, referred to as local transactions. Almost all mature relational databases provide native support for local transactions. However, in a distributed application environment based on micro services, more and more application scenarios require that access to multiple services and corresponding multiple database resources can be incorporated into the same transaction, and distributed transactions are generated at the same time.
Relational databases, though, provide native support for perfect ACID properties for local transactions. However, in a distributed scenario, such native support can be a penalty for system performance. How to make the database meet the ACID characteristics or find corresponding alternatives in a distributed scene is the key work of distributed transactions.
Related distributed transaction processing techniques include XA protocol-based two-phase commit techniques and flexible transaction-based processing techniques. In the XA protocol based two-phase commit technique, the processing of a distributed transaction includes a request phase and a commit phase. And the coordinator commits or cancels the distributed transaction according to the feedback of the participants on the commit or cancel transaction in the request phase. In the processing technology based on the flexible transaction, the transaction is submitted before being completed, and whether other transactions related to the transaction are completed or submitted does not need to be judged.
Disclosure of Invention
The inventor thinks that: in the related processing technology of distributed transaction, XA protocol realizes strong consistency, but is limited by database types, and only a specific database can be used. Flexible transaction-based processing techniques cannot achieve strong consistency at commit.
In view of the above technical problems, the present disclosure provides a solution, which is not limited by the type of the database, and achieves strong consistency in submission.
According to a first aspect of the present disclosure, there is provided a method for processing a distributed transaction, including: receiving a service request initiated by a user, wherein the service request comprises a plurality of write operation requests; creating a global transaction according to the service request, and allocating a first identifier for the global transaction; packaging each write operation request into a branch transaction, and allocating a second identifier to each branch transaction; for each branch transaction, inquiring original data before executing each branch transaction from a database corresponding to each branch transaction, and generating first snapshot data corresponding to the original data in a memory; for each branch transaction, executing each branch transaction on the first snapshot data in the memory to generate second snapshot data; and in response to the request initiated by the user for submitting the global transaction, writing the second snapshot data of each branch transaction into the database corresponding to each branch transaction.
In some embodiments, writing the second snapshot data of each branch transaction to the database corresponding to each branch transaction comprises: writing the second snapshot data of each branch transaction into a buffer pool; and writing the second snapshot data of each branch transaction in the buffer pool into the database corresponding to each branch transaction respectively at regular time.
In some embodiments, the service request further includes a read operation request, and the method for processing a distributed transaction further includes: determining an isolation level corresponding to the global transaction and a current version chain of a data primary key corresponding to the read operation request, wherein the current version chain is stored in the memory; and reading data corresponding to the data primary key from the current version chain according to the isolation level.
In some embodiments, reading data corresponding to the data primary key from the current version chain according to the isolation level comprises: reading the latest data in a committed state from the current version chain under the condition that the isolation level is read committed; reading the latest data in an uncommitted state from the current version chain under the condition that the isolation level is read uncommitted; and under the condition that the isolation level is repeatable reading or serializing, determining an identification interval of a visible global transaction of the global transaction, reading the latest data of the global transaction, the identification of which is located in the identification interval and is in a committed state, from the current version chain, wherein the minimum value of the identification interval is the sum of 1 on the identification of the global transaction in an uncommitted state, and the maximum value of the identification interval is the identification of the global transaction.
In some embodiments, the method for processing a distributed transaction further comprises: and reading the data corresponding to the data primary key from the database corresponding to the read operation request under the condition that the current version chain of the data primary key corresponding to the read operation request does not exist.
In some embodiments, the method for processing a distributed transaction further comprises: for the branch transaction with the write failure, the write operation is executed again until the failure times are equal to a preset threshold value; and if the failure times are equal to the preset threshold value, rolling back the written operation of other branch transactions except the branch transaction with the write failure of the global transaction.
In some embodiments, the operation of rolling back writes of other branch transactions of the global transaction other than the branch transaction in which the write failed comprises: writing the first snapshot data of other branch transactions into a database corresponding to the other branch transactions; and deleting the first snapshot data and the second snapshot data of each branch transaction of the global transaction.
In some embodiments, the method for processing a distributed transaction further comprises: after the global transaction is submitted, for each branch transaction, writing the corresponding relation of the first identifier, each second identifier, each first snapshot data and each second snapshot data into a pre-write log system WAL.
In some embodiments, the correspondence is stored in the memory through a remote dictionary service Redis and in a data structure of a structured merge tree LSMTree, and the method for processing a distributed transaction further includes: under the condition that the Redis is unavailable, writing each second snapshot data into a database corresponding to each second snapshot data according to the corresponding relation in the WAL; and deleting the corresponding relation in the memory under the condition that the Redis is available.
According to a second aspect of the present disclosure, there is provided a distributed transaction processing apparatus, including: the receiving module is configured to receive a service request initiated by a user, wherein the service request comprises a plurality of write operation requests; the creating module is configured to create a global transaction according to the service request and allocate a first identifier to the global transaction; the encapsulation module is configured to encapsulate each write operation request into one branch transaction and allocate a second identifier to each branch transaction; the first generation module is configured to query original data before executing each branch transaction from a database corresponding to each branch transaction and generate first snapshot data corresponding to the original data in a memory; a second generating module configured to execute, in the memory, each branch transaction on the first snapshot data and generate second snapshot data for each branch transaction; a writing module configured to write the second snapshot data of each branch transaction into a database corresponding to each branch transaction in response to the user-initiated request to commit the global transaction.
According to a third aspect of the present disclosure, there is provided a distributed transaction processing apparatus, including: a memory; and a processor coupled to the memory, the processor configured to execute the method for processing distributed transactions according to any of the above embodiments based on the instructions stored in the memory.
According to a fourth aspect of the present disclosure, a computer-storable medium has stored thereon computer program instructions which, when executed by a processor, implement a method of processing a distributed transaction as described in any of the embodiments above.
In the above embodiment, there is no limitation on the type of database, and strong consistency at the time of commit is achieved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.
The present disclosure may be more clearly understood from the following detailed description, taken with reference to the accompanying drawings, in which:
FIG. 1 illustrates a flow diagram of a method of processing a distributed transaction according to some embodiments of the present disclosure;
FIG. 2 illustrates a block diagram of a distributed transaction processing apparatus, in accordance with some embodiments of the present disclosure;
FIG. 3 illustrates a deployment architecture diagram of a distributed transactional processing system, in accordance with some embodiments of the present disclosure;
FIG. 4 illustrates a schematic diagram of a distributed transactional processing system, in accordance with some embodiments of the present disclosure;
FIG. 5 illustrates a block diagram of a distributed transaction processing apparatus, in accordance with some embodiments of the present disclosure;
FIG. 6 illustrates a block diagram of a computer system for implementing some embodiments of the present disclosure.
Detailed Description
Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
Fig. 1 illustrates a flow diagram of a method of processing a distributed transaction according to some embodiments of the present disclosure.
As shown in fig. 1, the processing method of the distributed transaction includes steps S110 to S160. In some embodiments, the method of processing a distributed transaction is performed by distributed transaction middleware.
In step S110, a service request initiated by a user is received. In some embodiments, the user initiates a transaction using a Connection command of JDBC (Java DataBase Connectivity) before initiating a service request.
For example, the database operations "transfer 100 yuan from account a to account B, search for the balance of account B, transfer 5 yuan from account B to account C, and search for the balance of account C" are performed for the database, and the databases corresponding to account A, B, C are database a, database B, and database C, respectively. Specifically, for the database operation "transfer 100 elements from account a to account B", the user initiated service request includes Update account set account-100 where account id is a. account is the data table name. The data primary key name of the data table account is account _ id. When the primary key value of the primary key of the data is a, the operation on the database a is indicated. and, amount represents the amount of money.
The service request includes a plurality of write operation requests. For example, the write operation request is the amount of money of database A minus 100, the amount of money of database B plus 100 yuan, the amount of money of database B minus 5 yuan, and the amount of money of database C plus 5 yuan, respectively. It should be appreciated that the various write operations should be performed sequentially.
In step S120, a global transaction is created according to the service request, and a first identifier is assigned to the global transaction. For example, the first identifier is gtx-1.
In step S130, each write operation request is encapsulated as a branch transaction, and each branch transaction is assigned a second identifier. For example, the branch transaction corresponding to the "database A less 100" write operation request has the second identifier branch-1.
In step S140, for each branch transaction, original data before executing each branch transaction is queried from the database corresponding to each branch transaction, and first snapshot data corresponding to the original data is generated in the memory. For example, the original amount of database A is 200 before executing the branch transaction corresponding to the write operation request "amount of database A minus 100". In some embodiments, the first snapshot data corresponding to the write operation request "amount of database a minus 100" is equal to 200. For example, the memory may be a memory of a distributed transaction processing apparatus, or may be a memory designated by a user.
In step S150, for each branch transaction, each branch transaction is executed on the first snapshot data in the memory, and second snapshot data is generated. For example, in the memory, the corresponding branch transaction is executed for the first snapshot data corresponding to the sum of the write operation request "database a minus 100", and the second snapshot data is generated. The second snapshot data is, for example, amount 100.
In some embodiments, the distributed transaction middleware parses the branch transaction corresponding to the write operation request "database a amount minus 100", and generates a data tuple in the memory, where the data tuple includes the first identifier, the data table name, the data key name, the key value, the first snapshot data, and the second snapshot data. The data tuples are represented, for example, using lists.
In some embodiments, the in-memory formed data structure of the branch-transaction branch-1 includes a database name and a data tuple. The database name is A, and the data tuple comprises a first identifier, a data table name, a data primary key name, a primary key value, first snapshot data and second snapshot data.
In some embodiments, a data structure for a global transaction is generated in memory, the data structure for the global transaction including a first identification and a second identification list for a branch transaction. For example, the first identification is gtx-1 and the second identification list of branch transactions is [ branch-1 ].
In step S160, in response to the request initiated by the user to commit the global transaction, the second snapshot data of each branch transaction is written into the database corresponding to each branch transaction. In the transaction processing scenario, the states of the global transaction and the individual branch transactions, which are visible to the user, are managed and recorded by the transaction manager. For example, the user initiates a request to commit the global transaction via a commit command.
According to the method and the device, direct operation on the database is avoided by generating the first snapshot data and the second snapshot data in the memory, the second snapshot data are written into the corresponding database under the condition that all branch transactions are completely finished in response to a submission request initiated by a user, and strong consistency in submission is realized. In addition, the method and the system interact with the database in a snapshot data mode through the memory, are not limited by the type of the database, and have universality.
In some embodiments, the second snapshot data for each branch transaction is written to the database corresponding to each branch transaction as follows.
First, the second snapshot data of each branch transaction is written to a buffer pool.
Then, the second snapshot data of each branch transaction in the buffer pool is written into the database corresponding to each branch transaction respectively at regular time.
The embodiment realizes asynchronous disk refreshing by using the buffer pool, and improves the submission performance of the distributed transaction.
In some embodiments, the service request further comprises a read operation request. For example, the database operates a read operation request of "transfer 100 yuan from account a to account B, inquire balance of account B, transfer 5 yuan from account B to account C, inquire balance of account C" to inquire balance of account B.
The processing method of the distributed transaction further comprises the following steps.
First, the isolation level corresponding to the global transaction and the current version chain of the primary key of the data corresponding to the read operation request are determined. The current version chain is stored in the memory. It should be appreciated that controlling the isolation level using the MVCC (Multi-Version concurrent control) algorithm will generate a chain of versions of different data primary keys.
Then, according to the isolation level, data corresponding to the data primary key is read from the current version chain. In some embodiments, the isolation level is preset by the user. Reading the data corresponding to the data primary key from the current version chain according to the isolation level is achieved, for example, as follows.
In the case where the isolation level is read committed, the most recent data in the committed state is read from the current version chain.
And reading the latest data in an uncommitted state from the current version chain under the condition that the isolation level is read uncommitted.
And under the condition that the isolation level is repeatable reading or serializing, determining an identification interval of a visible global transaction of the global transaction, and reading the latest data of which the identification is positioned in the identification interval and in a committed state from the current version chain. The minimum value of the identification interval is the identification of the global transaction currently in the uncommitted state plus 1. The maximum value of the identification interval is the identification of the currently processed global transaction.
In some embodiments, whether the data is the latest data is determined by the version number in each piece of data in the current version chain. For example, the version number is the largest and is the latest data.
The MVCC algorithm is used for controlling the memory, so that the support of the isolation level is realized, and the problem that a distributed transaction processing method based on flexible transactions cannot provide complete isolation level semanteme is solved.
In some embodiments, the method for processing a distributed transaction further comprises: and reading the data corresponding to the data primary key from the database corresponding to the read operation request under the condition that the current version chain of the data primary key corresponding to the read operation request does not exist.
In some embodiments, the method of processing a distributed transaction further comprises the following steps.
And for the branch transaction with the write failure, the write operation is executed again until the failure times are equal to the preset threshold value. For example, the preset threshold is 5 times.
And rolling back the write operation of other branch transactions except the branch transaction with the write failure of the global transaction under the condition that the failure times are equal to a preset threshold value. For example, the write operations of other branch transactions of the global transaction except the branch transaction for which the write failed are rolled back as follows. First, the first snapshot data of other branch transactions is utilized to write into the database corresponding to other branch transactions. The first snapshot data and the second snapshot data of each branch transaction of the global transaction are then deleted.
The embodiment tries to refresh the disk again for the branch transaction with failed write, calls the reverse engine when the failure times reach the maximum retry times, deletes the record which has been successfully refreshed, provides a runtime recovery mechanism, and because the read operations are all reading the data of the memory, even if one part of the branch transaction fails to write and the other part of the branch transaction succeeds to write, the read data will not be inconsistent.
In some embodiments, the method for processing a distributed transaction further comprises: after the global transaction is committed, for each branch transaction, writing the corresponding relationship of the first identifier, the second identifiers, the first snapshot data, and the second snapshot data into the WAL (Write-Ahead Logging). By writing the WAL, the performance of the transaction is improved when the transaction is submitted.
In some embodiments, the correspondence is stored in the memory by a Remote Dictionary service (rdss) and LSM Tree (Log-Structured Merge Tree) data structure. The processing method of the distributed transaction further comprises the following steps.
And under the condition that Redis is unavailable, writing each second snapshot data into a database corresponding to each snapshot data according to the corresponding relation in the WAL. For example, a situation where Redis is unavailable includes a connection timeout. Under the condition that Redis is unavailable, the memory cannot be read, and the second snapshot data is written into the corresponding database, so that the second snapshot data is persisted to the database, and the risk that the data cannot be persisted under the condition that the memory cannot be read is avoided.
And deleting the corresponding relation in the memory under the condition that the Redis is available. Under the condition that Redis is available, it is indicated that the second snapshot data can be persisted to the corresponding database by reading the memory, and the memory can be emptied, so that the memory is provided for the subsequent snapshot data.
Fig. 2 illustrates a block diagram of a distributed transaction processing apparatus, according to some embodiments of the present disclosure.
As shown in fig. 2, the processing apparatus 2 for distributed transaction includes a receiving module 21, a creating module 22, an encapsulating module 23, a first generating module 24, a second generating module 25, and a writing module 26.
The receiving module 21 is configured to receive a service request initiated by a user, for example, execute step S110 shown in fig. 1. The service request includes a plurality of write operation requests.
The creation module 22 is configured to create a global transaction according to the service request and assign a first identifier to the global transaction, for example, execute step S120 shown in fig. 1.
The encapsulation module 23 is configured to encapsulate each write operation request into a branch transaction and assign a second identifier to each branch transaction, for example, execute step S130 shown in fig. 1.
The first generating module 24 is configured to, for each branch transaction, query the database corresponding to each branch transaction for original data before executing each branch transaction, and generate first snapshot data corresponding to the original data in the memory, for example, execute step S140 shown in fig. 1.
The second generating module 25 is configured to execute each branch transaction on the first snapshot data in the memory for each branch transaction, and generate the second snapshot data, for example, execute step S150 shown in fig. 1.
The writing module 26 is configured to write the second snapshot data of each branch transaction to the database corresponding to each branch transaction in response to a request initiated by a user to commit the global transaction, for example, execute step S160 shown in fig. 1.
FIG. 3 illustrates a deployment architecture diagram of a distributed transactional processing system, according to some embodiments of the present disclosure.
As shown in fig. 3, the processing system 3 for distributed transactions includes a processing device 31 for distributed transactions, and a database 32A, a database 32B, a database 32C, and a database 32D. A distributed transaction processing means 31 may be encapsulated in different subscriber service systems.
In some embodiments, the processing device 31 for the distributed transaction may be a distributed transaction middleware, which is embedded in a service system of a user, and the user may initiate a service request through the service system and complete processing of the distributed transaction by using the processing device 31 for the distributed transaction, so as to transparently process the distributed transaction. For example, the distributed transaction middleware accesses a business system of a user through an interface of the distributed transaction middleware, and interfaces the database 32A, the database 32B, the database 32C, and the database 32D, thereby integrating the processing capability of the distributed transaction into existing frameworks such as Spring, MyBatis, JPA (Java persistence API), and the like. The processing device 31 for distributed transaction executes, for example, the processing method for distributed transaction according to any embodiment of the present disclosure.
The implementation principle of the processing system for distributed transactions will be described in detail below in connection with a schematic diagram of the processing system for distributed transactions as shown in fig. 4.
FIG. 4 illustrates a schematic diagram of a distributed transactional processing system, according to some embodiments of the present disclosure.
As shown in fig. 4, the processing system 3 for distributed transactions includes a processing device 31 for distributed transactions, and a database 32A, a database 32B, a database 32C, and a database 32D.
The processing means 31 of the distributed transaction are configured to perform the following steps.
Receiving a service request initiated by a user, for example, executing step S110 shown in fig. 1. Such as a service code written by the user for the service request. It should be understood that the number of write operations and read operations herein is merely illustrative and does not represent an actual number.
A global transaction is created according to the service request and a first identifier is assigned to the global transaction, for example, step S120 shown in fig. 1 is performed.
Each write operation request is encapsulated as a branch transaction and each branch transaction is assigned a second identification, e.g. step S130 as shown in fig. 1 is performed.
For each branch transaction, the original data before executing each branch transaction is queried from the database 32A, the database 32B, the database 32C, or the database 32D corresponding to each branch transaction, and first snapshot data corresponding to the original data is generated in the memory, for example, step S140 shown in fig. 1 is executed.
For each branch transaction, executing each branch transaction on the first snapshot data in the memory to generate the second snapshot data, for example, executing step S150 shown in fig. 1.
In response to a request initiated by a user to commit the global transaction, the second snapshot data of each branch transaction is written to the database 32A, the database 32B, the database 32C, or the database 32D corresponding to each branch transaction, for example, step S160 shown in fig. 1 is performed.
In some embodiments, the processing means 31 of the distributed transaction is further configured to perform write log operations and recovery operations.
The log writing operation is that after the global transaction is submitted, for each branch transaction, the corresponding relationship between the first identifier, each second identifier, each first snapshot data, and each second snapshot data is written into the WAL.
And the recovery operation is to write each second snapshot data into the database corresponding to each second snapshot data according to the corresponding relation in the WAL under the condition that the Redis is unavailable. For example, a situation where Redis is unavailable includes a connection timeout. And under the condition that Redis is unavailable, the memory cannot be read, and the second snapshot data is written into the corresponding database.
Fig. 5 illustrates a block diagram of a distributed transaction processing apparatus, according to some embodiments of the present disclosure.
As shown in fig. 5, the processing apparatus 5 for distributed transactions includes a memory 51; and a processor 52 coupled to the memory 51. The memory 51 is used for storing instructions for performing a corresponding embodiment of the processing method of the distributed transaction. The processor 52 is configured to perform a method of processing a distributed transaction in any of the embodiments of the present disclosure based on instructions stored in the memory 51.
FIG. 6 illustrates a block diagram of a computer system for implementing some embodiments of the present disclosure.
As shown in FIG. 6, computer system 60 may take the form of a general purpose computing device. Computer system 60 includes a memory 610, a processor 620, and a bus 600 that connects the various system components.
The memory 610 may include, for example, system memory, non-volatile storage media, and the like. The system memory stores, for example, an operating system, an application program, a Boot Loader (Boot Loader), and other programs. The system memory may include volatile storage media such as Random Access Memory (RAM) and/or cache memory. The non-volatile storage medium stores, for instance, instructions for performing corresponding embodiments of at least one of the processing methods of the distributed transaction. Non-volatile storage media include, but are not limited to, magnetic disk storage, optical storage, flash memory, and the like.
The processor 620 may be implemented as discrete hardware components, such as a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gates or transistors, or the like. Accordingly, each of the modules, such as the judging module and the determining module, may be implemented by a Central Processing Unit (CPU) executing instructions in a memory for performing the corresponding step, or may be implemented by a dedicated circuit for performing the corresponding step.
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable apparatus to produce a machine, such that the execution of the instructions by the processor results in an apparatus that implements the functions specified in the flowchart and/or block diagram block or blocks.
These computer-readable program instructions may also be stored in a computer-readable memory that can direct a computer to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and/or block diagram block or blocks.
The present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects.
By the processing method and the processing device for the distributed transaction and the computer storage medium in the embodiment, the limitation of the database type is avoided, and the strong consistency during submission is realized.
Thus far, a distributed transaction processing method and apparatus, and a computer-readable storage medium according to the present disclosure have been described in detail. Some details that are well known in the art have not been described in order to avoid obscuring the concepts of the present disclosure. It will be fully apparent to those skilled in the art from the foregoing description how to practice the presently disclosed embodiments.
Claims (12)
1. A method of processing a distributed transaction, comprising:
receiving a service request initiated by a user, wherein the service request comprises a plurality of write operation requests;
creating a global transaction according to the service request, and allocating a first identifier for the global transaction;
packaging each write operation request into a branch transaction, and allocating a second identifier to each branch transaction;
for each branch transaction, inquiring original data before executing each branch transaction from a database corresponding to each branch transaction, and generating first snapshot data corresponding to the original data in a memory;
for each branch transaction, executing each branch transaction on the first snapshot data in the memory to generate second snapshot data;
and in response to the request initiated by the user for submitting the global transaction, writing the second snapshot data of each branch transaction into the database corresponding to each branch transaction.
2. The distributed transaction processing method of claim 1, wherein writing the second snapshot data of each branch transaction to the database corresponding to each branch transaction comprises:
writing the second snapshot data of each branch transaction into a buffer pool;
and writing the second snapshot data of each branch transaction in the buffer pool into the database corresponding to each branch transaction respectively at regular time.
3. The method for processing the distributed transaction according to claim 1, wherein the service request further includes a read operation request, and the method for processing the distributed transaction further includes:
determining an isolation level corresponding to the global transaction and a current version chain of a data primary key corresponding to the read operation request, wherein the current version chain is stored in the memory;
and reading data corresponding to the data primary key from the current version chain according to the isolation level.
4. The method of processing a distributed transaction according to claim 3, wherein reading data corresponding to the data primary key from the current version chain according to the isolation level comprises:
reading the latest data in a committed state from the current version chain under the condition that the isolation level is read committed;
reading the latest data in an uncommitted state from the current version chain under the condition that the isolation level is read uncommitted;
and under the condition that the isolation level is repeatable reading or serializing, determining an identification interval of a visible global transaction of the global transaction, reading the latest data of the global transaction, the identification of which is located in the identification interval and is in a committed state, from the current version chain, wherein the minimum value of the identification interval is the sum of 1 on the identification of the global transaction in an uncommitted state, and the maximum value of the identification interval is the identification of the global transaction.
5. The method of processing a distributed transaction of claim 3, further comprising:
and reading the data corresponding to the data primary key from the database corresponding to the read operation request under the condition that the current version chain of the data primary key corresponding to the read operation request does not exist.
6. The method of processing a distributed transaction of claim 3, further comprising:
for the branch transaction with the write failure, the write operation is executed again until the failure times are equal to a preset threshold value;
and if the failure times are equal to the preset threshold value, rolling back the written operation of other branch transactions except the branch transaction with the write failure of the global transaction.
7. The method of distributed transaction processing of claim 6, wherein the operation of rolling back writes of other branch transactions of the global transaction except the branch transaction in which the write failed comprises:
writing the first snapshot data of other branch transactions into a database corresponding to the other branch transactions;
and deleting the first snapshot data and the second snapshot data of each branch transaction of the global transaction.
8. The method of processing a distributed transaction of claim 1, further comprising:
after the global transaction is submitted, for each branch transaction, writing the corresponding relation of the first identifier, each second identifier, each first snapshot data and each second snapshot data into a pre-write log system WAL.
9. The distributed transaction processing method according to claim 8, wherein the correspondence is stored in the memory by a remote dictionary service (Redis) in a data structure of a structured Merge Tree (LSM Tree), and the distributed transaction processing method further comprises:
under the condition that the Redis is unavailable, writing each second snapshot data into a database corresponding to each second snapshot data according to the corresponding relation in the WAL;
and deleting the corresponding relation in the memory under the condition that the Redis is available.
10. A distributed transaction processing apparatus, comprising:
the receiving module is configured to receive a service request initiated by a user, wherein the service request comprises a plurality of write operation requests;
the creating module is configured to create a global transaction according to the service request and allocate a first identifier to the global transaction;
the encapsulation module is configured to encapsulate each write operation request into one branch transaction and allocate a second identifier to each branch transaction;
the first generation module is configured to query original data before executing each branch transaction from a database corresponding to each branch transaction and generate first snapshot data corresponding to the original data in a memory;
a second generating module configured to execute, in the memory, each branch transaction on the first snapshot data and generate second snapshot data for each branch transaction;
a writing module configured to write the second snapshot data of each branch transaction into a database corresponding to each branch transaction in response to the user-initiated request to commit the global transaction.
11. A distributed transaction processing apparatus, comprising:
a memory; and
a processor coupled to the memory, the processor configured to perform the method of processing a distributed transaction of any of claims 1 to 9 based on instructions stored in the memory.
12. A computer-storable medium having stored thereon computer program instructions which, when executed by a processor, implement a method of processing a distributed transaction as claimed in any of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910991987.0A CN110765178B (en) | 2019-10-18 | 2019-10-18 | Distributed transaction processing method and device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910991987.0A CN110765178B (en) | 2019-10-18 | 2019-10-18 | Distributed transaction processing method and device and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110765178A true CN110765178A (en) | 2020-02-07 |
| CN110765178B CN110765178B (en) | 2021-03-05 |
Family
ID=69332492
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910991987.0A Active CN110765178B (en) | 2019-10-18 | 2019-10-18 | Distributed transaction processing method and device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110765178B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259083A (en) * | 2020-02-13 | 2020-06-09 | 神州数码融信软件有限公司 | Distributed transaction processing method and device |
| CN112015766A (en) * | 2020-08-24 | 2020-12-01 | 京东数字科技控股股份有限公司 | Data processing method and device based on pipelining and data processing system |
| CN112395102A (en) * | 2020-10-15 | 2021-02-23 | 北京仿真中心 | Distributed database middleware solution method |
| CN112433886A (en) * | 2020-11-24 | 2021-03-02 | 厦门美图之家科技有限公司 | Data processing method and device, server and storage medium |
| CN112732454A (en) * | 2020-12-30 | 2021-04-30 | 北京懿医云科技有限公司 | Medical service appointment method, device, storage medium and equipment |
| WO2021169243A1 (en) * | 2020-02-29 | 2021-09-02 | 华为技术有限公司 | Method, apparatus and system for processing distributed transaction |
| CN113495872A (en) * | 2020-04-08 | 2021-10-12 | 北京万里开源软件有限公司 | Transaction processing method and system in distributed database |
| WO2023231665A1 (en) * | 2022-05-31 | 2023-12-07 | 易保网络技术(上海)有限公司 | Distributed transaction processing method, system and device, and readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104793988A (en) * | 2014-01-20 | 2015-07-22 | 阿里巴巴集团控股有限公司 | Cross-database distributed transaction implementation method and device |
| CN106033437A (en) * | 2015-03-13 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Method and system for processing distributed transaction |
| CN106598992A (en) * | 2015-10-15 | 2017-04-26 | 中兴通讯股份有限公司 | Database operating method and device |
| JP2018045286A (en) * | 2016-09-12 | 2018-03-22 | 富士通株式会社 | Pretreatment device, index addition tree data correction method and index addition tree data correction program |
| US20180107695A1 (en) * | 2016-10-19 | 2018-04-19 | Futurewei Technologies, Inc. | Distributed fp-growth with node table for large-scale association rule mining |
| CN109710388A (en) * | 2019-01-09 | 2019-05-03 | 腾讯科技（深圳）有限公司 | Method for reading data, device, electronic equipment and storage medium |
| CN110019530A (en) * | 2017-12-29 | 2019-07-16 | 百度在线网络技术（北京）有限公司 | Transaction methods and device based on distributed data base |
-
2019
- 2019-10-18 CN CN201910991987.0A patent/CN110765178B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104793988A (en) * | 2014-01-20 | 2015-07-22 | 阿里巴巴集团控股有限公司 | Cross-database distributed transaction implementation method and device |
| CN106033437A (en) * | 2015-03-13 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Method and system for processing distributed transaction |
| CN106598992A (en) * | 2015-10-15 | 2017-04-26 | 中兴通讯股份有限公司 | Database operating method and device |
| JP2018045286A (en) * | 2016-09-12 | 2018-03-22 | 富士通株式会社 | Pretreatment device, index addition tree data correction method and index addition tree data correction program |
| US20180107695A1 (en) * | 2016-10-19 | 2018-04-19 | Futurewei Technologies, Inc. | Distributed fp-growth with node table for large-scale association rule mining |
| CN110019530A (en) * | 2017-12-29 | 2019-07-16 | 百度在线网络技术（北京）有限公司 | Transaction methods and device based on distributed data base |
| CN109710388A (en) * | 2019-01-09 | 2019-05-03 | 腾讯科技（深圳）有限公司 | Method for reading data, device, electronic equipment and storage medium |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259083A (en) * | 2020-02-13 | 2020-06-09 | 神州数码融信软件有限公司 | Distributed transaction processing method and device |
| WO2021169243A1 (en) * | 2020-02-29 | 2021-09-02 | 华为技术有限公司 | Method, apparatus and system for processing distributed transaction |
| CN113495872A (en) * | 2020-04-08 | 2021-10-12 | 北京万里开源软件有限公司 | Transaction processing method and system in distributed database |
| CN112015766A (en) * | 2020-08-24 | 2020-12-01 | 京东数字科技控股股份有限公司 | Data processing method and device based on pipelining and data processing system |
| CN112395102A (en) * | 2020-10-15 | 2021-02-23 | 北京仿真中心 | Distributed database middleware solution method |
| CN112433886A (en) * | 2020-11-24 | 2021-03-02 | 厦门美图之家科技有限公司 | Data processing method and device, server and storage medium |
| CN112732454A (en) * | 2020-12-30 | 2021-04-30 | 北京懿医云科技有限公司 | Medical service appointment method, device, storage medium and equipment |
| WO2023231665A1 (en) * | 2022-05-31 | 2023-12-07 | 易保网络技术(上海)有限公司 | Distributed transaction processing method, system and device, and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110765178B (en) | 2021-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110765178B (en) | Distributed transaction processing method and device and computer storage medium | |
| CN111143389B (en) | Transaction execution method and device, computer equipment and storage medium | |
| US11556396B2 (en) | Structure linked native query database management system and methods | |
| US7930274B2 (en) | Dual access to concurrent data in a database management system | |
| US11132350B2 (en) | Replicable differential store data structure | |
| EP2797014B1 (en) | Database update execution according to power management schemes | |
| CN111522631A (en) | Distributed transaction processing method, device, server and medium | |
| US11874816B2 (en) | Lock free distributed transaction coordinator for in-memory database participants | |
| CN114925084A (en) | Distributed transaction processing method, system, device and readable storage medium | |
| WO2022197462A1 (en) | Snapshot isolation query transactions in distributed systems | |
| US7827135B2 (en) | Method and apparatus for relaxed transactional isolation in a client-server caching architecture | |
| US11797523B2 (en) | Schema and data modification concurrency in query processing pushdown | |
| US20230315721A1 (en) | Snapshot isolation query transactions in distributed systems | |
| CN112559496A (en) | Distributed database transaction atomicity realization method and device | |
| WO2014061847A1 (en) | Apparatus and method for logging and recovering transaction of database built in mobile environment | |
| CN112100186A (en) | Data processing method and device based on distributed system and computer equipment | |
| US11914613B2 (en) | Data visibility for nested transactions in distributed systems | |
| US7284023B2 (en) | Database management method and apparatus for addressing to transaction completion from/in a stored program written in an interpreted language and an implementing program therefor | |
| US20230394028A1 (en) | Method and system for lock after qualification for update queries | |
| CN114416201B (en) | Snapshot isolation implementation method based on distributed database | |
| CN114579604B (en) | Database transaction implementation method and system of application layer | |
| US20230376479A1 (en) | Schema and data modification concurrency in query processing pushdown | |
| WO2024082693A1 (en) | Data processing method, and apparatus | |
| CN115422213A (en) | Data updating method and device and electronic equipment | |
| Narasani | Clustering implementation for H2 database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Patentee after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Patentee before: Jingdong Digital Technology Holding Co.,Ltd. Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Patentee after: Jingdong Digital Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Patentee before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |