CN113296895B - Transaction processing system, transaction processing method and device - Google Patents

Transaction processing system, transaction processing method and device Download PDF

Info

Publication number
CN113296895B
CN113296895B CN202010525627.4A CN202010525627A CN113296895B CN 113296895 B CN113296895 B CN 113296895B CN 202010525627 A CN202010525627 A CN 202010525627A CN 113296895 B CN113296895 B CN 113296895B
Authority
CN
China
Prior art keywords
transaction
data
processed
data source
identifier
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010525627.4A
Other languages
Chinese (zh)
Other versions
CN113296895A (en
Inventor
张包峰
陈希
陈云星
潘岳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN202010525627.4A priority Critical patent/CN113296895B/en
Publication of CN113296895A publication Critical patent/CN113296895A/en
Application granted granted Critical
Publication of CN113296895B publication Critical patent/CN113296895B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/466Transaction processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2365Ensuring data consistency and integrity
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/254Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/62Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Security & Cryptography (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the specification provides a transaction processing system, a transaction processing method and a transaction processing device, wherein after receiving a processing request of a to-be-processed transaction, a transaction coordination module of the transaction processing system sends a to-be-processed data source obtained by analyzing the processing request to a transaction management module to request a global lock and a transaction identifier under the condition that the processing request is determined to be of a first type, and the transaction management module adds the global lock corresponding to the processing request to the to-be-processed data source and establishes a first data execution transaction under the condition that the to-be-processed data source is determined to be not executed by other transactions after receiving the to-be-processed data source, so that other identical processing requests are prevented from operating on the to-be-processed data source, data in the to-be-processed data source is polluted, and the security of the to-be-processed data source is ensured when the transaction execution module executes the transaction processing on the first data.

Description

Transaction processing system, transaction processing method and device
Technical Field
Embodiments of the present disclosure relate to the field of data processing technologies, and in particular, to a transaction processing system. One or more embodiments of the present specification relate to a transaction method, a transaction apparatus, a computing device, and a computer-readable storage medium.
Background
In the traditional database system, all data read-write defaults meet the transaction, the data instantaneity is good, but the throughput performance of data processing is relatively poor, the ultra-large ETL scene is difficult to meet, the small-batch writing or updating is performed, the time delay of single data is high, and the instantaneity is poor.
In a big data scene, data reading and writing are performed simultaneously, data isolation cannot be achieved, and dirty reading and dirty writing of data are easy to cause.
Accordingly, there is an urgent need to provide a transaction processing system that can meet both transactional and non-transactional requirements, and can meet the isolation of data in process.
Disclosure of Invention
In view of this, the present description embodiments provide a transaction processing system. One or more embodiments of the present specification are also directed to a transaction method, a transaction apparatus, a computing device, and a computer-readable storage medium, which address the technical deficiencies in the prior art.
According to a first aspect of embodiments of the present specification, there is provided a transaction processing system comprising:
the transaction coordination module is configured to receive a transaction request, and send a data source to be processed obtained by analyzing the transaction request to the transaction management module when the transaction request belongs to a first type;
The transaction management module is configured to set a first operation lock for the received data source to be processed, establish a first data execution transaction based on the data source to be processed, generate a first identifier for the first data execution transaction, and return the first identifier and the first operation lock to the transaction coordination module;
and the transaction execution module is configured to generate a second identifier for the data source to be processed according to the first identifier under the condition that a transaction execution instruction carrying the first identifier and sent by the transaction coordination module is received, and process the first data to execute the transaction according to the second identifier and the data source to be processed.
According to a second aspect of embodiments of the present specification, there is provided a transaction processing method, comprising:
receiving a transaction request, and sending a data source to be processed obtained by analyzing the transaction request to a transaction management module under the condition that the transaction request belongs to a first type;
receiving a first identification of a first data transaction and the first operation lock, wherein the first identification is returned by the transaction management module and is established based on a data source to be processed provided with the first operation lock;
And sending a transaction execution instruction carrying the first identifier to a transaction execution module, receiving a second identifier generated by the transaction execution module for the data source to be processed according to the first identifier, and receiving a processing result of processing the first data transaction according to the second identifier and the data source to be processed.
According to a third aspect of embodiments of the present specification, there is provided a transaction processing apparatus comprising:
the first receiving module is configured to receive a transaction request, and send a data source to be processed obtained by analyzing the transaction request to the transaction management module when the transaction request belongs to a first type;
the second receiving module is configured to receive a first identification of a first data transaction which is returned by the transaction management module and is established based on a data source to be processed provided with a first operation lock and the first operation lock;
the third receiving module is configured to send a transaction execution instruction carrying the first identifier to the transaction execution module, receive a second identifier generated by the transaction execution module for the data source to be processed according to the first identifier, and receive a processing result of processing the first data transaction according to the second identifier and the data source to be processed.
According to a fourth aspect of embodiments of the present specification, there is provided a computing device comprising:
a memory and a processor;
the memory is configured to store computer-executable instructions and the processor is configured to execute the computer-executable instructions, wherein the processor, when executing the computer-executable instructions, performs the steps of the transaction method.
According to a fifth aspect of embodiments of the present description, there is provided a computer-readable storage medium storing computer-executable instructions which, when executed by a processor, implement the steps of the transaction method.
The transaction coordination module of the transaction processing system sends a to-be-processed data source obtained by analyzing the processing request to the transaction management module to request a global lock and a transaction identifier under the condition that the processing request is determined to be of a first type after receiving the processing request of the to-be-processed transaction, and the transaction management module holds the global lock corresponding to the processing request for the to-be-processed data source and establishes a first data execution transaction under the condition that the to-be-processed data source is determined not to be executed by other transactions after receiving the to-be-processed data source, so that other identical processing requests are prevented from operating on the to-be-processed data source, data in the to-be-processed data source is polluted, and the security of the to-be-processed data source is ensured when the transaction execution module executes the transaction processing on the first data.
Drawings
FIG. 1 is a schematic diagram of a transaction processing system according to one embodiment of the present disclosure;
FIG. 2 is a flow chart of a transaction method provided in one embodiment of the present disclosure;
FIG. 3 is a flowchart of a transaction processing method applied to a distributed database according to one embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a transaction device according to one embodiment of the present disclosure;
FIG. 5 is a block diagram of a computing device provided in one embodiment of the present description.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many other forms than described herein and similarly generalized by those skilled in the art to whom this disclosure pertains without departing from the spirit of the disclosure and, therefore, this disclosure is not limited by the specific implementations disclosed below.
The terminology used in the one or more embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the specification. As used in this specification, one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that, although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first may also be referred to as a second, and similarly, a second may also be referred to as a first, without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
First, terms related to one or more embodiments of the present specification will be explained.
ETL: extract Transform Load, describing the process of extracting (extract), transposing (transform), loading (load) data from a source to a destination; for example, batch data write/update/delete includes insert into select, insert overwrite, update set write, delete from, etc. sql pattern.
TC: the transaction coordinator comprises one of distributed transaction components, an initiator of the distributed transaction and execution control of the distributed transaction, wherein the execution control comprises the steps of starting the transaction, submitting the transaction and aborting the transaction.
TM: the TransactionManager, one of the distributed transaction components, maintains execution of a single data node local transaction engine, providing local transaction capabilities of the XA-like protocol.
GTM: globalTransactionManager, one of the distributed transaction components, provides the ability to apply for global TXID, provides the ability to manage active transactions, provides global lock capabilities.
GXID: global Transaction ID, global transaction ID, one of distributed transaction IDs, distributed transaction ID, is independently applied for allocation by GTM and ensures global monotonic increment.
XID: transaction ID, one of the (local) Transaction IDs, TM local Transaction ID, is independently assigned by the TM and guaranteed to monotonically increment within the TM.
Transactional (read-write): data processing satisfying database ACID characteristics.
Non-transaction (read-write): data processing that does not fully satisfy the database ACID characteristics is mainly referred to herein as not satisfying isolation, and it is understood that the isolation level is read uncompensated, i.e., dirty reads and dirty writes may be caused.
LockTable: is locked (write lock, mutex) by the table.
Globalsrapshot: global snapshot.
Begin: the first step in the distributed transaction control flow, the transaction is started.
Prepore: pre-commit/ready-to-commit.
checkpoint: the snapshot is persisted for failure recovery.
Commit: the commit of the distributed transaction control flow.
Rollback: rollback of distributed transaction control flow (which occurs after Begin, prepare when a TM or TC fails).
Client: client, here MySQL client.
control: the SQL executing controller is a process with the TC and is responsible for SQL analysis, plan generation and issuing and data routing.
In this specification, a transaction processing system is provided. One or more embodiments of the present specification relate to a transaction processing method, a transaction processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments.
The transaction processing system provided by the embodiment of the specification is applied to a distributed real-time data warehouse, such as a AnalyticDB for MySQL distributed real-time data warehouse, which ensures high-throughput ETL data writing and additionally supports ETL transactions, and the design of the ETL transactions of the distributed real-time data warehouse can meet the requirement that a source table can be repeatedly read in the ETL process; after ETL fails, rollback, batch pruning and no dirty writing can be performed; when the ETL is finished, new data among all the data nodes are visible at the same time; ETL leads in high performance, and ensures throughput.
Based on the implementation of the above ETL transaction, there is a further need to ensure real-time visibility and low-latency processing of data in a scenario with low requirements for the transaction, and the introduction of the ETL transaction cannot cause a large delay overhead in the distributed write processing (i.e. non-transaction) across data nodes due to the distributed transaction, while the read-write of the non-transaction mainly satisfies: semantically, the isolation of data processing is not satisfied, and the isolation level is read uncompensated (the lowest isolation level), namely dirty reading and dirty writing can be caused; in implementation, a set of distributed transaction components and processing logic: that is, there is no global lock contention on the GTM, no concurrent mutex or serialization processing of read and write on the TC, and only concurrent control of read and write on the TM.
Referring to fig. 1, fig. 1 is a schematic diagram of a transaction processing system according to an embodiment of the present disclosure, which is specifically applied to a distributed real-time data warehouse.
In FIG. 1, the transaction processing system includes a Client, a transaction coordination module 102, a transaction management module 104, and a transaction execution module 106; wherein, the Client corresponds to Client in fig. 1, the transaction coordination module 102 corresponds to the distributed transaction component TC in fig. 1, the transaction management module 104 corresponds to the distributed transaction component GTM in fig. 1, and the transaction execution module 106 corresponds to the distributed transaction component TM in fig. 1.
In practice, the Client (Client) in fig. 1 sends a transaction request to the TC, i.e. STMT (Statement), e.g. Insert into a select from b table, i.e. write the data in table b to table a.
After receiving the SQL statement of the transaction request, the TC determines that the SQL statement is a statement capable of being used as an ETL transaction, at the moment, the SQL statement is analyzed to obtain a to-be-processed data source a table and b table, then a write transaction (i.e. a first data executing transaction) for the a table is initiated to the GTM, after receiving the request sent by the TC, the GTM checks that other write transactions for the a table do not exist at present according to the table identification of the a table, namely that the a table does not exist at present, if the a table is being written, information of a request failure is returned to the TC, if the a table is not written at present, a global lock (i.e. a first operation lock) is set for the a table, then the GTM also creates a corresponding global transaction number GXID (i.e. a first data executing transaction) for the write transaction, and then returns the GXID to the TC according to the table identification of the a table, namely that the first transaction number of the a table is not included in the a table, and a corresponding transaction is completely returned to the corresponding table of the a table (i.e. a snapshot) is completely based on the corresponding table of the first snapshot b, and the corresponding table of the snapshot b is completely executed in the snapshot table (i.e. a snapshot table of the snapshot b).
In practical application, the operation of TC is executed by a controller and is responsible for SQL analysis, plan generation and issuing and data routing.
Referring to fig. 1, another embodiment of the present disclosure provides a transaction processing system, including a transaction coordination module 102, a transaction management module 104, and a transaction execution module 106;
wherein the transaction coordination module 102 is configured to receive a transaction request, and in case the transaction request belongs to a first type, send a pending data source obtained by parsing the transaction request to the transaction management module 104.
In particular, the transaction requests include, but are not limited to, a data write transaction processing request, a data delete transaction processing request, a data modify transaction processing request, and a data update transaction processing request.
The first type is a transaction type, that is, a transaction in a transaction request belongs to an ETL transaction type, and data processing meeting the characteristic of database ACID is performed, for example, the transaction request is insert into select, insert overwrite, update set write, delete from and other sql pattern, that is, the transaction type.
For example, if the transaction request is a insert into a select from b table processing request, it may be determined that the transaction request is insert into select, which is of the first type.
And in the case that the transaction request is determined to be of the first type, sending a to-be-processed data source obtained by analyzing the transaction request to the transaction management module 104, so as to request a global lock for the to-be-processed data source and establish a first data execution transaction for the to-be-processed data source from the transaction management module 104.
In practical application, after receiving the transaction request, the transaction coordination module 102 further determines whether the transaction request can be used as an executable ETL transaction request, so as to avoid resource waste caused by the execution of the invalid transaction request by the system, and the specific implementation manner is as follows:
the transaction coordination module 102 is further configured to:
determining whether the transaction request is an executable transaction request,
if so, under the condition that the transaction request belongs to the first type, sending a data source to be processed obtained by analyzing the transaction request to a transaction management module,
if not, returning the execution failure information.
Specifically, after the transaction coordination module 102 receives the transaction request, it first determines whether the transaction request is an executable transaction request, in this embodiment, the executable transaction request may be understood as a transaction request that satisfies the ETL transaction format, for example, the format of the ETL transaction request is insert into select, the insert overlay, update set, delete from, etc., and if the received transaction request is in the sql language of other formats, it may be determined that the transaction request is an executable transaction request of the system, and then execution failure information is returned to the client, where the cause of the execution failure of the transaction request may be reflected, and the client may request the system to execute again after modifying the transaction request based on the execution failure information.
And in the case that the transaction request is an ETL transaction request executable by the system and the transaction request is of the ETL transaction request type, the data source to be processed obtained by parsing the transaction request is sent to the transaction management module 104 to request a global lock for the data source to be processed and to establish a first data execution transaction for the data source to be processed from the transaction management module 104.
The transaction management module 104 is configured to set a first operation lock for the received data source to be processed, establish a first data execution transaction based on the data source to be processed with the first operation lock set, generate a first identifier for the first data execution transaction, and return the first identifier and the first operation lock to the transaction coordination module 102.
Specifically, after receiving the application of the global lock carrying the data source to be processed sent by the transaction coordination module 102, the transaction management module 104 first checks whether the data source to be processed is currently executed, so as to avoid data confusion caused by repeated execution of other transactions with the data source to be processed, and the specific implementation manner is as follows:
the transaction management module 104 is further configured to:
Receiving the data source to be processed, judging whether the data source to be processed is an executing data source,
if yes, returning the execution failure information,
if not, setting a first operation lock for the received data source to be processed, and establishing a first data execution transaction based on the data source to be processed with the first operation lock.
In practical application, after receiving a data source to be processed, the transaction management module 104 checks whether the data source to be processed is executed by other transactions, if so, returns execution failure information, and returns the execution failure information to the transaction coordination module 102; if not, the data source to be processed can be executed, at this time, a first operation lock is set for the data source to be processed, and a first data execution transaction is established for the data source to be processed, so that the data source to be processed cannot execute the same transaction in the process of executing the first data execution transaction, data isolation is ensured, interference to the execution of the transaction is avoided, and user experience is enhanced.
For example, if the transaction request is a data delete transaction request, the transaction management module 104 returns execution failure information to the transaction coordination module 102 when checking that the data table in the data delete transaction request is being deleted by other delete transactions, if the transaction management module 104 checks that the data table in the data delete transaction is not being executed by other delete transactions, a global lock may be written to the data table at this time, and a data delete transaction is established in the transaction management module 104 based on the data table, so that the data table is not executed by other data delete transactions in the process of being executed by the data delete transaction, that is, the same data source to be processed is not executed by the same type of transaction at the same time.
For a better understanding of the transaction system provided in the embodiments of the present disclosure, the following description will take the transaction request as an example of a data writing transaction.
In the case that the transaction request is a processing request of a data writing transaction, the data source to be processed includes a data source to be read and a data source to be written, and the processing request of the transaction request as Insert into a select from b table is still taken as an example, at this time, the data source to be read is a table b, and the data source to be written is a table a.
Then, the transaction management module 104 is further configured to:
receiving the data source to be written, judging whether the data source to be written is an executing data source,
if yes, returning the execution failure information,
if not, setting a first operation lock for the received data source to be written, and establishing a first data execution transaction based on the data source to be written with the first operation lock.
Specifically, it is determined whether the table a is being written, if yes, execution failure information is returned to the transaction coordination module 102, where the execution failure information may carry information that the table a is being executed by a certain write transaction, if no, the transaction management module 104 may set a write lock for the table a, and establish a first data execution transaction based on the table a, so that when the system executes the first data execution transaction, the table a will not be executed by other write transactions, but at this time, it will not prevent the table a from being read by other tables, and the table b will not be written and read by other data, by adopting this way, the table a will not be executed by other write transactions in the process of executing the write by the first data execution transaction, so as to avoid significant confusion caused by data in the table a, and cause influence on subsequent specific applications based on the data.
While the transaction management module 104 establishes a first data execution transaction for the data source to be processed, a unique first identifier is generated for the first data execution transaction, and the first identifier and the first operation lock are returned to the transaction coordination module 102.
In order to make the system distinguish whether the transaction to be processed belongs to the first type or the second type when executing the transaction, the distinction is made on the established identification of the data processing transaction, and the specific implementation manner is as follows:
the transaction management module is further configured to:
generating an incremental first identification for the first data execution transaction, wherein the first identification and the second identification are positive integers.
Specifically, if the transaction request belongs to the ETL transaction request of the first type, an incremental first identifier is generated for the first data execution transaction, and the first identifier and the second identifier are positive integers.
I.e., the transaction management module 104 will generate a unique, incremented positive integer identification for all ETL transactions belonging to the first type.
In another embodiment of the present disclosure, in a case where the transaction request is a processing request of a non-transaction type, i.e. a processing request of a data processing that does not fully satisfy the database ACID characteristic, for example insert into values, i.e. a non-ETL or the transaction to be processed is a real-time synchronous link, in order to ensure that the system supports, under a set of distributed transaction components, both an ETL transaction with high throughput performance and an execution of the transaction to be processed of the non-transaction, the transaction coordination module 102 is further configured to:
And under the condition that the transaction request is of a second type, establishing a second data execution transaction based on a to-be-processed data source obtained by analyzing the transaction request, and generating a third identification for the second data execution transaction.
In particular, when the transaction request belongs to a transaction request of a non-transaction type, the transaction coordination module 102 establishes a second data execution transaction capable of being executed simultaneously with the first data execution transaction directly based on the data source to be processed obtained by parsing, and generates a third identifier different from the first identifier for the second data execution transaction.
For example, the transaction request is a second type of data write transaction processing request, such as insert into a values processing request, i.e., a value of 100 is inserted into table a, at which time the transaction coordination module 102 establishes a second data execution transaction for table a, such as a write transaction for table a, based on the transaction request, and then generates a third identification for the second data execution transaction, wherein the third identification is negative.
The first data execution transaction and the second data execution transaction can be executed in the system at the same time, and no write-write conflict and no read-write conflict exist; in practical application, insert into values is non-ETL, so that low delay of writing is ensured, and overhead caused by distributed transactions is avoided under the condition that isolation level and read-write visibility are allowed to be relaxed; and the data subscribed to AnalyticDB for MySQL by the synchronization tool is used for avoiding that the writing cannot interrupt tasks due to the failure of integrity check of the transaction ACID and the like, so that the real-time synchronization link defaults to be used as non-transaction writing, the transaction of the first type and the non-transaction of the second type can be processed based on the transaction management system at the same time, and the real-time performance of the database and the high throughput of the data warehouse are considered.
The transaction execution module 106 is configured to generate a second identifier for the data source to be processed according to the first identifier, and process the first data to execute a transaction according to the second identifier and the data source to be processed, when receiving a transaction execution instruction carrying the first identifier sent by the transaction coordination module 102.
Specifically, after receiving the first operation lock returned by the transaction management module 104 and the first identifier of the first data execution transaction, the transaction coordination module 102 sends a transaction execution instruction to the transaction execution module 106, where the transaction execution instruction carries the first identifier of the first data execution transaction, and after receiving the first identifier, the transaction execution module 106 sets a table lock for a data source to be processed in order to ensure that the data source to be processed is not executed by other same transaction when executing the first data execution transaction, and the specific implementation manner is as follows:
the transaction execution module 106 is further configured to:
setting a second operation lock corresponding to the transaction processing request for the data source to be processed;
generating a data snapshot according to the data source to be processed, wherein the data source to be processed is provided with the second operation lock, and generating at least one corresponding second identifier for the data snapshot according to the first identifier;
And processing the first data to execute the transaction according to the second identification and the data snapshot.
Specifically, under the condition that a transaction execution instruction carrying the first identifier sent by the transaction coordination module 102 is received, a second operation station corresponding to the transaction processing request is set for a data source to be processed, for example, the transaction processing request is a processing request of a data writing transaction, and then a writing lock is set for the data source to be processed; and then generating a data snapshot according to the data source to be processed, on which the second operation lock is arranged, and generating at least one corresponding second identifier for the data snapshot according to the first identifier.
In practical applications, the transaction execution modules 106 are multiple, the transaction coordination module 102 issues begin instructions to each transaction execution module 106, each transaction execution module 106 sets its own data source to be processed to be locked and generates a data snapshot, each transaction execution module 106 generates a corresponding second identifier XID according to the first identifier GXID, the first identifier and the second identifier are both established based on the first data execution transaction, the second identifier records the unique identifier of the first data execution transaction on the single transaction execution module 106, if the first data execution transaction is to write the data of the table b into the table a, the data snapshot generated by the data source to be processed is generated, if the first data processing transaction of the transaction execution module 106 is to read the table b from the table b, the second data processing transaction is to read the table b from the table b, the XID of the first data processing transaction is different from the XID of the second data processing transaction, for the table b, but when the two data snapshots of each data processing transaction are read from the table b, the two XID of the transaction are distinguished when the two data snapshots of each transaction are read together.
For example, when the first data executing transaction is writing the data of the b table into the a table and the transaction executing module 106 receives the transaction executing instruction carrying the first identifier sent by the transaction coordination module 102, a second operation write lock corresponding to the transaction to be processed is set for the data source a table to be processed, then the data snapshot is generated for the a table and the b table, so that the data of the data snapshot of the b table is written into the data snapshot of the a table, after the data snapshot is generated for the a table and the b table, the corresponding second identifier is generated for each transaction executing module 106 based on the first identifier, and the second identifier corresponds to the data snapshots of the a table and the b table, finally, when the first data executing transaction is executed, the transaction executing module 106 determines the second identifier of the transaction executing module 106 according to the first identifier of the first data executing transaction, and then the data in the data snapshot of the b table corresponding to the second identifier is written into the data snapshot of the a table corresponding to the second identifier, so as to realize the processing of the first data executing transaction.
According to the transaction processing system provided by the embodiment of the specification, the data is read and written by performing snapshot creation on the transaction execution module 106, namely the TM side, so that the source table data cannot be lost under the condition of data reading or writing failure, and the safety and traceability of the source table data are ensured.
In another embodiment of the present disclosure, at the transaction execution module 106, it is further configured to:
and under the condition that a transaction execution instruction carrying the third identifier is received, which is sent by the transaction coordination module, generating a corresponding fourth identifier according to the third identifier, and processing the second data to execute the transaction according to the fourth identifier, wherein the third identifier and the fourth identifier are negative numbers.
Specifically, in the case that the transaction to be processed is of the second type, the transaction coordination module 102 establishes a second data execution transaction based on a data source to be processed obtained by analyzing the transaction processing request, generates a third identifier which is not mutually exclusive with the first identifier for the second data execution transaction, then sends a transaction execution instruction carrying the third identifier to the transaction execution module 106, and when the transaction execution instruction is received, the transaction execution module 106 generates a corresponding fourth identifier according to the third identifier, and then processes the second data execution transaction according to the fourth identifier.
For example, when the second data execution transaction is insert into a values, that is, the value 100 is inserted into the table a, and the transaction execution module 106 receives the transaction execution instruction carrying the third identifier sent by the transaction coordination module 102, each transaction execution module 106 that receives the transaction execution instruction generates the fourth identifier associated with the third identifier, and when the transaction execution module 106 executes the second data execution transaction, the transaction execution module 106 writes the data in the second data execution transaction into the table a of the transaction execution module 106 of the fourth identifier associated with the third identifier. In implementation, before writing the data into the table a, the transaction execution module 106 also creates a new data snapshot for the table a, so that the data can be written into the new data snapshot of the table a, thereby avoiding data confusion of the table a and enhancing user experience.
In a specific implementation, the transaction coordination module 102 fails after receiving the first identifier returned by the transaction management module 104 and the first operation lock, for example, the first data execution transaction established by the transaction management module 104 is restarted, and if the first data execution transaction is not timely cleaned, the data source to be processed is affected by execution of other transactions, for example, cannot be written by other transactions, so that in the case of the failure of the transaction coordination module 102, the transaction management module 104 needs to be notified to delete the first data execution transaction and unlock the data source to be processed by the first operation lock, which is specifically implemented as follows:
the transaction management module 104 is further configured to:
and under the condition that the fault indication sent by the transaction coordination module 102 is received, unlocking the data source to be processed, which is set with the first operation lock, and deleting the first data to execute the transaction.
In a specific implementation, after receiving a start instruction of the transaction coordination module 102, the transaction execution module 106 performs persistence on a data snapshot of a data source to be processed, if a certain transaction execution module 106 fails in the execution process of performing a first data execution transaction, the transaction execution module 106 can find data written in the data snapshot of the data source to be processed after receiving the start instruction from a log file of the persistence of the data snapshot, and then play back the data from the log file, so that the first data execution transaction can be continuously performed together with other transaction execution modules 106, thereby avoiding all data rollbacks, improving processing efficiency, and the specific implementation manner is as follows:
The transaction execution module is further configured to:
and under the condition that the fault operation is detected, acquiring a log file for realizing the persistence of the data snapshot, and continuing to process the first data execution transaction based on the log file.
In another embodiment of the present disclosure, the transaction management module 104 is further configured to:
and setting a first operation lock for the received data source to be processed under the condition that the data source to be processed is determined to be the data source in execution and the data source to be processed is executed within a preset time period, and establishing a first data execution transaction based on the data source to be processed with the first operation lock set.
Specifically, when the transaction management module 104 determines that the data source to be processed is the data source under execution, but in a preset period of time, the data source to be processed may still be operated, that is, the transaction management module 104 may set a waiting time for the data source to be processed, and after the data source to be processed is executed in the waiting period of time, the operation may be performed based on the data source to be processed, because in practical application, the data source to be processed may be executed by the last transaction in a very short period of time, if only other transactions on the data source to be processed are detected, resource waste may be caused under the condition that the operation is abandoned, and in the next operation of the data source to be processed, there is a blank period, it cannot be guaranteed that no other transaction occupies the data source to be processed again.
In another embodiment of the present disclosure, the transaction execution module 106 is further configured to:
and returning a processing result of the transaction processing request to the transaction coordination module 102 under the condition that the first data is processed according to the second identifier and the data source to be processed, and unlocking the data source to be processed, in which the second operation lock is set.
Specifically, after the transaction execution module 106 successfully executes the first data execution transaction, the execution result is returned to the transaction coordination module 102, and then the transaction coordination module 102 returns the execution result to the client, for example, the first data execution transaction is that the data of the table b is written into the table a, then the transaction execution module 106 returns the data snapshot written into the table a to the transaction coordination module 102, and then the transaction coordination module 102 returns the data snapshot to the client, and the data is processed under the condition that the user of the client is not felt, and only the processed result is returned to the client, so that the user experience is enhanced.
In another embodiment of the present disclosure, the transaction execution module 106 is further configured to:
and under the condition that the first data is processed according to the second identification and the data source to be processed, performing data Rollback, namely data Rollback, and sending transaction processing failure information to the transaction coordination module 102.
Specifically, in the case that the first data execution transaction fails, the data of the transaction execution module 106 rolls back into the persisted data snapshot, that is, when the transaction execution module 106 starts to execute, because the data snapshot of the data source to be processed is persisted when the first data execution transaction starts to execute, the first data execution transaction can be executed again based on the persisted result, and the execution of the second data execution transaction is not affected, so that the normal execution of the second data execution transaction is ensured.
In another embodiment of the present disclosure, the transaction execution module 106 is further configured to:
and carrying out asynchronous expiration deletion on the data snapshot corresponding to the expired second identifier.
Specifically, under the condition that the first data is processed according to the second identifier and the data source to be processed, and the execution transaction is successful, asynchronous expiration deletion is performed on the data snapshot corresponding to the expired second identifier, so that the fact that the past data snapshot occupies a large system storage space and the processing efficiency of a submitting system is avoided.
The transaction processing system provided by the embodiment of the specification can support the ETL with high throughput and high performance under a set of distributed transaction components, the ETL meets transaction semantics, non-transaction reading and writing can be allowed under the scene that the transaction requirement is not high and the data real-time requirement is high, the overhead of the distributed transaction is removed, low-delay data writing and updating are realized, and transaction and non-transaction processing is based on a set of storage media and a set of distributed transaction processing logic, so that the transaction processing system can realize the transaction processing system with both database real-time performance and data warehouse high throughput.
In addition, the non-transactional implementation logic of the embodiments of the present description is:
1. when the GXID of the default non-transaction is-1 and the XID is-1, and the TC processes the data read-write of the non-transaction, the GTM is not required to process the global read-write lock, and the XLOG of the XID= -1 is directly issued, namely, the GTM is not required to acquire the GXID and the global lock.
2. When the TM processes XLOG of XID= -1, on the read-write lock judgment of a single transaction engine (i.e. TM), the shared lock is adopted to directly read and write the latest storage snapshot, and a table can be written simultaneously by a plurality of non-transactional write (insert into select) tasks without considering that the process is in progress, because the XIDs of the tasks are all-1 and are not mutually exclusive by the table lock of the TM.
Referring to fig. 2, fig. 2 shows a flow chart of a transaction processing method according to an embodiment of the present specification, including the following steps:
specifically, the transaction processing method is applied to the transaction processing system and is described in terms of a transaction coordination module.
Step 202: and receiving a transaction request, and sending a data source to be processed obtained by analyzing the transaction request to a transaction management module under the condition that the transaction request belongs to a first type.
Step 204: and receiving a first identification of a first data transaction which is returned by the transaction management module and is established based on a data source to be processed provided with a first operation lock and the first operation lock.
Step 206: and sending a transaction execution instruction carrying the first identifier to a transaction execution module, receiving a second identifier generated by the transaction execution module for the data source to be processed according to the first identifier, and receiving a processing result of processing the first data transaction according to the second identifier and the data source to be processed.
Optionally, after receiving the transaction request, the method further includes:
in the event that the transaction request is of a second type, a second data transaction is established based on a source of pending data obtained by parsing the transaction request, and a third identification is generated for the second data transaction.
Optionally, after generating the third identifier for the second data transaction, the method further includes:
and sending a transaction execution instruction carrying the third identifier to the transaction execution module, receiving a fourth identifier generated by the transaction execution module for the data source to be processed according to the third identifier, and receiving a processing result of processing the second data transaction according to the fourth identifier and the data source to be processed, wherein the third identifier and the fourth identifier are negative numbers.
Optionally, the receiving, by the transaction management module, the first identification of the first data transaction that is set up based on the pending data source with the first operation lock set up includes:
And receiving a first data transaction which is returned by the transaction management module and is established based on a data source to be processed with a first operation lock, and generating an incremental first identification for the first data transaction, wherein the first identification and the second identification are positive integers.
Optionally, after receiving the transaction request, the method further includes:
determining whether the transaction request is an executable transaction request,
if so, under the condition that the transaction request belongs to the first type, sending a data source to be processed obtained by analyzing the transaction request to a transaction management module,
if not, returning the execution failure information.
Optionally, after sending the pending data source obtained by parsing the transaction request to the transaction management module, the method further includes:
and receiving the returned execution failure information of the transaction management module under the condition that the data source to be processed is determined to be the executing data source.
Optionally, the receiving, by the transaction management module, the first identifier of the first data transaction that is set up based on the pending data source with the first operation lock, and the first operation lock includes:
And receiving a first identification of a first data transaction established by the transaction management module based on the data source to be processed provided with a first operation lock and the first operation lock, wherein the first identification is returned by the transaction management module under the condition that the data source to be processed is determined to be the data source to be executed.
Optionally, after sending the transaction execution instruction carrying the first identifier to the transaction execution module, the method further includes:
and receiving a second operation lock corresponding to the transaction processing request, which is set by the transaction execution module for the data source to be processed.
Optionally, the receiving the second identifier generated by the transaction execution module for the data source to be processed according to the first identifier includes:
receiving at least one second identifier generated by the transaction execution module for the data snapshot corresponding to the data source to be processed according to the first identifier; and receiving a processing result of processing the first data transaction according to the second identifier and the data snapshot, wherein the data snapshot is generated according to a data source to be processed, in which the second operation lock is set.
Optionally, after receiving the first data transaction that is returned by the transaction management module and is established based on the data source to be processed and the first identifier that is generated for the first data transaction, the method further includes:
And under the condition that the fault indication is detected, unlocking the data source to be processed, which is provided with the first operation lock, and issuing a deleting instruction for deleting the first data transaction to the transaction management module.
Optionally, the transaction request includes a processing request of a data writing transaction, a processing request of a data deleting transaction, a processing request of a data modifying transaction, and a processing request of a data updating transaction.
Optionally, in the case that the transaction request is a processing request of a data writing transaction, the data source to be processed includes a data source to be read and a data source to be written.
The transaction processing method provided by the embodiment of the specification is applied to an ETL transaction processing device which is AnalyticDB for MySQL and allows non-transaction reading and writing, and under a set of distributed transaction components, not only is the ETL with high throughput and high performance supported, but also the ETL meets transaction semantics, and can allow the non-transaction reading and writing under the scene that the transaction requirement is not high and the data real-time requirement is high, so that the overhead of the distributed transaction is removed, and the low-delay data writing and updating are realized; and transaction and non-transaction processing is based on a set of storage media and a set of distributed transaction processing logic, and is a transaction processing method which takes database instantaneity and data warehouse high throughput into consideration.
In addition, in the transaction processing method provided in the embodiment of the present disclosure, the non-transaction may read and write the data snapshot that is doing the transaction, the non-transaction data may be successfully written into the data snapshot of the transaction, the transaction rollback success may not affect the non-transaction writing, the transaction commit success, the non-transaction writing may also be committed to the online, and the non-transaction writing may also be visible to the client during the period.
The above is an exemplary scheme of a transaction processing method of the present embodiment. It should be noted that, the technical solution of the transaction processing method and the technical solution of the transaction processing system belong to the same concept, and details of the technical solution of the transaction processing method, which are not described in detail, can be referred to the description of the technical solution of the transaction processing system.
The following describes, with reference to fig. 3, an example of application of the transaction processing method provided in the present specification to a distributed database, where a data writing transaction is performed based on the transaction processing method. Fig. 3 shows a process flow chart of a transaction processing method according to an embodiment of the present disclosure, which specifically includes the following steps:
where TC corresponds to the transaction coordination module 102 in the transaction system, gtm corresponds to the transaction management module 104 in the transaction system, TM corresponds to the transaction execution module 106 in the transaction system, and TE is a plurality of TABLEs controlled under TM, i.e., TABLE.
Step 302: the TC receives a transaction request sent by a client: insert into T select from S.
Step 304: the TC requests a global lock and GXID from the GTM based on the transaction request.
Specifically, the TC requests a global lock for the transaction from the GTM based on the transaction request, such that other TCs can no longer open the same transaction on the GTM, and such that the GTM establishes a write transaction of the T table based on the transaction request, and generates a unique identifier, GXID, for the write transaction of the T table in an incremental manner.
Step 306: the GTM establishes a write transaction of the T table based on the transaction request, generates GXID for the write transaction of the T table, and sets a global lock for the write transaction.
Step 308: the GTM returns the global lock and GXID to TC.
Step 310: the TC issues to the TM a start execution instruction of GXID for a write transaction carrying a T-table.
Step 312: and after receiving an instruction for starting execution, the TM locks the table on the T table.
Specifically, the TM is a table lock on the T table to avoid writing to the T table by other TCs.
Step 314: TM generates XID based on GXID of write transaction of T table, and takes table snapshot of T table and S table and persists T table.
The table snapshots of the T table and the S table may also be referred to as globalshapshot, i.e., global snapshot; XID is the local transaction id of this time, and mapping relation exists among the table snapshots of GXID-XID, XID-T table and S table.
The obtained snapshots of the T table and the S table correspond to the current XID, and the data of the S table snapshot which is needed to be written into the T table snapshot in the current writing transaction can be determined through the current XID.
In practical applications, the S-table snapshot may correspond to a plurality of read transactions, and the contents of the data to be read of each read transaction may be different, so when the transaction operation is executed, the GXID determines the corresponding XID first, and then determines which data of the S-table are written into the T-table snapshot corresponding to the XID based on the XID when the data is read based on the S-table snapshot.
For example, the S table corresponds to two read transactions: a first transaction: writing the data of the S table into the T table, and performing a second transaction: the data of the S table is written into the C table, see table 1.
TABLE 1
As can be seen from Table 1, the GXID of the first transaction is 1, the corresponding XIDs are 1.1 and 1.2, and the XIDs correspond to Table Snapshot 1 and Table Snapshot 2 of the S table, table Snapshot 1 and Table Snapshot 2 of the T table; GXID for the second transaction is 2, with corresponding XIDs of 2.1 and 2.2, and table snapshot 1 and table snapshot 2 for the S table, table snapshot 1 and table snapshot 2 for the c table; then, when the first transaction is executed, it may be determined which ones of the TMs execute the first transaction based on the mapping relationship between GXID and XID, and then determine which data in S-table snapshot 1 and table snapshot 2 (which data is obtained from GXID of the first transaction) are written into T-table snapshot 1 and table snapshot 2 based on the mapping relationship between XID and S-table and T-table in table 1; and the execution of the second transaction is the same as the execution of the first transaction.
Step 316: and the TM returns the mapping relation of the XID, the acquired table snapshots of the T table and the S table and the mapping relation of the GXID and the XID, and the mapping relation of the XID, the table snapshots of the T table and the S table to the TC.
Specifically, the TM includes a first TM and a second TM, where the first TM represents a TM associated with the S table, i.e., a TM where the S table is located, and the second TM represents a TM associated with the T table, i.e., a TM where the T table is located.
Step 318: and the TC sends a reading instruction for reading the data in the S table snapshot to the first TM, wherein the reading instruction carries GXID of the writing transaction.
Step 320: the first TM reads the read data in the S table snapshot corresponding to the writing transaction in the TE table based on the read instruction and returns the read data to the TC.
Step 322: and the TC sends a write instruction for writing the T table snapshot to the second TM, wherein the write instruction carries GXID of the write transaction and read data.
Step 324: and the second TM writes the read data into the T table snapshot corresponding to the writing transaction of the corresponding TE table based on the writing instruction, and returns the T table snapshot checkpoint of the written read data to the TC after persistence.
When the read data is written into the corresponding T table snapshot of the TE table corresponding to the writing transaction, the TM locks the lockTable on the TE, namely, the TE locks the TE, so that the TE does not execute other writing transactions when the writing transaction is executed.
Specifically, when the first TM performs a persistence on the data written by the T table snapshot, if there is a TC/TM fail (failure, for example, restart or other failure), the TC may repeat and continue to issue a commit (commit instruction) to complete a new data entry of the current write transaction.
Step 326: the TC sends a preparation, i.e., a Prepare commit instruction, to the TM, wherein the Prepare commit instruction carries a T-table snapshot of write read data and a GXID of a write transaction.
Step 328: and after receiving a ready-to-commit instruction sent by the TC, the TM controls the TE to return writing success information to the TC under the condition that all the T-table snapshots written with the read data are successfully written.
Step 330: and the TC sends a commit instruction to the GTM after receiving the write success information returned by the TM.
Specifically, the TC sends a commit instruction to the GTM to inform the GTM that the execution of the current write transaction is complete, and may delete the current write transaction from its transaction list and release the global lock.
Step 332: and the TC sends a commit instruction to the TM after receiving the write success information returned by the TM, wherein the commit instruction carries a T table snapshot for writing read data and GXID for writing transactions.
Step 334: after receiving the commit instruction of the TC, the TM returns the T table snapshot written with the read data to the TC, and releases the table lock at the TM side.
Step 336: after receiving the T table snapshot of the write-read data returned by the TM, the TC sends information for releasing the global lock to the GTM, and informs the GTM of releasing the global lock.
Step 338: and the TM performs the completion of the write transaction, returns to the TM the end of performing the write transaction, and writes how many pieces of data into the transaction execution information of the T table snapshot.
Step 340: and the TC returns the transaction execution information and the T-table snapshot which receives the written read data to the client after receiving the transaction execution information returned by the TM.
Step 342: the TM asynchronously deletes expired T-table snapshots and S-table snapshots in the TE.
The transaction processing method provided by the embodiment of the specification is applied to an ETL transaction processing device which is AnalyticDB for MySQL and allows non-transaction reading and writing, and under a set of distributed transaction components, not only is the ETL with high throughput and high performance supported, but also the ETL meets transaction semantics, and can allow the non-transaction reading and writing under the scene that the transaction requirement is not high and the data real-time requirement is high, so that the overhead of the distributed transaction is removed, and the low-delay data writing and updating are realized; and transaction and non-transaction processing is based on a set of storage media and a set of distributed transaction processing logic, and is a transaction processing method which takes database instantaneity and data warehouse high throughput into consideration.
Corresponding to the above method embodiments, the present disclosure further provides an embodiment of a transaction processing device, and fig. 4 shows a schematic structural diagram of a transaction processing device provided in one embodiment of the present disclosure. As shown in fig. 4, the apparatus includes:
a first receiving module 402, configured to receive a transaction request, and in a case that the transaction request belongs to a first type, send a data source to be processed obtained by parsing the transaction request to a transaction management module;
a second receiving module 404, configured to receive, returned by the transaction management module, a first identifier of a first data transaction established based on a data source to be processed in which a first operation lock is set, and the first operation lock;
and a third receiving module 406, configured to send a transaction execution instruction carrying the first identifier to a transaction execution module, receive a second identifier generated by the transaction execution module for the data source to be processed according to the first identifier, and receive a processing result of processing the first data transaction according to the second identifier and the data source to be processed.
Optionally, the apparatus further includes:
and the second transaction establishing module is configured to establish a second data transaction based on a to-be-processed data source obtained by analyzing the transaction request and generate a third identification for the second data transaction under the condition that the transaction request belongs to a second type.
Optionally, the apparatus further includes:
the identifier generating module is configured to send a transaction execution instruction carrying the third identifier to the transaction execution module, receive a fourth identifier generated by the transaction execution module for the data source to be processed according to the third identifier, and receive a processing result of processing the second data transaction according to the fourth identifier and the data source to be processed, wherein the third identifier and the fourth identifier are negative numbers.
Optionally, the second receiving module is further configured to:
and receiving a first data transaction which is returned by the transaction management module and is established based on a data source to be processed with a first operation lock, and generating an incremental first identification for the first data transaction, wherein the first identification and the second identification are positive integers.
Optionally, the apparatus further includes:
a first determination module configured to determine whether the transaction request is an executable transaction request,
if so, under the condition that the transaction request belongs to the first type, sending a data source to be processed obtained by analyzing the transaction request to a transaction management module,
If not, returning the execution failure information.
Optionally, the apparatus further includes:
and the fourth receiving module is configured to receive the execution failure information returned by the transaction management module when the data source to be processed is determined to be the executing data source.
Optionally, the second receiving module is further configured to:
and receiving a first identification of a first data transaction established by the transaction management module based on the data source to be processed provided with a first operation lock and the first operation lock, wherein the first identification is returned by the transaction management module under the condition that the data source to be processed is determined to be the data source to be executed.
Optionally, the apparatus further includes:
and a fifth receiving module configured to receive a second operation lock corresponding to the transaction request, which is set by the transaction executing module for the pending data source.
Optionally, the third receiving module is further configured to:
receiving at least one second identifier generated by the transaction execution module for the data snapshot corresponding to the data source to be processed according to the first identifier; and receiving a processing result of processing the first data transaction according to the second identifier and the data snapshot, wherein the data snapshot is generated according to a data source to be processed, in which the second operation lock is set.
Optionally, the apparatus further includes:
and the unlocking module is configured to unlock the data source to be processed, which is provided with the first operation lock, under the condition that the fault indication is detected, and issue a deleting instruction for deleting the first data transaction to the transaction management module.
Optionally, the transaction request includes a processing request of a data writing transaction, a processing request of a data deleting transaction, a processing request of a data modifying transaction, and a processing request of a data updating transaction.
Optionally, in the case that the transaction request is a processing request of a data writing transaction, the data source to be processed includes a data source to be read and a data source to be written.
The above is an exemplary scheme of a transaction processing device of the present embodiment. It should be noted that, the technical solution of the transaction processing device and the technical solution of the transaction processing system belong to the same concept, and details of the technical solution of the transaction processing device, which are not described in detail, can be referred to the description of the technical solution of the transaction processing system.
Fig. 5 illustrates a block diagram of a computing device 500 provided in accordance with one embodiment of the present description. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. Processor 520 is coupled to memory 510 via bus 530 and database 550 is used to hold data.
Computing device 500 also includes access device 540, access device 540 enabling computing device 500 to communicate via one or more networks 560. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. The access device 540 may include one or more of any type of network interface, wired or wireless (e.g., a Network Interface Card (NIC)), such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present description, the above-described components of computing device 500, as well as other components not shown in FIG. 5, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device shown in FIG. 5 is for exemplary purposes only and is not intended to limit the scope of the present description. Those skilled in the art may add or replace other components as desired.
Computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smart phone), wearable computing device (e.g., smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 500 may also be a mobile or stationary server.
Wherein the processor 520 is configured to execute computer-executable instructions that, when executed by the processor, perform the steps of the transaction method.
The foregoing is a schematic illustration of a computing device of this embodiment. It should be noted that, the technical solution of the computing device and the technical solution of the transaction processing method belong to the same concept, and details of the technical solution of the computing device, which are not described in detail, can be referred to the description of the technical solution of the transaction processing method.
An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the transaction method.
The above is an exemplary version of a computer-readable storage medium of the present embodiment. It should be noted that, the technical solution of the storage medium and the technical solution of the transaction processing method belong to the same concept, and details of the technical solution of the storage medium which are not described in detail can be referred to the description of the technical solution of the transaction processing method.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The computer instructions include computer program code that may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
It should be noted that, for simplicity of description, the foregoing method embodiments are all expressed as a series of combinations of actions, but it should be understood by those skilled in the art that the embodiments are not limited by the order of actions described, as some steps may be performed in other order or simultaneously according to the embodiments of the present disclosure. Further, those skilled in the art will appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily all required for the embodiments described in the specification.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
The preferred embodiments of the present specification disclosed above are merely used to help clarify the present specification. Alternative embodiments are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the teaching of the embodiments. The embodiments were chosen and described in order to best explain the principles of the embodiments and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. This specification is to be limited only by the claims and the full scope and equivalents thereof.

Claims (26)

1. A transaction processing system, comprising:
the transaction coordination module is configured to receive a transaction request, and send a data source to be processed obtained by analyzing the transaction request to the transaction management module when the transaction request belongs to a first type;
the transaction management module is configured to set a first operation lock for the received data source to be processed, establish a first data execution transaction based on the data source to be processed, generate a first identifier for the first data execution transaction, and return the first identifier and the first operation lock to the transaction coordination module;
And the transaction execution module is configured to generate a second identifier for the data source to be processed according to the first identifier under the condition that a transaction execution instruction carrying the first identifier and sent by the transaction coordination module is received, and process the first data to execute the transaction according to the second identifier and the data source to be processed.
2. The transaction system of claim 1, the transaction coordination module further configured to:
and under the condition that the transaction request is of a second type, establishing a second data execution transaction based on a to-be-processed data source obtained by analyzing the transaction request, and generating a third identification for the second data execution transaction.
3. The transaction processing system of claim 2, the transaction execution module further configured to:
and under the condition that a transaction execution instruction carrying the third identifier is received, which is sent by the transaction coordination module, generating a corresponding fourth identifier according to the third identifier, and processing the second data to execute the transaction according to the fourth identifier, wherein the third identifier and the fourth identifier are negative numbers.
4. The transaction processing system of claim 3, the transaction management module further configured to:
Generating an incremental first identification for the first data execution transaction, wherein the first identification and the second identification are positive integers.
5. The transaction system of claim 1, the transaction coordination module further configured to:
determining whether the transaction request is an executable transaction request,
if so, under the condition that the transaction request belongs to the first type, sending a data source to be processed obtained by analyzing the transaction request to a transaction management module,
if not, returning the execution failure information.
6. The transaction processing system of claim 1, the transaction management module further configured to:
receiving the data source to be processed, judging whether the data source to be processed is an executing data source,
if yes, returning the execution failure information,
if not, setting a first operation lock for the received data source to be processed, and establishing a first data execution transaction based on the data source to be processed with the first operation lock.
7. The transaction system of claim 1, the transaction execution module further configured to:
Setting a second operation lock corresponding to the transaction processing request for the data source to be processed;
generating a data snapshot according to the data source to be processed, wherein the data source to be processed is provided with the second operation lock, and generating at least one corresponding second identifier for the data snapshot according to the first identifier;
and processing the first data to execute the transaction according to the second identification and the data snapshot.
8. The transaction processing system of claim 6, the transaction management module further configured to:
and under the condition that a fault indication sent by the transaction coordination module is received, unlocking the data source to be processed, which is provided with the first operation lock, and deleting the first data execution transaction.
9. The transaction system of claim 7, the transaction execution module further configured to:
and under the condition that the fault operation is detected, acquiring a log file for realizing the persistence of the data snapshot, and continuing to process the first data execution transaction based on the log file.
10. The transaction processing system of claim 1, the transaction management module further configured to:
and setting a first operation lock for the received data source to be processed under the condition that the data source to be processed is determined to be the data source in execution and the data source to be processed is executed within a preset time period, and establishing a first data execution transaction based on the data source to be processed with the first operation lock set.
11. The transaction system of claim 7, the transaction execution module further configured to:
and returning a processing result aiming at the transaction processing request to the transaction coordination module under the condition that the first data is processed according to the second identification and the data source to be processed, and unlocking the data source to be processed, on which the second operation lock is arranged.
12. The transaction system of claim 11, the transaction execution module further configured to:
and under the condition that the first data is processed according to the second identification and the data source to be processed and the first data fails to execute the transaction, carrying out data rollback, and sending transaction processing failure information to the transaction coordination module.
13. The transaction system of claim 7, the transaction execution module further configured to:
and carrying out asynchronous expiration deletion on the data snapshot corresponding to the expired second identifier.
14. A transaction processing method, comprising:
receiving a transaction request, and sending a data source to be processed obtained by analyzing the transaction request to a transaction management module under the condition that the transaction request belongs to a first type;
Receiving a first identification of a first data transaction and the first operation lock, wherein the first identification is returned by the transaction management module and is established based on a data source to be processed provided with the first operation lock;
and sending a transaction execution instruction carrying the first identifier to a transaction execution module, receiving a second identifier generated by the transaction execution module for the data source to be processed according to the first identifier, and receiving a processing result of processing the first data transaction according to the second identifier and the data source to be processed.
15. The transaction method of claim 14, further comprising, after receiving the transaction request:
in the event that the transaction request is of a second type, a second data transaction is established based on a source of pending data obtained by parsing the transaction request, and a third identification is generated for the second data transaction.
16. The transaction processing method of claim 15, further comprising, after generating a third identification for the second data transaction:
and sending a transaction execution instruction carrying the third identifier to the transaction execution module, receiving a fourth identifier generated by the transaction execution module for the data source to be processed according to the third identifier, and receiving a processing result of processing the second data transaction according to the fourth identifier and the data source to be processed, wherein the third identifier and the fourth identifier are negative numbers.
17. The transaction processing method of claim 16, wherein receiving a first identification of a first data transaction established by the transaction management module based on a pending data source having a first operation lock set comprises:
and receiving a first data transaction which is returned by the transaction management module and is established based on a data source to be processed with a first operation lock, and generating an incremental first identification for the first data transaction, wherein the first identification and the second identification are positive integers.
18. The transaction method of claim 14, further comprising, after receiving the transaction request:
determining whether the transaction request is an executable transaction request,
if so, under the condition that the transaction request belongs to the first type, sending a data source to be processed obtained by analyzing the transaction request to a transaction management module,
if not, returning the execution failure information.
19. The transaction method according to claim 14, further comprising, after the sending the pending data source obtained by parsing the transaction request to a transaction management module:
and receiving the returned execution failure information of the transaction management module under the condition that the data source to be processed is determined to be the executing data source.
20. The transaction processing method of claim 19, the receiving a first identification of a first data transaction established by the transaction management module based on a pending data source having a first operation lock set, and the first operation lock comprising:
and receiving a first identification of a first data transaction established by the transaction management module based on the data source to be processed provided with a first operation lock and the first operation lock, wherein the first identification is returned by the transaction management module under the condition that the data source to be processed is determined to be the data source to be executed.
21. The transaction processing method of claim 14, further comprising, after sending the transaction execution instruction carrying the first identification to a transaction execution module:
and receiving a second operation lock corresponding to the transaction processing request, which is set by the transaction execution module for the data source to be processed.
22. The transaction method of claim 21, the receiving a second identification generated by the transaction execution module for the pending data source based on the first identification comprising:
receiving at least one second identifier generated by the transaction execution module for the data snapshot corresponding to the data source to be processed according to the first identifier; and receiving a processing result of processing the first data transaction according to the second identifier and the data snapshot, wherein the data snapshot is generated according to a data source to be processed, in which the second operation lock is set.
23. The transaction processing method according to claim 20, further comprising, after receiving a first data transaction established based on the pending data source and a first identification generated for the first data transaction returned by the transaction management module:
and under the condition that the fault indication is detected, unlocking the data source to be processed, which is provided with the first operation lock, and issuing a deleting instruction for deleting the first data transaction to the transaction management module.
24. A transaction processing apparatus, comprising:
the first receiving module is configured to receive a transaction request, and send a data source to be processed obtained by analyzing the transaction request to the transaction management module when the transaction request belongs to a first type;
the second receiving module is configured to receive a first identification of a first data transaction which is returned by the transaction management module and is established based on a data source to be processed provided with a first operation lock and the first operation lock;
the third receiving module is configured to send a transaction execution instruction carrying the first identifier to the transaction execution module, receive a second identifier generated by the transaction execution module for the data source to be processed according to the first identifier, and receive a processing result of processing the first data transaction according to the second identifier and the data source to be processed.
25. A computing device, comprising:
a memory and a processor;
the memory is configured to store computer executable instructions and the processor is configured to execute the computer executable instructions, wherein the processor, when executing the computer executable instructions, performs the steps of the transaction method of any of claims 14 to 23.
26. A computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the transaction method of any of claims 14 to 23.
CN202010525627.4A 2020-06-10 2020-06-10 Transaction processing system, transaction processing method and device Active CN113296895B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010525627.4A CN113296895B (en) 2020-06-10 2020-06-10 Transaction processing system, transaction processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010525627.4A CN113296895B (en) 2020-06-10 2020-06-10 Transaction processing system, transaction processing method and device

Publications (2)

Publication Number Publication Date
CN113296895A CN113296895A (en) 2021-08-24
CN113296895B true CN113296895B (en) 2024-01-23

Family

ID=77318654

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010525627.4A Active CN113296895B (en) 2020-06-10 2020-06-10 Transaction processing system, transaction processing method and device

Country Status (1)

Country Link
CN (1) CN113296895B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018223789A1 (en) * 2017-06-05 2018-12-13 中兴通讯股份有限公司 Method and system for performing operation with respect to transaction id, and computer readable storage medium
CN111259083A (en) * 2020-02-13 2020-06-09 神州数码融信软件有限公司 Distributed transaction processing method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7289992B2 (en) * 2003-05-01 2007-10-30 International Business Machines Corporation Method, system, and program for lock and transaction management

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018223789A1 (en) * 2017-06-05 2018-12-13 中兴通讯股份有限公司 Method and system for performing operation with respect to transaction id, and computer readable storage medium
CN111259083A (en) * 2020-02-13 2020-06-09 神州数码融信软件有限公司 Distributed transaction processing method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
多数据库系统中的一致性维护;卢正鼎, 杨玉萍, 李长磊, 肖卫军;计算机研究与发展(第02期);全文 *

Also Published As

Publication number Publication date
CN113296895A (en) 2021-08-24

Similar Documents

Publication Publication Date Title
CN107977376B (en) Distributed database system and transaction processing method
CN109739935B (en) Data reading method and device, electronic equipment and storage medium
US9418135B2 (en) Primary database system, replication database system and method for replicating data of a primary database system
EP3120261B1 (en) Dependency-aware transaction batching for data replication
US7376675B2 (en) Simulating multi-user activity while maintaining original linear request order for asynchronous transactional events
EP2595068B1 (en) System and method of performing snapshot isolation in distributed databases
CN110196856B (en) Distributed data reading method and device
CN111858629B (en) Implementation method and device for two-stage submitting distributed transaction update database
KR101993432B1 (en) Systems and methods for supporting transaction recovery based on a strict ordering of two-phase commit calls
CN105574187B (en) A kind of Heterogeneous Database Replication transaction consistency support method and system
EP3722973B1 (en) Data processing method and device for distributed database, storage medium, and electronic device
EP4276651A1 (en) Log execution method and apparatus, and computer device and storage medium
CN109783578B (en) Data reading method and device, electronic equipment and storage medium
WO2023082992A1 (en) Data processing method and system
WO2022213526A1 (en) Transaction processing method, distributed database system, cluster, and medium
CN107533474A (en) A kind of transaction methods and device
TWI774643B (en) Database operation method and device
CN110955719B (en) Data access processing equipment, system and method
CN113297159B (en) Data storage method and device
CN115495495A (en) Transaction processing method, distributed database system, cluster and medium
CN113296895B (en) Transaction processing system, transaction processing method and device
CN113297231A (en) Database processing method and device
CN114217978B (en) Database transaction processing method, system, computing device and computer readable storage medium based on optimistic lock
CN115774754A (en) Metadata management method, device, equipment and medium based on distributed transaction
CN112612647B (en) Log parallel replay method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40057487

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant