CN115509694A - Transaction processing method and device, electronic equipment and storage medium - Google Patents

Transaction processing method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115509694A
CN115509694A CN202211237963.4A CN202211237963A CN115509694A CN 115509694 A CN115509694 A CN 115509694A CN 202211237963 A CN202211237963 A CN 202211237963A CN 115509694 A CN115509694 A CN 115509694A
Authority
CN
China
Prior art keywords
transaction
key
executed
write set
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211237963.4A
Other languages
Chinese (zh)
Other versions
CN115509694B (en
Inventor
邓云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Volcano Engine Technology Co Ltd
Original Assignee
Beijing Volcano Engine Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Volcano Engine Technology Co Ltd filed Critical Beijing Volcano Engine Technology Co Ltd
Priority to CN202211237963.4A priority Critical patent/CN115509694B/en
Publication of CN115509694A publication Critical patent/CN115509694A/en
Application granted granted Critical
Publication of CN115509694B publication Critical patent/CN115509694B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/466—Transaction processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/242—Query formulation
    • G06F16/2433—Query languages
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present disclosure provides a transaction processing method, an apparatus, an electronic device, and a storage medium, wherein a first write set of a currently executed transaction is determined according to transaction record log information of a primary database, and the first write set includes key values of multiple types of keys, which are used for uniquely characterizing the currently executed transaction and statements in the currently executed transaction; determining a to-be-executed transaction in an executable state, and determining a second write set of the to-be-executed transaction according to transaction record log information of the primary database, wherein the second write set comprises key values of multiple types of keys and is used for uniquely representing the to-be-executed transaction and each statement in the to-be-executed transaction; and judging whether each statement included in the transaction to be executed conflicts with the current execution transaction or not according to the key values of the various keys in the first write set and the key values of the various keys in the second write set, and executing any statement in a front-end mode when any statement is determined not to conflict with the current execution transaction.

Description

Transaction processing method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a transaction processing method and apparatus, an electronic device, and a storage medium.
Background
The core problem affecting the synchronization performance of Data replication between instances of database MySQL, data Transmission Service (DTS), and the like is the concurrency analysis of transactions, which needs to be able to coordinate conflicts between transactions and to improve the concurrency of playback transactions.
In the related technology, transaction update data and transaction concurrency related information can be recorded in a binary log (Binlog) on a master node based on a method of a write set (WriteSet) and a logic clock, and then a slave node plays back transactions concurrently according to the Binlog of the master node, so that the purpose of data consistency with the master node is achieved.
Disclosure of Invention
The embodiment of the disclosure at least provides a transaction processing method, a transaction processing device, an electronic device and a storage medium.
In a first aspect, an embodiment of the present disclosure provides a transaction processing method, including:
determining a first write set of a currently executed transaction according to transaction record log information of a primary database, wherein the first write set comprises key values of multiple types of keys and is used for uniquely representing the currently executed transaction and statements in the currently executed transaction;
determining a transaction to be executed in an executable state, and determining a second write set of the transaction to be executed according to transaction record log information of the primary database, wherein the second write set comprises key values of multiple types of keys and is used for uniquely characterizing the transaction to be executed and each statement in the transaction to be executed;
and judging whether each statement included in the transaction to be executed conflicts with the current execution transaction or not according to the key values of the various types of keys in the first write set and the key values of the various types of keys in the second write set, and pre-executing any statement when determining that any statement does not conflict with the current execution transaction.
In an optional implementation manner, if the first write set or the second write set is used as a target write set, and the currently executed transaction or the to-be-executed transaction is used as a target transaction, determining a target write set of the target transaction according to transaction record log information of a master database, includes:
determining a global key, a table key and a main key and/or a unique key of each statement included in a target writing set of the target transaction according to the transaction record log information of the main database;
wherein the global key represents a global lock of the target transaction, and the table key represents a table-level lock of at least one table corresponding to the target transaction; the primary key and the unique key characterize a statement lock; and a lock priority is such that the global key is larger than the table key, which is larger than the primary key and the unique key.
In an optional embodiment, determining, according to the transaction record log information of the master database, that the target write set of the target transaction includes a global key, a table key, and a master key and/or a unique key of each statement includes:
if a preset target condition exists, determining a global key or a table key included in a target writing set of the target transaction; if the target condition does not exist, determining the main key and/or the unique key of each statement included in the target write set of the target transaction;
wherein the target condition comprises: if the target transaction comprises a Database Definition Language (DDL) statement or a first transaction in a transaction record log file included in the transaction record log information, determining that a target write set of the target transaction comprises the global key; if the target transaction does not have a primary key table or comprises at least one target type table, determining that the target write set of the target transaction at least comprises the table key, wherein the target type table is: there is a table of unique index statements, a parent table of foreign key constraints.
In an optional implementation manner, determining whether each statement included in the transaction to be executed conflicts with the currently executed transaction according to the key values of the various types of keys in the first write set and the key values of the various types of keys in the second write set includes:
respectively comparing a second write set corresponding to each statement included in the transaction to be executed with the first write set of the current transaction to be executed;
and sequentially judging whether the global key, the table key and the main key or the unique key of the transaction to be executed have intersection with the first write set according to the priority of the lock, determining that a corresponding statement conflicts with the current execution transaction under the condition of determining that the intersection exists, and determining that the corresponding statement does not conflict with the current execution transaction under the condition of determining that the intersection does not exist.
In an optional implementation manner, the sequentially determining, according to the lock priority, whether the global key, the table key, and the primary key or the unique key of the transaction to be executed intersect with the first write set, determining that a corresponding statement conflicts with the currently executed transaction if it is determined that the intersection exists, and determining that the corresponding statement does not conflict with the currently executed transaction if it is determined that no intersection exists, includes:
according to the priority of the lock, if the global key of the transaction to be executed has an intersection with the first write set, determining that all statements included in the transaction to be executed conflict with the current transaction to be executed;
if the global key of the to-be-executed transaction does not have an intersection with the first write set and the table key of the to-be-executed transaction has an intersection with the first write set, all statements under the table key of the to-be-executed transaction having the intersection conflict with the currently-executed transaction;
if the global key, the table key and the first write set of the transaction to be executed do not have an intersection, and the unique key or the main key of the transaction to be executed has an intersection with the first write set, determining that the statement corresponding to the unique key or the main key having the intersection conflicts with the currently executed transaction.
In an alternative embodiment, before determining the first write set of the currently executed transaction according to the transaction record log information of the master database, the method further includes:
and after any transaction is executed, determining the current execution transaction according to the logic clock of the transaction record log information of the master database, wherein the current execution transaction at least comprises the next execution sequence transaction corresponding to any transaction after execution is completed.
In an optional embodiment, determining a transaction to be executed in an executable state includes:
and determining the next transaction in the execution sequence corresponding to the current execution transaction as the to-be-executed transaction in an executable state according to the logic clock of the transaction record log information of the main database.
In a second aspect, an embodiment of the present disclosure further provides a transaction processing apparatus, including:
a first determining module, configured to determine a first write set of a currently executed transaction according to transaction record log information of a primary database, where the first write set includes key values of multiple types of keys, and is used to uniquely characterize the currently executed transaction and statements in the currently executed transaction;
a second determining module, configured to determine a to-be-executed transaction in an executable state, and determine a second write set of the to-be-executed transaction according to transaction record log information of the primary database, where the second write set includes key values of multiple types of keys, and is used to uniquely characterize the to-be-executed transaction and statements in the to-be-executed transaction;
and the execution module is used for judging whether each statement included in the transaction to be executed conflicts with the current execution transaction or not according to the key values of the various types of keys in the first write set and the key values of the various types of keys in the second write set, and pre-executing any statement when determining that any statement does not conflict with the current execution transaction.
In a third aspect, this disclosure also provides an electronic device including a processor, a memory, and a processor, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions are executed by the processor to perform the steps in the first aspect or any possible implementation manner of the first aspect.
In a fourth aspect, alternative implementations of the present disclosure also provide a computer-readable storage medium having a computer program stored thereon, where the computer program is executed by a processor to perform the steps of the first aspect or any one of the possible implementations of the first aspect.
In the embodiment of the disclosure, a first write set of a currently executed transaction is determined according to transaction record log information of a primary database, the transaction to be executed in an executable state is determined, a second write set of the transaction to be executed is determined according to the transaction record log information of the primary database, and then, according to key values of various types of keys in the first write set and key values of various types of keys in the second write set, whether each statement included in the transaction to be executed conflicts with the currently executed transaction is judged, and when it is determined that any statement does not conflict with the currently executed transaction, the statement is executed in a pre-positioned manner.
For the description of the effects of the transaction processing apparatus, the electronic device, and the computer-readable storage medium, reference is made to the description of the transaction processing method, which is not repeated herein.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the technical aspects of the disclosure.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is to be understood that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art to which the disclosure pertains without the benefit of the inventive faculty, and that additional related drawings may be derived therefrom.
FIG. 1 shows a schematic diagram of MySQL master-slave replication principle in the related art;
FIG. 2 is a flow chart illustrating a transaction processing method provided by an embodiment of the present disclosure;
FIG. 3 illustrates a lock priority diagram provided by an embodiment of the present disclosure;
FIG. 4 is a schematic diagram illustrating an application scenario of transaction processing provided by an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a transaction device provided by an embodiment of the disclosure;
fig. 6 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
Detailed Description
It is understood that, before the technical solutions disclosed in the embodiments of the present disclosure are used, the user should be informed of the type, the use range, the use scene, etc. of the personal information related to the present disclosure in a proper manner according to the relevant laws and regulations and obtain the authorization of the user.
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of embodiments of the present disclosure, as generally described and illustrated herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making any creative effort, shall fall within the protection scope of the disclosure.
For the convenience of understanding the technical solution of the present disclosure, technical terms in the embodiments of the present disclosure are first explained:
data Transmission Service (DTS): the data interaction method is a real-time data stream service provided by a certain manufacturer, supports data interaction among data sources such as a relational database, a non-relational database, data multidimensional analysis and the like, and comprises services such as data synchronization, migration, subscription, integration, processing and the like.
Transaction: the system is a logical unit of a database management system in the process of executing operation, is composed of a limited database operation sequence, is a minimum execution unit of the database management system operation, and can be understood as a minimum non-separable work unit, usually one transaction corresponds to a complete service, and a complete service is usually completed by jointly combining a plurality of Data Manipulation Language (DML) statements.
Binary journal (Binlog): binlog is a very important log of MySQL, records all Database Definition Language (DDL) and DML statements (except for a Data query statement select), and also includes the time consumed by the execution of the statements, and generally, binlog mainly includes two types of files: the system comprises a binary log index file and a binary log file, wherein the binary log index file is used for recording all binary files and the binary log file is used for recording all DDL (data query statement select) and DML (data query statement select) statement events of a database, for example, the transaction log information of a main database in the embodiment of the disclosure can be Binlog, and the transaction log information comprises a plurality of transaction log files.
DDL statement: the main commands include create (create), modify (alter), drop (drop), etc., and the DDL is mainly an initial job for defining or changing the structure of tables, data types, connections and constraints between tables, etc., and is generally used when building tables.
DML statement: the main commands include select, update, insert, delete, etc. for operating on data in the database.
Log sequence number (Log sequence number, LSN): the logical sequence number representing the transaction is mainly used for identifying the position in the Binlog, maintaining the consistency and the integrity of the database, and plays a very important role in the implementation and rollback operation, the crash and the forward roll recovery and the synchronization of the database operation in the partitioned database environment, the value of the LSN is gradually increased along with the writing of each transaction, and the value of the LSN in the new Binlog restarts counting each time.
Log sequence number of Last commit completed (Last Committed LSN): indicating that the current transaction is at the maximum Committed LSN before completion Prepare (Prepare) and start Commit (Commit), i.e. the LSN of the Last transaction on which the current transaction depends, it can be simply understood that if the transactions less than or equal to Last Committed LSN of the current transaction have been executed and completed, the current transaction can start executing, and the transactions with the same Last Committed LSN can be grouped to indicate that they can be executed or played back in parallel without conflict.
Write set (WriteSet): writeSet is a transaction conflict detection technology with finer granularity, and is used for processing Last Committed LSN of a transaction on the basis of a logic clock, and the basic principle is as follows: a) Transactions without data conflicts can be written concurrently; b) The existence of an intersection of the WriteSet of a transaction indicates a data conflict; c) The WriteSet represents a set of row key Hash values (RowKey Hash) of transaction changes or dependencies, and specifically includes the following expression: if a transaction is T, which includes n rows (n statements), T = { r = { r } 1 ,…,r n }，
Figure BDA0003878787170000081
Figure BDA0003878787170000082
Wherein r is i For line i of transaction T, priKey represents the primary key, uniqKey represents the unique key, and ForeignKey represents the foreign key.
d) For ease of calculation, each RowKey may be derived to compute the Hash value in a set format, for example as follows:
table 1.
Figure BDA0003878787170000083
Primary Key (Primary Key): if a column uniquely identifies each row in a table, that column may be referred to as the primary key of the table, and only one primary key is allowed in a table, i.e., only one column can be designated as the primary key, the primary key does not accept duplicate values and NULL values, i.e., the key values of the primary keys of each row are not the same and are not NULL values.
Unique bond (Unique Key): a unique key may also uniquely identify each row in the table, unlike a primary key, which allows NULL values, and multiple unique keys in a table, i.e., multiple columns may be used as unique keys.
External bond (Foreign Key): foreign keys are used to associate with other tables.
Research shows that the core problem that the synchronization performance is affected by Data replication between examples of the database MySQL, data Transmission Service (DTS) and the like is concurrent analysis of transactions, and not only can the conflict between the transactions be coordinated, but also the concurrency of the playback transactions can be improved.
In the related art, a logic clock and a WriteSet may be introduced on the basis of Binlog to handle a multi-transaction concurrency problem, for example, referring to fig. 1, as a schematic diagram of a MySQL master-slave replication principle in the related art, a master node (master) records transaction update and transaction concurrency related information in the Binlog, the Binlog may be multiple, when a slave node (slave) is normally connected to the master node, the master node may synchronize the updated Binlog to the slave node, an Input/Output (I/O) thread of the slave node may read the Binlog of the master node and copy the Binlog to a local relay log (RelayLog), an SQL thread of the slave node reads the RelayLog from the local, plays back a transaction submitted on a database locally, and may execute the transaction through multiple Worker (Worker) threads, thereby achieving the purpose of keeping data consistent with the master node.
To improve the transaction concurrency performance, mysql5.7 introduces a Logical Clock (LC) mechanism, and the principle of the method is mainly to add two hint (hit) fields of LSN and Last committed LSN in Binlog for providing the information of data concurrency for the slave nodes. After mysql5.7.21, on the basis of LC, a WriteSet mechanism is newly added, and the WriteSet mechanism is a switchable function, if it is turned on, the master node can perform WriteSet analysis on the transaction to improve the concurrency effect of the logical clock, the basic principle is to obtain the WriteSet of the transaction by calculating its Primary Key, uniqueKey and ForeignKey, and then determine the conflict dependency between the transactions by comparing the writesets, if the writesets of two transactions intersect, the two transactions have conflict, but because of some legacy characteristics of MySQL, the data conflict of the transaction cannot be completed by only the WriteSet analysis, such as the operations of case, UPDATE (UPDATE), DELETE (DELETE) and the like of foreign Key, mySQL WriteSet in the related art is only an optimization method used on the master node, mySQL optimizes the Last in Binlog through the WriteSet, and optimizes the Last information in Binlog, and does not sense the concurrency effect of the WriteSet, and does not actually improve the concurrency effect of the transaction, and MySQL does not sense the concurrency information on the WriteSet.
However, in the method of the related art, the slave node only depends on the logic clock of Binlog to determine the execution sequence of each transaction, for a transaction with a conflict, the transaction needs to be executed until the execution of the transaction with the conflict is finished, and one transaction may involve operations of reading data, checking service, updating data and the like, so that the whole execution process may take a long time, the throughput of service processing is limited, the concurrency of transaction playback is limited, and the processing efficiency is reduced.
Based on the above research, the present disclosure provides a transaction processing method, which determines a first write set of a currently executed transaction according to transaction record log information of a primary database, determines a to-be-executed transaction in an executable state, determines a second write set of the to-be-executed transaction according to transaction record log information of the primary database, and further determines whether each statement included in the to-be-executed transaction conflicts with the currently executed transaction according to key values of various types of keys in the first write set and key values of various types of keys in the second write set, and performs pre-execution of any statement when it is determined that any statement does not conflict with the currently executed transaction.
The above drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above problems and the solutions proposed by the present disclosure in the following description should be the contribution of the inventor to the present disclosure in the course of the present disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
To facilitate understanding of this embodiment, first, a transaction processing method disclosed in an embodiment of the present disclosure is described in detail, where an execution subject of the transaction processing method provided in the embodiment of the present disclosure is generally an electronic device with certain computing capability, and the electronic device includes, for example: a terminal device or a server or other processing devices, where the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, and the like, where the PDA is a handheld electronic device having some functions of an electronic computer, and may be used to manage Personal information, browse through the internet, send and receive electronic mails, and the like, and is generally not equipped with a keyboard or may be called a palm computer. In some possible implementations, the transaction processing method may be implemented by a processor calling computer readable instructions stored in a memory.
The following describes a transaction processing method provided by the embodiment of the present disclosure by taking an execution subject as a server.
Referring to fig. 2, a flowchart of a transaction processing method provided in an embodiment of the present disclosure is shown, where the method includes:
s201: and determining a first write set of the currently executed transaction according to the transaction record log information of the primary database, wherein the first write set comprises key values of various types of keys and is used for uniquely characterizing the currently executed transaction and statements in the currently executed transaction.
In the related art, only conflict analysis at the transaction level can be realized in the process of replaying the transaction, and in the embodiment of the disclosure, in order to improve the concurrency to the maximum extent, the WriteSet of the transaction is directly reconstructed, so that conflict judgment at the statement level is realized.
Specifically, when the step S201 is executed, the embodiment of the present disclosure provides a possible manner: acquiring the transaction record log information of a master database, and determining a first write set of the currently executed transaction according to the transaction record log information of the master database.
In which, the transaction record log information is, for example, binlog, and changes of all data in the main database are recorded in the Binlog, so that, in the embodiment of the present disclosure, the WriteSet may be calculated based on the Binlog.
Certainly, in the embodiment of the present disclosure, the number of currently executed transactions is not limited to one or multiple, if there are multiple currently executed transactions, at this time, the first write set of each current transaction needs to be calculated, and then during conflict analysis, a conflict judgment may be performed on the to-be-executed transaction and multiple currently executed transactions, for example, it is determined that multiple transactions can be executed concurrently, there are multiple currently executed transactions, and usually the number of transactions in the transaction queue is limited, and there are at most multiple transactions that satisfy the concurrency, so the calculation amount of the WriteSet is also controllable.
S202: determining a to-be-executed transaction in an executable state, and determining a second write set of the to-be-executed transaction according to the transaction record log information of the main database, wherein the second write set comprises key values of multiple types of keys and is used for uniquely representing the to-be-executed transaction and each statement in the to-be-executed transaction.
When the step S202 is executed, the method includes:
s1, determining a transaction to be executed in an executable state.
In particular, the present disclosure provides one possible implementation: and determining the next transaction in the execution sequence corresponding to the currently executed transaction as the transaction to be executed in an executable state according to the logic clock of the transaction record log information of the main database.
For example, the transaction log information is Binlog, the LSN and the Last Committed LSN of each transaction are recorded in the Binlog, and the order of each transaction may be determined according to the value size of the LSN, a transaction with the same Last Committed LSN indicates that no conflict exists and that the transaction with the same logical clock concurrency condition is satisfied, that is, the transaction with the executable state, may be determined according to the LSN of the currently executed transaction and the LSNs and Last Committed LSNs of all unexecuted transactions, and may be a transaction that does not satisfy the logical clock concurrency condition with the currently executed transaction, that is, different from the Last Committed LSN of the currently executed transaction, and is a transaction with the next execution order corresponding to the determined maximum LSN of the currently executed transaction, that is, and is marked as the executable state.
In addition, the transaction log information may include a plurality of transaction log files, each transaction log file has a file identifier, the LSNs of the transactions in each transaction log file may restart counting, and when the execution sequence of each transaction is determined, the value size of the LSN and the Last Committed LSN is not only based on, but also according to the transaction log file in which the transaction is located.
And S2, determining a second write set of the transaction to be executed.
Specifically, the method comprises a possible implementation: and determining a second writing set of the transaction to be executed according to the transaction record log information of the main database.
S203: and judging whether each statement included in the transaction to be executed conflicts with the current execution transaction or not according to the key values of the various keys in the first write set and the key values of the various keys in the second write set, and executing any statement in a preposed manner when determining that any statement does not conflict with the current execution transaction.
In the embodiment of the disclosure, the first write set of the currently executed transaction and the second write set of the to-be-executed transaction can uniquely represent the transaction and statements included in the transaction, where an intersection is the same key value, which indicates that there is a conflict, and an intersection is not the same, which indicates that there is no conflict, and by constructing the write sets, the conflicts between the statements in the to-be-executed transaction and the currently executed transaction can be determined, instead of determining that the conflict between the to-be-executed transaction and the currently executed transaction is determined, so that the transaction can be started first, and the statements can be executed in advance as long as the statements in the to-be-executed transaction and the currently executed transaction do not conflict.
That is to say, in the embodiment of the present disclosure, the following condition needs to be satisfied when any statement in the transaction to be executed can be executed: 1) The transaction to which any statement belongs is in an executable state, and whether the transaction is in the executable state can be determined based on a logic clock; 2) Any of these statements do not conflict with all transactions that are currently executing and have LSNs less than the Last Committed LSN of the affiliated transaction.
Further, an embodiment of the present disclosure further provides a trigger condition for executing the method, that is, before the step S201 is executed to determine the first write set of the currently executed transaction according to the transaction record log information of the master database, the method further includes: and after responding to the completion of any transaction, determining the current execution transaction according to the logic clock of the transaction record log information of the main database, wherein the current execution transaction at least comprises the next execution sequence transaction corresponding to any transaction after the completion of the execution.
That is, in the embodiment of the present disclosure, each time there is a transaction execution completed, the scheduler is triggered to recalculate the first write set of the currently executed transaction and notify the to-be-executed transaction in the executable state, which also means that there is a new transaction or a new sentence that needs to be executed when there is a transaction execution completed, and the currently executed transaction and the to-be-executed transaction in the executable state change, so that recalculation is required to perform conflict analysis to determine statements in the concurrently executable to-be-executed transaction, so that non-conflicting statements can be pre-executed.
In the embodiment of the disclosure, a first write set of a currently executed transaction is determined according to transaction record log information of a primary database, a to-be-executed transaction in an executable state is determined, a second write set of the to-be-executed transaction is determined according to transaction record log information of the primary database, and then, whether statements included in the to-be-executed transaction conflict with the currently executed transaction is determined according to key values of various types of keys in the first write set and key values of various types of keys in the second write set, and when it is determined that any statement does not conflict with the currently executed transaction, the statement is executed in advance.
Based on the foregoing embodiments, in an embodiment of the present disclosure, in a data replication scenario, when a node is replicated or a transaction is played back from a node, a first write set of a currently executed transaction and a second write set of a transaction to be executed in an executable state may be determined, so that conflict comparison of statement granularity may be performed, so as to improve a concurrency to the maximum extent.
For the sake of understanding, the basic principle of how to determine write sets in the embodiments of the present disclosure is described first, the basic principle of the MySQL WriteSet method in the related art is that a master node can optimize LSN and Last Committed LSN information about transaction concurrency in Binlog based on WriteSet, and then a replication node or a slave node can play back a transaction concurrently through these information in Binlog, and the embodiments of the present disclosure consider that the idea of MySQL WriteSet in the related art can be used for reference, and the WriteSet of a transaction is reconstructed on the replication node or the slave node through Binlog to perform transaction concurrency analysis, but it is found through research that the WriteSet cannot be constructed directly based on Binlog, because some implicit semantics in MySQL are not contained in Binlog, and there are cases where information is incomplete when the WriteSet is calculated, for example, there are several possible cases where information is incomplete in Binlog: 1) Unique function index statements, such as a statement: CREATE TABLE t1 (c 1 int, c2 int, UNIQUE INDEX ((c 1+ c 2))), where only the values of c1 and c2 are recorded in Billog, and UNIQUE INDEX (c 1+ c 2) is stored in MySQL as a hidden generated column, but not recorded in Billog, so that UNIQUE INDEX (c 1+ c 2) is difficult to restore when the function INDEX is a UNIQUE INDEX, especially if the expression can support the embedded MySQL function. 2) There is a foreign key constraint statement, such as a cascade command, and the deletion operation of the command automatically deletes the slave table when deleting the master table, but only the deletion record of the master table exists in the Binlog, and there is no deletion record of the sub-table. 3) There is a UNIQUE multi-valued INDEX statement, such AS UNIQUE INDEX idx1 ((CAST (c 1- > $. Arr' AS UNSIGNED ARRAY))), and the INDEX is built by only the attribute value of the data type, and repeated insertion errors occur when values are inserted for multiple times.
Therefore, in the embodiment of the disclosure, aiming at the problems occurring when the WriteSet is constructed based on Binlog, the WriteSet in the related technology is improved, the semantics of the WriteSet are expanded, and the concepts of global keys and table keys are provided, so that the semantics of statements can be properly amplified aiming at scenes with incomplete information in the Binlog, and no conflict exists between the concurrently executed transactions.
Specifically, in the embodiment of the present disclosure, a possible specific implementation manner is provided for determining a first write set of a currently executed transaction and determining a second write set of a transaction to be executed, where the first write set and the second write set are calculated in the same manner, and for convenience of unified description, the first write set or the second write set is used as a target write set, and the currently executed transaction or the transaction to be executed is used as a target transaction, which are described by taking the target write set and the target transaction as examples, where the first write set corresponds to the currently executed transaction and the second write set corresponds to the transaction to be executed, and then specifically, according to the transaction record log information of the master database, determining the target write set of the target transaction may include the following implementation manners:
and determining a global key, a table key and a main key and/or a unique key of each statement included in a target write set of the target transaction according to the transaction record log information of the main database.
The global key represents a global lock of a target transaction, and the table key represents a table-level lock of at least one table corresponding to the target transaction; the primary key and the unique key characterize a statement lock; and the lock priority is that the global key is larger than the table key, and the table key is larger than the primary key and the unique key.
In the embodiment of the present disclosure, the target write set includes key values of multiple types of keys, for example, a global key, a table key, a primary key, and a unique key, and different types of keys may represent locks of different semantic levels, so that when a statement level lock of a statement cannot be uniquely determined for a scene with incomplete information in a Binlog, to avoid a collision, semantics may be appropriately amplified to generate a global key or a table key.
Determining a global key, a table key, and a primary key and/or a unique key of each statement included in a target write set of a target transaction according to transaction record log information of a master database, which may be specifically divided into the following scenarios:
the first scenario is: and if the preset target condition exists, determining a global key or a table key included in a target write set of the target transaction.
Wherein the target conditions include: 1) If the target transaction comprises a DDL statement or a first transaction in a transaction record log file included in the transaction record log information, determining that the target write set of the target transaction comprises a global key.
For example, since a DDL statement is usually the initial work for defining or updating the structure, data type, connection and constraint between tables, etc. of a table, the DDL statement usually involves multiple transactions or tables, so to avoid concurrent execution of conflicting transactions, a global key may be set for the DDL statement, and transactions with the same global key indicate conflict and cannot be executed concurrently, for example, the global key may be a randomly generated character string.
2) If the target transaction does not have the primary key table or comprises at least one of the following target type tables, determining that the target write set of the target transaction at least comprises a table key, wherein the target type table is as follows: there is a table of unique index statements, a parent table of foreign key constraints.
The unique index statement is, for example, a unique function index statement, or a unique multi-valued index statement.
That is, in the embodiment of the present disclosure, a table lock of the table may be constructed when incomplete information may occur in the table, semantics of a table-level lock are increased, and a possible implementation is provided when the table lock is calculated, 1) a primary key and/or a Unique key of a statement where incomplete information may occur may be used as a table key, for example, if a transaction includes a uniform Functional Index line, the primary key and the Unique key of the line may be directly upgraded to the table key. 2) If the table is a table without a primary key, the system will default to add a hidden primary key, and if no record is recorded in the Binlog, a table key can be generated to identify the whole table, so as to avoid data operation conflict in the table.
That is, in the embodiment of the present disclosure, for the implicit semantics that cannot be recorded in Binlog, the semantics of the current recording can be properly amplified, and a global key and a table key are added on the basis of the existing WriteSet, so that when a statement cannot be uniquely determined or recording is incomplete due to an external key, the semantics of a global lock or a table lock of a transaction can be expanded, thereby avoiding a conflict of transaction concurrency and ensuring the consistency and integrity of data.
The second scenario is as follows: and if the target condition does not exist, determining the primary key and/or the unique key of each statement included in the target write set of the target transaction.
That is, for a statement with certain semantics, the primary key and/or the unique key of the statement may be directly constructed to implement a lock on the statement.
Based on the foregoing embodiment, in the embodiment of the present disclosure, a lock priority definition is further provided, specifically referring to fig. 3, which is a schematic diagram of a lock priority in the embodiment of the present disclosure, as shown in fig. 3, a mutual exclusion relationship between different types of keys may be defined by a tree-shaped lock structure, where the lock priority is that a global key is larger than a table key, the table key is larger than a primary key and a unique key, a conflict exists between transactions of the same global key, a conflict exists between tables of the same table key, and a conflict exists between statements of the same unique key or the primary key, and when performing concurrent conflict analysis, comparisons may be performed in sequence according to the lock priority, for example, for a lock priority including only a global key (RootKey, RK), a write lock may be defined as: WLock (RK), the lock priority for a lock including a Table Key (TK) may be defined as: a read lock RLock (RK) - > write lock WLock (TK), a lock priority for a lock including a Primary Key (PK) and/or a Unique Key (UK) may be defined as: reading-locked RLock (RK) - > reading-locked RLock (TK) - > locked Lock (PK/UK).
Therefore, the write set is reconstructed during transaction playback, the write set in the related technology is improved, the lock semantics can be expanded under the condition that information in the Binlog is incomplete, the concurrent playback transaction is ensured not to conflict, the accuracy of transaction conflict analysis is improved, and the consistency and the integrity of data in the main database are ensured.
Further, in the embodiment of the present disclosure, a conflict analysis may be performed based on the determined write set and the types of keys in the write set, and specifically, for determining whether each statement included in the to-be-executed transaction conflicts with the currently executed transaction according to the key values of the types of keys in the first write set and the key values of the types of keys in the second write set in step S203, the present disclosure provides a possible implementation manner:
s1, respectively comparing a second write set corresponding to each statement included in the transaction to be executed with a first write set of the currently executed transaction.
In the embodiment of the present disclosure, when performing the transaction conflict analysis, the statement of the to-be-executed transaction and the currently executed transaction may be compared and analyzed, so as to determine whether the statement of the to-be-executed transaction and the currently executed transaction can be executed concurrently.
And S2, sequentially judging whether the global key, the table key and the main key or the unique key of the transaction to be executed have intersection with the first write set according to the lock priority, determining that the corresponding statement conflicts with the currently executed transaction under the condition that the intersection is determined, and determining that the corresponding statement does not conflict with the currently executed transaction under the condition that the intersection is not determined.
The method specifically comprises the following conditions:
1) And according to the priority of the lock, if the global key of the transaction to be executed has intersection with the first write set, determining that all statements included in the transaction to be executed conflict with the current transaction to be executed.
That is to say, in the embodiment of the present disclosure, when performing transaction conflict analysis, it is first determined whether the global key of the transaction to be executed conflicts with the first write set according to the lock priority, and if the global key conflicts, it is not necessary to determine the table key, the primary key, and the unique key, and it may be considered that all statements in the global key transaction conflict with the currently executed transaction.
2) If the global key of the to-be-executed transaction does not have an intersection with the first write set and the table key of the to-be-executed transaction has an intersection with the first write set, all statements under the table key of the to-be-executed transaction having the intersection conflict with the currently-executed transaction.
After the global keys are determined to be different, whether the table key of the transaction to be executed has conflict is judged, if the table keys are the same, the conflict exists only in the transaction of which the table is updated, the statements related to the table are considered to have conflict, and the conflict cannot be executed concurrently with the currently executed transaction.
3) If the global key, the table key and the first write set of the transaction to be executed do not have intersection, and the unique key or the main key of the transaction to be executed has intersection with the first write set, determining that the statement corresponding to the unique key or the main key with intersection conflicts with the current execution transaction.
Therefore, in the embodiment of the disclosure, a write set of a transaction can be constructed according to the transaction record log information of the main database, and when the write set is constructed, the problem of incomplete information in the transaction record log information is solved by expanding the semantics of the write set, so that the transaction concurrency analysis between the statement of the transaction to be executed and the currently executed transaction can be performed based on the constructed write set, so that the conflict-free statement can be executed in the front, the transaction execution can be started first without waiting until no conflict exists with the currently executed transaction, and the transaction concurrency analysis can be executed first as long as the statement of the transaction to be executed and the currently executed transaction do not conflict, thereby greatly reducing the conflict waiting time.
The foregoing embodiment is described below with reference to a specific application scenario, and referring to fig. 4, which is a schematic view of an application scenario of transaction processing in the embodiment of the present disclosure, as shown in fig. 4, for example, there are nine transactions, which are Trx1, trx2, \8230;, and Trx9, and the transaction processing method in the embodiment of the present disclosure is described below with reference to the scenario.
1) At time t1, trx2, trx3 are concurrently executing, and the currently executed transaction includes Trx1, trx2, trx3.
2) If the Trx1 and Trx3 are executed at the time t2, the Trx2 is still executed, it is determined that Trx4 can continue to be executed according to the logic clock of the transaction record log information in the master database, trx5 does not meet the logic clock concurrency condition because of logic clock dependency, the threshold is temporarily held, trx5 is marked as an executable state but cannot be executed concurrently, at this time, the currently executed transaction is updated to be Trx2 and Trx4, and the transaction to be executed in the executable state is updated to be Trx5.
For example, as in fig. 4, trx5 contains two statements: insert r3 operation I (r 3) and update r1 operation U (r 1), trx2 comprises a statement: insert r1 operation I (r 1), trx4 includes a statement: if r2 operation I (r 2) is inserted, based on the transaction processing method in the embodiment of the present disclosure, by comparing the statement included in Trx5 with the conflict of the currently executed transactions Trx2 and Trx4, it is determined that the I (r 3) statement in Trx5 does not conflict with Trx2 and Trx4, and it is determined that the U (r 1) statement in Trx5 conflicts with Trx2, then the I (r 3) statement in Trx5 may be concurrently executed, and the U (r 1) statement in Trx5 is not executed.
In this way, in the embodiment of the present disclosure, during the transaction playback process, based on the transaction concurrency information in the Binlog, for the transactions meeting the logic clock concurrency condition, each transaction meeting the logic clock concurrency condition is directly and concurrently executed, and for the transactions not meeting the logic clock concurrency condition and marked as an executable state, the statement conflict analysis is performed, and the conflict-free statement may be executed first, that is, the first statement of the Trx5 in the executable state as described above, and may not be executed after the currently executed transaction is executed, and may be executed concurrently with the currently executed transaction, and the conflict analysis is performed at the statement granularity, so that the concurrency is improved, and the processing efficiency and performance are also improved.
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same inventive concept, a transaction processing apparatus corresponding to the transaction processing method is also provided in the embodiments of the present disclosure, and since the principle of the apparatus in the embodiments of the present disclosure for solving the problem is similar to the transaction processing method described above in the embodiments of the present disclosure, the implementation of the apparatus may refer to the implementation of the method, and repeated details are not described again.
Referring to fig. 5, a schematic diagram of a transaction processing provided in an embodiment of the present disclosure, where the apparatus includes:
a first determining module 51, configured to determine a first write set of a currently executed transaction according to transaction record log information of a primary database, where the first write set includes key values of multiple types of keys, and is used to uniquely characterize the currently executed transaction and statements in the currently executed transaction;
a second determining module 52, configured to determine a transaction to be executed in an executable state, and determine a second write set of the transaction to be executed according to the transaction record log information in the primary database, where the second write set includes key values of multiple types of keys, and is used to uniquely characterize the transaction to be executed and each statement in the transaction to be executed;
and the execution module 53 is configured to determine whether each statement included in the to-be-executed transaction conflicts with the currently executed transaction according to the key values of the various types of keys in the first write set and the key values of the various types of keys in the second write set, and when it is determined that any statement does not conflict with the currently executed transaction, perform the any statement in advance.
In an optional implementation manner, when the first write set or the second write set is used as a target write set, and the currently executed transaction or the to-be-executed transaction is used as a target transaction, and the target write set of the target transaction is determined according to the transaction record log information of the master database, the first determining module 51 or the second determining module 52 is configured to:
determining a global key, a table key and a main key and/or a unique key of each statement included in a target writing set of the target transaction according to the transaction record log information of the main database;
wherein the global key represents a global lock of the target transaction, and the table key represents a table-level lock of at least one table corresponding to the target transaction; the primary key and the unique key characterize a statement lock; and a lock priority is such that the global key is larger than the table key, which is larger than the primary key and the unique key.
In an optional embodiment, when determining, according to the transaction record log information of the master database, that the target write set of the target transaction includes a global key, a table key, and a primary key and/or a unique key of each statement, the first determining module 51 or the second determining module 52 is configured to:
if a preset target condition exists, determining a global key or a table key included in a target write set of the target transaction; if the target condition does not exist, determining the main key and/or the unique key of each statement included in the target write set of the target transaction;
wherein the target condition comprises: if the target transaction comprises a Database Definition Language (DDL) statement or a first transaction in a transaction record log file included in the transaction record log information, determining that a target write set of the target transaction comprises the global key; if the target transaction does not have a primary key table or comprises at least one target type table, determining that the target write set of the target transaction at least comprises the table key, wherein the target type table is: there is a table of unique index statements, a parent table of foreign key constraints.
In an optional implementation manner, when determining whether each statement included in the to-be-executed transaction conflicts with the currently executed transaction according to the key values of the various types of keys in the first write set and the key values of the various types of keys in the second write set, the execution module 53 is configured to:
respectively comparing a second write set corresponding to each statement included in the transaction to be executed with the first write set of the current transaction to be executed;
and sequentially judging whether the global key, the table key and the main key or the unique key of the transaction to be executed have intersection with the first write set according to the priority of the lock, determining that a corresponding statement conflicts with the current execution transaction under the condition of determining that the intersection exists, and determining that the corresponding statement does not conflict with the current execution transaction under the condition of determining that the intersection does not exist.
In an optional implementation manner, the executing module 53 is configured to sequentially determine, according to the lock priority, whether the global key, the table key, and the primary key or the unique key of the transaction to be executed intersect with the first write set, determine that a corresponding statement conflicts with the currently executed transaction when it is determined that the intersection exists, and determine that the corresponding statement does not conflict with the currently executed transaction when it is determined that the intersection does not exist, where:
according to the priority of the lock, if the global key of the transaction to be executed has an intersection with the first write set, determining that all statements included in the transaction to be executed conflict with the current transaction to be executed;
if the global key of the to-be-executed transaction does not have an intersection with the first write set and the table key of the to-be-executed transaction has an intersection with the first write set, all statements under the table key of the to-be-executed transaction having the intersection conflict with the currently-executed transaction;
if the global key, the table key and the first write set of the transaction to be executed do not have an intersection, and the unique key or the main key of the transaction to be executed has an intersection with the first write set, determining that the statement corresponding to the unique key or the main key having the intersection conflicts with the currently executed transaction.
In an alternative embodiment, the first determining module 51 is further configured to: before determining a first write set of a currently executing transaction according to the transaction record log information of the master database, performing the following operations:
and after any transaction is executed, determining the current execution transaction according to the logic clock of the transaction record log information of the master database, wherein the current execution transaction at least comprises the next execution sequence transaction corresponding to any transaction after execution is completed.
In an alternative embodiment, when determining a transaction to be executed in an executable state, the second determining module 52 is configured to: and determining the next transaction in the execution sequence corresponding to the currently executed transaction as the to-be-executed transaction in an executable state according to the logic clock of the transaction record log information of the main database.
The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.
An embodiment of the present disclosure further provides an electronic device, as shown in fig. 6, which is a schematic structural diagram of the electronic device provided in the embodiment of the present disclosure, and includes:
a processor 61 and a memory 62; the memory 62 stores machine-readable instructions executable by the processor 61, the processor 61 being configured to execute the machine-readable instructions stored in the memory 62, the processor 61 performing the following steps when the machine-readable instructions are executed by the processor 61:
determining a first write set of a currently executed transaction according to transaction record log information of a primary database, wherein the first write set comprises key values of multiple types of keys and is used for uniquely representing the currently executed transaction and statements in the currently executed transaction;
determining a to-be-executed transaction in an executable state, and determining a second write set of the to-be-executed transaction according to transaction record log information of the primary database, wherein the second write set comprises key values of multiple types of keys and is used for uniquely representing the to-be-executed transaction and each statement in the to-be-executed transaction;
and judging whether each statement included in the transaction to be executed conflicts with the current execution transaction or not according to the key values of the various types of keys in the first write set and the key values of the various types of keys in the second write set, and executing any statement in a front-end mode when any statement is determined not to conflict with the current execution transaction.
In an optional implementation manner, when the first write set or the second write set is used as a target write set, and the currently executed transaction or the to-be-executed transaction is used as a target transaction, and when the target write set of the target transaction is determined according to the transaction record log information of the master database, the processor 61 is configured to:
determining a global key, a table key and a main key and/or a unique key of each statement included in a target writing set of the target transaction according to the transaction record log information of the main database;
the global key represents a global lock of the target transaction, and the table key represents a table-level lock of at least one table corresponding to the target transaction; the primary key and the unique key characterize a statement lock; and a lock priority is such that the global key is larger than the table key, which is larger than the primary key and the unique key.
In an alternative embodiment, when determining, according to the transaction record log information of the master database, that the target write set of the target transaction includes a global key, a table key, and a master key and/or a unique key of each statement, the processor 61 is configured to:
if a preset target condition exists, determining a global key or a table key included in a target write set of the target transaction; if the target condition does not exist, determining the primary key and/or the unique key of each statement included in the target write set of the target transaction;
wherein the target condition comprises: if the target transaction comprises a Database Definition Language (DDL) statement or a first transaction in a transaction record log file included in the transaction record log information, determining that a target write set of the target transaction comprises the global key; if the target transaction does not have a primary key table or comprises at least one of the following target type tables, determining that the target write set of the target transaction at least comprises the table key, wherein the target type table is: there is a table of unique index statements, a parent table of foreign key constraints.
In an optional implementation manner, when determining whether each statement included in the to-be-executed transaction conflicts with the currently executed transaction according to key values of various types of keys in the first write set and key values of various types of keys in the second write set, the processor 61 is configured to:
respectively comparing a second write set corresponding to each statement included in the transaction to be executed with the first write set of the current transaction to be executed;
and sequentially judging whether the global key, the table key and the main key or the unique key of the transaction to be executed have intersection with the first write set according to the priority of the lock, determining that a corresponding statement conflicts with the current execution transaction under the condition of determining that the intersection exists, and determining that the corresponding statement does not conflict with the current execution transaction under the condition of determining that the intersection does not exist.
In an optional implementation manner, the processor 61 is configured to sequentially determine, according to the lock priority, whether the global key, the table key, and the primary key or the unique key of the transaction to be executed intersect with the first write set, determine that a corresponding statement conflicts with the currently executed transaction when it is determined that the intersection exists, and determine that the corresponding statement does not conflict with the currently executed transaction when it is determined that the intersection does not exist, where:
according to the lock priority, if the global key of the transaction to be executed has an intersection with the first write set, determining that all statements included in the transaction to be executed conflict with the current execution transaction;
if the global key of the transaction to be executed does not have an intersection with the first write set and the table key of the transaction to be executed has an intersection with the first write set, all statements under the table key of the transaction to be executed having the intersection conflict with the current execution transaction;
if the global key, the table key and the first write set of the transaction to be executed do not have an intersection, and the unique key or the main key of the transaction to be executed has an intersection with the first write set, determining that the statement corresponding to the unique key or the main key having the intersection conflicts with the currently executed transaction.
In an alternative embodiment, the processor 61 is further configured to: before determining a first write set of a currently executing transaction according to the transaction record log information of the master database, performing the following operations:
and after responding to the execution of any transaction, determining the current execution transaction according to the logic clock of the transaction record log information of the main database, wherein the current execution transaction at least comprises the transaction of the next execution sequence corresponding to the any transaction after the execution is completed.
In an alternative embodiment, when determining a transaction to be executed in an executable state, the processor 61 is configured to: and determining the next transaction in the execution sequence corresponding to the currently executed transaction as the to-be-executed transaction in an executable state according to the logic clock of the transaction record log information of the main database.
The memory 62 includes a memory 621 and an external memory 622; the memory 621 is also referred to as an internal memory, and temporarily stores operation data in the processor 61 and data exchanged with the external memory 622 such as a hard disk, and the processor 61 exchanges data with the external memory 622 via the memory 621.
For the specific execution process of the instruction, reference may be made to the steps of the transaction processing method described in the embodiments of the present disclosure, and details are not described here.
The embodiments of the present disclosure also provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the transaction processing method in the foregoing method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, and instructions included in the program code may be used to execute the steps of the transaction processing method in the foregoing method embodiments, which may be referred to specifically in the foregoing method embodiments, and are not described herein again.
The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working process of the system and the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and details are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in software functional units and sold or used as a stand-alone product, may be stored in a non-transitory computer-readable storage medium executable by a processor. Based on such understanding, the technical solutions of the present disclosure, which are essential or part of the technical solutions contributing to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, but not to limit the technical solutions, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes and substitutions do not depart from the spirit and scope of the embodiments disclosed herein, and they should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims (10)

1. A transaction processing method, comprising:
determining a first write set of a current execution transaction according to transaction record log information of a primary database, wherein the first write set comprises key values of multiple types of keys and is used for uniquely representing the current execution transaction and each statement in the current execution transaction;
determining a transaction to be executed in an executable state, and determining a second write set of the transaction to be executed according to transaction record log information of the primary database, wherein the second write set comprises key values of multiple types of keys and is used for uniquely characterizing the transaction to be executed and each statement in the transaction to be executed;
and judging whether each statement included in the transaction to be executed conflicts with the current execution transaction or not according to the key values of the various types of keys in the first write set and the key values of the various types of keys in the second write set, and pre-executing any statement when determining that any statement does not conflict with the current execution transaction.
2. The method of claim 1, wherein the determining the target write set of the target transaction according to transaction record log information of a master database by using the first write set or the second write set as a target write set and using the currently executed transaction or the to-be-executed transaction as the target transaction comprises:
determining a global key, a table key and a main key and/or a unique key of each statement included in a target writing set of the target transaction according to the transaction record log information of the main database;
the global key represents a global lock of the target transaction, and the table key represents a table-level lock of at least one table corresponding to the target transaction; the primary key and the unique key characterize a statement lock; and a lock priority is such that the global key is larger than the table key, which is larger than the primary key and the unique key.
3. The method according to claim 2, wherein determining, according to the transaction record log information of the master database, a global key, a table key, and a master key and/or a unique key of each statement included in the target write set of the target transaction comprises:
if a preset target condition exists, determining a global key or a table key included in a target write set of the target transaction; if the target condition does not exist, determining the main key and/or the unique key of each statement included in the target write set of the target transaction;
wherein the target condition comprises: if the target transaction comprises a Database Definition Language (DDL) statement or a first transaction in a transaction record log file included in the transaction record log information, determining that a target write set of the target transaction comprises the global key; if the target transaction does not have a primary key table or comprises at least one target type table, determining that the target write set of the target transaction at least comprises the table key, wherein the target type table is: there is a table of unique index statements, a parent table of foreign key constraints.
4. The method according to claim 2 or 3, wherein determining whether each statement included in the transaction to be executed conflicts with the currently executed transaction according to the key values of the various types of keys in the first write set and the key values of the various types of keys in the second write set comprises:
respectively comparing a second write set corresponding to each statement included in the transaction to be executed with the first write set of the current transaction to be executed;
and sequentially judging whether the global key, the table key and the main key or the unique key of the transaction to be executed have intersection with the first write set according to the priority of the lock, determining that a corresponding statement conflicts with the current execution transaction under the condition of determining that the intersection exists, and determining that the corresponding statement does not conflict with the current execution transaction under the condition of determining that the intersection does not exist.
5. The method of claim 4, wherein the determining, according to the lock priority, whether the global key, the table key, and the primary key or the unique key of the transaction to be executed intersect with the first write set in sequence, determining that a corresponding statement conflicts with the currently executed transaction if it is determined that there is an intersection, and determining that a corresponding statement does not conflict with the currently executed transaction if it is determined that there is no intersection, comprises:
according to the priority of the lock, if the global key of the transaction to be executed has an intersection with the first write set, determining that all statements included in the transaction to be executed conflict with the current transaction to be executed;
if the global key of the to-be-executed transaction does not have an intersection with the first write set and the table key of the to-be-executed transaction has an intersection with the first write set, all statements under the table key of the to-be-executed transaction having the intersection conflict with the currently-executed transaction;
if the global key, the table key and the first write set of the transaction to be executed do not have an intersection, and the unique key or the primary key of the transaction to be executed has an intersection with the first write set, determining that the statement corresponding to the unique key or the primary key having the intersection conflicts with the currently executed transaction.
6. The method of claim 1, wherein determining, based on transaction log information of a master database, that a first write set of a currently executing transaction precedes, further comprises:
and after any transaction is executed, determining the current execution transaction according to the logic clock of the transaction record log information of the master database, wherein the current execution transaction at least comprises the next execution sequence transaction corresponding to any transaction after execution is completed.
7. The method of any of claims 1-6, wherein determining a transaction to be executed that is in an executable state comprises:
and determining the next transaction in the execution sequence corresponding to the current execution transaction as the to-be-executed transaction in an executable state according to the logic clock of the transaction record log information of the main database.
8. A transaction processing apparatus, comprising:
the first determining module is used for determining a first write set of a currently executed transaction according to transaction record log information of a main database, wherein the first write set comprises key values of various types of keys and is used for uniquely representing the currently executed transaction and statements in the currently executed transaction;
a second determining module, configured to determine a transaction to be executed in an executable state, and determine a second write set of the transaction to be executed according to transaction record log information in the primary database, where the second write set includes key values of multiple types of keys, and is used to uniquely characterize the transaction to be executed and statements in the transaction to be executed;
and the execution module is used for judging whether each statement included in the to-be-executed transaction conflicts with the currently executed transaction according to the key values of the various types of keys in the first write set and the key values of the various types of keys in the second write set, and pre-executing any statement when any statement does not conflict with the currently executed transaction.
9. An electronic device, comprising: a processor, a memory storing machine-readable instructions executable by the processor, the processor to execute machine-readable instructions stored in the memory, the machine-readable instructions, when executed by the processor, the processor to perform the steps of the transaction processing method of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when executed by a processor, performs the steps of the transaction processing method according to any one of claims 1 to 7.
CN202211237963.4A 2022-10-08 2022-10-08 Transaction processing method, device, electronic equipment and storage medium Active CN115509694B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211237963.4A CN115509694B (en) 2022-10-08 2022-10-08 Transaction processing method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211237963.4A CN115509694B (en) 2022-10-08 2022-10-08 Transaction processing method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN115509694A true CN115509694A (en) 2022-12-23
CN115509694B CN115509694B (en) 2024-04-30

Family

ID=84508208

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211237963.4A Active CN115509694B (en) 2022-10-08 2022-10-08 Transaction processing method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115509694B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116089403A (en) * 2023-04-11 2023-05-09 北京奥星贝斯科技有限公司 Flow playback method and system for database
CN117076547A (en) * 2023-10-16 2023-11-17 江苏量界数据科技有限公司 Data storage method based on virtual distributed server

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102306205A (en) * 2011-09-30 2012-01-04 苏州大学 Method and device for allocating transactions
US20160110439A1 (en) * 2014-10-21 2016-04-21 International Business Machines Corporation Database management system and method of operation
CN107958023A (en) * 2017-11-06 2018-04-24 北京华宇信息技术有限公司 Method of data synchronization, data synchronization unit and computer-readable recording medium
CN111143389A (en) * 2019-12-27 2020-05-12 腾讯科技（深圳）有限公司 Transaction execution method and device, computer equipment and storage medium
CN112256442A (en) * 2019-07-22 2021-01-22 中兴通讯股份有限公司 Database deadlock detection method and device
CN112818053A (en) * 2019-11-18 2021-05-18 Sap欧洲公司 Method and system for real-time cross-system database replication
CN115098537A (en) * 2021-10-19 2022-09-23 腾讯科技（深圳）有限公司 Transaction execution method and device, computing equipment and storage medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102306205A (en) * 2011-09-30 2012-01-04 苏州大学 Method and device for allocating transactions
US20160110439A1 (en) * 2014-10-21 2016-04-21 International Business Machines Corporation Database management system and method of operation
CN107958023A (en) * 2017-11-06 2018-04-24 北京华宇信息技术有限公司 Method of data synchronization, data synchronization unit and computer-readable recording medium
CN112256442A (en) * 2019-07-22 2021-01-22 中兴通讯股份有限公司 Database deadlock detection method and device
CN112818053A (en) * 2019-11-18 2021-05-18 Sap欧洲公司 Method and system for real-time cross-system database replication
CN111143389A (en) * 2019-12-27 2020-05-12 腾讯科技（深圳）有限公司 Transaction execution method and device, computer equipment and storage medium
CN115098537A (en) * 2021-10-19 2022-09-23 腾讯科技（深圳）有限公司 Transaction execution method and device, computing equipment and storage medium

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116089403A (en) * 2023-04-11 2023-05-09 北京奥星贝斯科技有限公司 Flow playback method and system for database
CN117076547A (en) * 2023-10-16 2023-11-17 江苏量界数据科技有限公司 Data storage method based on virtual distributed server
CN117076547B (en) * 2023-10-16 2024-01-02 江苏量界数据科技有限公司 Data storage method based on virtual distributed server

Also Published As

Publication number Publication date
CN115509694B (en) 2024-04-30

Similar Documents

Publication Publication Date Title
US11429641B2 (en) Copying data changes to a target database
CN115509694B (en) Transaction processing method, device, electronic equipment and storage medium
EP3111347B1 (en) Efficient methods and systems for consistent read in record-based multi-version concurrency control
US7240054B2 (en) Techniques to preserve data constraints and referential integrity in asynchronous transactional replication of relational tables
US9507822B2 (en) Methods and systems for optimizing queries in a database system
US8572027B2 (en) Performing synchronization among relational database tables with minimal contention
US20070288537A1 (en) Method and apparatus for replicating data across multiple copies of a table in a database system
US9020916B2 (en) Database server apparatus, method for updating database, and recording medium for database update program
US8090700B2 (en) Method for updating databases
EP4006740A1 (en) Method for indexing data in storage engines, and related device
US20110145201A1 (en) Database mirroring
CN107665255B (en) Method, device, equipment and storage medium for key value database data change
CN109871386A (en) Multi version concurrency control (MVCC) in nonvolatile memory
Kleppmann Designing data-intensive applications
US20180300147A1 (en) Database Operating Method and Apparatus
CN115617571A (en) Data backup method, device, system, equipment and storage medium
CN110647421B (en) Database processing method, device and system and electronic equipment
CN113760902A (en) Data splitting method, device, equipment, medium and program product
WO2020192663A1 (en) Data management method and related device
KR20120082176A (en) Data processing method of database management system and system thereof
CN111240891A (en) Data recovery method and device based on data consistency among multiple tables of database
CN113360495B (en) Database query interruption recovery method, device, equipment and readable medium
CN112084161B (en) Database-based data processing method and device and readable storage medium
CN114461605B (en) Transaction data multi-version implementation method, device and equipment of memory multi-dimensional database
WO2024082693A1 (en) Data processing method, and apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant