WO2024098363A1

WO2024098363A1 - Multicore-processor-based concurrent transaction processing method and system

Info

Publication number: WO2024098363A1
Application number: PCT/CN2022/131298
Authority: WO
Inventors: 樊文飞; 曹洋; 欧伟杰; 谢锐
Original assignee: 深圳计算科学研究院
Priority date: 2022-11-09
Filing date: 2022-11-11
Publication date: 2024-05-16
Also published as: CN115629822A; CN115629822B

Abstract

Provided in the present application are a multicore-processor-based concurrent transaction processing method and system. The method comprises: when transaction concurrency occurs within a target time period, acquiring a read-write set of transactions that occur within the target time period; according to the read-write set, determining a conflict relationship between the transactions and execution costs corresponding to the transactions; according to the conflict relationship, determining partitions to which the transactions correspond; according to the conflict relationship and the execution costs, determining queue types of the transactions in the partitions, wherein the queue types comprise a conflict-free queue and a conflict queue; and when there is a conflict between a transaction in the conflict queue and a transaction in another partition, delaying the processing of the transaction in the conflict queue. Runtime scheduling can reduce transaction conflicts and is also effective for high-conflict concurrent transactions. Conflicts are reduced by means of delaying the execution, and there is no need to consider a pessimistic lock for all transactions, thereby reducing the overall lock wait delay. For an estimated execution cost deviation, concurrency control can be relied upon to ensure the correctness of a result.

Description

A concurrent transaction processing method and system based on multi-core processor

Technical Field

The present application relates to the field of concurrent transaction processing, and in particular to a concurrent transaction processing method and system based on a multi-core processor.

Background technique

Previously, only one user accessed the data. When multiple users accessed the same data, this was called "concurrency". When one user attempts to modify the data that another user is retrieving or modifying, concurrency becomes a problem. Modern computer systems are based on multi-core central processing units (CPUs). In order to handle concurrent transaction requests, database systems need to distribute transactions to different CPU cores for calculation.

Different transactions may access the same data, and a lock mechanism is needed to protect specific data records concurrently. However, such a lock mechanism can only avoid concurrent access, which may cause waiting between different transaction processing, and thus the overall processing capacity of the system is severely restricted by the transaction lock granularity, resulting in poor scalability.

In order to improve parallel processing capabilities, a type of database system has proposed lock-free concurrency control, which binds data partitions to specific CPU cores. When a transaction only acts on a specific data partition, there is no need to consider concurrent access with other CPU cores. This type of method is called optimistic concurrency control (OCC). However, the OCC method has a high cost for the interruption and retry mechanism of concurrent access for transactions across partitions, and its cost is higher than that of the lock protection-based method.

Summary of the invention

In view of the above problems, the present application is proposed to provide a concurrent transaction processing method based on a multi-core processor and a system thereof that overcomes the above problems or at least partially solves the above problems, including:

A concurrent transaction processing method based on a multi-core processor comprises the following steps:

When transaction concurrency occurs within the target time period, the read and write sets of transactions occurring within the target time period are obtained;

Determining the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;

Determining the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;

Determine the queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;

When a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.

Furthermore, the execution cost is the time taken by the corresponding processor to execute the transaction; and the step of determining the queue type of the transaction in the partition according to the conflict relationship and the execution cost includes:

Determining a start time of the transaction according to the read-write set;

Determine an execution time slice of the transaction according to the start time and the execution cost;

Determine whether the execution time slice of the target transaction in the target partition overlaps with the execution time slice of transactions in other partitions according to the conflict relationship. If so, the target transaction is in the conflict queue; if not, the target transaction is in the non-conflict queue.

Furthermore, when a transaction in the conflict queue conflicts with a transaction in another partition, the step of delaying the processing of the transaction in the conflict queue includes:

Determining the order of the transaction in the conflict queue according to the execution time slice;

When the execution time slices of the transactions in the conflict queue overlap with the execution time slices of the transactions in other partitions, the transactions in the conflict queue are delayed.

Furthermore, the step of determining the order of the transactions in the conflict queue according to the execution time slice includes:

The execution time slices of the transactions in the conflict queue are arranged in a staggered manner with the execution time slices of the transactions in other partitions according to the conflict relationship.

Furthermore, the step of determining the partition corresponding to the transaction according to the conflict relationship includes:

Determining the data type accessed by the transaction according to the read-write set;

The partition of the transaction is determined according to the data type and the conflict relationship.

Furthermore, it also includes:

When the actual execution time of the transaction in the conflict queue is not equal to the corresponding execution cost, the transaction is locked for access.

Furthermore, it also includes:

When a new transaction is added to the processing, the partition and queue type corresponding to the new transaction are determined according to the read and write sets of the new transaction;

When the new transaction conflicts with transactions in other partitions in the corresponding queue, the processing of the new transaction is delayed.

A concurrent transaction processing system based on a multi-core processor, comprising:

An acquisition module, used for acquiring a read-write set of transactions occurring within a target time period when transactions occur concurrently within the target time period;

A calculation module, used to determine the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;

A partitioning module, used for determining the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;

A scheduling module, used to determine the queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;

The processing module is used for delaying the processing of transactions in the conflict queue when there is a conflict between the transactions in the conflict queue and the transactions in other partitions.

A computer device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of a concurrent transaction processing method based on a multi-core processor as described above are implemented.

A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a concurrent transaction processing method based on a multi-core processor as described above are implemented.

This application has the following advantages:

In the embodiments of the present application, relative to the problem of "limitations in transaction concurrency processing" in the prior art, the present application provides a solution to the scheduling and processing flow of transaction processing based on transaction cost and multi-core architecture, specifically: when transaction concurrency occurs within the target time period, the read-write set of the transaction occurring within the target time period is obtained; the conflict relationship between the transactions and the execution cost corresponding to the transaction are determined based on the read-write set; the partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; the queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes a conflict-free queue and a conflict queue; when the transaction in the conflict queue conflicts with the transaction in other partitions, the transaction in the conflict queue is delayed. Transaction conflicts can be reduced through runtime scheduling, which is also effective for high-conflict concurrent transactions. Conflicts can be reduced by delayed execution, and pessimistic locks do not need to be considered for all transactions, reducing the overall lock waiting delay. For execution cost estimation deviations, concurrency control can be relied on to ensure the correctness of the results.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a flowchart of a method for concurrent transaction processing based on a multi-core processor according to an embodiment of the present application;

FIG2 is a queue diagram of a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the application;

FIG3 is a block diagram of a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the application;

FIG. 4 is a schematic diagram of the compatibility of a read-write lock according to a prior art 1 provided by an embodiment of the present application.

FIG. 5 is a schematic diagram of the use of a read-write lock at a commit read level according to a first prior art provided by an embodiment of the present application.

FIG6 is a structural block diagram of a concurrent transaction processing system based on a multi-core processor provided in an embodiment of the present application;

7 is a structural block diagram of a concurrent transaction processing system based on a multi-core processor provided in an embodiment of the present application;

FIG. 8 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention.

Detailed ways

In order to make the objects, features and advantages of the present application more obvious and understandable, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

The inventors analyzed the prior art and found that: Prior Art 1 and Prior Art 2 are described below:

Existing technology 1: MySQL lock technology and MVCC foundation, the read-write lock compatibility is shown in Figure 4.

1. MySQL lock technology

When there are multiple requests to read data in the table, no action can be taken, but when there are read requests and modification requests among the multiple requests, a measure must be taken to perform concurrency control. Otherwise, inconsistency is likely to occur.

Solving the above problem is simple. You only need to use a combination of two locks to control read and write requests. These two locks are called:

A shared lock, also called a "read lock", can be shared, or multiple read requests can share a lock to read data without causing blocking.

An exclusive lock, also called a "write lock," excludes all other requests to acquire the lock and blocks the lock until the write is completed and the lock is released.

Read-write locks can be used to perform read-read operations in parallel, but write-read operations or write-write operations in parallel are not possible. Transaction isolation is achieved based on read-write locks.

2. MVCC Implementation

MVCC (MultiVersion Concurrency Control). The MVCC of the MySQL storage engine InnoDB is implemented by storing two hidden columns at the end of each row. One of these two columns stores the creation time of the row, and the other stores the expiration time of the row. Of course, what is stored is not the actual time value, but the system version number.

According to the definition of MVCC in the book "High Performance MySQL", its main implementation idea is to achieve read-write separation through multiple versions of data, so as to achieve lock-free reading and read-write parallelism.

The use of read-write locks at the committed read level is shown in Figure 5. The implementation of MVCC in MySQL relies on undo log and read view. Undo log: undo log records multiple versions of a row of data; readview: used to determine the visibility of the current version of data.

Prior Art 2: OCC Transaction Processing in Silo,OCC transaction tracks the records it reads and writes in thread local storage. At commit time, after confirming that no concurrent transaction's write set overlaps with the read set, the transaction immediately executes all write records. If the verification fails, the transaction will be aborted.

Silos are based on time periods called epochs, which are used to ensure serializable recovery to remove garbage and provide read-only snapshots. Each epoch has an epoch number. The global epoch number E is visible to all threads. Designated threads update E periodically; other threads access E when committing transactions. E should be updated frequently because the epoch period affects transaction latency, but the epoch should change infrequently compared to the transaction duration so that the value of E is usually cached. The current implementation updates every 40 milliseconds; shorter times are also possible. No locks are required to process E.

TID (thread identifier) is a 64-bit integer, which is divided into several parts to represent different meanings. The highest few bits represent epoch E when the current transaction is committed. The middle bits represent the timestamp calculated based on the current epoch. The lowest three bits represent the lock bit, the lastest-version bit, and the absent bit. Tid is written directly into the tuple (array), and each record contains the TID of the transaction that most recently modified it. And meet the conditions:

1. The timestamp assigned by the current epoch must be smaller than that assigned by the next epoch;

2. In the same worker thread, the assigned timestamps are strictly monotonically increasing;

3. For transactions that modify the same record, the timestamps are strictly monotonically increasing.

Silo records contain the following information: TID word (TID plus status bits), previous version pointer, and data.

Commitprotocol (commit protocol):

When a worker (executor) runs a transaction, it maintains a read set that identifies all records that were read, and the TID of each record when it was accessed. For modified records, it maintains a write set that stores the new state of the record (instead of the previous TID). Records that are read and modified appear in both the read set and the write set. In normal operation, all records in the write set also appear in the read set. At transaction completion, the worker attempts to commit using the following protocol:

Phase 1: Check all records in the transaction's write set and lock each record by acquiring the record's lock bit. To avoid deadlock, lock records in global order. After acquiring all write locks, the worker takes a snapshot of the global epoch number using a single memory access. To ensure that the read is in main memory (and not out of date). The snapshot of the global epoch number is the serialization point for committing transactions.

Phase 2: Check all records in the transaction's read set (which may include some records that have already been read and written). If a record's TID is different from what it observed during execution, is no longer the latest version, or is locked by a different transaction, the transaction releases its lock and aborts. If the TIDs of all read records are unchanged, the transaction is allowed to commit, knowing that all its reads are consistent. The worker assigns a TID to the transaction using the snapshot of the global epoch number obtained in phase 1.

Phase 3: The worker writes its modified records into the data blocks and updates their TIDs to the transaction IDs calculated in the previous phase. Each lock can be released immediately after its record is written. It must be ensured that the new TID is visible as soon as the lock is released.

In summary, the concurrent transaction processing mechanism for multi-cores is one of the key challenges of modern OLTP (online transaction processing) database management systems. Whether the database system can meet more parallel transaction requests as the number of hardware cores increases is a key indicator for measuring the database processing capability. Commercial database Oracle and open source databases MySQL and PostgreSQL all rely on traditional transaction processing mechanisms based on lock waiting. Their implementation is relatively simple and reliable, but the scalability problem is not well solved.

In the research field, there are also many new database systems based on OCC that continue to try to solve such problems. Among them, the more representative ones are Silo proposed by MIT and TICTOC proposed by Carnegie Mellon University. They verified the scalability of OCC in low-conflict scenarios, but it is not suitable for high-conflict scenarios. When cross-data partition transactions based on the OCC method cause concurrent access, the current transaction needs to be interrupted and retried. Transactions can be parallelized at the partition granularity, but the interruption and retry mechanism for concurrent access to cross-partition transactions is more expensive. When the transaction conflict is large, it may cause a large number of transactions to be retried repeatedly.

The present invention focuses on solving how to efficiently handle concurrent transactions in a database management system under a multi-core architecture. The key is how to maximize parallelism and avoid concurrent access between transactions for a batch of transactions within a specific time. The present invention provides a scheduling mechanism that supports transaction cost-based scheduling to solve the above problem, that is, for a specific transaction set, it is assigned to different CPU cores according to the data sets it accesses for conflict delay processing. For a specific CPU core, a specific transaction conflicts with other CPU core transactions, and the premise of parallel execution is time staggered. Therefore, based on the advance identification of conflicts and costs between transactions, parallel transaction processing between different CPU cores is achieved through time slice delay scheduling.

1 and 2 , a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the present application is shown;

The method comprises:

S110, when concurrent transactions occur within the target time period, obtaining a read-write set of transactions occurring within the target time period;

S120, determining the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;

S130, determining the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;

S140, determining a queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;

S150: When a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is processed in a delayed manner.

In the embodiments of the present application, relative to the problem of "limitations in transaction concurrency processing" in the prior art, the present application provides a solution for scheduling and processing flow of transaction processing based on transaction cost and multi-core architecture, specifically: when transaction concurrency occurs within the target time period, the read-write set of transactions occurring within the target time period is obtained; the conflict relationship between the transactions and the execution cost corresponding to the transactions are determined based on the read-write set; the partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; the queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes conflict-free queue and conflict queue; when the transaction in the conflict queue conflicts with the transaction in other partitions, the transaction in the conflict queue is delayed. Transaction conflicts can be reduced through runtime scheduling, which is also effective for high-conflict concurrent transactions. Conflicts can be reduced by delayed execution, and pessimistic locks do not need to be considered for all transactions, reducing the overall lock waiting delay. For execution cost estimation deviations, concurrency control can be relied on to ensure the correctness of the results.

Next, a concurrent transaction processing method based on a multi-core processor in this exemplary embodiment will be further described.

As described in step S110, when transaction concurrency occurs within the target time period, the read and write sets of transactions occurring within the target time period are acquired.

As an example, in data synchronization, the synchronization system is deployed on the source database and the target database. The source data synchronization system reads logs from the source database, while the target data synchronization system is responsible for applying the synchronization operations sent from the source to the target database. A set of operations is an indivisible unit of work. A transaction submits or cancels all operations as a whole to the system, that is, these operations either succeed or fail at the same time.

The source database generates data changes by executing transactions. Each transaction contains one or more database operations, and each operation generates a data change. Operations include reading data, writing data, updating and modifying data, and deleting data. In specific implementation scenarios, one operation can correspond to an SQL statement. A transaction is a set of statements for database operations. For a batch of transactions within a specific time period, the database system will generate a set of read data and a set of write data for the current transaction.

As described in step S120, the conflict relationship between the transactions and the execution cost corresponding to the transaction are determined according to the read-write set.

As an example, the transaction cost refers to the time required by the CPU to execute the transaction, which is usually related to the amount of data modified. The database system generates a set of read and write data for the current transaction, through which its execution cost can be estimated. A transaction contains an execution cost, which is a fixed value and is only related to the modified data record.

After determining the read and write sets of each transaction, a conflict relationship graph is established between transactions in the same batch. For any transaction, a list of transactions that conflict with it can be quickly found. The relationship graph is used to determine whether related transactions conflict, which is used for subsequent partitioning and scheduling. If two transactions conflict, ensure that their running times do not overlap.

In a specific implementation, referring to FIG2, assuming that a batch task includes T1, T2, ..., Tn transactions, the conflict graph between transactions is first formed from the read and write data sets involved in each transaction, where each transaction includes the execution cost of its conflicting transaction. As shown in FIG2, Txn* on the right is Txn1, and it can be concluded that Txn1 conflicts with Txn2, Txn4, and Txn6.

As described in step S130, the partition corresponding to the transaction is determined according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same.

In an embodiment of the present invention, the specific process of "determining the partition corresponding to the transaction according to the conflict relationship" in step S130 can be further explained in combination with the following description.

As described in the following steps, determining the data type accessed by the transaction according to the read-write set;

As described in the following steps, the partition of the transaction is determined according to the data type and the conflict relationship.

As an example, for batch transaction processes, after determining the read and write sets of each transaction, divide it according to the data set it accesses to ensure that transactions accessing the same data belong to the same partition as much as possible to avoid conflicts between cross-partition transactions. In order to improve the efficiency of transaction execution on the destination side, the destination side uses parallel execution for transactions when synchronizing data. Therefore, it is necessary to create multiple execution threads that can be executed in parallel and assign transactions that need to execute the corresponding partitions to different execution threads for execution.

In a specific implementation, the conflict relationship graph is partitioned by a partitioning algorithm (such as a minimum cut algorithm). The purpose of the partitioning algorithm is to reduce the number of conflicting transactions across partitions as much as possible. Ideally, transactions in each partition will not access the same data as transactions in other partitions.

As described in step S140, the queue type of the transaction in the partition is determined according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue.

In an embodiment of the present invention, the specific process of "determining the queue type of the transaction in the partition according to the conflict relationship and the execution cost" in step S140 can be further explained in combination with the following description.

As described in the following steps, determining the start time of the transaction according to the read-write set;

As described in the following steps, determining the execution time slice of the transaction according to the start time and the execution cost;

As described in the following steps, it is determined whether the execution time slice of the target transaction in the target partition overlaps with the execution time slices of transactions in other partitions based on the conflict relationship. If so, the target transaction is in the conflict queue; if not, the target transaction is in the non-conflict queue.

As an example, the execution start time of each transaction is obtained from the read-write set of the transaction, and then the execution time slice of each transaction is obtained according to the time required for the corresponding execution thread to execute the transaction. The transactions in the partition can be placed in different queues through the execution time slice and the conflict relationship diagram. The thread responsible for execution decides to put the transaction into different queues according to the execution cost of the transaction and the relevant transaction conflict information. The transactions in each partition that do not conflict with the transactions of other partitions are added to the corresponding conflict-free queue P, where the transactions in the conflict-free queue P are completely independent of the transactions of other partitions, while the transactions in the conflict queue Q need to be scheduled and sorted. Since the transactions of each partition are independent, the execution order is independent of other partitions, and multiple transaction execution threads can be executed in parallel.

In a specific implementation, the number of queues is consistent with the number of partitions. Assume that there are 50 partitions, and each partition has two types of queues: conflict-free queue P and conflict queue Q. Then, there are a total of 100 queues in the 50 partitions, including 50 conflict-free queues P and 50 conflict queues Q, which are equal to the number of partitions.

As described in step S150, when a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.

In an embodiment of the present invention, the specific process of "when a transaction in the conflict queue conflicts with a transaction in another partition, delaying the processing of the transaction in the conflict queue" in step S150 can be further explained in combination with the following description.

As described in the following steps, determining the order of the transaction in the conflict queue according to the execution time slice;

As described in the following steps, when the execution time slices of transactions in the conflict queue overlap with the execution time slices of transactions in other partitions, the transactions in the conflict queue are delayed.

In an embodiment of the present invention, the specific process of “determining the order of the transaction in the conflict queue according to the execution time slice” may be further explained in combination with the following description.

As described in the following steps, the execution time slices of the transactions in the conflict queue and the execution time slices of the transactions in other partitions are arranged in a staggered manner according to the conflict relationship.

As an example, in the conflict queue Q, the transactions therein need to be scheduled and sorted. The time slices of the transactions in the conflict queue Q are staggered and non-overlapping with the time slices of the transactions in other queues. If there is still a specific transaction that is already at the end of the conflict queue Q and overlaps with the execution time slices of other threads, the specific transaction is delayed to achieve conflict-free concurrent execution.

In a specific implementation, referring to FIG. 2 , the black shaded portion in FIG. 2 represents the time slices in the queue that have been occupied by other transactions. When a transaction is assigned to a scheduling queue Q of a specific partition, it is necessary to confirm the execution time slices of other transactions through the conflict relationship diagram. In principle, conflicting transactions cannot have overlapping time slices in order to be executed in parallel. For example, the start time of T2 joining Q3 is later than the end time of T1, or the end time of T2 joining Qk is earlier than the start time of T1. When a specific queue Q can no longer find a time slice queue that is staggered with the conflicting transaction, conflict-free concurrent execution is achieved by delaying the start.

In this embodiment, the concurrent transaction processing method further includes:

As an example, for the scheduled transactions, concurrent access control needs to be considered during the final execution. By locking the transaction access, it is possible to avoid runtime cost deviations that cause data to be modified by multiple transactions at the same time. Locking can construct a row lock for each operation based on the unique identifier of each operation, and the value of the unique identifier is used as the value of the row lock.

As an example, for each instant transaction, obtain its read-write set, and determine the conflict relationship and execution cost between the instant transaction and the current transaction based on its read-write set, perform certain random processing based on the data set it accesses, determine the partition and queue corresponding to the instant transaction, and assign the transaction to the corresponding execution thread. Schedule the instant transaction together with the current partitioned transaction. The subsequent steps are similar to batch transaction processing. If the execution time slice of the instant transaction overlaps with that of transactions in other partition queues, the instant transaction will be delayed.

Referring to Figure 3, there is a block diagram of a scheme of a concurrent transaction processing method based on a multi-core processor of the present invention. In Figure 3, bundled workload is a batch transaction request, which is a routine business; unbundled transactions are instant transaction requests; Transaction-to-thread assignment is to distribute transactions to specific threads (i.e., CPU cores); for heavier batch requests, consider partitioning transactions according to relevance, and for lighter instant requests, consider assigning them through a random algorithm. After the transactions are partitioned, Thread-local buffers are used to save transactions assigned to specific threads. TSkd is mainly used for transaction execution scheduling within each thread, including a scheduler and a delayer. Their function is to stagger the execution time slices of transactions that conflict between threads. The specific execution unit is Execution (CC), which can support various concurrent transaction processing mechanisms, such as lock waiting and OCC.

Example 1

Batch transaction process:

1. After determining the read and write sets of each transaction, a conflict relationship diagram is established for transactions in the same batch. For any Txn, a list of transactions that conflict with it can be quickly found for subsequent partitioning and scheduling.

2. After determining the read and write sets of each transaction, divide them according to the data sets they access, ensuring that transactions accessing the same data belong to the same partition as much as possible to avoid conflicts between cross-partition transactions. Assign transactions of the corresponding partitions to the corresponding execution threads.

3. The thread responsible for execution decides to put the transaction into different queues based on the execution cost of the transaction and the related transaction conflict information. The transactions in the conflict-free queue P are completely independent of the transactions in other partitions, while the transactions in the conflict queue Q need to be scheduled and sorted.

4. If a specific transaction is already at the end of the conflict queue and still conflicts with other threads, choose to delay processing.

5. For transactions that have completed scheduling, concurrent access control needs to be considered during the final execution to avoid runtime cost deviations that cause data to be modified by multiple transactions at the same time.

Real-time transaction process:

For each instant transaction, a certain randomization is performed based on the data set it accesses, and the transaction is assigned to the corresponding execution thread. It is scheduled together with the currently partitioned transactions. The subsequent steps are similar to batch transaction processing and will not be repeated here.

As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

6 , a concurrent transaction processing system based on a multi-core processor provided in an embodiment of the present application is shown;

Specifically include:

The acquisition module 610 is used to acquire the read and write sets of the transactions occurring in the target time period when transactions are concurrently occurring in the target time period;

A calculation module 620, configured to determine the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;

A partition module 630 is used to determine the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;

A scheduling module 640, configured to determine a queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;

The processing module 650 is used to delay the processing of the transaction in the conflict queue when there is a conflict between the transaction in the conflict queue and the transaction in other partitions.

In one embodiment of the present invention, the execution cost is the time taken by the corresponding processor to execute the transaction; the scheduling module 640 includes:

A start time determination submodule, used to determine the start time of the transaction according to the read-write set;

A time slice determination submodule, used to determine the execution time slice of the transaction according to the start time and the execution cost;

The first judgment submodule is used to determine whether the execution time slice of the target transaction in the target partition overlaps with the execution time slices of transactions in other partitions according to the conflict relationship. If so, the target transaction is in the conflict queue; if not, the target transaction is in the non-conflict queue.

In one embodiment of the present invention, the processing module 650 includes:

A sorting submodule, used to determine the sorting of the transaction in the conflict queue according to the execution time slice;

The delay processing submodule is used to delay the processing of transactions in the conflict queue when the execution time slice of the transaction in the conflict queue overlaps with the execution time slice of the transaction in other partitions.

In one embodiment of the present invention, the sorting submodule includes:

The peak-shifting sorting unit is used to sort the execution time slices of the transactions in the conflict queue and the execution time slices of the transactions in other partitions in a peak-shifting manner according to the conflict relationship.

In one embodiment of the present invention, the partition module 630 includes:

An access type determination submodule, used to determine the data type accessed by the transaction according to the read-write set;

The partition determination submodule is used to determine the partition of the transaction according to the data type and the conflict relationship.

In one embodiment of the present invention, it also includes:

The protection module 660 is used to lock access to the transaction when the actual execution time of the transaction in the conflict queue is not equal to the corresponding execution cost.

7, in one embodiment of the present invention, the concurrent transaction processing system further includes:

The real-time transaction scheduling module 710 is used to determine the partition and queue type corresponding to a new transaction according to the read-write set of the new transaction when a new transaction is added to the processing;

The instant transaction processing module 720 is used to delay the processing of the new transaction when the new transaction conflicts with transactions in other partitions in the corresponding queue.

8, a computer device of a concurrent transaction processing method based on a multi-core processor of the present invention is shown, which may specifically include the following:

The computer device 12 is in the form of a general-purpose computing device, and the components of the computer device 12 may include but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

The bus 18 represents one or more of several types of bus 18 structures, including a memory bus 18 or memory controller, a peripheral bus 18, an accelerated graphics port, a processor or a local bus 18 using any of a variety of bus 18 architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus 18, a Micro Channel Architecture (MAC) bus 18, an Enhanced ISA bus 18, an Audio Video Electronics Standards Association (VESA) local bus 18, and a Peripheral Component Interconnect (PCI) bus 18.

Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32. The computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as "hard drives"). Although not shown in Figure 8, a disk drive for reading and writing removable non-volatile disks (such as "floppy disks"), and an optical disk drive for reading and writing removable non-volatile optical disks (such as CD-ROMs, DVD-ROMs or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42 that are configured to perform the functions of various embodiments of the present invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored in, for example, a memory, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules 42, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and/or methods of the embodiments described herein.

The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, cameras, etc.), may communicate with one or more devices that enable an operator to interact with the computer device 12, and/or may communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input/output (I/O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs)), wide area networks (WANs), and/or public networks (e.g., the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer device 12 via a bus 18. It should be understood that, although not shown in FIG. 8 , other hardware and/or software modules may be used in conjunction with the computer device 12, including, but not limited to, microcode, device drivers, redundant processing units 16, external disk drive arrays, RAID systems, tape drives, and data backup storage systems 34, etc.

The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the present invention.

That is, when the processing unit 16 executes the program, the following is achieved: when concurrent transactions occur within a target time period, a read-write set of transactions occurring within the target time period is obtained; a conflict relationship between the transactions and an execution cost corresponding to the transaction are determined based on the read-write set; a partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; a queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue; when a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.

In an embodiment of the present invention, the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a concurrent transaction processing method based on a multi-core processor as provided in all embodiments of the present application is implemented:

That is, when the program is executed by the processor, it is implemented as follows: when concurrent transactions occur within a target time period, a read-write set of transactions occurring within the target time period is obtained; a conflict relationship between the transactions and an execution cost corresponding to the transaction are determined based on the read-write set; a partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; a queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue; when a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.

Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

The computer program code for performing the operation of the present invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The program code can be executed entirely on the operator's computer, partially on the operator's computer, as an independent software package, partially on the operator's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the operator's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet). The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other.

Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

The above is a detailed introduction to a concurrent transaction processing method and system based on a multi-core processor provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

A concurrent transaction processing method based on a multi-core processor, characterized by comprising the steps of:

When transaction concurrency occurs within the target time period, the read and write sets of transactions occurring within the target time period are obtained;

Determining the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;

Determining the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;

Determine the queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;

When a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.
The method according to claim 1, characterized in that the execution cost is the time taken by the corresponding processor to execute the transaction; the step of determining the queue type of the transaction in the partition based on the conflict relationship and the execution cost comprises:

Determining a start time of the transaction according to the read-write set;

Determine an execution time slice of the transaction according to the start time and the execution cost;

Determine whether the execution time slice of the target transaction in the target partition overlaps with the execution time slice of transactions in other partitions according to the conflict relationship. If so, the target transaction is in the conflict queue; if not, the target transaction is in the non-conflict queue.
The method according to claim 2, characterized in that when a transaction in the conflict queue conflicts with a transaction in another partition, the step of delaying the processing of the transaction in the conflict queue comprises:

Determining the order of the transaction in the conflict queue according to the execution time slice;

When the execution time slices of the transactions in the conflict queue overlap with the execution time slices of the transactions in other partitions, the transactions in the conflict queue are delayed.
The method according to claim 3, characterized in that the step of determining the order of the transactions in the conflict queue according to the execution time slice comprises:

The execution time slices of the transactions in the conflict queue and the execution time slices of the transactions in other partitions are arranged in a staggered order according to the conflict relationship.
The method according to claim 1, characterized in that the step of determining the partition corresponding to the transaction based on the conflict relationship comprises:

Determining the data type accessed by the transaction according to the read-write set;

The partition of the transaction is determined according to the data type and the conflict relationship.
The method according to claim 1, further comprising:

When the actual execution time of the transaction in the conflict queue is not equal to the corresponding execution cost, the transaction is locked for access.
The method according to claim 1, further comprising:

When a new transaction is added to the processing, the partition and queue type corresponding to the new transaction are determined according to the read and write sets of the new transaction;

When the new transaction conflicts with transactions in other partitions in the corresponding queue, the processing of the new transaction is delayed.
A concurrent transaction processing system based on a multi-core processor, characterized by comprising:

An acquisition module, used for acquiring a read-write set of transactions occurring within a target time period when transactions occur concurrently within the target time period;

A calculation module, used to determine the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;

A partitioning module, used to determine the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;

A scheduling module, used to determine the queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;

The processing module is used for delaying the processing of transactions in the conflict queue when there is a conflict between the transactions in the conflict queue and the transactions in other partitions.
A computer device, characterized in that it comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.