WO2024098363A1 - Multicore-processor-based concurrent transaction processing method and system - Google Patents

Multicore-processor-based concurrent transaction processing method and system Download PDF

Info

Publication number
WO2024098363A1
WO2024098363A1 PCT/CN2022/131298 CN2022131298W WO2024098363A1 WO 2024098363 A1 WO2024098363 A1 WO 2024098363A1 CN 2022131298 W CN2022131298 W CN 2022131298W WO 2024098363 A1 WO2024098363 A1 WO 2024098363A1
Authority
WO
WIPO (PCT)
Prior art keywords
transaction
conflict
transactions
queue
execution
Prior art date
Application number
PCT/CN2022/131298
Other languages
French (fr)
Chinese (zh)
Inventor
樊文飞
曹洋
欧伟杰
谢锐
Original Assignee
深圳计算科学研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳计算科学研究院 filed Critical 深圳计算科学研究院
Publication of WO2024098363A1 publication Critical patent/WO2024098363A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/445Program loading or initiating
    • G06F9/44552Conflict resolution, i.e. enabling coexistence of conflicting executables
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/52Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present application relates to the field of concurrent transaction processing, and in particular to a concurrent transaction processing method and system based on a multi-core processor.
  • a type of database system has proposed lock-free concurrency control, which binds data partitions to specific CPU cores.
  • This type of method is called optimistic concurrency control (OCC).
  • OCC optimistic concurrency control
  • the OCC method has a high cost for the interruption and retry mechanism of concurrent access for transactions across partitions, and its cost is higher than that of the lock protection-based method.
  • the present application is proposed to provide a concurrent transaction processing method based on a multi-core processor and a system thereof that overcomes the above problems or at least partially solves the above problems, including:
  • a concurrent transaction processing method based on a multi-core processor comprises the following steps:
  • the queue type includes a non-conflict queue and a conflict queue
  • the execution cost is the time taken by the corresponding processor to execute the transaction; and the step of determining the queue type of the transaction in the partition according to the conflict relationship and the execution cost includes:
  • the step of delaying the processing of the transaction in the conflict queue includes:
  • step of determining the order of the transactions in the conflict queue according to the execution time slice includes:
  • the execution time slices of the transactions in the conflict queue are arranged in a staggered manner with the execution time slices of the transactions in other partitions according to the conflict relationship.
  • step of determining the partition corresponding to the transaction according to the conflict relationship includes:
  • the partition of the transaction is determined according to the data type and the conflict relationship.
  • the partition and queue type corresponding to the new transaction are determined according to the read and write sets of the new transaction;
  • a concurrent transaction processing system based on a multi-core processor comprising:
  • An acquisition module used for acquiring a read-write set of transactions occurring within a target time period when transactions occur concurrently within the target time period
  • a calculation module used to determine the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set
  • a partitioning module used for determining the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;
  • a scheduling module used to determine the queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;
  • the processing module is used for delaying the processing of transactions in the conflict queue when there is a conflict between the transactions in the conflict queue and the transactions in other partitions.
  • a computer device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of a concurrent transaction processing method based on a multi-core processor as described above are implemented.
  • a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a concurrent transaction processing method based on a multi-core processor as described above are implemented.
  • the present application provides a solution to the scheduling and processing flow of transaction processing based on transaction cost and multi-core architecture, specifically: when transaction concurrency occurs within the target time period, the read-write set of the transaction occurring within the target time period is obtained; the conflict relationship between the transactions and the execution cost corresponding to the transaction are determined based on the read-write set; the partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; the queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes a conflict-free queue and a conflict queue; when the transaction in the conflict queue conflicts with the transaction in other partitions, the transaction in the conflict queue is delayed.
  • Transaction conflicts can be reduced through runtime scheduling, which is also effective for high-conflict concurrent transactions. Conflicts can be reduced by delayed execution, and pessimistic locks do not need to be considered for all transactions, reducing the overall lock waiting delay. For execution cost estimation deviations, concurrency control can be relied on to ensure the correctness of the results.
  • FIG1 is a flowchart of a method for concurrent transaction processing based on a multi-core processor according to an embodiment of the present application
  • FIG2 is a queue diagram of a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the application;
  • FIG3 is a block diagram of a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the application;
  • FIG. 4 is a schematic diagram of the compatibility of a read-write lock according to a prior art 1 provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of the use of a read-write lock at a commit read level according to a first prior art provided by an embodiment of the present application.
  • FIG6 is a structural block diagram of a concurrent transaction processing system based on a multi-core processor provided in an embodiment of the present application
  • FIG. 7 is a structural block diagram of a concurrent transaction processing system based on a multi-core processor provided in an embodiment of the present application
  • FIG. 8 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention.
  • Prior Art 1 and Prior Art 2 are described below:
  • a shared lock also called a "read lock”
  • An exclusive lock also called a "write lock,” excludes all other requests to acquire the lock and blocks the lock until the write is completed and the lock is released.
  • Read-write locks can be used to perform read-read operations in parallel, but write-read operations or write-write operations in parallel are not possible. Transaction isolation is achieved based on read-write locks.
  • MVCC MultiVersion Concurrency Control
  • the MVCC of the MySQL storage engine InnoDB is implemented by storing two hidden columns at the end of each row. One of these two columns stores the creation time of the row, and the other stores the expiration time of the row. Of course, what is stored is not the actual time value, but the system version number.
  • OCC Transaction Processing in Silo,OCC transaction tracks the records it reads and writes in thread local storage. At commit time, after confirming that no concurrent transaction's write set overlaps with the read set, the transaction immediately executes all write records. If the verification fails, the transaction will be aborted.
  • Silos are based on time periods called epochs, which are used to ensure serializable recovery to remove garbage and provide read-only snapshots.
  • Each epoch has an epoch number.
  • the global epoch number E is visible to all threads. Designated threads update E periodically; other threads access E when committing transactions. E should be updated frequently because the epoch period affects transaction latency, but the epoch should change infrequently compared to the transaction duration so that the value of E is usually cached.
  • the current implementation updates every 40 milliseconds; shorter times are also possible. No locks are required to process E.
  • TID thread identifier
  • epoch E when the current transaction is committed.
  • the middle bits represent the timestamp calculated based on the current epoch.
  • the lowest three bits represent the lock bit, the lastest-version bit, and the absent bit. Tid is written directly into the tuple (array), and each record contains the TID of the transaction that most recently modified it. And meet the conditions:
  • the timestamp assigned by the current epoch must be smaller than that assigned by the next epoch;
  • TID word TID plus status bits
  • previous version pointer previous version pointer
  • a worker executes a transaction, it maintains a read set that identifies all records that were read, and the TID of each record when it was accessed. For modified records, it maintains a write set that stores the new state of the record (instead of the previous TID). Records that are read and modified appear in both the read set and the write set. In normal operation, all records in the write set also appear in the read set.
  • the worker attempts to commit using the following protocol:
  • Phase 1 Check all records in the transaction's write set and lock each record by acquiring the record's lock bit. To avoid deadlock, lock records in global order. After acquiring all write locks, the worker takes a snapshot of the global epoch number using a single memory access. To ensure that the read is in main memory (and not out of date). The snapshot of the global epoch number is the serialization point for committing transactions.
  • Phase 2 Check all records in the transaction's read set (which may include some records that have already been read and written). If a record's TID is different from what it observed during execution, is no longer the latest version, or is locked by a different transaction, the transaction releases its lock and aborts. If the TIDs of all read records are unchanged, the transaction is allowed to commit, knowing that all its reads are consistent. The worker assigns a TID to the transaction using the snapshot of the global epoch number obtained in phase 1.
  • Phase 3 The worker writes its modified records into the data blocks and updates their TIDs to the transaction IDs calculated in the previous phase. Each lock can be released immediately after its record is written. It must be ensured that the new TID is visible as soon as the lock is released.
  • the concurrent transaction processing mechanism for multi-cores is one of the key challenges of modern OLTP (online transaction processing) database management systems. Whether the database system can meet more parallel transaction requests as the number of hardware cores increases is a key indicator for measuring the database processing capability.
  • OLTP online transaction processing
  • Commercial database Oracle and open source databases MySQL and PostgreSQL all rely on traditional transaction processing mechanisms based on lock waiting. Their implementation is relatively simple and reliable, but the scalability problem is not well solved.
  • the present invention focuses on solving how to efficiently handle concurrent transactions in a database management system under a multi-core architecture.
  • the key is how to maximize parallelism and avoid concurrent access between transactions for a batch of transactions within a specific time.
  • the present invention provides a scheduling mechanism that supports transaction cost-based scheduling to solve the above problem, that is, for a specific transaction set, it is assigned to different CPU cores according to the data sets it accesses for conflict delay processing.
  • a specific transaction conflicts with other CPU core transactions, and the premise of parallel execution is time staggered. Therefore, based on the advance identification of conflicts and costs between transactions, parallel transaction processing between different CPU cores is achieved through time slice delay scheduling.
  • the method comprises:
  • S140 determining a queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;
  • the present application provides a solution for scheduling and processing flow of transaction processing based on transaction cost and multi-core architecture, specifically: when transaction concurrency occurs within the target time period, the read-write set of transactions occurring within the target time period is obtained; the conflict relationship between the transactions and the execution cost corresponding to the transactions are determined based on the read-write set; the partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; the queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes conflict-free queue and conflict queue; when the transaction in the conflict queue conflicts with the transaction in other partitions, the transaction in the conflict queue is delayed.
  • Transaction conflicts can be reduced through runtime scheduling, which is also effective for high-conflict concurrent transactions. Conflicts can be reduced by delayed execution, and pessimistic locks do not need to be considered for all transactions, reducing the overall lock waiting delay. For execution cost estimation deviations, concurrency control can be relied on to ensure the correctness of the results.
  • step S110 when transaction concurrency occurs within the target time period, the read and write sets of transactions occurring within the target time period are acquired.
  • the synchronization system in data synchronization, is deployed on the source database and the target database.
  • the source data synchronization system reads logs from the source database, while the target data synchronization system is responsible for applying the synchronization operations sent from the source to the target database.
  • a set of operations is an indivisible unit of work.
  • a transaction submits or cancels all operations as a whole to the system, that is, these operations either succeed or fail at the same time.
  • the source database generates data changes by executing transactions.
  • Each transaction contains one or more database operations, and each operation generates a data change. Operations include reading data, writing data, updating and modifying data, and deleting data.
  • one operation can correspond to an SQL statement.
  • a transaction is a set of statements for database operations. For a batch of transactions within a specific time period, the database system will generate a set of read data and a set of write data for the current transaction.
  • step S120 the conflict relationship between the transactions and the execution cost corresponding to the transaction are determined according to the read-write set.
  • the transaction cost refers to the time required by the CPU to execute the transaction, which is usually related to the amount of data modified.
  • the database system generates a set of read and write data for the current transaction, through which its execution cost can be estimated.
  • a transaction contains an execution cost, which is a fixed value and is only related to the modified data record.
  • a conflict relationship graph is established between transactions in the same batch. For any transaction, a list of transactions that conflict with it can be quickly found. The relationship graph is used to determine whether related transactions conflict, which is used for subsequent partitioning and scheduling. If two transactions conflict, ensure that their running times do not overlap.
  • Txn* on the right is Txn1, and it can be concluded that Txn1 conflicts with Txn2, Txn4, and Txn6.
  • the partition corresponding to the transaction is determined according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same.
  • step S130 the specific process of "determining the partition corresponding to the transaction according to the conflict relationship" in step S130 can be further explained in combination with the following description.
  • the partition of the transaction is determined according to the data type and the conflict relationship.
  • the conflict relationship graph is partitioned by a partitioning algorithm (such as a minimum cut algorithm).
  • the purpose of the partitioning algorithm is to reduce the number of conflicting transactions across partitions as much as possible. Ideally, transactions in each partition will not access the same data as transactions in other partitions.
  • the queue type of the transaction in the partition is determined according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue.
  • step S140 the specific process of "determining the queue type of the transaction in the partition according to the conflict relationship and the execution cost" in step S140 can be further explained in combination with the following description.
  • the execution start time of each transaction is obtained from the read-write set of the transaction, and then the execution time slice of each transaction is obtained according to the time required for the corresponding execution thread to execute the transaction.
  • the transactions in the partition can be placed in different queues through the execution time slice and the conflict relationship diagram.
  • the thread responsible for execution decides to put the transaction into different queues according to the execution cost of the transaction and the relevant transaction conflict information.
  • the transactions in each partition that do not conflict with the transactions of other partitions are added to the corresponding conflict-free queue P, where the transactions in the conflict-free queue P are completely independent of the transactions of other partitions, while the transactions in the conflict queue Q need to be scheduled and sorted. Since the transactions of each partition are independent, the execution order is independent of other partitions, and multiple transaction execution threads can be executed in parallel.
  • the number of queues is consistent with the number of partitions. Assume that there are 50 partitions, and each partition has two types of queues: conflict-free queue P and conflict queue Q. Then, there are a total of 100 queues in the 50 partitions, including 50 conflict-free queues P and 50 conflict queues Q, which are equal to the number of partitions.
  • step S150 when a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.
  • step S150 the specific process of "when a transaction in the conflict queue conflicts with a transaction in another partition, delaying the processing of the transaction in the conflict queue" in step S150 can be further explained in combination with the following description.
  • the execution time slices of the transactions in the conflict queue and the execution time slices of the transactions in other partitions are arranged in a staggered manner according to the conflict relationship.
  • the transactions therein need to be scheduled and sorted.
  • the time slices of the transactions in the conflict queue Q are staggered and non-overlapping with the time slices of the transactions in other queues. If there is still a specific transaction that is already at the end of the conflict queue Q and overlaps with the execution time slices of other threads, the specific transaction is delayed to achieve conflict-free concurrent execution.
  • the black shaded portion in FIG. 2 represents the time slices in the queue that have been occupied by other transactions.
  • a transaction is assigned to a scheduling queue Q of a specific partition, it is necessary to confirm the execution time slices of other transactions through the conflict relationship diagram.
  • conflicting transactions cannot have overlapping time slices in order to be executed in parallel.
  • the start time of T2 joining Q3 is later than the end time of T1, or the end time of T2 joining Qk is earlier than the start time of T1.
  • conflict-free concurrent execution is achieved by delaying the start.
  • the concurrent transaction processing method further includes:
  • Locking can construct a row lock for each operation based on the unique identifier of each operation, and the value of the unique identifier is used as the value of the row lock.
  • the concurrent transaction processing method further includes:
  • the partition and queue type corresponding to the new transaction are determined according to the read and write sets of the new transaction;
  • each instant transaction obtain its read-write set, and determine the conflict relationship and execution cost between the instant transaction and the current transaction based on its read-write set, perform certain random processing based on the data set it accesses, determine the partition and queue corresponding to the instant transaction, and assign the transaction to the corresponding execution thread. Schedule the instant transaction together with the current partitioned transaction. The subsequent steps are similar to batch transaction processing. If the execution time slice of the instant transaction overlaps with that of transactions in other partition queues, the instant transaction will be delayed.
  • FIG. 3 there is a block diagram of a scheme of a concurrent transaction processing method based on a multi-core processor of the present invention.
  • bundled workload is a batch transaction request, which is a routine business; unbundled transactions are instant transaction requests; Transaction-to-thread assignment is to distribute transactions to specific threads (i.e., CPU cores); for heavier batch requests, consider partitioning transactions according to relevance, and for lighter instant requests, consider assigning them through a random algorithm.
  • Thread-local buffers are used to save transactions assigned to specific threads.
  • TSkd is mainly used for transaction execution scheduling within each thread, including a scheduler and a delayer. Their function is to stagger the execution time slices of transactions that conflict between threads.
  • the specific execution unit is Execution (CC), which can support various concurrent transaction processing mechanisms, such as lock waiting and OCC.
  • CC Execution
  • the thread responsible for execution decides to put the transaction into different queues based on the execution cost of the transaction and the related transaction conflict information.
  • the transactions in the conflict-free queue P are completely independent of the transactions in other partitions, while the transactions in the conflict queue Q need to be scheduled and sorted.
  • the acquisition module 610 is used to acquire the read and write sets of the transactions occurring in the target time period when transactions are concurrently occurring in the target time period;
  • a calculation module 620 configured to determine the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set
  • a partition module 630 is used to determine the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;
  • a scheduling module 640 configured to determine a queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;
  • the processing module 650 is used to delay the processing of the transaction in the conflict queue when there is a conflict between the transaction in the conflict queue and the transaction in other partitions.
  • the execution cost is the time taken by the corresponding processor to execute the transaction;
  • the scheduling module 640 includes:
  • a start time determination submodule used to determine the start time of the transaction according to the read-write set
  • a time slice determination submodule used to determine the execution time slice of the transaction according to the start time and the execution cost
  • the first judgment submodule is used to determine whether the execution time slice of the target transaction in the target partition overlaps with the execution time slices of transactions in other partitions according to the conflict relationship. If so, the target transaction is in the conflict queue; if not, the target transaction is in the non-conflict queue.
  • the processing module 650 includes:
  • a sorting submodule used to determine the sorting of the transaction in the conflict queue according to the execution time slice
  • the delay processing submodule is used to delay the processing of transactions in the conflict queue when the execution time slice of the transaction in the conflict queue overlaps with the execution time slice of the transaction in other partitions.
  • the sorting submodule includes:
  • the peak-shifting sorting unit is used to sort the execution time slices of the transactions in the conflict queue and the execution time slices of the transactions in other partitions in a peak-shifting manner according to the conflict relationship.
  • the partition module 630 includes:
  • An access type determination submodule used to determine the data type accessed by the transaction according to the read-write set
  • the partition determination submodule is used to determine the partition of the transaction according to the data type and the conflict relationship.
  • it also includes:
  • the protection module 660 is used to lock access to the transaction when the actual execution time of the transaction in the conflict queue is not equal to the corresponding execution cost.
  • the concurrent transaction processing system further includes:
  • the real-time transaction scheduling module 710 is used to determine the partition and queue type corresponding to a new transaction according to the read-write set of the new transaction when a new transaction is added to the processing;
  • the instant transaction processing module 720 is used to delay the processing of the new transaction when the new transaction conflicts with transactions in other partitions in the corresponding queue.
  • a computer device of a concurrent transaction processing method based on a multi-core processor of the present invention is shown, which may specifically include the following:
  • the computer device 12 is in the form of a general-purpose computing device, and the components of the computer device 12 may include but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
  • the bus 18 represents one or more of several types of bus 18 structures, including a memory bus 18 or memory controller, a peripheral bus 18, an accelerated graphics port, a processor or a local bus 18 using any of a variety of bus 18 architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus 18, a Micro Channel Architecture (MAC) bus 18, an Enhanced ISA bus 18, an Audio Video Electronics Standards Association (VESA) local bus 18, and a Peripheral Component Interconnect (PCI) bus 18.
  • ISA Industry Standard Architecture
  • MAC Micro Channel Architecture
  • VESA Audio Video Electronics Standards Association
  • PCI Peripheral Component Interconnect
  • Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and nonvolatile media, removable and non-removable media.
  • the system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32.
  • the computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
  • the storage system 34 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as "hard drives").
  • a disk drive for reading and writing removable non-volatile disks such as "floppy disks”
  • an optical disk drive for reading and writing removable non-volatile optical disks (such as CD-ROMs, DVD-ROMs or other optical media) may be provided.
  • each drive may be connected to the bus 18 via one or more data medium interfaces.
  • the memory may include at least one program product having a set (e.g., at least one) of program modules 42 that are configured to perform the functions of various embodiments of the present invention.
  • a program/utility 40 having a set (at least one) of program modules 42 may be stored in, for example, a memory, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules 42, and program data, each of which or some combination may include an implementation of a network environment.
  • the program modules 42 generally perform the functions and/or methods of the embodiments described herein.
  • the computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, cameras, etc.), may communicate with one or more devices that enable an operator to interact with the computer device 12, and/or may communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input/output (I/O) interface 22.
  • the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs)), wide area networks (WANs), and/or public networks (e.g., the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer device 12 via a bus 18.
  • LANs local area networks
  • WANs wide area networks
  • public networks e.g., the Internet
  • the processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the present invention.
  • the processing unit 16 executes the program, the following is achieved: when concurrent transactions occur within a target time period, a read-write set of transactions occurring within the target time period is obtained; a conflict relationship between the transactions and an execution cost corresponding to the transaction are determined based on the read-write set; a partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; a queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue; when a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.
  • the present invention further provides a computer-readable storage medium on which a computer program is stored.
  • a concurrent transaction processing method based on a multi-core processor as provided in all embodiments of the present application is implemented:
  • the program when executed by the processor, it is implemented as follows: when concurrent transactions occur within a target time period, a read-write set of transactions occurring within the target time period is obtained; a conflict relationship between the transactions and an execution cost corresponding to the transaction are determined based on the read-write set; a partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; a queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue; when a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.
  • a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
  • a computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof.
  • Computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.
  • Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • the computer program code for performing the operation of the present invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages.
  • the program code can be executed entirely on the operator's computer, partially on the operator's computer, as an independent software package, partially on the operator's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer can be connected to the operator's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
  • LAN local area network
  • WAN wide area network
  • Internet service provider for example, using an Internet service provider to connect through the Internet.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Provided in the present application are a multicore-processor-based concurrent transaction processing method and system. The method comprises: when transaction concurrency occurs within a target time period, acquiring a read-write set of transactions that occur within the target time period; according to the read-write set, determining a conflict relationship between the transactions and execution costs corresponding to the transactions; according to the conflict relationship, determining partitions to which the transactions correspond; according to the conflict relationship and the execution costs, determining queue types of the transactions in the partitions, wherein the queue types comprise a conflict-free queue and a conflict queue; and when there is a conflict between a transaction in the conflict queue and a transaction in another partition, delaying the processing of the transaction in the conflict queue. Runtime scheduling can reduce transaction conflicts and is also effective for high-conflict concurrent transactions. Conflicts are reduced by means of delaying the execution, and there is no need to consider a pessimistic lock for all transactions, thereby reducing the overall lock wait delay. For an estimated execution cost deviation, concurrency control can be relied upon to ensure the correctness of a result.

Description

一种基于多核处理器的并发事务处理方法及其系统A concurrent transaction processing method and system based on multi-core processor 技术领域Technical Field
本申请涉及并发事务处理领域,特别是一种基于多核处理器的并发事务处理方法及其系统。The present application relates to the field of concurrent transaction processing, and in particular to a concurrent transaction processing method and system based on a multi-core processor.
背景技术Background technique
之前都只有一个用户访问数据,出现多个用户访问相同数据的情况,这被称为“并发”(concurrency),当一个用户企图修改另一个用户正在检索或修改的数据时,并发会成为一个问题。现代计算机系统都是基于多核架构的中央处理器(Central Processing Unit,简称CPU),而数据库系统为了处理并发事务请求,需要将事务分配到不同CPU的核心上进行计算。Previously, only one user accessed the data. When multiple users accessed the same data, this was called "concurrency". When one user attempts to modify the data that another user is retrieving or modifying, concurrency becomes a problem. Modern computer systems are based on multi-core central processing units (CPUs). In order to handle concurrent transaction requests, database systems need to distribute transactions to different CPU cores for calculation.
不同事务间可能存在访问相同数据的情况,需要通过类似锁机制针对特定数据记录进行并发保护。而类似的锁机制仅能避免并发访问,其可能导致不同事务处理之间的等待,进而使得系统整体处理能力严重受制于事务锁粒度,扩展性较差。Different transactions may access the same data, and a lock mechanism is needed to protect specific data records concurrently. However, such a lock mechanism can only avoid concurrent access, which may cause waiting between different transaction processing, and thus the overall processing capacity of the system is severely restricted by the transaction lock granularity, resulting in poor scalability.
有一类数据库系统为了提升并行处理能力,提出了无锁并发控制,将数据分区与特定CPU核绑定,当事务仅作用于特定数据分区时,不用考虑与其他CPU核的并发访问问题,这类方法被称为乐观并发控制(Optimistic Concurrency Control,简称OCC)。但当OCC方法对于跨分区的事务其并发访问的中断重试机制成本较高,其成本相较基于锁保护的方法更高。In order to improve parallel processing capabilities, a type of database system has proposed lock-free concurrency control, which binds data partitions to specific CPU cores. When a transaction only acts on a specific data partition, there is no need to consider concurrent access with other CPU cores. This type of method is called optimistic concurrency control (OCC). However, the OCC method has a high cost for the interruption and retry mechanism of concurrent access for transactions across partitions, and its cost is higher than that of the lock protection-based method.
发明内容Summary of the invention
鉴于所述问题,提出了本申请以便提供克服所述问题或者至少部分地解决所述问题的一种基于多核处理器的并发事务处理方法及其系统,包括:In view of the above problems, the present application is proposed to provide a concurrent transaction processing method based on a multi-core processor and a system thereof that overcomes the above problems or at least partially solves the above problems, including:
一种基于多核处理器的并发事务处理方法,包括步骤:A concurrent transaction processing method based on a multi-core processor comprises the following steps:
当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;When transaction concurrency occurs within the target time period, the read and write sets of transactions occurring within the target time period are obtained;
依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务 的执行代价;Determining the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;
依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;Determining the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;
依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;Determine the queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;
当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。When a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.
进一步地,所述执行代价为对应的处理器执行所述事务的时间;所述依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型的步骤,包括:Furthermore, the execution cost is the time taken by the corresponding processor to execute the transaction; and the step of determining the queue type of the transaction in the partition according to the conflict relationship and the execution cost includes:
依据所述读写集合确定所述事务的启动时间;Determining a start time of the transaction according to the read-write set;
依据所述启动时间和所述执行代价确定所述事务的执行时间片;Determine an execution time slice of the transaction according to the start time and the execution cost;
依据所述冲突关系确定目标分区中目标事务的执行时间片与其他分区中事务的执行时间片是否有重叠,若是,则所述目标事务在所述冲突队列中,若否,则所述目标事务在所述无冲突队列中。Determine whether the execution time slice of the target transaction in the target partition overlaps with the execution time slice of transactions in other partitions according to the conflict relationship. If so, the target transaction is in the conflict queue; if not, the target transaction is in the non-conflict queue.
进一步地,所述当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理的步骤,包括:Furthermore, when a transaction in the conflict queue conflicts with a transaction in another partition, the step of delaying the processing of the transaction in the conflict queue includes:
依据所述执行时间片确定所述事务在所述冲突队列中的排序;Determining the order of the transaction in the conflict queue according to the execution time slice;
当所述冲突队列中事务的执行时间片与其他分区中事务的执行时间片重叠时,则所述冲突队列中的事务延迟处理。When the execution time slices of the transactions in the conflict queue overlap with the execution time slices of the transactions in other partitions, the transactions in the conflict queue are delayed.
进一步地,所述依据所述执行时间片确定所述事务在所述冲突队列中的排序的步骤,包括:Furthermore, the step of determining the order of the transactions in the conflict queue according to the execution time slice includes:
依据所述冲突关系将所述冲突队列中事务的执行时间片与其他分区中事务的执行时间片错峰排序。The execution time slices of the transactions in the conflict queue are arranged in a staggered manner with the execution time slices of the transactions in other partitions according to the conflict relationship.
进一步地,所述依据所述冲突关系确定所述事务对应的分区的步骤,包括:Furthermore, the step of determining the partition corresponding to the transaction according to the conflict relationship includes:
依据所述读写集合确定所述事务访问的数据类型;Determining the data type accessed by the transaction according to the read-write set;
依据所述数据类型和所述冲突关系确定所述事务的分区。The partition of the transaction is determined according to the data type and the conflict relationship.
进一步地,还包括:Furthermore, it also includes:
当所述冲突队列中事务的实际执行时间与对应的所述执行代价不相等时,则对所述事务进行加锁访问。When the actual execution time of the transaction in the conflict queue is not equal to the corresponding execution cost, the transaction is locked for access.
进一步地,还包括:Furthermore, it also includes:
当有新事务加入处理时,依据所述新事务的读写集合确定所述新事务对应的分区及队列类型;When a new transaction is added to the processing, the partition and queue type corresponding to the new transaction are determined according to the read and write sets of the new transaction;
当所述新事务在对应的队列中与其他分区中的事务存在冲突时,则所述新事务延迟处理。When the new transaction conflicts with transactions in other partitions in the corresponding queue, the processing of the new transaction is delayed.
一种基于多核处理器的并发事务处理系统,包括:A concurrent transaction processing system based on a multi-core processor, comprising:
获取模块,用于当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;An acquisition module, used for acquiring a read-write set of transactions occurring within a target time period when transactions occur concurrently within the target time period;
计算模块,用于依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价;A calculation module, used to determine the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;
分区模块,用于依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;A partitioning module, used for determining the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;
调度模块,用于依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;A scheduling module, used to determine the queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;
处理模块,用于当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。The processing module is used for delaying the processing of transactions in the conflict queue when there is a conflict between the transactions in the conflict queue and the transactions in other partitions.
一种计算机设备,包括处理器、存储器及存储在所述存储器上并能够在所述处理器上运行的计算机程序,所述计算机程序被所述处理器执行时实现如上所述的一种基于多核处理器的并发事务处理方法的步骤。A computer device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of a concurrent transaction processing method based on a multi-core processor as described above are implemented.
一种计算机可读存储介质,所述计算机可读存储介质上存储计算机程序,所述计算机程序被处理器执行时实现如上所述的一种基于多核处理器的并发事务处理方法的步骤。A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a concurrent transaction processing method based on a multi-core processor as described above are implemented.
本申请具有以下优点:This application has the following advantages:
在本申请的实施例中,相对于现有技术中的“事务并发处理存在局限性”的问题,本申请提供了基于事务代价和多核架构下事务处理的调度和处理流程的解决方案,具体为:当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价;依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。通过运行时调度可以降低事务的冲突,对于高冲突并发事务同样有效,通过延迟执行降低冲突,且不需要针对所有事务都考虑悲观锁,降低整体锁等待时延,对于执行代价估计偏差,可以依赖并发控制确保结果的正确性。In the embodiments of the present application, relative to the problem of "limitations in transaction concurrency processing" in the prior art, the present application provides a solution to the scheduling and processing flow of transaction processing based on transaction cost and multi-core architecture, specifically: when transaction concurrency occurs within the target time period, the read-write set of the transaction occurring within the target time period is obtained; the conflict relationship between the transactions and the execution cost corresponding to the transaction are determined based on the read-write set; the partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; the queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes a conflict-free queue and a conflict queue; when the transaction in the conflict queue conflicts with the transaction in other partitions, the transaction in the conflict queue is delayed. Transaction conflicts can be reduced through runtime scheduling, which is also effective for high-conflict concurrent transactions. Conflicts can be reduced by delayed execution, and pessimistic locks do not need to be considered for all transactions, reducing the overall lock waiting delay. For execution cost estimation deviations, concurrency control can be relied on to ensure the correctness of the results.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了更清楚地说明本申请的技术方案,下面将对本申请的描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
图1是本申请一实施例提供的一种基于多核处理器的并发事务处理方法的步骤流程图;FIG1 is a flowchart of a method for concurrent transaction processing based on a multi-core processor according to an embodiment of the present application;
图2是申请一实施例提供的一种基于多核处理器的并发事务处理方法的队列示意图;FIG2 is a queue diagram of a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the application;
图3是申请一实施例提供的一种基于多核处理器的并发事务处理方法的方案框图;FIG3 is a block diagram of a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the application;
图4是本申请一实施例提供的现有技术一的读写锁兼容性示意图。FIG. 4 is a schematic diagram of the compatibility of a read-write lock according to a prior art 1 provided by an embodiment of the present application.
图5是本申请一实施例提供的现有技术一的提交读级别下的读写锁的使用示意图。FIG. 5 is a schematic diagram of the use of a read-write lock at a commit read level according to a first prior art provided by an embodiment of the present application.
图6是本申请一实施例提供的一种基于多核处理器的并发事务处理系统的结构框图;FIG6 is a structural block diagram of a concurrent transaction processing system based on a multi-core processor provided in an embodiment of the present application;
图7是本申请一实施例提供的一种基于多核处理器的并发事务处理系统的结构框图;7 is a structural block diagram of a concurrent transaction processing system based on a multi-core processor provided in an embodiment of the present application;
图8是本发明一实施例提供的一种计算机设备的结构示意图。FIG. 8 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention.
具体实施方式Detailed ways
为使本申请的所述目的、特征和优点能够更加明显易懂,下面结合附图和具体实施方式对本申请作进一步详细的说明。显然,所描述的实施例是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。In order to make the objects, features and advantages of the present application more obvious and understandable, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
发明人通过分析现有技术发现:现有技术一和现有技术二,下面分别阐述:The inventors analyzed the prior art and found that: Prior Art 1 and Prior Art 2 are described below:
现有技术一:MySQL锁技术以及MVCC基础,读写锁兼容性如图4所示。Existing technology 1: MySQL lock technology and MVCC foundation, the read-write lock compatibility is shown in Figure 4.
1、MySQL锁技术1. MySQL lock technology
当有多个请求来读取表中的数据时可以不采取任何操作,但是多个请求里有读请求,又有修改请求时必须有一种措施来进行并发控制。不然很有可能会造成不一致。When there are multiple requests to read data in the table, no action can be taken, but when there are read requests and modification requests among the multiple requests, a measure must be taken to perform concurrency control. Otherwise, inconsistency is likely to occur.
解决上述问题很简单,只需用两种锁的组合来对读写请求进行控制即可,这两种锁被称为:Solving the above problem is simple. You only need to use a combination of two locks to control read and write requests. These two locks are called:
共享锁(shared lock),又叫做“读锁”,读锁是可以共享的,或者说多个读请求可以共享一把锁读数据,不会造成阻塞。A shared lock, also called a "read lock", can be shared, or multiple read requests can share a lock to read data without causing blocking.
排他锁(exclusive lock),又叫做“写锁”,写锁会排斥其他所有获取锁的请求,一直阻塞,直到写入完成释放锁。An exclusive lock, also called a "write lock," excludes all other requests to acquire the lock and blocks the lock until the write is completed and the lock is released.
通过读写锁,可以做到读读并行,但不能做到写读,写写并行。事务的隔离性就是根据读写锁来实现的。Read-write locks can be used to perform read-read operations in parallel, but write-read operations or write-write operations in parallel are not possible. Transaction isolation is achieved based on read-write locks.
2、MVCC实现2. MVCC Implementation
MVCC(MultiVersion Concurrency Control,多版本并发控制)。MySQL存储引擎InnoDB的MVCC,是通过在每行记录的后面保存两个隐藏的列来 实现的。这两个列,一个保存了行的创建时间,一个保存了行的过期时间,当然存储的并不是实际的时间值,而是系统版本号。MVCC (MultiVersion Concurrency Control). The MVCC of the MySQL storage engine InnoDB is implemented by storing two hidden columns at the end of each row. One of these two columns stores the creation time of the row, and the other stores the expiration time of the row. Of course, what is stored is not the actual time value, but the system version number.
根据《高性能Mysql》这本书对MVCC的定义。他的主要实现思想是通过数据多版本来做到读写分离。从而实现不加锁读进而做到读写并行。According to the definition of MVCC in the book "High Performance MySQL", its main implementation idea is to achieve read-write separation through multiple versions of data, so as to achieve lock-free reading and read-write parallelism.
提交读级别下的读写锁的使用如图5所示,MVCC在MySQL中的实现依赖的是undo log与read view。其中,undo log:undo log中记录某行数据的多个版本的数据;readview:用来判断当前版本数据的可见性。The use of read-write locks at the committed read level is shown in Figure 5. The implementation of MVCC in MySQL relies on undo log and read view. Undo log: undo log records multiple versions of a row of data; readview: used to determine the visibility of the current version of data.
现有技术二:Silo中的OCC事务处理,OCC事务跟踪它在线程本地存储中读写的记录。在提交时,在确认没有并发事务的写集与读集重叠之后,事务立即执行所有写记录。如果验证失败,事务将中止。Prior Art 2: OCC Transaction Processing in Silo,OCC transaction tracks the records it reads and writes in thread local storage. At commit time, after confirming that no concurrent transaction's write set overlaps with the read set, the transaction immediately executes all write records. If the verification fails, the transaction will be aborted.
Silo基于称为epoch(纪元号)的时间段,用于确保可序列化的恢复,以删除垃圾并提供只读快照。每个epoch都有一个epoch号。全局纪元号E对所有线程都可见。指定线程定期更新E;其他线程在提交事务时访问E。E应该经常更新,因为epoch周期会影响事务延迟,但与事务持续时间相比,epoch的变化应该很少,以便通常缓存E的值。目前的实现是40毫秒更新一次;更短的时间也可以。处理E不需要锁。Silos are based on time periods called epochs, which are used to ensure serializable recovery to remove garbage and provide read-only snapshots. Each epoch has an epoch number. The global epoch number E is visible to all threads. Designated threads update E periodically; other threads access E when committing transactions. E should be updated frequently because the epoch period affects transaction latency, but the epoch should change infrequently compared to the transaction duration so that the value of E is usually cached. The current implementation updates every 40 milliseconds; shorter times are also possible. No locks are required to process E.
TID(线程控制符)是一个64bit的integer,被分成几个部分来表示不同的含义。最高的几个位次代表当前事务提交时的epoch E。中间位次表示基于当前epoch算出来的timestamp。最低的三个bit分别代表lock bit、lastest-versionbit和absentbit。Tid是直接写到tuple(数组)里的,每条记录都包含最近修改它的事务的TID。并满足条件:TID (thread identifier) is a 64-bit integer, which is divided into several parts to represent different meanings. The highest few bits represent epoch E when the current transaction is committed. The middle bits represent the timestamp calculated based on the current epoch. The lowest three bits represent the lock bit, the lastest-version bit, and the absent bit. Tid is written directly into the tuple (array), and each record contains the TID of the transaction that most recently modified it. And meet the conditions:
1.当前epoch分配的时间戳一定小于下一个epoch分配的;1. The timestamp assigned by the current epoch must be smaller than that assigned by the next epoch;
2.同一个工作线程中,分配的时间戳严格单调递增;2. In the same worker thread, the assigned timestamps are strictly monotonically increasing;
3.修改同一条记录的事务,时间戳严格单调递增。3. For transactions that modify the same record, the timestamps are strictly monotonically increasing.
Silo的记录包含以下信息:TID word(TID加status bits),前版本指针,数据。Silo records contain the following information: TID word (TID plus status bits), previous version pointer, and data.
Commitprotocol(提交协议):Commitprotocol (commit protocol):
当worker(执行器)运行一个事务时,它维护一个读集,该读集标识所 有被读取的记录,以及每条记录在被访问时的TID。对于已修改的记录,它维护一个写集,用于存储记录的新状态(而不是以前的TID)。被读取和修改的记录同时出现在读集和写集中。在正常操作中,写集中的所有记录也出现在读集中。在事务完成时,worker尝试使用以下协议提交:When a worker (executor) runs a transaction, it maintains a read set that identifies all records that were read, and the TID of each record when it was accessed. For modified records, it maintains a write set that stores the new state of the record (instead of the previous TID). Records that are read and modified appear in both the read set and the write set. In normal operation, all records in the write set also appear in the read set. At transaction completion, the worker attempts to commit using the following protocol:
阶段1:检查事务的写集中的所有记录,并通过获取记录的锁位来锁定每条记录。为了避免死锁,按全局顺序锁定记录。在获得所有的写锁之后,worker使用一次内存访问获取全局纪元号的快照。为了确保读到主存(而不是过期的)。全局纪元号的快照是提交事务的序列化点。Phase 1: Check all records in the transaction's write set and lock each record by acquiring the record's lock bit. To avoid deadlock, lock records in global order. After acquiring all write locks, the worker takes a snapshot of the global epoch number using a single memory access. To ensure that the read is in main memory (and not out of date). The snapshot of the global epoch number is the serialization point for committing transactions.
阶段2:检查事务的读集中的所有记录(其中可能包含一些已读和已写的记录)。如果某个记录的TID与其在执行过程中观察到的不同,不再是最新版本,或者被不同的事务锁定,那么事务将释放其锁并中止。如果所有读取记录的tid不变,那么事务就允许提交,因为知道它的所有读取都是一致的。worker使用在阶段1中获得的全局纪元号的快照为事务分配一个TID。Phase 2: Check all records in the transaction's read set (which may include some records that have already been read and written). If a record's TID is different from what it observed during execution, is no longer the latest version, or is locked by a different transaction, the transaction releases its lock and aborts. If the TIDs of all read records are unchanged, the transaction is allowed to commit, knowing that all its reads are consistent. The worker assigns a TID to the transaction using the snapshot of the global epoch number obtained in phase 1.
阶段3:worker将其修改过的记录写入数据块中,并将它们的TIDs更新为上一阶段计算的事务ID。每个锁都可以在其记录被写入之后立即被释放。必须确保锁一释放,新的TID就可见。Phase 3: The worker writes its modified records into the data blocks and updates their TIDs to the transaction IDs calculated in the previous phase. Each lock can be released immediately after its record is written. It must be ensured that the new TID is visible as soon as the lock is released.
综上,面向多核的并发事务处理机制是现代OLTP(联机事务处理)数据库管理系统的关键挑战之一。数据库系统是否能随硬件核数的增加而满足更多的并行事务请求,是衡量数据库处理能力的关键指标。商业数据库Oracle和开源数据库MySQL、PostgreSQL在都是依靠传统基于锁等待的事务处理机制,其实现相对来说简单可靠,但是扩展性问题并未很好解决。In summary, the concurrent transaction processing mechanism for multi-cores is one of the key challenges of modern OLTP (online transaction processing) database management systems. Whether the database system can meet more parallel transaction requests as the number of hardware cores increases is a key indicator for measuring the database processing capability. Commercial database Oracle and open source databases MySQL and PostgreSQL all rely on traditional transaction processing mechanisms based on lock waiting. Their implementation is relatively simple and reliable, but the scalability problem is not well solved.
在研究领域也有不少基于OCC的新型数据库系统不断尝试解决这类问题。其中比较有代表性的如麻省理工学院提出的Silo和卡内基梅隆大学提出的TICTOC等。他们验证了OCC在低冲突场景下的扩展性,但并不适用于高冲突场景。基于OCC方法跨数据分区事务导致并发访问时,需要将当前事务中断后再重试,可以将事务按分区粒度并行,但对于跨分区的事务其并发访问的中断重试机制成本较高,当事务冲突较大时可能导致大量事务反复重试。In the research field, there are also many new database systems based on OCC that continue to try to solve such problems. Among them, the more representative ones are Silo proposed by MIT and TICTOC proposed by Carnegie Mellon University. They verified the scalability of OCC in low-conflict scenarios, but it is not suitable for high-conflict scenarios. When cross-data partition transactions based on the OCC method cause concurrent access, the current transaction needs to be interrupted and retried. Transactions can be parallelized at the partition granularity, but the interruption and retry mechanism for concurrent access to cross-partition transactions is more expensive. When the transaction conflict is large, it may cause a large number of transactions to be retried repeatedly.
本发明重点解决在多核架构下数据库管理系统中如何高效处理并发事务,其关键是对于特定时间内的一批事务,如何能够最大程度实现并行且避免事物间的并发访问。本发明提供了支持基于事务代价的调度机制来解决上述问题,即对于特定事务集合根据其访问的数据集分配到不同的CPU核进行冲突延迟处理。对于特定CPU核来说,特定事务因与其他CPU核事务间存在冲突,并行执行的前提是时间上错峰。因此基于提前识别事务之间的冲突和代价,通过时间片延迟调度实现不同CPU核之间的并行事务处理。The present invention focuses on solving how to efficiently handle concurrent transactions in a database management system under a multi-core architecture. The key is how to maximize parallelism and avoid concurrent access between transactions for a batch of transactions within a specific time. The present invention provides a scheduling mechanism that supports transaction cost-based scheduling to solve the above problem, that is, for a specific transaction set, it is assigned to different CPU cores according to the data sets it accesses for conflict delay processing. For a specific CPU core, a specific transaction conflicts with other CPU core transactions, and the premise of parallel execution is time staggered. Therefore, based on the advance identification of conflicts and costs between transactions, parallel transaction processing between different CPU cores is achieved through time slice delay scheduling.
参照图1和图2,示出了本申请一实施例提供的一种基于多核处理器的并发事务处理方法;1 and 2 , a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the present application is shown;
所述方法包括:The method comprises:
S110、当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;S110, when concurrent transactions occur within the target time period, obtaining a read-write set of transactions occurring within the target time period;
S120、依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价;S120, determining the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;
S130、依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;S130, determining the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;
S140、依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;S140, determining a queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;
S150、当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。S150: When a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is processed in a delayed manner.
在本申请的实施例中,相对于现有技术中的“事务并发处理存在局限性”的问题,本申请提供了基于事务代价和多核架构下事务处理的调度和处理流程的解决方案,具体为:当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价;依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;当所述冲突队列中的事务与其他分区中 的事务存在冲突时,则所述冲突队列中的事务延迟处理。通过运行时调度可以降低事务的冲突,对于高冲突并发事务同样有效,通过延迟执行降低冲突,且不需要针对所有事务都考虑悲观锁,降低整体锁等待时延,对于执行代价估计偏差,可以依赖并发控制确保结果的正确性。In the embodiments of the present application, relative to the problem of "limitations in transaction concurrency processing" in the prior art, the present application provides a solution for scheduling and processing flow of transaction processing based on transaction cost and multi-core architecture, specifically: when transaction concurrency occurs within the target time period, the read-write set of transactions occurring within the target time period is obtained; the conflict relationship between the transactions and the execution cost corresponding to the transactions are determined based on the read-write set; the partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; the queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes conflict-free queue and conflict queue; when the transaction in the conflict queue conflicts with the transaction in other partitions, the transaction in the conflict queue is delayed. Transaction conflicts can be reduced through runtime scheduling, which is also effective for high-conflict concurrent transactions. Conflicts can be reduced by delayed execution, and pessimistic locks do not need to be considered for all transactions, reducing the overall lock waiting delay. For execution cost estimation deviations, concurrency control can be relied on to ensure the correctness of the results.
下面,将对本示例性实施例中一种基于多核处理器的并发事务处理方法作进一步地说明。Next, a concurrent transaction processing method based on a multi-core processor in this exemplary embodiment will be further described.
如所述步骤S110所述,当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合。As described in step S110, when transaction concurrency occurs within the target time period, the read and write sets of transactions occurring within the target time period are acquired.
作为一种示例,在数据同步中,在源端数据库及目的端数据库部署同步系统,源端数据同步系统从源端数据库读取日志,而目的端数据同步系统则是负责把源端发过来的同步操作应用到目的端数据库。一组操作的集合,它是一个不可分割的工作单位,事务会把所有的操作作为一个整体一起向系统提交或撤销操作请求,即这些操作要么同时成功,要么同时失败。As an example, in data synchronization, the synchronization system is deployed on the source database and the target database. The source data synchronization system reads logs from the source database, while the target data synchronization system is responsible for applying the synchronization operations sent from the source to the target database. A set of operations is an indivisible unit of work. A transaction submits or cancels all operations as a whole to the system, that is, these operations either succeed or fail at the same time.
源端数据库通过执行事务产生数据变化,每个事务中包含一个或多个数据库操作,每个操作会产生一个数据变化。操作包括读取数据、写数据、更新修改数据、删除数据等,在具体实施场景中,一个操作可以对应一个SQL语句。事务就是一系列对数据库操作的语句集合,对于特定时间段内的一批事务,数据库系统中会生成当前事务读数据的集合和写数据的集合。The source database generates data changes by executing transactions. Each transaction contains one or more database operations, and each operation generates a data change. Operations include reading data, writing data, updating and modifying data, and deleting data. In specific implementation scenarios, one operation can correspond to an SQL statement. A transaction is a set of statements for database operations. For a batch of transactions within a specific time period, the database system will generate a set of read data and a set of write data for the current transaction.
如所述步骤S120所述,依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价。As described in step S120, the conflict relationship between the transactions and the execution cost corresponding to the transaction are determined according to the read-write set.
作为一种示例,事务代价是指执行该事务对应CPU需要的时间,这个时间通常与修改的数据量有相关性。数据库系统中会生成当前事务的读写数据集合,通过这个集合可以估算其执行代价。一个事务中包含一个执行代价,执行代价是一个固定值,仅与修改数据记录相关。As an example, the transaction cost refers to the time required by the CPU to execute the transaction, which is usually related to the amount of data modified. The database system generates a set of read and write data for the current transaction, through which its execution cost can be estimated. A transaction contains an execution cost, which is a fixed value and is only related to the modified data record.
在确定每个事务的读写集合后,对于同一批中事务建立事物之间冲突关系图,对于任一事务可以快速查找和它冲突的事务列表,利用关系图判断相关事务是否冲突,用于后续步骤分区和调度,如果两个事务冲突了则要确保它们的运行时间不重叠。After determining the read and write sets of each transaction, a conflict relationship graph is established between transactions in the same batch. For any transaction, a list of transactions that conflict with it can be quickly found. The relationship graph is used to determine whether related transactions conflict, which is used for subsequent partitioning and scheduling. If two transactions conflict, ensure that their running times do not overlap.
在一具体实现中,参照图2,假设一个批量任务包含有T1,T2,…,Tn个事务,先从各事务涉及的读写数据集来形成事务之间的冲突关系图Conflict Graph,其中对于每个事务包含其冲突事务的执行代价。如图2,右边中Txn*为Txn1,可以得出Txn1与Txn2、Txn4以及Txn6冲突。In a specific implementation, referring to FIG2, assuming that a batch task includes T1, T2, ..., Tn transactions, the conflict graph between transactions is first formed from the read and write data sets involved in each transaction, where each transaction includes the execution cost of its conflicting transaction. As shown in FIG2, Txn* on the right is Txn1, and it can be concluded that Txn1 conflicts with Txn2, Txn4, and Txn6.
如所述步骤S130所述,依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同。As described in step S130, the partition corresponding to the transaction is determined according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same.
在本发明一实施例中,可以结合下列描述进一步说明步骤S130所述“依据所述冲突关系确定所述事务对应的分区”的具体过程。In an embodiment of the present invention, the specific process of "determining the partition corresponding to the transaction according to the conflict relationship" in step S130 can be further explained in combination with the following description.
如下列步骤所述,依据所述读写集合确定所述事务访问的数据类型;As described in the following steps, determining the data type accessed by the transaction according to the read-write set;
如下列步骤所述,依据所述数据类型和所述冲突关系确定所述事务的分区。As described in the following steps, the partition of the transaction is determined according to the data type and the conflict relationship.
作为一种示例,对于批量事务流程,在确定每个事务的读写集合后,根据其访问的数据集合进行划分,确保访问相同数据的事务尽量属于同一分区,避免跨分区事务之间存在冲突。为了提高目的端事务执行效率,进行数据同步时目的端对事务采用并行执行的方式,因此,需要创建多个能够并行执行的执行线程,将需要执行对应分区的事务分配至不同的执行线程进行执行。As an example, for batch transaction processes, after determining the read and write sets of each transaction, divide it according to the data set it accesses to ensure that transactions accessing the same data belong to the same partition as much as possible to avoid conflicts between cross-partition transactions. In order to improve the efficiency of transaction execution on the destination side, the destination side uses parallel execution for transactions when synchronizing data. Therefore, it is necessary to create multiple execution threads that can be executed in parallel and assign transactions that need to execute the corresponding partitions to different execution threads for execution.
在一具体实现中,通过分区算法(如最小割算法)对冲突关系图进行分区,分区算法的目的就是让跨分区的冲突事务尽可能地少,理想状态下各分区中的事务与其他分区事务之间不会访问相同的数据。In a specific implementation, the conflict relationship graph is partitioned by a partitioning algorithm (such as a minimum cut algorithm). The purpose of the partitioning algorithm is to reduce the number of conflicting transactions across partitions as much as possible. Ideally, transactions in each partition will not access the same data as transactions in other partitions.
如所述步骤S140所述,依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列。As described in step S140, the queue type of the transaction in the partition is determined according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue.
在本发明一实施例中,可以结合下列描述进一步说明步骤S140所述“依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型”的具体过程。In an embodiment of the present invention, the specific process of "determining the queue type of the transaction in the partition according to the conflict relationship and the execution cost" in step S140 can be further explained in combination with the following description.
如下列步骤所述,依据所述读写集合确定所述事务的启动时间;As described in the following steps, determining the start time of the transaction according to the read-write set;
如下列步骤所述,依据所述启动时间和所述执行代价确定所述事务的执行时间片;As described in the following steps, determining the execution time slice of the transaction according to the start time and the execution cost;
如下列步骤所述,依据所述冲突关系确定目标分区中目标事务的执行时 间片与其他分区中事务的执行时间片是否有重叠,若是,则所述目标事务在所述冲突队列中,若否,则所述目标事务在所述无冲突队列中。As described in the following steps, it is determined whether the execution time slice of the target transaction in the target partition overlaps with the execution time slices of transactions in other partitions based on the conflict relationship. If so, the target transaction is in the conflict queue; if not, the target transaction is in the non-conflict queue.
作为一种示例,从所述事务的读写集合中获取每个事务的执行启动时间,再依据对应执行线程执行该事务所需的时间,即可得到每个事务的执行时间片,通过执行时间片和冲突关系图可以对分区中的事务放入不同的队列中。负责执行的线程,根据事务的执行代价和相关事务冲突信息,决定将事务放入不同队列,各分区中不与其他分区事务冲突的事务加入对应的无冲突队列P中,其中无冲突队列P中事务与其他分区的事务是完全独立的,而冲突队列Q中的事务则需要进行调度排序。由于各分区的事务都是独立的,因此执行顺序与其他分区无关,多个事务执行线程可以并行执行。As an example, the execution start time of each transaction is obtained from the read-write set of the transaction, and then the execution time slice of each transaction is obtained according to the time required for the corresponding execution thread to execute the transaction. The transactions in the partition can be placed in different queues through the execution time slice and the conflict relationship diagram. The thread responsible for execution decides to put the transaction into different queues according to the execution cost of the transaction and the relevant transaction conflict information. The transactions in each partition that do not conflict with the transactions of other partitions are added to the corresponding conflict-free queue P, where the transactions in the conflict-free queue P are completely independent of the transactions of other partitions, while the transactions in the conflict queue Q need to be scheduled and sorted. Since the transactions of each partition are independent, the execution order is independent of other partitions, and multiple transaction execution threads can be executed in parallel.
在一具体实现中,队列的个数与分区数保持一致,假设有50个分区,每个分区中都有无冲突队列P和冲突队列Q两个类型的队列,则50个分区中共有100个队列,其中无冲突队列P有50个,冲突队列Q也有50个,数量与分区数相等。In a specific implementation, the number of queues is consistent with the number of partitions. Assume that there are 50 partitions, and each partition has two types of queues: conflict-free queue P and conflict queue Q. Then, there are a total of 100 queues in the 50 partitions, including 50 conflict-free queues P and 50 conflict queues Q, which are equal to the number of partitions.
如所述步骤S150所述,当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。As described in step S150, when a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.
在本发明一实施例中,可以结合下列描述进一步说明步骤S150所述“当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理”的具体过程。In an embodiment of the present invention, the specific process of "when a transaction in the conflict queue conflicts with a transaction in another partition, delaying the processing of the transaction in the conflict queue" in step S150 can be further explained in combination with the following description.
如下列步骤所述,依据所述执行时间片确定所述事务在所述冲突队列中的排序;As described in the following steps, determining the order of the transaction in the conflict queue according to the execution time slice;
如下列步骤所述,当所述冲突队列中事务的执行时间片与其他分区中事务的执行时间片重叠时,则所述冲突队列中的事务延迟处理。As described in the following steps, when the execution time slices of transactions in the conflict queue overlap with the execution time slices of transactions in other partitions, the transactions in the conflict queue are delayed.
在本发明一实施例中,可以结合下列描述进一步说明“依据所述执行时间片确定所述事务在所述冲突队列中的排序”的具体过程。In an embodiment of the present invention, the specific process of “determining the order of the transaction in the conflict queue according to the execution time slice” may be further explained in combination with the following description.
如下列步骤所述,依据所述冲突关系将所述冲突队列中事务的执行时间片与其他分区中事务的执行时间片错峰排序。As described in the following steps, the execution time slices of the transactions in the conflict queue and the execution time slices of the transactions in other partitions are arranged in a staggered manner according to the conflict relationship.
作为一种示例,在所述冲突队列Q中,需要对其中的事务进行调度排序。 将冲突队列Q中的事务的时间片与其他队列中事务的时间片错开不重叠,如果仍存在特定事务已排在冲突队列Q的队尾且与其他线程的执行时间片有重叠的,则该特定事务选择延迟处理,实现无冲突并发执行。As an example, in the conflict queue Q, the transactions therein need to be scheduled and sorted. The time slices of the transactions in the conflict queue Q are staggered and non-overlapping with the time slices of the transactions in other queues. If there is still a specific transaction that is already at the end of the conflict queue Q and overlaps with the execution time slices of other threads, the specific transaction is delayed to achieve conflict-free concurrent execution.
在一具体实现中,参照图2,图2中黑色阴影部分代表队列中已被其他事务占用的时间片。当事务分配到特定分区的调度队列Q中时,需要通过冲突关系图确认其他事务的执行时间片。原则上冲突事务不能有时间片重叠,才能并行执行。如T2加入Q3的启动时间晚于T1结束时间,或者T2加入Qk的结束时间早于T1的开始时间。当特定队列Q已经无法找到与冲突事务错开的时间片队列时,通过延迟启动实现无冲突并发执行。In a specific implementation, referring to FIG. 2 , the black shaded portion in FIG. 2 represents the time slices in the queue that have been occupied by other transactions. When a transaction is assigned to a scheduling queue Q of a specific partition, it is necessary to confirm the execution time slices of other transactions through the conflict relationship diagram. In principle, conflicting transactions cannot have overlapping time slices in order to be executed in parallel. For example, the start time of T2 joining Q3 is later than the end time of T1, or the end time of T2 joining Qk is earlier than the start time of T1. When a specific queue Q can no longer find a time slice queue that is staggered with the conflicting transaction, conflict-free concurrent execution is achieved by delaying the start.
在本实施例中,所述并发事务处理方法还包括:In this embodiment, the concurrent transaction processing method further includes:
当所述冲突队列中事务的实际执行时间与对应的所述执行代价不相等时,则对所述事务进行加锁访问。When the actual execution time of the transaction in the conflict queue is not equal to the corresponding execution cost, the transaction is locked for access.
作为一种示例,对于已完成调度的事务,最终执行时需要考虑并发访问控制,通过对事务进行加锁访问,避免运行时代价偏差导致数据被多事务同时修改。加锁可以根据每个操作的唯一标识符,对每个操作构造行锁,唯一标识符的值作为行锁的值。As an example, for the scheduled transactions, concurrent access control needs to be considered during the final execution. By locking the transaction access, it is possible to avoid runtime cost deviations that cause data to be modified by multiple transactions at the same time. Locking can construct a row lock for each operation based on the unique identifier of each operation, and the value of the unique identifier is used as the value of the row lock.
在本实施例中,所述并发事务处理方法还包括:In this embodiment, the concurrent transaction processing method further includes:
当有新事务加入处理时,依据所述新事务的读写集合确定所述新事务对应的分区及队列类型;When a new transaction is added to the processing, the partition and queue type corresponding to the new transaction are determined according to the read and write sets of the new transaction;
当所述新事务在对应的队列中与其他分区中的事务存在冲突时,则所述新事务延迟处理。When the new transaction conflicts with transactions in other partitions in the corresponding queue, the processing of the new transaction is delayed.
作为一种示例,对于每个即时事务,获取其读写集合,并根据其读写集合确定出即时事务与当前事务之间的冲突关系和执行代价,根据其访问的数据集合进行一定随机化处理,确定出即时事务对应的分区和队列,并将事务分配给对应的执行线程。将即时事务与当前已分区的事务一并调度。后续步骤与批量事务处理类似,若即时事务与其他分区队列中的事务的执行时间片重叠时,则该即时事务延迟处理。As an example, for each instant transaction, obtain its read-write set, and determine the conflict relationship and execution cost between the instant transaction and the current transaction based on its read-write set, perform certain random processing based on the data set it accesses, determine the partition and queue corresponding to the instant transaction, and assign the transaction to the corresponding execution thread. Schedule the instant transaction together with the current partitioned transaction. The subsequent steps are similar to batch transaction processing. If the execution time slice of the instant transaction overlaps with that of transactions in other partition queues, the instant transaction will be delayed.
参照图3,为本发明的一种基于多核处理器的并发事务处理方法的方案 框图。图3中,bundled workload为批量事务请求,是例行业务;unbundled transactions为即时事务请求;Transaction-to-thread assignment为将事务分发给特定的线程(即CPU核);对于较重的批量请求,则考虑将事务按相关性进行分区,而对于较轻量的即时请求,可以考虑通过随机算法进行分配。将事务分区后,通过Thread-local buffers,用于保存分配给特定线程的事务。TSkd为主要用于各线程内的事务执行调度,包括调度器和延迟器两部分。它们的作用是将各线程之间存在冲突的事务执行时间片错开。具体执行单元为Execution(CC),可以支持各类并发事务处理机制,如锁等待和OCC等。Referring to Figure 3, there is a block diagram of a scheme of a concurrent transaction processing method based on a multi-core processor of the present invention. In Figure 3, bundled workload is a batch transaction request, which is a routine business; unbundled transactions are instant transaction requests; Transaction-to-thread assignment is to distribute transactions to specific threads (i.e., CPU cores); for heavier batch requests, consider partitioning transactions according to relevance, and for lighter instant requests, consider assigning them through a random algorithm. After the transactions are partitioned, Thread-local buffers are used to save transactions assigned to specific threads. TSkd is mainly used for transaction execution scheduling within each thread, including a scheduler and a delayer. Their function is to stagger the execution time slices of transactions that conflict between threads. The specific execution unit is Execution (CC), which can support various concurrent transaction processing mechanisms, such as lock waiting and OCC.
实施例1Example 1
批量事务流程:Batch transaction process:
1、在确定每个事务的读写集合后,对于同一批中事务建立事物之间冲突关系图,对于任一Txn可以快速查找和它冲突的事务列表,用于后续步骤分区和调度。1. After determining the read and write sets of each transaction, a conflict relationship diagram is established for transactions in the same batch. For any Txn, a list of transactions that conflict with it can be quickly found for subsequent partitioning and scheduling.
2、在确定每个事务的读写集合后,根据其访问的数据集合进行划分,确保访问相同数据的事务尽量属于同一分区,避免跨分区事务之间存在冲突。将对应分区的事务分配给对应的执行线程。2. After determining the read and write sets of each transaction, divide them according to the data sets they access, ensuring that transactions accessing the same data belong to the same partition as much as possible to avoid conflicts between cross-partition transactions. Assign transactions of the corresponding partitions to the corresponding execution threads.
3、负责执行的线程,根据事务的执行代价和相关事务冲突信息,决定将事务放入不同队列,其中无冲突队列P中事务与其他分区的事务是完全独立的,而冲突队列Q中的事务则需要进行调度排序。3. The thread responsible for execution decides to put the transaction into different queues based on the execution cost of the transaction and the related transaction conflict information. The transactions in the conflict-free queue P are completely independent of the transactions in other partitions, while the transactions in the conflict queue Q need to be scheduled and sorted.
4、如果存在特定事务已排在冲突队列的队尾且与其他线程仍有冲突则选择延迟处理。4. If a specific transaction is already at the end of the conflict queue and still conflicts with other threads, choose to delay processing.
5、对于已完成调度的事务,最终执行时让需要考虑并发访问控制,避免运行时代价偏差导致数据被多事务同时修改。5. For transactions that have completed scheduling, concurrent access control needs to be considered during the final execution to avoid runtime cost deviations that cause data to be modified by multiple transactions at the same time.
即时事务流程:Real-time transaction process:
对于每个即时事务,根据其访问的数据集合进行一定随机化处理,将事务分配给对应的执行线程。与当前已分区的事务一并调度。后续步骤与批量事务处理类似,不再赘述。For each instant transaction, a certain randomization is performed based on the data set it accesses, and the transaction is assigned to the corresponding execution thread. It is scheduled together with the currently partitioned transactions. The subsequent steps are similar to batch transaction processing and will not be repeated here.
对于系统实施例而言,由于其与方法实施例基本相似,所以描述的比较 简单,相关之处参见方法实施例的部分说明即可。As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
参照图6,示出了本申请一实施例提供的一种基于多核处理器的并发事务处理系统;6 , a concurrent transaction processing system based on a multi-core processor provided in an embodiment of the present application is shown;
具体包括:Specifically include:
获取模块610,用于当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;The acquisition module 610 is used to acquire the read and write sets of the transactions occurring in the target time period when transactions are concurrently occurring in the target time period;
计算模块620,用于依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价;A calculation module 620, configured to determine the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;
分区模块630,用于依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;A partition module 630 is used to determine the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;
调度模块640,用于依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;A scheduling module 640, configured to determine a queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;
处理模块650,用于当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。The processing module 650 is used to delay the processing of the transaction in the conflict queue when there is a conflict between the transaction in the conflict queue and the transaction in other partitions.
在本发明一实施例中,所述执行代价为对应的处理器执行所述事务的时间;所述调度模块640,包括:In one embodiment of the present invention, the execution cost is the time taken by the corresponding processor to execute the transaction; the scheduling module 640 includes:
启动时间确定子模块,用于依据所述读写集合确定所述事务的启动时间;A start time determination submodule, used to determine the start time of the transaction according to the read-write set;
时间片确定子模块,用于依据所述启动时间和所述执行代价确定所述事务的执行时间片;A time slice determination submodule, used to determine the execution time slice of the transaction according to the start time and the execution cost;
第一判断子模块,用于依据所述冲突关系确定目标分区中目标事务的执行时间片与其他分区中事务的执行时间片是否有重叠,若是,则所述目标事务在所述冲突队列中,若否,则所述目标事务在所述无冲突队列中。The first judgment submodule is used to determine whether the execution time slice of the target transaction in the target partition overlaps with the execution time slices of transactions in other partitions according to the conflict relationship. If so, the target transaction is in the conflict queue; if not, the target transaction is in the non-conflict queue.
在本发明一实施例中,所述处理模块650,包括:In one embodiment of the present invention, the processing module 650 includes:
排序子模块,用于依据所述执行时间片确定所述事务在所述冲突队列中的排序;A sorting submodule, used to determine the sorting of the transaction in the conflict queue according to the execution time slice;
延迟处理子模块,用于当所述冲突队列中事务的执行时间片与其他分区中事务的执行时间片重叠时,则所述冲突队列中的事务延迟处理。The delay processing submodule is used to delay the processing of transactions in the conflict queue when the execution time slice of the transaction in the conflict queue overlaps with the execution time slice of the transaction in other partitions.
在本发明一实施例中,所述排序子模块,包括:In one embodiment of the present invention, the sorting submodule includes:
错峰排序单元,用于依据所述冲突关系将所述冲突队列中事务的执行时间片与其他分区中事务的执行时间片错峰排序。The peak-shifting sorting unit is used to sort the execution time slices of the transactions in the conflict queue and the execution time slices of the transactions in other partitions in a peak-shifting manner according to the conflict relationship.
在本发明一实施例中,所述分区模块630,包括:In one embodiment of the present invention, the partition module 630 includes:
访问类型确定子模块,用于依据所述读写集合确定所述事务访问的数据类型;An access type determination submodule, used to determine the data type accessed by the transaction according to the read-write set;
分区确定子模块,用于依据所述数据类型和所述冲突关系确定所述事务的分区。The partition determination submodule is used to determine the partition of the transaction according to the data type and the conflict relationship.
在本发明一实施例中,还包括:In one embodiment of the present invention, it also includes:
保护模块660,用于当所述冲突队列中事务的实际执行时间与对应的所述执行代价不相等时,则对所述事务进行加锁访问。The protection module 660 is used to lock access to the transaction when the actual execution time of the transaction in the conflict queue is not equal to the corresponding execution cost.
参照图7,在本发明一实施例中,所述并发事务处理系统,还包括:7, in one embodiment of the present invention, the concurrent transaction processing system further includes:
即时事务调度模块710,用于当有新事务加入处理时,依据所述新事务的读写集合确定所述新事务对应的分区及队列类型;The real-time transaction scheduling module 710 is used to determine the partition and queue type corresponding to a new transaction according to the read-write set of the new transaction when a new transaction is added to the processing;
即时事务处理模块720,用于当所述新事务在对应的队列中与其他分区中的事务存在冲突时,则所述新事务延迟处理。The instant transaction processing module 720 is used to delay the processing of the new transaction when the new transaction conflicts with transactions in other partitions in the corresponding queue.
参照图8,示出了本发明的一种基于多核处理器的并发事务处理方法的计算机设备,具体可以包括如下:8, a computer device of a concurrent transaction processing method based on a multi-core processor of the present invention is shown, which may specifically include the following:
上述计算机设备12以通用计算设备的形式表现,计算机设备12的组件可以包括但不限于:一个或者多个处理器或者处理单元16,系统存储器28,连接不同系统组件(包括系统存储器28和处理单元16)的总线18。The computer device 12 is in the form of a general-purpose computing device, and the components of the computer device 12 may include but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
总线18表示几类总线18结构中的一种或多种,包括存储器总线18或者存储器控制器,外围总线18,图形加速端口,处理器或者使用多种总线18结构中的任意总线18结构的局域总线18。举例来说,这些体系结构包括但不限于工业标准体系结构(ISA)总线18,微通道体系结构(MAC)总线18,增强型ISA总线18、音视频电子标准协会(VESA)局域总线18以及外围组件互连(PCI)总线18。The bus 18 represents one or more of several types of bus 18 structures, including a memory bus 18 or memory controller, a peripheral bus 18, an accelerated graphics port, a processor or a local bus 18 using any of a variety of bus 18 architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus 18, a Micro Channel Architecture (MAC) bus 18, an Enhanced ISA bus 18, an Audio Video Electronics Standards Association (VESA) local bus 18, and a Peripheral Component Interconnect (PCI) bus 18.
计算机设备12典型地包括多种计算机系统可读介质。这些介质可以是任何能够被计算机设备12访问的可用介质,包括易失性和非易失性介质, 可移动的和不可移动的介质。 Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and nonvolatile media, removable and non-removable media.
系统存储器28可以包括易失性存储器形式的计算机系统可读介质,例如随机存取存储器(RAM)30和/或高速缓存存储器32。计算机设备12可以进一步包括其他移动/不可移动的、易失性/非易失性计算机体统存储介质。仅作为举例,存储系统34可以用于读写不可移动的、非易失性磁介质(通常称为“硬盘驱动器”)。尽管图8中未示出,可以提供用于对可移动非易失性磁盘(如“软盘”)读写的磁盘驱动器,以及对可移动非易失性光盘(例如CD-ROM,DVD-ROM或者其他光介质)读写的光盘驱动器。在这些情况下,每个驱动器可以通过一个或者多个数据介质界面与总线18相连。存储器可以包括至少一个程序产品,该程序产品具有一组(例如至少一个)程序模块42,这些程序模块42被配置以执行本发明各实施例的功能。The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32. The computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as "hard drives"). Although not shown in Figure 8, a disk drive for reading and writing removable non-volatile disks (such as "floppy disks"), and an optical disk drive for reading and writing removable non-volatile optical disks (such as CD-ROMs, DVD-ROMs or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42 that are configured to perform the functions of various embodiments of the present invention.
具有一组(至少一个)程序模块42的程序/实用工具40,可以存储在例如存储器中,这样的程序模块42包括——但不限于——操作系统、一个或者多个应用程序、其他程序模块42以及程序数据,这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块42通常执行本发明所描述的实施例中的功能和/或方法。A program/utility 40 having a set (at least one) of program modules 42 may be stored in, for example, a memory, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules 42, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and/or methods of the embodiments described herein.
计算机设备12也可以与一个或多个外部设备14(例如键盘、指向设备、显示器24、摄像头等)通信,还可与一个或者多个使得操作人员能与该计算机设备12交互的设备通信,和/或与使得该计算机设备12能与一个或多个其他计算设备进行通信的任何设备(例如网卡,调制解调器等等)通信。这种通信可以通过输入/输出(I/O)界面22进行。并且,计算机设备12还可以通过网络适配器20与一个或者多个网络(例如局域网(LAN)),广域网(WAN)和/或公共网络(例如因特网)通信。如图所示,网络适配器20通过总线18与计算机设备12的其他模块通信。应当明白,尽管图8中未示出,可以结合计算机设备12使用其他硬件和/或软件模块,包括但不限于:微代码、设备驱动器、冗余处理单元16、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统34等。The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, cameras, etc.), may communicate with one or more devices that enable an operator to interact with the computer device 12, and/or may communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input/output (I/O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs)), wide area networks (WANs), and/or public networks (e.g., the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer device 12 via a bus 18. It should be understood that, although not shown in FIG. 8 , other hardware and/or software modules may be used in conjunction with the computer device 12, including, but not limited to, microcode, device drivers, redundant processing units 16, external disk drive arrays, RAID systems, tape drives, and data backup storage systems 34, etc.
处理单元16通过运行存储在系统存储器28中的程序,从而执行各种功 能应用以及数据处理,例如实现本发明实施例所提供的一种基于多核处理器的并发事务处理方法。The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a concurrent transaction processing method based on a multi-core processor provided in an embodiment of the present invention.
也即,上述处理单元16执行上述程序时实现:当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价;依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。That is, when the processing unit 16 executes the program, the following is achieved: when concurrent transactions occur within a target time period, a read-write set of transactions occurring within the target time period is obtained; a conflict relationship between the transactions and an execution cost corresponding to the transaction are determined based on the read-write set; a partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; a queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue; when a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.
在本发明实施例中,本发明还提供一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现如本申请所有实施例提供的一种基于多核处理器的并发事务处理方法:In an embodiment of the present invention, the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a concurrent transaction processing method based on a multi-core processor as provided in all embodiments of the present application is implemented:
也即,给程序被处理器执行时实现:当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价;依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。That is, when the program is executed by the processor, it is implemented as follows: when concurrent transactions occur within a target time period, a read-write set of transactions occurring within the target time period is obtained; a conflict relationship between the transactions and an execution cost corresponding to the transaction are determined based on the read-write set; a partition corresponding to the transaction is determined based on the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same; a queue type of the transaction in the partition is determined based on the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue; when a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.
可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线或半导体的系统、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件或者上述的任意合适的组合。在本文件中,计算机 可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括——但不限于——电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
可以以一种或多种程序设计语言或其组合来编写用于执行本发明操作的计算机程序代码,上述程序设计语言包括面向对象的程序设计语言——诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言——诸如“C”语言或类似的程序设计语言。程序代码可以完全地在操作人员计算机上执行、部分地在操作人员计算机上执行、作为一个独立的软件包执行、部分在操作人员计算机上部分在远程计算机上执行或者完全在远程计算机或者服务器上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)——连接到操作人员计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。本说明书中的各个实施例均采用递进的方式描述,每个实施例重点说明的都是与其他实施例的不同之处,各个实施例之间相同相似的部分互相参见即可。The computer program code for performing the operation of the present invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The program code can be executed entirely on the operator's computer, partially on the operator's computer, as an independent software package, partially on the operator's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the operator's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet). The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other.
尽管已描述了本申请实施例的优选实施例,但本领域内的技术人员一旦得知了基本创造性概念,则可对这些实施例做出另外的变更和修改。所以,所附权利要求意欲解释为包括优选实施例以及落入本申请实施例范围的所有变更和修改。Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.
最后,还需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而 使得包括一系列要素的过程、方法、物品或者终端设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者终端设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者终端设备中还存在另外的相同要素。Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
以上对本申请所提供的一种基于多核处理器的并发事务处理方法及其系统,进行了详细介绍,本文中应用了具体个例对本申请的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本申请的方法及其核心思想;同时,对于本领域的一般技术人员,依据本申请的思想,在具体实施方式及应用范围上均会有改变之处,综上所述,本说明书内容不应理解为对本申请的限制。The above is a detailed introduction to a concurrent transaction processing method and system based on a multi-core processor provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims (10)

  1. 一种基于多核处理器的并发事务处理方法,其特征在于,包括步骤:A concurrent transaction processing method based on a multi-core processor, characterized by comprising the steps of:
    当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;When transaction concurrency occurs within the target time period, the read and write sets of transactions occurring within the target time period are obtained;
    依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价;Determining the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;
    依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;Determining the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;
    依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;Determine the queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;
    当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。When a transaction in the conflict queue conflicts with a transaction in another partition, the transaction in the conflict queue is delayed.
  2. 根据权利要求1所述的方法,其特征在于,所述执行代价为对应的处理器执行所述事务的时间;所述依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型的步骤,包括:The method according to claim 1, characterized in that the execution cost is the time taken by the corresponding processor to execute the transaction; the step of determining the queue type of the transaction in the partition based on the conflict relationship and the execution cost comprises:
    依据所述读写集合确定所述事务的启动时间;Determining a start time of the transaction according to the read-write set;
    依据所述启动时间和所述执行代价确定所述事务的执行时间片;Determine an execution time slice of the transaction according to the start time and the execution cost;
    依据所述冲突关系确定目标分区中目标事务的执行时间片与其他分区中事务的执行时间片是否有重叠,若是,则所述目标事务在所述冲突队列中,若否,则所述目标事务在所述无冲突队列中。Determine whether the execution time slice of the target transaction in the target partition overlaps with the execution time slice of transactions in other partitions according to the conflict relationship. If so, the target transaction is in the conflict queue; if not, the target transaction is in the non-conflict queue.
  3. 根据权利要求2所述的方法,其特征在于,所述当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理的步骤,包括:The method according to claim 2, characterized in that when a transaction in the conflict queue conflicts with a transaction in another partition, the step of delaying the processing of the transaction in the conflict queue comprises:
    依据所述执行时间片确定所述事务在所述冲突队列中的排序;Determining the order of the transaction in the conflict queue according to the execution time slice;
    当所述冲突队列中事务的执行时间片与其他分区中事务的执行时间片重叠时,则所述冲突队列中的事务延迟处理。When the execution time slices of the transactions in the conflict queue overlap with the execution time slices of the transactions in other partitions, the transactions in the conflict queue are delayed.
  4. 根据权利要求3所述的方法,其特征在于,所述依据所述执行时间片确定所述事务在所述冲突队列中的排序的步骤,包括:The method according to claim 3, characterized in that the step of determining the order of the transactions in the conflict queue according to the execution time slice comprises:
    依据所述冲突关系将所述冲突队列中事务的执行时间片与其他分区中 事务的执行时间片错峰排序。The execution time slices of the transactions in the conflict queue and the execution time slices of the transactions in other partitions are arranged in a staggered order according to the conflict relationship.
  5. 根据权利要求1所述的方法,其特征在于,所述依据所述冲突关系确定所述事务对应的分区的步骤,包括:The method according to claim 1, characterized in that the step of determining the partition corresponding to the transaction based on the conflict relationship comprises:
    依据所述读写集合确定所述事务访问的数据类型;Determining the data type accessed by the transaction according to the read-write set;
    依据所述数据类型和所述冲突关系确定所述事务的分区。The partition of the transaction is determined according to the data type and the conflict relationship.
  6. 根据权利要求1所述的方法,其特征在于,还包括:The method according to claim 1, further comprising:
    当所述冲突队列中事务的实际执行时间与对应的所述执行代价不相等时,则对所述事务进行加锁访问。When the actual execution time of the transaction in the conflict queue is not equal to the corresponding execution cost, the transaction is locked for access.
  7. 根据权利要求1所述的方法,其特征在于,还包括:The method according to claim 1, further comprising:
    当有新事务加入处理时,依据所述新事务的读写集合确定所述新事务对应的分区及队列类型;When a new transaction is added to the processing, the partition and queue type corresponding to the new transaction are determined according to the read and write sets of the new transaction;
    当所述新事务在对应的队列中与其他分区中的事务存在冲突时,则所述新事务延迟处理。When the new transaction conflicts with transactions in other partitions in the corresponding queue, the processing of the new transaction is delayed.
  8. 一种基于多核处理器的并发事务处理系统,其特征在于,包括:A concurrent transaction processing system based on a multi-core processor, characterized by comprising:
    获取模块,用于当目标时间段内出现事务并发时,则获取在所述目标时间段内出现的事务的读写集合;An acquisition module, used for acquiring a read-write set of transactions occurring within a target time period when transactions occur concurrently within the target time period;
    计算模块,用于依据所述读写集合确定所述事务之间的冲突关系以及对应于所述事务的执行代价;A calculation module, used to determine the conflict relationship between the transactions and the execution cost corresponding to the transactions according to the read-write set;
    分区模块,用于依据所述冲突关系确定所述事务对应的分区;其中,访问相同数据的所述事务对应的分区相同;A partitioning module, used to determine the partition corresponding to the transaction according to the conflict relationship; wherein the partitions corresponding to the transactions accessing the same data are the same;
    调度模块,用于依据所述冲突关系和所述执行代价确定所述事务在所述分区的队列类型;其中,所述队列类型包括无冲突队列和冲突队列;A scheduling module, used to determine the queue type of the transaction in the partition according to the conflict relationship and the execution cost; wherein the queue type includes a non-conflict queue and a conflict queue;
    处理模块,用于当所述冲突队列中的事务与其他分区中的事务存在冲突时,则所述冲突队列中的事务延迟处理。The processing module is used for delaying the processing of transactions in the conflict queue when there is a conflict between the transactions in the conflict queue and the transactions in other partitions.
  9. 一种计算机设备,其特征在于,包括处理器、存储器及存储在所述存储器上并能够在所述处理器上运行的计算机程序,所述计算机程序被所述处理器执行时实现如权利要求1至7中任一项所述的方法。A computer device, characterized in that it comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.
  10. 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储计算机程序,所述计算机程序被处理器执行时实现如权利要求1至7中任一项所述的方法。A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
PCT/CN2022/131298 2022-11-09 2022-11-11 Multicore-processor-based concurrent transaction processing method and system WO2024098363A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211399303.6A CN115629822B (en) 2022-11-09 2022-11-09 Concurrent transaction processing method and system based on multi-core processor
CN202211399303.6 2022-11-09

Publications (1)

Publication Number Publication Date
WO2024098363A1 true WO2024098363A1 (en) 2024-05-16

Family

ID=84908215

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/131298 WO2024098363A1 (en) 2022-11-09 2022-11-11 Multicore-processor-based concurrent transaction processing method and system

Country Status (2)

Country Link
CN (1) CN115629822B (en)
WO (1) WO2024098363A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116775315B (en) * 2023-08-22 2024-01-02 北京遥感设备研究所 Multi-core CPU concurrent transaction allocation method based on dependency graph

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102354289A (en) * 2011-09-21 2012-02-15 苏州大学 Concurrent transaction scheduling method and related device
US20130097607A1 (en) * 2011-10-14 2013-04-18 Brian T. Lewis Method, apparatus, and system for adaptive thread scheduling in transactional memory systems
CN115080670A (en) * 2022-06-21 2022-09-20 东北大学 Deterministic transaction concurrency control method based on GPU acceleration

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8555016B2 (en) * 2008-12-17 2013-10-08 Intel Corporation Unified optimistic and pessimistic concurrency control for a software transactional memory (STM) system
CN114217978B (en) * 2022-02-21 2022-05-17 在线途游(北京)科技有限公司 Database transaction processing method, system, computing device and computer readable storage medium based on optimistic lock
CN115309515B (en) * 2022-10-10 2023-01-31 北京理工大学 Cross-chain transaction processing method, device and equipment based on block chain

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102354289A (en) * 2011-09-21 2012-02-15 苏州大学 Concurrent transaction scheduling method and related device
US20130097607A1 (en) * 2011-10-14 2013-04-18 Brian T. Lewis Method, apparatus, and system for adaptive thread scheduling in transactional memory systems
CN115080670A (en) * 2022-06-21 2022-09-20 东北大学 Deterministic transaction concurrency control method based on GPU acceleration

Also Published As

Publication number Publication date
CN115629822A (en) 2023-01-20
CN115629822B (en) 2023-07-25

Similar Documents

Publication Publication Date Title
US11003689B2 (en) Distributed database transaction protocol
US10180946B2 (en) Consistent execution of partial queries in hybrid DBMS
US10261862B2 (en) Data replication in a database management system
Thomson et al. The case for determinism in database systems
US10572510B2 (en) Distributed database transaction protocol
US9904721B1 (en) Source-side merging of distributed transactions prior to replication
JP5660693B2 (en) Hybrid OLTP and OLAP high performance database system
Ren et al. An evaluation of the advantages and disadvantages of deterministic database systems
US11386065B2 (en) Database concurrency control through hash-bucket latching
Lin et al. Towards a non-2pc transaction management in distributed database systems
US8438571B2 (en) Thread speculative execution and asynchronous conflict
Wu et al. Transaction healing: Scaling optimistic concurrency control on multicores
Yabandeh et al. A critique of snapshot isolation
Mahmoud et al. Maat: Effective and scalable coordination of distributed transactions in the cloud
US9652491B2 (en) Out-of-order execution of strictly-ordered transactional workloads
Chairunnanda et al. ConfluxDB: Multi-master replication for partitioned snapshot isolation databases
WO2024098363A1 (en) Multicore-processor-based concurrent transaction processing method and system
Yuan et al. Rubato DB: A highly scalable staged grid database system for OLTP and big data applications
Yao et al. Dgcc: A new dependency graph based concurrency control protocol for multicore database systems
Dong et al. Fine-Grained Re-Execution for Efficient Batched Commit of Distributed Transactions
Tanabe et al. Integration of parallel write ahead logging and cicada concurrency control method
Jones Fault-tolerant distributed transactions for partitioned OLTP databases
Yang et al. Natto: Providing distributed transaction prioritization for high-contention workloads
Ragunathan et al. Improving the performance of read-only transactions through asynchronous speculation
Shacham et al. Taking omid to the clouds: Fast, scalable transactions for real-time cloud analytics