CN114297291A - Transaction combination-based parallel execution method and device - Google Patents
Transaction combination-based parallel execution method and device Download PDFInfo
- Publication number
- CN114297291A CN114297291A CN202111503817.7A CN202111503817A CN114297291A CN 114297291 A CN114297291 A CN 114297291A CN 202111503817 A CN202111503817 A CN 202111503817A CN 114297291 A CN114297291 A CN 114297291A
- Authority
- CN
- China
- Prior art keywords
- transaction
- lsn
- execution
- parallel
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a transaction combination-based parallel execution method and device. The method mainly comprises the following steps: merging the single transaction into multiple transactions and setting a commit LSN, a first transaction A and a parallel LSN of the multiple transactions; setting a first transaction A as a current transaction B; entering a corresponding step by judging the next operation of the current transaction B and searching an execution thread C of which the parallel LSN is smaller than the current operation LSN to be executed; and judging whether the current single transaction is the last single transaction, if so, executing thread submission operation, otherwise, taking down one single transaction to be set as a first transaction A, and re-entering the step of setting the current transaction B. The method adopted by the invention can optimize the parallelism of conflict-free data synchronization operation after transaction combination, thereby improving the performance of data synchronization.
Description
Technical Field
The invention relates to the technical field of database data processing, in particular to a parallel execution method and equipment based on transaction merging.
Background
At present, heterogeneous database replication technology based on log analysis is widely applied. The technology captures the logs of the database at the source end, analyzes INSERT, UPDATE and DELETE operations in the logs, then sends the logs to the target end, and the target end carries out reverse generation on the log information, restores the log information into an SQL statement mode, and then uses a database general interface to be applied to the target database to realize data replication. Therefore, in the real-time database replication process, the execution efficiency of the target end is one of the important factors influencing the data synchronization performance.
When a synchronous transaction is executed on a target-end database, transaction operations need to be executed based on the submission sequence of the transaction, otherwise, errors occur when operations related to each other are executed, and therefore the accuracy of synchronization is affected. In order to pursue the performance of data synchronization, a transaction merging strategy is often adopted when a synchronization transaction is executed, small transactions are merged into large transactions, the execution times can be reduced by merging the same operations of different transactions, the transaction submission times can also be reduced, and the aim of improving the synchronization performance is fulfilled. In addition, the number of threads for executing the transaction is increased, and a multi-thread parallel strategy is adopted to simultaneously store a plurality of merged transactions in a warehouse, so that the synchronization performance is further improved. However, after the strategies of transaction merging and parallel execution are adopted at the same time, in order to prevent and solve the problem of data conflict caused by parallel execution, the existing scheme executes operations according to the order of transaction submission, but the efficiency of parallel execution is reduced to a certain extent.
In view of this, how to overcome the defects existing in the prior art and ensure the efficient execution of the merging transaction and the consistency of the data synchronization is a difficult problem to be solved in the technical field.
Disclosure of Invention
In view of the above deficiencies and needs in the art, the present invention provides a method and apparatus for transaction merge based parallel execution. For convenience of presentation, the transaction in the source log is called "single transaction", and the transaction formed after the multiple single transactions are merged is called "multiple transaction". The invention relates to a method for improving the synchronization performance of data synchronization, which comprises the steps of customizing a strategy for parallel execution by taking a first execution interval (an interval before parallel LSN) as an example, skipping the execution of the current single transaction if the execution of the current single transaction is constrained by other parallel execution transactions and waits according to the submission sequence of the single transaction, taking the next single transaction to judge whether the execution meets the condition of the parallel execution, executing if the execution meets the condition, and traversing to the last single transaction if the execution does not meet the condition, so as to achieve the aim of improving the synchronization performance.
The embodiment of the invention adopts the following technical scheme:
in a first aspect, the present invention provides a transaction merging-based parallel execution method, including:
merging the single transaction into multiple transactions and setting a commit LSN, a first transaction A and a parallel LSN of the multiple transactions;
setting a current transaction B step: setting a first transaction A as a current transaction B;
entering a corresponding step by judging the next operation of the current transaction B and searching an execution thread C of which the parallel LSN is smaller than the current operation LSN to be executed;
a thread submitting step: and judging whether the current single transaction is the last single transaction, if so, executing thread submission operation, otherwise, taking down one single transaction to be set as a first transaction A, and re-entering the step of setting the current transaction B.
Further, the method also comprises a deployment preparation step:
data synchronization service is deployed in both a source end database and a target end database;
after the data synchronization service of the target terminal is started, initializing a linked list to be executed, an execution thread, a wake-up linked list and an execution thread linked list;
after receiving the source end operation, the target end data synchronization service carries out classification management according to the transaction ID of the operation, and when a certain transaction is submitted, the transaction is stored into the chain table to be executed according to the LSN size sequence of the submitted operation.
Further, the merging the single transaction into multiple transactions and setting the commit LSN, the first transaction a, and the parallel LSN of the multiple transactions specifically includes:
the execution thread extracts N single transactions from the chain table to be executed according to the sequence of transaction submission LSN and combines the single transactions into a multi-transaction, sets the submission LSN of the last single transaction as the submission LSN of the multi-transaction, and then registers in the execution thread chain table according to the submission LSN of the multi-transaction;
the execution thread sets the next single transaction in the multi-transaction to a first transaction A and sets the commit LSN of the first single transaction to a parallel LSN.
Further, the entering of the corresponding step by judging the next operation of the current transaction B and searching for the execution thread C with the parallel LSN smaller than the current operation to be executed LSN specifically includes:
a first submission operation judgment step: selecting to enter a searching step or a submitting step of the execution thread C according to the operation type of the next operation of the current transaction B;
and C, executing a thread C searching step: entering a first parallel execution step or a loop judgment step by searching an execution thread C with a parallel LSN smaller than the current LSN to be executed;
a first parallel execution step: jumping to a first submitting operation judging step after executing the current operation;
and a circulating judgment step: a loop decision is made for all single transactions to determine if they can be executed in parallel.
Further, the first submitting operation determining step specifically includes:
extracting the next operation of the current transaction B, and judging whether the extracted operation is a commit operation;
if yes, entering a thread submitting step, and if not, entering a thread C searching step.
Further, the step of executing the thread C search specifically includes:
obtaining a current LSN to be executed, and searching an execution thread C of which the parallel LSN is smaller than the LSN to be executed forward in an execution thread linked list;
if the result is found, entering a circulating judgment step, and if the result is not found, entering a first parallel execution step.
Further, the loop judging step specifically includes:
a rollback step: backing the current operation position and pointing the current transaction B to the next single transaction, and then entering a second submitting operation judgment step;
a second submitting operation judging step: extracting the next operation of the current transaction B and judging whether the extracted operation is a commit operation, if so, entering a rollback step, and if not, entering a parallel LSN judgment step;
parallel LSN judging step: judging whether the parallel LSNs of the execution threads arranged before the current execution thread are all larger than the current LSN to be executed, if so, entering a second parallel execution step, and otherwise, entering a rollback step;
a second parallel execution step: and entering a second submitting operation judgment step after the current operation is executed.
Further, when the current transaction B points to the next single transaction in the rollback step, if there is no next single transaction, the first transaction a is set as the current transaction B, and the current execution thread is added to the wakeup linked list of the execution thread C to wait for the execution thread C to be committed and then to be woken up.
Furthermore, when the thread submitting operation is executed in the thread submitting step, all the other execution threads waiting in the wake-up linked list of the execution thread are woken up, and then the execution thread is removed from the execution thread linked list.
On the other hand, the invention provides a parallel execution device based on transaction merging, which specifically comprises: the parallel execution method based on the transaction combination comprises at least one processor and a memory, wherein the at least one processor and the memory are connected through a data bus, and the memory stores instructions capable of being executed by the at least one processor, and the instructions are used for completing the parallel execution method based on the transaction combination in the first aspect after being executed by the processor.
Compared with the prior art, the invention has the beneficial effects that:
firstly, when two execution threads execute multiple transactions in parallel, different parallel intervals are divided by adopting the commit LSN of a single transaction, and in the first divided parallel interval (namely the interval before the parallel LSN), the operation of operating the LSN in the two execution threads in the interval can be unconditionally parallel, because the resources accessed by the operations are not conflicted.
Secondly, after the affair merging strategy is adopted, a plurality of single affairs are merged into a plurality of affairs, if the single affair operations are executed one by one according to the affair merging sequence during the execution of the plurality of affairs, the execution process is suspended after the first single affair operation does not satisfy the parallel execution condition, and in this case, the operations which satisfy the parallel execution in the following single affairs lose the opportunity of the parallel execution. The invention carries out targeted optimization aiming at the situation, when finding that the operation does not meet the condition of parallel execution, the operation is backed off, a single transaction is taken down, whether the operation meets the condition of parallel execution is judged, if so, the operation is executed, and the process is repeated until the last single transaction. The scheme can optimize the parallelism of conflict-free data synchronization operation after transaction merging, thereby improving the performance of data synchronization and simultaneously ensuring the high-efficiency execution of merged transactions and the consistency of data synchronization.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a flowchart of a parallel execution method based on transaction merging according to embodiment 1 of the present invention;
fig. 2 is a detailed flowchart of deployment preparation steps provided in embodiment 1 of the present invention;
FIG. 3 is a flowchart illustrating a step 100 provided in embodiment 1 of the present invention;
FIG. 4 is a flowchart illustrating a step 300 according to embodiment 1 of the present invention;
FIG. 5 is a flowchart illustrating a step 400 provided in embodiment 1 of the present invention;
FIG. 6 is a flowchart specifically illustrating the steps 600 provided in embodiment 1 of the present invention;
fig. 7 is a flowchart of a specific transaction execution provided in embodiment 1 of the present invention;
fig. 8 is a schematic structural diagram of a parallel execution device based on transaction merging according to embodiment 3 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The present invention is a system structure of a specific function system, so the functional logic relationship of each structural module is mainly explained in the specific embodiment, and the specific software and hardware implementation is not limited.
In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other. The invention will be described in detail below with reference to the figures and examples.
Example 1:
as shown in fig. 1, an embodiment of the present invention provides a parallel execution method based on transaction merging, which includes the following specific steps.
Step 100: the single transaction is merged into multiple transactions and the commit LSN, the first transaction A, and the parallel LSN of the multiple transactions are set.
Step 200 (set current transaction B step): the first transaction a is set to the current transaction B. The current transaction B in this step refers to the single transaction that is currently fetching the operation and is ready for execution.
Step 300-step 600: and entering corresponding steps by judging the next operation of the current transaction B and searching the execution thread C of which the parallel LSN is smaller than the current operation LSN to be executed.
Step 700 (thread commit step): and judging whether the current single transaction is the last single transaction, if so, executing thread submission operation, otherwise, taking down one single transaction to be set as a first transaction A, and re-entering the step of setting the current transaction B.
The steps 300 to 600 specifically include the steps 300, 400, 500 and 600. The method comprises the following steps:
step 300 (first commit operation determination step): and selecting to enter the step of executing the thread C to search or the step of submitting the thread according to the operation type of the next operation of the current transaction B.
Step 400 (execution thread C lookup step): and entering a first parallel execution step or a loop judgment step by searching the execution thread C with the parallel LSN smaller than the current operation LSN to be executed.
Step 500 (first parallel execution step): and jumping to the first submitting operation judgment step after the current operation is executed. The determination of the next operation of the current transaction B is continued in a first commit operation determination step.
Step 600 (loop determination step): a loop decision is made for all single transactions to determine if they can be executed in parallel.
Through the above steps 100 to 700, the embodiment of the present invention determines whether to execute the current operation in parallel by searching for the execution thread whose parallel LSN is smaller than the current operation to be executed LSN, and performing loop judgment on all single transactions. In the scheme, because a plurality of single transactions exist in the multiple transactions executed by the execution thread, the LSN of the first single transaction is the parallel LSN, the operation in the transaction is submitted later, and if the LSN of the operation is less than the parallel LSN, the unconditional execution can be carried out. When multiple transactions are executed, each single transaction in the multiple transactions is traversed, and the operations meeting the conditions are found out, so that the data synchronization efficiency can be improved through parallel execution. The scheme can optimize the parallelism of conflict-free data synchronization operation after transaction merging, thereby improving the performance of data synchronization and simultaneously ensuring the high-efficiency execution of merged transactions and the consistency of data synchronization.
In the preferred embodiment, the method further includes a deployment preparation step, as shown in fig. 2, the deployment preparation step specifically includes:
step 10: and deploying data synchronization service in both the source end database and the target end database. In the step, a log analysis module of the data synchronization service is deployed in a source end database and is responsible for analyzing log operation, a data synchronization receiving service is deployed in a target end database and is responsible for receiving operation sent by source end data synchronously and storing the operation in a warehouse.
Step 20: after the data synchronization service of the target terminal is started, initializing a chain table to be executed, an execution thread, a wake-up chain table and an execution thread chain table. In this step, the to-be-executed linked list is used to store the to-be-executed single transactions in the linked list according to the LSN size order of the transaction commit operation, and to wait for execution to be put in storage. Each execution thread is used for extracting a plurality of single transactions from the chain table to be executed and combining the single transactions into a plurality of transactions, setting the submission LSN of the last single transaction as the submission LSN of the plurality of transactions, and then executing the transaction into a storage. The execution thread linked list is used for registering the execution threads in the linked list according to the size sequence of the current multiple-transaction submission LSN to be executed, and the multiple-transaction operation can be executed only after the registration is completed.
Step 30: after receiving the source end operation, the target end data synchronization service carries out classification management according to the transaction ID of the operation, and when a certain transaction is submitted, the transaction is stored into the chain table to be executed according to the LSN size sequence of the submitted operation.
As shown in fig. 3, in the preferred embodiment, the step 100 (combining single transaction into multiple transaction and setting commit LSN, first transaction a and parallel LSN of multiple transaction) specifically includes:
step 101: the execution thread extracts N single transactions from the chain table to be executed according to the sequence of transaction submission LSN and combines the single transactions into a multi-transaction, sets the submission LSN of the last single transaction as the submission LSN of the multi-transaction, and then registers in the execution thread chain table according to the submission LSN of the multi-transaction.
Step 102: the execution thread sets the next single transaction in the multi-transaction to a first transaction A and sets the commit LSN of the first single transaction to a parallel LSN.
In the preferred embodiment, parallel LSNs refer to: the threads that are listed after the thread in the thread list are ready to execute operations LSN that are less than the parallel LSN, and thus may be executed in parallel, since the operations are also parallel when executing on the source.
In the preferred embodiment, the first transaction a refers to the first single transaction in the multi-transaction that has not been executed, and if a single transaction has been executed, the first transaction a is moved backwards to point to its next single transaction.
In the preferred embodiment, since the execution thread executes multiple transactions combined from multiple single transactions, it executes the single transactions in the order of combination in the multiple transactions when fetching the transactions.
As shown in fig. 4, in the preferred embodiment, the step 300 (the first commit operation determining step) specifically includes:
step 301: and extracting the next operation of the current transaction B, and judging whether the extracted operation is a commit operation.
Step 302: if yes, entering a thread submitting step, and if not, entering a thread C searching step.
As shown in fig. 5, in the preferred embodiment, the step 400 (the step of executing the thread C search) specifically includes:
step 401: and obtaining the current LSN to be executed, and searching an execution thread C of which the parallel LSN is smaller than the LSN to be executed forward in an execution thread linked list.
Step 402: if the result is found, entering a circulating judgment step, and if the result is not found, entering a first parallel execution step.
As shown in fig. 6, in the preferred embodiment, the step 600 (loop judgment step) specifically includes:
step 601 (rollback step): and returning the current operation position and pointing the current transaction B to the next single transaction, and then entering a second commit operation judgment step. In this step, it is necessary to rollback the position of the extraction operation of the current transaction B to the previous position, and point the current transaction B to the next single transaction, if there is no next single transaction, set the first transaction a as the current transaction B, and then add the current execution thread to the wakeup linked list of the execution thread C to wait for the execution thread C to be committed and then to be woken up.
In the preferred embodiment, after all the operations meeting the parallel execution condition in the single transaction are executed, the position of the current transaction B needs to be reset to the position of the first transaction a, and then the next cycle can be continued after the operations are waken up until all the operations in the multiple transactions are executed.
Step 602 (second commit operation determination step): and extracting the next operation of the current transaction B and judging whether the extracted operation is a commit operation, if so, entering a rollback step, and if not, entering a parallel LSN judgment step.
Step 603 (parallel LSN determination step): and judging whether the parallel LSNs of the execution threads arranged before the current execution thread are all larger than the current LSN of the operation to be executed, if so, entering a second parallel execution step, and otherwise, entering a rollback step. In this step, the current to-be-executed operation LSN needs to be acquired first, and then whether the parallel LSNs of the execution threads arranged before the current execution thread are all larger than the current to-be-executed operation LSN is determined forward in the execution thread linked list.
Step 604 (second parallel execution step): and entering a second submitting operation judgment step after the current operation is executed. The next operation of the current transaction B continues to be fetched for decision in the second commit operation decision step.
In the preferred embodiment, after the thread commit operation is performed in step 700 (thread commit step), all other execution threads waiting in the wake-up chain table of the execution thread are also woken up, and then the execution thread is removed from the execution thread chain table to complete the multi-transaction execution.
The above is a detailed description of each step of the method provided in this embodiment, and a specific transaction execution flow after the steps are combined is shown in fig. 7, where the specific transaction execution flow in fig. 7 is as follows: starting to execute the transaction; merging the single transaction into a multi-transaction and registering the multi-transaction to an execution thread linked list; setting a commit LSN of a first single transaction as a parallel LSN; setting a first transaction A (the first transaction A at the beginning) as a single transaction to be processed next and a current transaction B; extracting the next operation of the current transaction B and judging whether the next operation is a commit operation or not; if the last single transaction is the commit operation, judging whether the last single transaction is the commit operation, if the last single transaction is the commit operation, executing the commit operation, awakening all the execution threads in the awakening linked list, then finishing the execution of the multiple transactions, and if the last single transaction is not the commit operation, returning to the step of setting the first transaction A as the next single transaction to be processed and the current transaction B; if the current operation is not a commit operation, judging whether the current execution LSNs are all smaller than the parallel LSNs of the previous execution thread, if so, executing the operation to enter the database and returning to the step of extracting the next operation of the current transaction B and judging whether the operation is the commit operation, and if not, returning the current operation position and pointing the current transaction B to the next single transaction; and then judging whether the current transaction B is the last single transaction, if so, resetting the single transaction pointed by the first transaction A and pointing to the single transaction B by the current transaction B, then adding the single transaction B to an awakening linked list of a conflict execution thread (namely, a certain execution thread of which the current execution LSN is larger than the parallel LSN in the previous judgment) to wait for awakening, if not, extracting the next operation of the current transaction B for the second time, then judging whether the next operation is a commit operation, if so, returning to the step of returning to the current operation position, if not, judging whether the current execution LSN is smaller than the parallel LSN of the previous execution thread, if so, executing the operation to put in storage and returning to the step of extracting the next operation of the current transaction B for the second time, and if not, returning to the step of returning to the current operation position.
In summary, in this embodiment, when two execution threads execute multiple transactions in parallel, a commit LSN of a single transaction is used to partition different parallel intervals, and in the first partitioned parallel interval (i.e. the interval before the parallel LSN), the operations in the two execution threads whose operation LSN falls in this interval can be unconditionally parallel, because the resources accessed by these operations are not conflicted. Secondly, after the affair merging strategy is adopted, a plurality of single affairs are merged into a plurality of affairs, if the single affair operations are executed one by one according to the affair merging sequence during the execution of the plurality of affairs, the execution process is suspended after the first single affair operation does not satisfy the parallel execution condition, and in this case, the operations which satisfy the parallel execution in the following single affairs lose the opportunity of the parallel execution. The embodiment performs targeted optimization for the situation, when finding that the operation does not meet the condition of parallel execution, backs the operation, takes down a single transaction, judges whether the operation meets the condition of parallel execution, executes the operation if the operation meets the condition of parallel execution, and repeats the process until the last single transaction. The scheme can optimize the parallelism of conflict-free data synchronization operation after transaction merging, thereby improving the performance of data synchronization and simultaneously ensuring the high-efficiency execution of merged transactions and the consistency of data synchronization.
Example 2:
based on the transaction merging-based parallel execution method provided in embodiment 1, embodiment 2 describes the present invention in more detail through a specific application scenario.
In this embodiment, taking the source database table T1(ID INT, DATA INT) as an example, there are 3 rows of DATA, and the DATA is distributed in the database as follows:
| ID column | DATA column |
| 1 | 1 |
| 2 | 2 |
| 3 | 3 |
The source has the following log operations:
specifically, based on the application scenario, the establishing step of the execution thread in embodiment 2 based on embodiment 1 is as follows:
1. two threads of execution a and B are created.
2. Wherein the execution thread A extracts the transactions with single transaction IDs 1 and 2 and combines the transactions into a multi-transaction, uses the commit LSN 6 of the single transaction 2 as the commit LSN of the multi-transaction, and registers in the execution thread chain table.
3. The execution thread B extracts the transactions with the single transaction IDs of 3 and 4 and combines the transactions into a multi-transaction, the submission LSN 8 of the single transaction 4 is used as the submission LSN of the multi-transaction and is registered in the execution thread chain table, at this time, the execution thread A already exists in the execution thread chain table, and the submission LSN of the B is larger than that of the A, so that the A is arranged in front of the B in the execution thread chain table.
4. The execution thread A partitions the parallel interval [1-3] according to the commit LSN of the first single transaction, and sets the commit LSN3 of the first single transaction to the parallel LSN.
5. Execution thread B partitions the parallel interval [6-7] according to the commit LSN of the first single transaction, and sets the commit LSN 7 of the first single transaction to the parallel LSN.
6. The execution threads a and B begin executing in parallel.
After the execution threads A and B are established and the preparation work is finished, formally entering a parallel execution stage, wherein the execution flow of the stage refers to the following table:
in the above example, in the existing scheme, when single transactions are merged into multiple transactions, the first operation of the single transaction 4 in the execution thread B needs to wait until the execution thread a commits, and the execution thread B may be stuck to the operation of the single transaction 3 when executed according to the order of the transaction merged transactions, and the operation in the single transaction 3 needs to wait until the single transaction 1 is executed and commits before starting, and in the existing scheme, serial execution may ensure data synchronization consistency but sacrifice synchronization performance.
It can be seen from the above parallel execution flow that in the embodiment of the present invention, when the execution thread B executes the single transaction 3, it finds that its first operation LSN is 5, which is greater than the parallel LSN3 of the execution thread a, so that it first rolls back this operation, then executes the next single transaction 4, and the first operation LSN of the single transaction 4 is 2, which is smaller than the parallel LSN3 of the execution thread a, so that unconditional parallelism, which is a point of improving synchronization performance, and so on, after all the single transactions are traversed, it adds itself to the wakeup linked list of the execution thread a, and after the execution thread a commits, executes the operation in the single transaction 3 with conflict again, thereby ensuring the consistency of data synchronization when the parallel execution has errors.
Example 3:
on the basis of the transaction merging-based parallel execution method provided in the foregoing embodiments 1 to 2, the present invention further provides a transaction merging-based parallel execution device that can be used for implementing the foregoing method, and as shown in fig. 8, the present invention is a schematic diagram of a device architecture in an embodiment of the present invention. The transaction consolidation based parallel execution device of the present embodiment includes one or more processors 21 and a memory 22. In fig. 8, one processor 21 is taken as an example.
The processor 21 and the memory 22 may be connected by a bus or other means, and fig. 8 illustrates the connection by a bus as an example.
The memory 22, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the transaction merge-based parallel execution method in embodiments 1 to 2. The processor 21 executes various functional applications and data processing of the transaction merge-based parallel execution device by executing the nonvolatile software program, instructions, and modules stored in the memory 22, that is, implements the transaction merge-based parallel execution method of embodiments 1 to 2.
The memory 22 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 22 may optionally include memory located remotely from the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Program instructions/modules are stored in the memory 22, and when executed by the one or more processors 21, perform the transaction merge-based parallel execution method in embodiments 1 to 2 described above, for example, perform the respective steps shown in fig. 1 to 7 described above.
Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the embodiments may be implemented by associated hardware as instructed by a program, which may be stored on a computer-readable storage medium, which may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. A transaction merge-based parallel execution method, comprising:
merging the single transaction into multiple transactions and setting a commit LSN, a first transaction A and a parallel LSN of the multiple transactions;
setting a current transaction B step: setting a first transaction A as a current transaction B;
entering a corresponding step by judging the next operation of the current transaction B and searching an execution thread C of which the parallel LSN is smaller than the current operation LSN to be executed;
a thread submitting step: and judging whether the current single transaction is the last single transaction, if so, executing thread submission operation, otherwise, taking down one single transaction to be set as a first transaction A, and re-entering the step of setting the current transaction B.
2. The transaction merging-based parallel execution method according to claim 1, further comprising a deployment preparation step of:
data synchronization service is deployed in both a source end database and a target end database;
after the data synchronization service of the target terminal is started, initializing a linked list to be executed, an execution thread, a wake-up linked list and an execution thread linked list;
after receiving the source end operation, the target end data synchronization service carries out classification management according to the transaction ID of the operation, and when a certain transaction is submitted, the transaction is stored into the chain table to be executed according to the LSN size sequence of the submitted operation.
3. The transaction merge-based parallel execution method of claim 2, wherein merging single transactions into multiple transactions and setting commit LSNs, first transaction a, and parallel LSNs of the multiple transactions specifically comprises:
the execution thread extracts N single transactions from the chain table to be executed according to the sequence of transaction submission LSN and combines the single transactions into a multi-transaction, sets the submission LSN of the last single transaction as the submission LSN of the multi-transaction, and then registers in the execution thread chain table according to the submission LSN of the multi-transaction;
the execution thread sets the next single transaction in the multi-transaction to a first transaction A and sets the commit LSN of the first single transaction to a parallel LSN.
4. The transaction merging-based parallel execution method of claim 3, wherein the entering the corresponding step by determining a next operation of the current transaction B and searching for the execution thread C with a parallel LSN less than the LSN of the current operation to be executed specifically comprises:
a first submission operation judgment step: selecting to enter a searching step or a submitting step of the execution thread C according to the operation type of the next operation of the current transaction B;
and C, executing a thread C searching step: entering a first parallel execution step or a loop judgment step by searching an execution thread C with a parallel LSN smaller than the current LSN to be executed;
a first parallel execution step: jumping to a first submitting operation judging step after executing the current operation;
and a circulating judgment step: a loop decision is made for all single transactions to determine if they can be executed in parallel.
5. The transaction merging-based parallel execution method of claim 4, wherein the first commit operation determining step specifically includes:
extracting the next operation of the current transaction B, and judging whether the extracted operation is a commit operation;
if yes, entering a thread submitting step, and if not, entering a thread C searching step.
6. The transaction merging-based parallel execution method of claim 5, wherein the step of searching for the execution thread C specifically comprises:
obtaining a current LSN to be executed, and searching an execution thread C of which the parallel LSN is smaller than the LSN to be executed forward in an execution thread linked list;
if the result is found, entering a circulating judgment step, and if the result is not found, entering a first parallel execution step.
7. The transaction merging-based parallel execution method of claim 6, wherein the loop judgment step specifically comprises:
a rollback step: backing the current operation position and pointing the current transaction B to the next single transaction, and then entering a second submitting operation judgment step;
a second submitting operation judging step: extracting the next operation of the current transaction B and judging whether the extracted operation is a commit operation, if so, entering a rollback step, and if not, entering a parallel LSN judgment step;
parallel LSN judging step: judging whether the parallel LSNs of the execution threads arranged before the current execution thread are all larger than the current LSN to be executed, if so, entering a second parallel execution step, and otherwise, entering a rollback step;
a second parallel execution step: and entering a second submitting operation judgment step after the current operation is executed.
8. The transaction merging-based parallel execution method of claim 7, wherein when the current transaction B points to a next single transaction in the rollback step, if there is no next single transaction, the first transaction a is set as the current transaction B, and the current execution thread is added to a wake-up linked list of the execution thread C to wait for the execution thread C to be committed and then wake up.
9. The parallel execution method based on transaction merging as claimed in any one of claims 2-8, wherein when the thread commit operation is performed in the thread commit step, all other execution threads waiting in the wake-up chain table of the execution thread are also woken up, and then the execution thread is removed from the execution thread chain table.
10. A transaction merge-based parallel execution device, characterized by:
comprises at least one processor and a memory, wherein the at least one processor and the memory are connected through a data bus, and the memory stores instructions which can be executed by the at least one processor and are used for completing the parallel execution method based on the transaction combination according to any one of claims 1-9 after being executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111503817.7A CN114297291B (en) | 2021-12-09 | 2021-12-09 | Transaction combination-based parallel execution method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111503817.7A CN114297291B (en) | 2021-12-09 | 2021-12-09 | Transaction combination-based parallel execution method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114297291A true CN114297291A (en) | 2022-04-08 |
| CN114297291B CN114297291B (en) | 2023-02-03 |
Family
ID=80967274
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111503817.7A Active CN114297291B (en) | 2021-12-09 | 2021-12-09 | Transaction combination-based parallel execution method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114297291B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030037029A1 (en) * | 2001-08-15 | 2003-02-20 | Iti, Inc. | Synchronization of plural databases in a database replication system |
| JP2007219598A (en) * | 2006-02-14 | 2007-08-30 | Nippon Telegr & Teleph Corp <Ntt> | Multiplex database system, its synchronization method, database server, and database server program |
| CN107026880A (en) * | 2016-02-01 | 2017-08-08 | 北京京东尚科信息技术有限公司 | Method of data synchronization and device |
| CN110597663A (en) * | 2019-09-16 | 2019-12-20 | 阿里巴巴集团控股有限公司 | Transaction processing method and device |
| CN111475480A (en) * | 2020-04-13 | 2020-07-31 | 上海达梦数据库有限公司 | Log processing method and system |
| CN111858503A (en) * | 2020-06-04 | 2020-10-30 | 武汉达梦数据库有限公司 | Parallel execution method and data synchronization system based on log analysis synchronization |
| CN111930692A (en) * | 2020-05-28 | 2020-11-13 | 武汉达梦数据库有限公司 | Transaction merging execution method and device based on log analysis synchronization |
-
2021
- 2021-12-09 CN CN202111503817.7A patent/CN114297291B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030037029A1 (en) * | 2001-08-15 | 2003-02-20 | Iti, Inc. | Synchronization of plural databases in a database replication system |
| JP2007219598A (en) * | 2006-02-14 | 2007-08-30 | Nippon Telegr & Teleph Corp <Ntt> | Multiplex database system, its synchronization method, database server, and database server program |
| CN107026880A (en) * | 2016-02-01 | 2017-08-08 | 北京京东尚科信息技术有限公司 | Method of data synchronization and device |
| CN110597663A (en) * | 2019-09-16 | 2019-12-20 | 阿里巴巴集团控股有限公司 | Transaction processing method and device |
| CN111475480A (en) * | 2020-04-13 | 2020-07-31 | 上海达梦数据库有限公司 | Log processing method and system |
| CN111930692A (en) * | 2020-05-28 | 2020-11-13 | 武汉达梦数据库有限公司 | Transaction merging execution method and device based on log analysis synchronization |
| CN111858503A (en) * | 2020-06-04 | 2020-10-30 | 武汉达梦数据库有限公司 | Parallel execution method and data synchronization system based on log analysis synchronization |
Non-Patent Citations (2)
| Title |
|---|
| KOJI NAKANO: "Asynchronous Memory Machine Models with Barrier Synchronization", 《2012 THIRD INTERNATIONAL CONFERENCE ON NETWORKING AND COMPUTING》 * |
| 吴树森 等: "UPPA：面向异构众核系统的统一并行编程架构", 《计算机学报》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114297291B (en) | 2023-02-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101807144A (en) | Prospective multi-threaded parallel execution optimization method | |
| CN111694798B (en) | Data synchronization method and data synchronization system based on log analysis | |
| CN109213576B (en) | Program deadlock detection method, storage medium, device and system | |
| Chen et al. | Efficient GPU hardware transactional memory through early conflict resolution | |
| CN101655783B (en) | Forward-looking multithreading partitioning method | |
| US11269693B2 (en) | Method, apparatus, and electronic device for improving CPU performance | |
| CN114296885A (en) | Transaction combination-based parallel execution method and device | |
| CN111858504B (en) | Operation merging execution method based on log analysis synchronization and data synchronization system | |
| CN114297291B (en) | Transaction combination-based parallel execution method and device | |
| CN103377292B (en) | Database result set caching method and device | |
| CN105045563B (en) | A kind of method for collision management for speculating nested software transaction storage | |
| CN111930692B (en) | Transaction merging execution method and device based on log analysis synchronization | |
| EP4309046A1 (en) | Snapshot isolation query transactions in distributed systems | |
| CN110990363A (en) | Distributed database multithreading collaborative transaction log playback method and system | |
| US20220300484A1 (en) | Snapshot isolation query transactions in distributed systems | |
| CN112035222B (en) | Transaction operation merging execution method and device based on log analysis synchronization | |
| CN111858503B (en) | Parallel execution method and data synchronization system based on log analysis synchronization | |
| CN114296887A (en) | Transaction combination-based parallel execution method and device | |
| CN114296886A (en) | Parallel execution method and device based on range row lock | |
| CN109753314B (en) | Configurable automatic flow execution method and system | |
| CN111858626A (en) | Data synchronization method and device based on parallel execution | |
| CN111930693B (en) | Transaction merging execution method and device based on log analysis synchronization | |
| CN102063291B (en) | Multilevel parallel execution method of speculation thread | |
| CN107608662B (en) | MongoDB-based distributed timing system | |
| CN111858505B (en) | Parallel execution method and data synchronization system based on log analysis synchronization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |