CN114297291B - Transaction combination-based parallel execution method and device - Google Patents
Transaction combination-based parallel execution method and device Download PDFInfo
- Publication number
- CN114297291B CN114297291B CN202111503817.7A CN202111503817A CN114297291B CN 114297291 B CN114297291 B CN 114297291B CN 202111503817 A CN202111503817 A CN 202111503817A CN 114297291 B CN114297291 B CN 114297291B
- Authority
- CN
- China
- Prior art keywords
- transaction
- lsn
- execution
- thread
- parallel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention relates to a transaction combination-based parallel execution method and device. The method mainly comprises the following steps: merging the single transaction into multiple transactions and setting a commit LSN, a first transaction A and a parallel LSN of the multiple transactions; setting a first transaction A as a current transaction B; entering a corresponding step by judging the next operation of the current transaction B and searching an execution thread C of which the parallel LSN is smaller than the current operation LSN to be executed; and judging whether the current single transaction is the last single transaction, if so, executing thread submission operation, otherwise, taking down one single transaction to be set as a first transaction A, and re-entering the step of setting the current transaction B. The method adopted by the invention can optimize the parallelism of conflict-free data synchronization operation after transaction combination, thereby improving the performance of data synchronization.
Description
Technical Field
The invention relates to the technical field of database data processing, in particular to a parallel execution method and equipment based on transaction merging.
Background
At present, heterogeneous database replication technology based on log analysis is widely applied. The technology captures the logs of the database at the source end, analyzes INSERT, UPDATE and DELETE operations in the logs, then sends the logs to the target end, and the target end carries out reverse generation on the log information, restores the log information into an SQL statement mode, and then uses a database general interface to be applied to the target database to realize data replication. Therefore, in the real-time database replication process, the execution efficiency of the target end is one of the important factors influencing the data synchronization performance.
When the synchronous transaction is executed on the target-end database, the transaction operation needs to be executed based on the submission sequence of the transaction, otherwise, the operation related to the former operation and the latter operation may be executed incorrectly, thereby affecting the correctness of the synchronization. In order to pursue the performance of data synchronization, a strategy of transaction combination is often adopted when a synchronization transaction is executed, small transactions are combined into large transactions, the execution times can be reduced by combining the same operations of different transactions, the transaction submission times can also be reduced, and the aim of improving the synchronization performance is fulfilled. In addition, the number of threads for executing the transaction can be increased, and a multi-thread parallel strategy is adopted to simultaneously store a plurality of merged transactions in a warehouse, so that the synchronization performance is further improved. However, after the strategies of transaction merging and parallel execution are adopted at the same time, in order to prevent and solve the problem of data conflict caused by parallel execution, the existing scheme executes operations according to the order of transaction submission, but the efficiency of parallel execution is reduced to a certain extent.
In view of this, how to overcome the defects existing in the prior art and ensure the efficient execution of the merging transaction and the consistency of the data synchronization is a difficult problem to be solved in the technical field.
Disclosure of Invention
In view of the above deficiencies and needs in the art, the present invention provides a method and apparatus for transaction merge based parallel execution. For convenience of presentation, the transaction in the source log is called "single transaction", and the transaction formed after the multiple single transactions are merged is called "multiple transaction". The invention relates to a method for improving the synchronization performance of data synchronization, which comprises the steps of customizing a strategy for parallel execution by taking a first execution interval (an interval before parallel LSN) as an example, skipping the execution of the current single transaction if the execution of the current single transaction is constrained by other parallel execution transactions and waits according to the submission sequence of the single transaction, taking the next single transaction to judge whether the execution meets the condition of the parallel execution, executing if the execution meets the condition, and traversing to the last single transaction if the execution does not meet the condition, so as to achieve the aim of improving the synchronization performance.
The embodiment of the invention adopts the following technical scheme:
in a first aspect, the present invention provides a transaction merging-based parallel execution method, including:
merging the single transaction into multiple transactions and setting a commit LSN, a first transaction A and a parallel LSN of the multiple transactions;
setting a current transaction B step: setting a first transaction A as a current transaction B;
entering a corresponding step by judging the next operation of the current transaction B and searching an execution thread C of which the parallel LSN is smaller than the current operation LSN to be executed;
a thread submitting step: and judging whether the current single transaction is the last single transaction, if so, executing a thread submitting operation, otherwise, taking down the next single transaction to be set as a first transaction A, and re-entering the step of setting the current transaction B.
Further, the method also comprises a deployment preparation step:
data synchronization service is deployed in both a source end database and a target end database;
after the data synchronization service of the target terminal is started, initializing a linked list to be executed, an execution thread, a wake-up linked list and an execution thread linked list;
after receiving the source end operation, the target end data synchronization service carries out classification management according to the transaction ID of the operation, and when a certain transaction is submitted, the transaction is stored into the chain table to be executed according to the LSN size sequence of the submitted operation.
Further, the merging of the single transaction into multiple transactions and setting the commit LSN, the first transaction a, and the parallel LSN of the multiple transactions specifically include:
the execution thread picks N single transactions from the chain table to be executed according to the sequence of transaction submission LSN and combines the single transactions into a multi-transaction, sets the submission LSN of the last single transaction as the submission LSN of the multi-transaction, and then registers in the execution thread chain table according to the submission LSN of the multi-transaction;
the execution thread sets the next single transaction in the multi-transaction to a first transaction A and sets the commit LSN of the first single transaction to a parallel LSN.
Further, the entering of the corresponding step by judging the next operation of the current transaction B and searching for the execution thread C with the parallel LSN smaller than the current operation to be executed LSN specifically includes:
a first submission operation judgment step: selecting to enter a searching step or a submitting step of the execution thread C according to the operation type of the next operation of the current transaction B;
and C, executing a thread C searching step: entering a first parallel execution step or a loop judgment step by searching an execution thread C of which the parallel LSN is smaller than the current LSN to be executed;
a first parallel execution step: skipping to a first submitting operation judging step after executing the current operation;
and a circulating judgment step: a loop decision is made for all single transactions to determine if they can be executed in parallel.
Further, the first submitting operation determining step specifically includes:
extracting the next operation of the current transaction B, and judging whether the extracted operation is a commit operation;
if yes, entering a thread submitting step, and if not, entering a thread C searching step.
Further, the step of executing the thread C search specifically includes:
obtaining a current LSN to be executed, and searching forward an execution thread C of which the parallel LSN is smaller than the LSN to be executed in an execution thread linked list;
if the result is found, entering a circulating judgment step, and if the result is not found, entering a first parallel execution step.
Further, the step of circularly judging specifically includes:
a rollback step: backing the current operation position and pointing the current transaction B to the next single transaction, and then entering a second submitting operation judgment step;
a second submission operation judgment step: extracting the next operation of the current transaction B and judging whether the extracted operation is a commit operation, if so, entering a rollback step, and if not, entering a parallel LSN judgment step;
a parallel LSN judging step: judging whether the parallel LSNs of the execution threads arranged before the current execution thread are all larger than the current LSN to be executed, if so, entering a second parallel execution step, and otherwise, entering a rollback step;
a second parallel execution step: and after the current operation is executed, a second submitting operation judging step is carried out.
Further, when the current transaction B points to the next single transaction in the rollback step, if there is no next single transaction, the first transaction a is set as the current transaction B, and the current execution thread is added to the wakeup linked list of the execution thread C to wait for the execution thread C to be committed and then to be woken up.
Furthermore, when the thread submitting operation is executed in the thread submitting step, all the other execution threads waiting in the wake-up linked list of the execution thread are woken up, and then the execution thread is removed from the execution thread linked list.
On the other hand, the invention provides a parallel execution device based on transaction merging, which specifically comprises: the parallel execution method based on the transaction combination comprises at least one processor and a memory, wherein the at least one processor and the memory are connected through a data bus, and the memory stores instructions capable of being executed by the at least one processor, and the instructions are used for completing the parallel execution method based on the transaction combination in the first aspect after being executed by the processor.
Compared with the prior art, the invention has the beneficial effects that:
firstly, when two execution threads execute multiple transactions in parallel, different parallel intervals are divided by adopting the commit LSN of a single transaction, and in the first divided parallel interval (namely the interval before the parallel LSN), the operation of operating the LSN in the two execution threads in the interval can be unconditionally parallel, because the resources accessed by the operations are not conflicted.
Secondly, after the affair merging strategy is adopted, a plurality of single affairs are merged into a plurality of affairs, if the single affair operations are executed one by one according to the affair merging sequence, the execution process is suspended after the first single affair operation does not satisfy the parallel execution condition, and in this case, the operations which satisfy the parallel execution in the following single affairs lose the opportunity of parallel execution. The invention carries out targeted optimization aiming at the situation, when the operation is found not to meet the condition of parallel execution, the operation is backed, a next single transaction is taken, whether the operation meets the condition of parallel execution or not is judged, if yes, the operation is executed, and the process is repeated until the last single transaction. The scheme can optimize the parallelism of conflict-free data synchronization operation after the transactions are merged, thereby improving the performance of data synchronization and simultaneously ensuring the high-efficiency execution of the merged transactions and the consistency of the data synchronization.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a flowchart of a parallel execution method based on transaction merging according to embodiment 1 of the present invention;
fig. 2 is a detailed flowchart of deployment preparation steps provided in embodiment 1 of the present invention;
FIG. 3 is a flowchart specifically illustrating the step 100 provided in embodiment 1 of the present invention;
FIG. 4 is a flowchart specifically illustrating the step 300 provided in embodiment 1 of the present invention;
FIG. 5 is a flowchart illustrating a step 400 provided in embodiment 1 of the present invention;
FIG. 6 is a flowchart specifically illustrating the steps 600 provided in embodiment 1 of the present invention;
fig. 7 is a flowchart of a specific transaction execution provided in embodiment 1 of the present invention;
fig. 8 is a schematic structural diagram of a parallel execution device based on transaction merging according to embodiment 3 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
The present invention is an architecture of a system with specific functions, and therefore, in the specific embodiment, the functional logic relationship of each structural module is mainly described, and the specific software and hardware implementation is not limited.
In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other. The invention will now be described in detail with reference to the figures and examples.
Example 1:
as shown in fig. 1, an embodiment of the present invention provides a parallel execution method based on transaction merging, which includes the following specific steps.
Step 100: the single transaction is merged into multiple transactions and the multi-transaction commit LSN, first transaction a, and parallel LSNs are set.
Step 200 (set current transaction B step): the first transaction a is set to the current transaction B. The current transaction B in this step refers to the single transaction that is currently fetching the operation and is ready for execution.
Step 300-step 600: and entering corresponding steps by judging the next operation of the current transaction B and searching the execution thread C of which the parallel LSN is less than the current operation LSN to be executed.
Step 700 (thread commit step): and judging whether the current single transaction is the last single transaction, if so, executing thread submission operation, otherwise, taking down one single transaction to be set as a first transaction A, and re-entering the step of setting the current transaction B.
The steps 300 to 600 specifically include the steps 300, 400, 500 and 600. The method comprises the following steps:
step 300 (first commit operation determination step): and selecting to enter the step of executing the thread C to search or the step of submitting the thread according to the operation type of the next operation of the current transaction B.
Step 400 (performing thread C lookup step): and entering a first parallel execution step or a loop judgment step by searching the execution thread C with the parallel LSN smaller than the current operation LSN to be executed.
Step 500 (first parallel execution step): and jumping to the first submitting operation judgment step after the current operation is executed. The determination of the next operation of the current transaction B is continued in a first commit operation determination step.
Step 600 (loop determination step): a loop decision is made for all single transactions to determine if they can be executed in parallel.
Through the steps 100 to 700, the embodiment of the present invention determines whether to execute the current operation in parallel by searching for the execution thread whose parallel LSN is smaller than the current operation to be executed LSN, and performing loop judgment on all single transactions. In the scheme, because a plurality of single transactions exist in the multiple transactions executed by the execution thread, the LSN of the first single transaction is the parallel LSN, the operation in the transaction is submitted later, and if the LSN of the operation is less than the parallel LSN, the unconditional execution can be carried out. During multi-transaction execution, each single transaction in the multi-transaction is traversed, and the operations meeting the conditions are found out, so that the data synchronization efficiency can be improved through parallel execution. The scheme can optimize the parallelism of conflict-free data synchronization operation after transaction merging, thereby improving the performance of data synchronization and simultaneously ensuring the high-efficiency execution of merged transactions and the consistency of data synchronization.
In the preferred embodiment, the method further includes a deployment preparation step, as shown in fig. 2, the deployment preparation step specifically includes:
step 10: and deploying data synchronization service in both the source end database and the target end database. In the step, a log analysis module of the data synchronization service is deployed in a source end database and is responsible for analyzing log operation, a data synchronization receiving service is deployed in a target end database and is responsible for receiving operation sent by source end data synchronously and storing the operation in a warehouse.
Step 20: and after the data synchronization service of the target end is started, initializing a linked list to be executed, an execution thread, an awakening linked list and an execution thread linked list. In this step, the chain table to be executed is used to store the single transaction to be executed in the chain table according to the LSN size order of the transaction commit operation, and to wait for execution to be put in storage. Each execution thread is used for extracting a plurality of single transactions from the chain table to be executed and combining the single transactions into a plurality of transactions, setting the submission LSN of the last single transaction as the submission LSN of the plurality of transactions, and then executing the transaction to be put in storage. The execution thread linked list is used for registering the execution threads in the linked list according to the size sequence of the current multi-transaction submission LSN to be executed, and the multi-transaction operation can be executed only after the registration is finished.
Step 30: after receiving the source end operation, the target end data synchronization service carries out classification management according to the transaction ID of the operation, and when a certain transaction is submitted, the transaction is stored into the chain table to be executed according to the LSN size sequence of the submitted operation.
As shown in fig. 3, in the preferred embodiment, the step 100 (combining single transaction into multiple transaction and setting commit LSN, first transaction a and parallel LSN of multiple transaction) specifically includes:
step 101: the execution thread extracts N single transactions from the chain table to be executed according to the sequence of transaction submission LSN and combines the single transactions into a multi-transaction, sets the submission LSN of the last single transaction as the submission LSN of the multi-transaction, and then registers in the execution thread chain table according to the submission LSN of the multi-transaction.
Step 102: the execution thread sets the next single transaction in the multi-transaction to a first transaction A and sets the commit LSN of the first single transaction to a parallel LSN.
In the preferred embodiment, parallel LSNs refer to: the threads that are listed after the thread in the thread list are ready to execute operations LSN that are less than the parallel LSN, and thus may be executed in parallel, since the operations are also parallel when executing on the source.
In the preferred embodiment, the first transaction a refers to the first single transaction in the multi-transaction that has not been executed, and if a single transaction has been executed, the first transaction a is moved backwards to point to its next single transaction.
In the preferred embodiment, since the execution thread executes multiple transactions combined by multiple single transactions, it executes the single transactions in the order of combination in the multiple transactions when fetching the transactions.
As shown in fig. 4, in the preferred embodiment, the step 300 (the first commit operation determining step) specifically includes:
step 301: and extracting the next operation of the current transaction B, and judging whether the extracted operation is a commit operation.
Step 302: if so, entering a thread submitting step, and if not, entering a thread C searching step.
As shown in fig. 5, in the preferred embodiment, the step 400 (the step of executing the thread C search) specifically includes:
step 401: and obtaining the current LSN to be executed, and searching an execution thread C of which the parallel LSN is smaller than the LSN to be executed forward in an execution thread linked list.
Step 402: if the result is found, entering a circulating judgment step, and if the result is not found, entering a first parallel execution step.
As shown in fig. 6, in the preferred embodiment, the step 600 (loop judgment step) specifically includes:
step 601 (rollback step): and returning the current operation position and pointing the current transaction B to the next single transaction, and then entering a second commit operation judgment step. In the step, the position of the extraction operation of the current transaction B needs to be rolled back to the previous position, the current transaction B is pointed to the next single transaction, if the next single transaction does not exist, the first transaction a is set as the current transaction B, and then the current execution thread is added to the wakeup linked list of the execution thread C to wait for the execution thread C to be submitted and then be awakened.
In the preferred embodiment, after all the operations meeting the parallel execution condition in the single transaction are executed, the position of the current transaction B needs to be reset to the position of the first transaction a, and then the next cycle can be continued after the operations are waken up until all the operations in the multiple transactions are executed.
Step 602 (second commit operation determining step): and extracting the next operation of the current transaction B and judging whether the extracted operation is a commit operation, if so, entering a backspacing step, and if not, entering a parallel LSN judging step.
Step 603 (parallel LSN determination step): and judging whether the parallel LSNs of the execution threads arranged before the current execution thread are all larger than the current LSN of the operation to be executed, if so, entering a second parallel execution step, and otherwise, entering a rollback step. In this step, the current LSN of the operation to be executed needs to be obtained first, and then it is determined forward in the execution thread linked list whether the parallel LSNs of the execution threads arranged before the current execution thread are all greater than the current LSN of the operation to be executed.
Step 604 (second parallel execution step): and entering a second submitting operation judgment step after the current operation is executed. The next operation of the current transaction B continues to be fetched for decision in the second commit operation decision step.
In the preferred embodiment, after the thread commit operation is performed in step 700 (thread commit step), all other execution threads waiting in the wake-up chain table of the execution thread are also woken up, and then the execution thread is removed from the execution thread chain table to complete the multi-transaction execution.
The above is a detailed description of each step of the method provided in this embodiment, and a specific transaction execution flow after the steps are combined is shown in fig. 7, where the specific transaction execution flow in fig. 7 is as follows: starting to execute the transaction; merging the single transaction into a multi-transaction and registering the multi-transaction to an execution thread linked list; setting a commit LSN of a first single transaction as a parallel LSN; setting a first transaction A (the first transaction A at the beginning) as a single transaction to be processed next and a current transaction B; extracting the next operation of the current transaction B and judging whether the operation is a commit operation or not; if the single transaction is the last single transaction, judging whether the single transaction is the last single transaction, if the single transaction is the last single transaction, executing the submission, awakening all the execution threads in the awakening linked list, then finishing the execution of the multiple transactions, and if the single transaction is not the last single transaction, returning to the step of setting the first transaction A as a next single transaction to be processed and the current transaction B; if the current operation is not a commit operation, judging whether the current LSNs in execution are all smaller than the parallel LSNs of the previous execution thread, if so, entering the execution operation and returning to the step of extracting the next operation of the current transaction B and judging whether the operation is the commit operation, and if not, returning to the current operation position and pointing the current transaction B to the next single transaction; and then judging whether the current transaction B is the last single transaction, if so, resetting the single transaction pointed by the first transaction A and pointing to the single transaction B by the current transaction B, then adding the single transaction B to an awakening linked list of a conflict execution thread (namely, a certain execution thread of which the current execution LSN is larger than the parallel LSN in the previous judgment) to wait for awakening, if not, extracting the next operation of the current transaction B for the second time, then judging whether the next operation is a commit operation, if so, returning to the step of returning to the current operation position, if not, judging whether the current execution LSN is smaller than the parallel LSN of the previous execution thread, if so, executing the operation to put in storage and returning to the step of extracting the next operation of the current transaction B for the second time, and if not, returning to the step of returning to the current operation position.
In summary, in this embodiment, when two execution threads execute multiple transactions in parallel, a commit LSN of a single transaction is used to partition different parallel intervals, and in the first partitioned parallel interval (i.e. the interval before the parallel LSN), the operations in the two execution threads whose operation LSN falls in this interval can be unconditionally parallel, because the resources accessed by these operations are not conflicted. Secondly, after the affair merging strategy is adopted, a plurality of single affairs are merged into a plurality of affairs, if the single affair operations are executed one by one according to the affair merging sequence during the execution of the plurality of affairs, the execution process is suspended after the first single affair operation does not satisfy the parallel execution condition, and in this case, the operations which satisfy the parallel execution in the following single affairs lose the opportunity of the parallel execution. The embodiment performs targeted optimization for the situation, when the operation is found not to meet the condition of parallel execution, the operation is rolled back, a next single transaction is taken, whether the operation meets the condition of parallel execution or not is judged, if yes, the process is executed, and the process is repeated until the last single transaction. The scheme can optimize the parallelism of conflict-free data synchronization operation after transaction merging, thereby improving the performance of data synchronization and simultaneously ensuring the high-efficiency execution of merged transactions and the consistency of data synchronization.
Example 2:
based on the transaction merging-based parallel execution method provided in embodiment 1, embodiment 2 describes the present invention in more detail through a specific application scenario.
In this embodiment, taking a source database table T1 (ID INT, DATA INT) as an example, there are 3 rows of DATA, and the DATA is distributed in the database as follows:
| ID column | DATA column |
| 1 | 1 |
| 2 | 2 |
| 3 | 3 |
The source has the following log operations:
specifically, based on the application scenario, the establishing step of the execution thread in embodiment 2 based on embodiment 1 is as follows:
1. two threads of execution a and B are created.
2. Wherein the execution thread A extracts the transactions with single transaction IDs 1 and 2 and combines the transactions into a multi-transaction, uses the commit LSN 6 of the single transaction 2 as the commit LSN of the multi-transaction, and registers in the execution thread chain table.
3. The execution thread B extracts the transactions with the single transaction IDs of 3 and 4 and combines the transactions into a multi-transaction, the submission LSN 8 of the single transaction 4 is used as the submission LSN of the multi-transaction and is registered in the execution thread chain table, at this time, the execution thread A already exists in the execution thread chain table, and the submission LSN of the B is larger than that of the A, so that the A is arranged in front of the B in the execution thread chain table.
4. The execution thread A divides the parallel interval [1-3] according to the commit LSN of the first single transaction, and sets the commit LSN3 of the first single transaction as the parallel LSN.
5. Execution thread B divides the parallel interval [6-7] according to the commit LSN of the first single transaction, and sets the commit LSN 7 of the first single transaction to the parallel LSN.
6. The execution threads a and B begin executing in parallel.
After the execution threads A and B are established and the preparation work is finished, formally entering a parallel execution stage, wherein the execution flow of the stage refers to the following table:
in the above example, in the existing scheme, when single transactions are merged into multiple transactions, the first operation of the single transaction 4 in the execution thread B needs to wait until the execution thread a commits, and the execution thread B may be stuck to the operation of the single transaction 3 when executed according to the order of the transaction merged transactions, and the operation in the single transaction 3 needs to wait until the single transaction 1 is executed and commits before starting, and in the existing scheme, serial execution may ensure data synchronization consistency but sacrifice synchronization performance.
It can be seen from the above parallel execution flow that when the execution thread B executes the single transaction 3, it finds that its first operation LSN is 5, which is greater than the parallel LSN3 of the execution thread a, so that it rolls back this operation first, and then executes the next single transaction 4, and the first operation LSN of the single transaction 4 is 2, which is smaller than the parallel LSN3 of the execution thread a, so that unconditional parallelism is possible, which is a point of improving synchronization performance, and so on, and after all the single transactions are traversed, it adds itself to the wakeup linked list of the execution thread a, and after the execution thread a commits, executes the operation in the conflicting single transaction 3 again, thereby ensuring the consistency of data synchronization when the parallel execution is faulty.
Example 3:
on the basis of the transaction merging-based parallel execution method provided in the foregoing embodiments 1 to 2, the present invention further provides a transaction merging-based parallel execution device that can be used for implementing the foregoing method, and as shown in fig. 8, the present invention is a schematic diagram of a device architecture in an embodiment of the present invention. The transaction consolidation based parallel execution device of the present embodiment includes one or more processors 21 and a memory 22. In fig. 8, one processor 21 is taken as an example.
The processor 21 and the memory 22 may be connected by a bus or other means, and fig. 8 illustrates the connection by a bus as an example.
The memory 22, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the transaction merge-based parallel execution method in embodiments 1 to 2. The processor 21 executes various functional applications and data processing of the transaction merge-based parallel execution device by executing the nonvolatile software program, instructions, and modules stored in the memory 22, that is, implements the transaction merge-based parallel execution method of embodiments 1 to 2.
The memory 22 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 22 may optionally include memory located remotely from the processor 21, which may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Program instructions/modules are stored in the memory 22, and when executed by the one or more processors 21, perform the transaction merge-based parallel execution method in embodiments 1 to 2 described above, for example, perform the respective steps shown in fig. 1 to 7 described above.
Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the embodiments may be performed by associated hardware as instructed by a program, which may be stored on a computer-readable storage medium, which may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (8)
1. A transaction merge-based parallel execution method, comprising:
merging the single transaction into multiple transactions and setting a commit LSN, a first transaction A and a parallel LSN of the multiple transactions; specifically, the execution thread picks N single transactions from the chain table to be executed according to the sequence of transaction submission LSN and combines the single transactions into a multi-transaction, sets the submission LSN of the last single transaction as the submission LSN of the multi-transaction, and then registers in the execution thread chain table according to the submission LSN of the multi-transaction; the execution thread sets the next single transaction in the multiple transactions as a first transaction A and sets the commit LSN of the first single transaction as a parallel LSN;
setting a current transaction B step: setting a first transaction A as a current transaction B;
entering a corresponding step by judging the next operation of the current transaction B and searching an execution thread C of which the parallel LSN is smaller than the current operation LSN to be executed; specifically, the first submitting operation judging step: selecting to enter a searching step of an execution thread C or a submitting step of the thread according to the operation type of the next operation of the current transaction B; and C, executing a thread C searching step: entering a first parallel execution step or a loop judgment step by searching an execution thread C with a parallel LSN smaller than the current LSN to be executed; a first parallel execution step: jumping to a first submitting operation judging step after executing the current operation; and a circulating judgment step: performing loop judgment on all the single transactions to determine whether the single transactions can be executed in parallel;
a thread submitting step: and judging whether the current single transaction is the last single transaction, if so, executing thread submission operation, otherwise, taking down one single transaction to be set as a first transaction A, and re-entering the step of setting the current transaction B.
2. The transaction merging-based parallel execution method according to claim 1, further comprising a deployment preparation step of:
data synchronization service is deployed in both a source end database and a target end database;
after the data synchronization service of the target terminal is started, initializing a linked list to be executed, an execution thread, a wake-up linked list and an execution thread linked list;
after receiving the source end operation, the target end data synchronization service carries out classification management according to the transaction ID of the operation, and when a certain transaction is submitted, the transaction is stored into the chain table to be executed according to the LSN size sequence of the submitted operation.
3. The transaction merging-based parallel execution method of claim 1, wherein the first commit operation determining step specifically includes:
extracting the next operation of the current transaction B, and judging whether the extracted operation is a commit operation;
if yes, entering a thread submitting step, and if not, entering a thread C searching step.
4. The transaction merging-based parallel execution method according to claim 3, wherein the step of searching the execution thread C specifically comprises:
obtaining a current LSN to be executed, and searching an execution thread C of which the parallel LSN is smaller than the LSN to be executed forward in an execution thread linked list;
if the result is found, entering a circulating judgment step, and if the result is not found, entering a first parallel execution step.
5. The transaction merging-based parallel execution method of claim 4, wherein the loop judgment step specifically comprises:
a rollback step: backing the current operation position and pointing the current transaction B to the next single transaction, and then entering a second submitting operation judgment step;
a second submission operation judgment step: extracting the next operation of the current transaction B and judging whether the extracted operation is a commit operation, if so, entering a rollback step, and if not, entering a parallel LSN judgment step;
parallel LSN judging step: judging whether the parallel LSNs of the execution threads arranged before the current execution thread are all larger than the current LSN to be executed, if so, entering a second parallel execution step, and otherwise, entering a rollback step;
a second parallel execution step: and entering a second submitting operation judgment step after the current operation is executed.
6. The transaction merging-based parallel execution method of claim 5, wherein when the current transaction B points to a next single transaction in the rollback step, if there is no next single transaction, the first transaction A is set as the current transaction B, and the current execution thread is added to a wake-up linked list of the execution thread C to wait for the execution thread C to be committed and then wake up.
7. The parallel execution method based on transaction merging as claimed in any one of claims 1-6, characterized in that when executing thread commit operation in said thread commit step, all other executing threads waiting in the executing thread's wake-up linked list are also woken up, and then the executing thread is removed from the executing thread linked list.
8. A transaction merging-based parallel execution device, characterized by:
comprises at least one processor and a memory, wherein the at least one processor and the memory are connected through a data bus, and the memory stores instructions which can be executed by the at least one processor and are used for completing the parallel execution method based on the transaction combination according to any one of claims 1-7 after being executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111503817.7A CN114297291B (en) | 2021-12-09 | 2021-12-09 | Transaction combination-based parallel execution method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111503817.7A CN114297291B (en) | 2021-12-09 | 2021-12-09 | Transaction combination-based parallel execution method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114297291A CN114297291A (en) | 2022-04-08 |
| CN114297291B true CN114297291B (en) | 2023-02-03 |
Family
ID=80967274
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111503817.7A Active CN114297291B (en) | 2021-12-09 | 2021-12-09 | Transaction combination-based parallel execution method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114297291B (en) |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6745209B2 (en) * | 2001-08-15 | 2004-06-01 | Iti, Inc. | Synchronization of plural databases in a database replication system |
| JP2007219598A (en) * | 2006-02-14 | 2007-08-30 | Nippon Telegr & Teleph Corp <Ntt> | Multiplex database system, its synchronization method, database server, and database server program |
| CN107026880A (en) * | 2016-02-01 | 2017-08-08 | 北京京东尚科信息技术有限公司 | Method of data synchronization and device |
| CN110597663A (en) * | 2019-09-16 | 2019-12-20 | 阿里巴巴集团控股有限公司 | Transaction processing method and device |
| CN111475480B (en) * | 2020-04-13 | 2023-08-15 | 上海达梦数据库有限公司 | Log processing method and system |
| CN111930692B (en) * | 2020-05-28 | 2022-05-13 | 武汉达梦数据库股份有限公司 | Transaction merging execution method and device based on log analysis synchronization |
| CN111858503B (en) * | 2020-06-04 | 2023-03-28 | 武汉达梦数据库股份有限公司 | Parallel execution method and data synchronization system based on log analysis synchronization |
-
2021
- 2021-12-09 CN CN202111503817.7A patent/CN114297291B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN114297291A (en) | 2022-04-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111694798B (en) | Data synchronization method and data synchronization system based on log analysis | |
| CN101807144A (en) | Prospective multi-threaded parallel execution optimization method | |
| CN103995691A (en) | Service state consistency maintenance method based on transactions | |
| CN103559017A (en) | Character string matching method and system based on graphic processing unit (GPU) heterogeneous computing platform | |
| US11269693B2 (en) | Method, apparatus, and electronic device for improving CPU performance | |
| CN114296885A (en) | Transaction combination-based parallel execution method and device | |
| CN100583047C (en) | Method for synchronizing real-time interruption with multiple progress states | |
| CN114297291B (en) | Transaction combination-based parallel execution method and device | |
| CN111858504B (en) | Operation merging execution method based on log analysis synchronization and data synchronization system | |
| CN103377292B (en) | Database result set caching method and device | |
| CN111930692B (en) | Transaction merging execution method and device based on log analysis synchronization | |
| CN105045563B (en) | A kind of method for collision management for speculating nested software transaction storage | |
| CN110990363A (en) | Distributed database multithreading collaborative transaction log playback method and system | |
| CN112035222B (en) | Transaction operation merging execution method and device based on log analysis synchronization | |
| CN112270290B (en) | Dynamic clustering method and device for face images, electronic equipment and storage medium | |
| CN111858503B (en) | Parallel execution method and data synchronization system based on log analysis synchronization | |
| CN117610320A (en) | Directed acyclic graph workflow engine cyclic scheduling method, device and equipment | |
| CN114296887A (en) | Transaction combination-based parallel execution method and device | |
| CN114296886A (en) | Parallel execution method and device based on range row lock | |
| CN109753314B (en) | Configurable automatic flow execution method and system | |
| CN111858626A (en) | Data synchronization method and device based on parallel execution | |
| CN102063291B (en) | Multilevel parallel execution method of speculation thread | |
| CN111930693B (en) | Transaction merging execution method and device based on log analysis synchronization | |
| CN111858505B (en) | Parallel execution method and data synchronization system based on log analysis synchronization | |
| Solaiman et al. | Later validation/earlier write: Concurrency control for resource-constrained systems with real-time properties |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |