CN110807046B - Novel distributed NEWSQL database intelligent transaction optimization method - Google Patents
Novel distributed NEWSQL database intelligent transaction optimization method Download PDFInfo
- Publication number
- CN110807046B CN110807046B CN201911053253.4A CN201911053253A CN110807046B CN 110807046 B CN110807046 B CN 110807046B CN 201911053253 A CN201911053253 A CN 201911053253A CN 110807046 B CN110807046 B CN 110807046B
- Authority
- CN
- China
- Prior art keywords
- transaction
- optimization
- read
- write
- setting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005457 optimization Methods 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000015556 catabolic process Effects 0.000 claims abstract description 11
- 238000006731 degradation reaction Methods 0.000 claims abstract description 11
- 238000002955 isolation Methods 0.000 claims description 32
- 206010000210 abortion Diseases 0.000 claims description 6
- 230000000903 blocking effect Effects 0.000 claims description 3
- 238000002360 preparation method Methods 0.000 claims 2
- 230000001105 regulatory effect Effects 0.000 claims 1
- 239000002253 acid Substances 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000002688 persistence Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention particularly relates to a novel distributed NEWSQL database intelligent transaction optimization method. The novel distributed NEWSQL database intelligent transaction optimization method is based on a manual setting switch starting optimization scheme and based on a data Flow transfer switch state; when a write-read conflict transaction occurs, firstly judging whether to optimize; if not, starting the original SSI transaction executor, if so, starting the RC transaction executor, and after the transaction execution is finished, starting the respective transaction submitter to submit or abort. The novel distributed NEWSQL database intelligent transaction optimization method solves the problem of low concurrency in the transaction writing and reading process existing in the existing distributed NewSQL database, can remarkably improve the data processing efficiency of the database on the premise of not damaging the ACID characteristics of the database, improves the concurrency performance of writing and reading conflict scenes, and achieves the purpose of transaction degradation.
Description
Technical Field
The invention relates to the technical field of database transactions, in particular to a novel intelligent transaction optimization method for a distributed NEWSQL database.
Background
A Transaction (Transaction) is a logical unit of program execution consisting of a series of operations that access and update data in the system. A transaction has four characteristics, Atomicity (Atomicity), Consistency (Consistency), Isolation (Isolation), and persistence (persistence), referred to as the ACID properties of the transaction.
The atomicity means that a transaction is an inseparable work unit, and operations in the transaction either all occur or none occur. Consistency means that the integrity of the data must remain consistent before and after a transaction. The isolation of the transaction means that when a plurality of users access the database concurrently, the database is a transaction opened by each user and cannot be interfered by operation data of other transactions, and the plurality of concurrent transactions are isolated from each other.
In a database system, multiple sessions (sessions) may access the same row of data of the same table of the same database. Thus, for data, it means that a plurality of sessions can apply operations (or read operations or write operations) to the data in the same time period, and different applying sequences of read and write operations may cause conflicting transactions, generate abnormal phenomena such as dirty reading, and be divided into the problems of read-write and write-write.
The concurrency control technology is one of the important technologies for realizing atomicity, consistency and isolation. The essence of concurrency control techniques is to implement correct (guarantee data consistency, guarantee atomicity of transaction operations) and efficient (implement concurrency with "serializability/recoverability/strictness", sacrifice consistency in some cases, or tolerate inconsistencies with low levels of isolation to improve concurrency execution efficiency) scheduling for concurrent transactions.
The multi-version concurrency control technology (MVCC) is not a transaction concurrency control technology that can be used independently, and needs to be implemented by relying on other concurrency control technologies to improve the concurrency control technology and increase the concurrency degree.
The multi-version timestamp ordering mechanism ensures serializability by ordering the commit order of things by the timestamp value of the starting transaction and the rules determined by the protocol. However, in the state that the existing isolation level is SSI, when a write-read transaction conflict occurs, a read transaction enters a "pending" state, which limits the concurrency degree to a great extent, and has a problem of low concurrency.
Because of the characteristics of the distributed multi-nodes, in the distributed system, although each node can know success or failure of its own operation, it cannot know success or failure of the operation of other nodes. When a transaction spans a plurality of nodes, in order to maintain the ACID characteristic of the transaction, a component serving as a coordinator needs to be introduced to uniformly master the operation results of all nodes (called participants) and finally indicate whether the nodes need to actually commit the operation results (such as writing updated data to a disk and the like). The Two-phase Commit (Two-phase Commit) algorithm idea can be summarized as: the participants inform the coordinator of success or failure of the operation, and the coordinator determines whether each participant needs to submit the operation or suspend the operation according to the feedback information of all the participants.
The invention provides a novel distributed NEWSQL database intelligent transaction optimization method, aiming at solving the problem of low concurrency in the transaction writing and reading process in the existing distributed NewSQL database.
Disclosure of Invention
In order to make up for the defects of the prior art, the invention provides a simple and efficient novel distributed NEWSQL database intelligent transaction optimization method.
The invention is realized by the following technical scheme:
a novel distributed NEWSQL database intelligent transaction optimization method is characterized by comprising the following steps: the method comprises the following steps:
step 1, setting a switch starting optimization scheme based on manual work, and transmitting a switch state based on data Flow;
step 2, when a write-read conflict transaction occurs, firstly judging whether optimization is carried out;
if not, starting an original SSI (serialized Snapshot Isolation) transaction executor, if so, starting an RC (Read Committed) transaction executor, and after the transaction execution is finished, starting respective transaction submitters to submit (commit) or abort (abort);
step 3, submitting Commit stage;
the gateway node counts the running state of each node, when all the node states are ready ' prepended ', a write transaction is submitted, if one node state is aborted ', when the optimization is not carried out, the serializable SSI transaction submitter isolated by snapshot aborts the whole write transaction, and after the write transaction is finished, the read transaction continues to run; while optimized, the read committed RC transaction submitter aborts the entire read transaction.
The step 1 specifically comprises the following steps:
(1) inputting SQL sentences of a specified opening optimization scheme at a client;
(2) constructing a distributed logic plan, and splitting a transaction request sent by a client into tree structures;
(3) passing the switch state to the child transaction leaf txn;
(4) transmitting a blocking request basic request into an execution node;
(5) the execution node obtains the sent lockout request basic request and obtains the switch state from the lockout request.
In the step 1, the system supports automatic identification and manual setting of two degradation starting modes; after the system is started, firstly, whether manual setting starting optimization is carried out or not is judged, if yes, the submitted RC transaction executor is directly read, otherwise, the automatic recognition mode is entered, and the system judges whether optimization is needed or not according to the existing transaction execution mode.
In the step 1, in order to meet a complex real scene, after a user starts a client, the time and the mode of the optimization can be selected independently through manual setting, and the optimization setting mode comprises statement setting and session setting;
the statement setting can enable the isolation level of the current transaction started by the user to be an RC isolation level;
the session setting can enable the isolation level of the transaction of the current session opened by the user to be an RC isolation level;
the priority levels of the statement setting and the session setting are sequentially increased, and when a setting mode with low priority level is not actively set, a setting result with high priority level is automatically inherited; the database management system carries out syntactic analysis and lexical analysis to decompose SQL layer statements by reading user optimization statements sent by the client, generates a syntactic analysis tree, and the transaction executor judges whether a degradation switch is released or not and which switch is released according to the analysis tree structure.
In the step 1, under the condition that a user does not carry out autonomous setting, the database management system is in an automatic identification mode, and the optimal scheme selection of the transaction isolation level is automatically carried out aiming at different transaction composition conditions and corresponding scenes; the user can automatically identify whether the optimization is needed or not by the system by opening an automatic identification mode switch of the global attribute in the database; when a write-read conflict occurs, the isolation level of the read transaction can be set according to the transaction condition and the security requirement under the condition that manual setting is not performed and the automatic identification mode is started.
In the step 2, whether optimization is performed or not is automatically judged, and the method specifically comprises the following steps:
(1) inputting an SQL statement which specifies to start an automatic recognition (auto) mode at a client;
(2) the read transaction finds that the read data has a write intention (write intent), and triggers a judgment mechanism;
(3) judging whether manual setting is set or not, if so, directly reading an actuator which already submits the RC affair, and otherwise, judging whether an automatic identification (auto) mode is set or not;
if an automatic identification (auto) mode is set, the read transaction obtains an automatic optimization authority, an optimization scheme is started, an old data version is obtained based on a write intention (write intent), the obtained old data version is endowed to the read transaction, and the read transaction continues to run;
if auto-recognition (auto) mode is not set, then the read transaction is not optimized and enters a wait "pending" state.
In the step 2, the read transaction T2 in the optimized transaction executor does not enter a waiting "pending" state, but obtains a timestamp (timestamp) T1 of the write transaction through a transaction id (txnid) generating a write intention stored in the write intention (write intent), and searches for data with the timestamp (timestamp) closest to T1, that is, the latest committed version, by searching for existing versions with the same index, so as to reduce the waiting time and improve the concurrency; after the operation of each sub-transaction of the read transaction on the execution node is finished, the operation success state of the node is changed from work to ready to prepare, and the operation failure state is changed to abort abortive, and then a read committed RC transaction submitter is started.
In the step 2, when an automatic identification (auto) mode is not set, the end of the write transaction is waited, after the operation of each sub-transaction of the write transaction on the execution node is ended, the operation success state of the node is changed from work to prepare, and the operation failure state is changed to abort, and then a serializable SSI transaction submitter with snapshot isolation is started.
The invention has the beneficial effects that: the novel distributed NEWSQL database intelligent transaction optimization method solves the problem of low concurrency in the transaction writing and reading process existing in the existing distributed NewSQL database, can remarkably improve the data processing efficiency of the database on the premise of not damaging the ACID characteristics of the database, improves the concurrency performance of writing and reading conflict scenes, and achieves the purpose of transaction degradation.
Drawings
FIG. 1 is a diagram of a write-read transaction conflict.
FIG. 2 is a schematic diagram of the novel distributed NEWSQL database intelligent transaction optimization method of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention more apparent, the present invention is described in detail below with reference to the embodiments. It should be noted that the specific embodiments described herein are only for explaining the present invention and are not used to limit the present invention.
A Timestamp (Timestamp) identifying the time of execution of the data, operation, or transaction.
The KV storage refers to the step of finally converting the MVCC multi-version control data into KV format data for storage. Each piece of data consists of two parts, a key (key) consisting of a timestamp (timestamp) and a lookup index (index) of the value (value). All historical versions (old versions of data) with the index, i.e., the value, can be traversed prefixed by the index (index) of the data.
When a Write transaction writes a piece of data, a Write intent (Write intent) is generated, which consists of the key to which the data was written and the transaction id (txnid) that generated the Write intent. When the transaction generating the write intention is ended, the write intention is not immediately eliminated, but when the new transaction performs a new operation on the data, the write intention is found to exist and the corresponding txn of the write intention is ended, and then the write intention is eliminated.
The process of forming a write-read transaction conflict can be seen in figure 1. When the client initiates a write transaction T1 for data a, it identifies that a write operation was performed on the piece of data by constructing a write intent. A read transaction T2 for data a is initiated, a T2 transaction first determines whether data a has a write intention, if not, data a is directly read, if so, the transaction is searched by a transaction ID which is stored by the write intention and generates the write intention, and when the transaction is not finished (commit or abort commit), the transactions T1 and T2 become conflict transactions.
The novel distributed NEWSQL database intelligent transaction optimization method comprises the following steps:
step 1, a switch starting optimization scheme is set manually, and a switch state is transmitted based on data Flow;
step 2, when a write-read conflict transaction occurs, judging whether optimization is carried out or not;
if not, starting an original SSI (serialized Snapshot Isolation) transaction executor, if so, starting an RC (Read Committed) transaction executor, and after the transaction execution is finished, starting respective transaction submitters to submit (commit) or abort (abort);
step 3, submitting Commit stage;
the gateway node counts the running state of each node, when all the node states are ready ' prepended ', a write transaction is submitted, if one node state is aborted ', when the optimization is not carried out, the serializable SSI transaction submitter isolated by snapshot aborts the whole write transaction, and after the write transaction is finished, the read transaction continues to run; while optimized, the read committed RC transaction submitter aborts the entire read transaction.
The step 1 specifically comprises the following steps:
(1) inputting a specified SQL statement for starting an optimization scheme at a client;
(2) constructing a distributed logic plan, and splitting a transaction request sent by a client into tree structures;
(3) passing the switch state to the child transaction leaf txn;
(4) transmitting a blocking request basic request into an execution node;
(5) the execution node obtains the sent lockout request basic request and obtains the switch state from the lockout request.
In the step 1, the system supports automatic identification and manual setting of two degradation starting modes; after the system is started, whether manual setting starting optimization is carried out or not is judged firstly, if yes, the system directly enters an executor for reading submitted RC affairs, otherwise, the system enters an automatic recognition mode, and whether optimization is needed or not is judged according to the existing affair execution mode.
In the step 1, in order to meet a complex real scene, after a user starts a client, the time and the mode of the optimization can be selected independently through manual setting, and the optimization setting mode comprises statement setting and session setting;
the statement setting can enable the isolation level of the current transaction started by the user to be the RC isolation level;
the session setting can enable the isolation level of the transaction of the current session opened by the user to be an RC isolation level;
the priority levels of the statement setting and the session setting are sequentially increased, and when a setting mode with low priority level is not actively set, a setting result with high priority level is automatically inherited; the database management system carries out syntactic analysis and lexical analysis to decompose SQL layer statements by reading user optimization statements sent by the client, generates a syntactic analysis tree, and the transaction executor judges whether a degradation switch is released or not and which switch is released according to the analysis tree structure.
In the step 1, under the condition that a user does not carry out autonomous setting, the database management system is in an automatic identification mode, and the optimal scheme selection of the transaction isolation level is automatically carried out aiming at different transaction composition conditions and corresponding scenes; the user can automatically identify whether the optimization is needed or not by the system by opening an automatic identification mode switch of the global attribute in the database; when a write-read conflict occurs, the isolation level of the read transaction can be set according to the transaction condition and the security requirement under the condition that manual setting is not performed and the automatic identification mode is started.
In the step 2, whether optimization is performed or not is automatically judged, and the method specifically comprises the following steps:
(1) inputting an SQL statement which specifies to start an automatic recognition (auto) mode at a client;
(2) the read transaction finds that the read data has a write intention (write intent), and triggers a judgment mechanism;
(3) judging whether manual setting is set or not, if so, directly reading an actuator which already submits the RC affair, and otherwise, judging whether an automatic identification (auto) mode is set or not;
if an automatic identification (auto) mode is set, the read transaction obtains an automatic optimization authority, an optimization scheme is started, an old data version is obtained based on a write intention (write intent), the obtained old data version is endowed to the read transaction, and the read transaction continues to run;
if auto-recognition (auto) mode is not set, then the read transaction is not optimized and enters a wait "pending" state.
In the step 2, the read transaction T2 in the optimized transaction executor does not enter a waiting "pending" state, but obtains a timestamp (timestamp) T1 of the write transaction through a transaction id (txnid) generating a write intention stored in the write intention (write intent), and searches for data with the timestamp (timestamp) closest to T1, that is, the latest committed version, by searching for existing versions with the same index, so as to reduce the waiting time and improve the concurrency; after the operation of each sub-transaction of the read transaction on the execution node is finished, the operation success state of the node is changed from work to ready to prepare, and the operation failure state is changed to abort abortive, and then a read committed RC transaction submitter is started.
In the step 2, when an automatic identification (auto) mode is not set, the end of the write transaction is waited, after the operation of each sub-transaction of the write transaction on the execution node is ended, the operation success state of the node is changed from work to prepare, and the operation failure state is changed to abort, and then a serializable SSI transaction submitter with snapshot isolation is started.
Example 1
According to the novel distributed NEWSQL database intelligent transaction optimization method, the passing rate in more than 50 designed scenes reaches 100%. Here, the simplest and classic scenario among them will be expressed, and the transaction scenario is shown in the following table:
TABLE x.1 READ-WRITE SCENES
| Affair 1 | Affairs 2 (demotion) |
| W(X)-Update | |
| R(X) | |
| Commit | |
| R(X) |
This is a classic write-read scenario in which transaction 1 writes to X and transaction 2 reads, and in the event of transaction downgrades, transaction 2 will be able to read the old version of data immediately before transaction 1 updates, and after transaction 1 commits, transaction 2 will be able to read the new version. Before the transaction is degraded, the first read operation of the transaction 2 is blocked until the transaction is committed, which greatly reduces the data processing efficiency of the database.
The transaction degradation scheme also passes OLTP test and TPCC test, and the test results are shown in the following table:
TABLE x.2 OLTP test results
| til | read | write | total | tps | qps |
| RC | 471184 | 0 | 538496 | 559.84(33656) | 8957.42(538496) |
| SSI | 387002 | 0 | 442288 | 459.82(27643) | 7357.11(442288) |
From the test results, it can be seen that the efficiency is improved by 17.86% by the fact that tps (transaction operand per second) for transaction degradation is significantly higher than that of the snapshot-isolated serializable SSI transaction executor.
TABLE x.3 TPCC test results
| elapsed | tpmC | efc | avg(ms) | p50(ms) | p90(ms) | p95(ms) | p99(ms) | pMax | |
| SSI | 279.2s | 125.9 | 97.9% | 114.3 | 75.5 | 176.2 | 352.3 | 805.3 | 838.9 |
| RC | 286.3s | 131.8 | 102.5% | 103.2 | 71.3 | 167.8 | 419.4 | 671.1 | 704.6 |
According to the test result, it can be seen that tps (1/AVG) of the read committed RC transaction executor is higher than that of the snapshot isolated serializable SSI transaction executor, which is improved by 9.7%.
In summary, the novel distributed NEWSQL database intelligent transaction optimization method can remarkably improve the data processing efficiency of the database on the premise of not destroying the ACID characteristics of the database, and perfectly achieves the purpose of transaction degradation.
Compared with the prior art, the novel distributed NEWSQL database intelligent transaction optimization method has the following characteristics:
firstly, aiming at a write-read operation scene capable of concurrent processing in a database, by reducing the isolation level, the conflict between write-read transactions is solved, and the concurrent operation of the write-read transactions is realized, while for a scene incapable of degrading the concurrency, the same processing mechanism as the original database is still maintained. On the premise of ensuring that the ACID characteristic of the database is not damaged, the concurrency performance of the writing and reading conflict scene is improved.
Secondly, the database can automatically judge the possibility of transaction optimization, so that the effect of automatically adjusting the isolation level of the transaction is achieved, and the user is allowed to actively set the isolation level at the terminal to meet the operation requirement of the user.
Thirdly, the disaster tolerance of the database is guaranteed, the database can be backtracked when data is damaged or even destroyed due to emergencies such as hardware damage, manual misoperation, sudden power failure and the like, and the data is restored to the state before the emergencies, so that the safety of the data of the database is guaranteed.
The above-described embodiment is only one specific embodiment of the present invention, and general changes and substitutions by those skilled in the art within the technical scope of the present invention are included in the protection scope of the present invention.
Claims (7)
1. A novel distributed NEWSQL database intelligent transaction optimization method is characterized by comprising the following steps:
step 1, a switch starting optimization scheme is set manually, and a switch state is transmitted based on data Flow;
the system supports automatic identification and manual setting of two degradation starting modes; after the system is started, firstly judging whether manual setting starting optimization is carried out or not, if so, directly entering into an executor for reading submitted RC affairs, otherwise, entering into an automatic identification mode, and judging whether optimization is needed or not by the system according to the existing affair execution mode;
step 2, when a write-read conflict transaction occurs, firstly judging whether optimization is carried out;
if not, starting the serializable SSI transaction executor isolated by the original snapshot, if so, starting a committed RC transaction executor, and starting respective transaction submitters to submit or abort after the transaction execution is finished;
step 3, a submitting stage;
the gateway node counts the running states of all nodes, submits the write transaction when all the node states are ready, if one node state is abort, the serialization SSI transaction submitter isolated by the snapshot aborts the whole write transaction when not optimized, and the read transaction continues to run after the write transaction is finished; while optimized, the read committed RC transaction submitter aborts the entire read transaction.
2. The novel distributed NEWSQL database intelligent transaction optimization method of claim 1, wherein: the step 1 specifically comprises the following steps:
(1) inputting SQL sentences of a specified opening optimization scheme at a client;
(2) constructing a distributed logic plan, and splitting a transaction request sent by a client into tree structures;
(3) passing the switch state to the child transaction leaf txn;
(4) transmitting a blocking request basic request into an execution node;
(5) the execution node obtains the sent lockout request basic request and obtains the switch state from the lockout request.
3. The novel distributed NEWSQL database intelligent transaction optimization method of claim 1, wherein: in the step 1, in order to meet a complex real scene, after a user starts a client, the time and the mode of the optimization can be selected independently through manual setting, and the optimization setting mode comprises statement setting and session setting;
the statement setting can enable the isolation level of the current transaction started by the user to be the RC isolation level;
the session setting can enable the isolation level of the transaction of the current session opened by the user to be an RC isolation level;
the priority levels of the statement setting and the session setting are sequentially increased, and when a setting mode with low priority level is not actively set, a setting result with high priority level is automatically inherited; the database management system carries out syntactic analysis and lexical analysis to decompose SQL layer statements by reading user optimization statements sent by the client, generates a syntactic analysis tree, and the transaction executor judges whether a degradation switch is released or not and which switch is released according to the analysis tree structure.
4. The novel distributed NEWSQL database intelligent transaction optimization method of claim 1, wherein: in the step 1, under the condition that a user does not carry out autonomous setting, the database management system is in an automatic identification mode, and the optimal scheme selection of the transaction isolation level is automatically carried out aiming at different transaction composition conditions and corresponding scenes; the user can automatically identify whether the optimization is needed or not by the system by opening an automatic identification mode switch of the global attribute in the database; when a write-read conflict occurs, the isolation level of the read transaction can be set according to the transaction condition and the security requirement under the condition that manual setting is not performed and the automatic identification mode is started.
5. The novel distributed NEWSQL database intelligent transaction optimization method of claim 1, wherein: in the step 2, whether optimization is performed or not is automatically judged, and the method specifically comprises the following steps:
(1) inputting SQL sentences which are regulated to start an automatic recognition mode at a client;
(2) the reading affairs find that the data read by the affairs has a writing intention, and trigger a judgment mechanism;
(3) judging whether manual setting is set or not, if so, directly reading an actuator which already submits the RC affair, and otherwise, judging whether an automatic identification mode is set or not;
if the automatic identification mode is set, the read transaction obtains automatic optimization permission, an optimization scheme is started, an old data version is obtained based on the writing intention, the obtained old data version is endowed to the read transaction, and the read transaction continues to run;
if the automatic identification mode is not set, the read transaction is not optimized and enters a waiting state.
6. The novel distributed NEWSQL database intelligent transaction optimization method of claim 5, wherein: in step 2, the read transaction T2 in the optimized transaction executor does not enter a wait state, but obtains the timestamp T1 of the write transaction by the transaction ID generating the write intent stored in the write intent, and reduces the wait time and improves the concurrency by searching for the existing version with the same index and finding the data with the timestamp closest to T1, that is, the latest committed version; after the sub-transactions of the read transaction are operated on the execution node, the operation success state of the node is changed from work to preparation, the failure state is changed to termination, and then the read committed RC transaction submitter is started.
7. The novel distributed NEWSQL database intelligent transaction optimization method of claim 5, wherein: in step 2, when the automatic identification mode is not set, the end of the write transaction is waited, after the operation of each sub-transaction of the write transaction on the execution node is ended, the operation success state of the node is changed from work to preparation, and the failure state is changed to termination, and then the serializable SSI transaction submitter with snapshot isolation is started.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911053253.4A CN110807046B (en) | 2019-10-31 | 2019-10-31 | Novel distributed NEWSQL database intelligent transaction optimization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911053253.4A CN110807046B (en) | 2019-10-31 | 2019-10-31 | Novel distributed NEWSQL database intelligent transaction optimization method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110807046A CN110807046A (en) | 2020-02-18 |
| CN110807046B true CN110807046B (en) | 2022-06-07 |
Family
ID=69489827
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911053253.4A Active CN110807046B (en) | 2019-10-31 | 2019-10-31 | Novel distributed NEWSQL database intelligent transaction optimization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110807046B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112966047B (en) * | 2021-03-05 | 2023-01-13 | 上海沄熹科技有限公司 | Method for realizing table copying function based on distributed database |
| CN113867634B (en) * | 2021-09-24 | 2022-09-23 | 北京百度网讯科技有限公司 | Data reading method and device, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101127045A (en) * | 2007-09-25 | 2008-02-20 | 中兴通讯股份有限公司 | Database repeatable reading implement method, device and database management system |
| CN101356543A (en) * | 2005-08-10 | 2009-01-28 | 洛克威尔自动控制技术股份有限公司 | Enhanced controller utilizing RFID technology |
| CN104793988A (en) * | 2014-01-20 | 2015-07-22 | 阿里巴巴集团控股有限公司 | Cross-database distributed transaction implementation method and device |
| CN108710638A (en) * | 2018-04-13 | 2018-10-26 | 上海交通大学 | A kind of Distributed concurrency control method and system based on mixing RDMA operation |
| CN109977171A (en) * | 2019-02-02 | 2019-07-05 | 中国人民大学 | A kind of distributed system and method guaranteeing transaction consistency and linear consistency |
| CN110018850A (en) * | 2017-12-30 | 2019-07-16 | 英特尔公司 | For can configure equipment, the method and system of the multicast in the accelerator of space |
| CN110196856A (en) * | 2019-05-09 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of distributed data read method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7051028B2 (en) * | 2000-11-15 | 2006-05-23 | Ndsu-Research Foundation | Concurrency control in high performance database systems |
-
2019
- 2019-10-31 CN CN201911053253.4A patent/CN110807046B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101356543A (en) * | 2005-08-10 | 2009-01-28 | 洛克威尔自动控制技术股份有限公司 | Enhanced controller utilizing RFID technology |
| CN101127045A (en) * | 2007-09-25 | 2008-02-20 | 中兴通讯股份有限公司 | Database repeatable reading implement method, device and database management system |
| CN104793988A (en) * | 2014-01-20 | 2015-07-22 | 阿里巴巴集团控股有限公司 | Cross-database distributed transaction implementation method and device |
| CN110018850A (en) * | 2017-12-30 | 2019-07-16 | 英特尔公司 | For can configure equipment, the method and system of the multicast in the accelerator of space |
| CN108710638A (en) * | 2018-04-13 | 2018-10-26 | 上海交通大学 | A kind of Distributed concurrency control method and system based on mixing RDMA operation |
| CN109977171A (en) * | 2019-02-02 | 2019-07-05 | 中国人民大学 | A kind of distributed system and method guaranteeing transaction consistency and linear consistency |
| CN110196856A (en) * | 2019-05-09 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of distributed data read method and device |
Non-Patent Citations (1)
| Title |
|---|
| 分布式数据库中一致性与可用性的关系;朱涛等;《软件学报》;20171204;第131-149页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110807046A (en) | 2020-02-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9892153B2 (en) | Detecting lost writes | |
| US10503699B2 (en) | Metadata synchronization in a distrubuted database | |
| US20180075086A1 (en) | Asynchronous Shared Application Upgrade | |
| CN104793988B (en) | The implementation method and device of integration across database distributed transaction | |
| US8868577B2 (en) | Generic database manipulator | |
| US5953719A (en) | Heterogeneous database system with dynamic commit procedure control | |
| JP4114861B2 (en) | Method for managing distributed savepoints across multiple DBMSs in a distributed transaction | |
| US7233947B2 (en) | Timestamping in databases | |
| US9747356B2 (en) | Eager replication of uncommitted transactions | |
| US9626394B2 (en) | Method for mass-deleting data records of a database system | |
| CN109992628B (en) | Data synchronization method, device, server and computer readable storage medium | |
| CN110807046B (en) | Novel distributed NEWSQL database intelligent transaction optimization method | |
| CN107870954B (en) | Method and device for expanding distributed database | |
| CA2167790A1 (en) | Relational database system and method with high data availability during table data restructuring | |
| JPH08241235A (en) | Process for detection of write collision | |
| JPH08241234A (en) | Database management method | |
| CN112883045B (en) | Database transaction splitting execution method and device | |
| US7225206B2 (en) | System and method for reorganizing stored data | |
| JP2022531867A (en) | Data reading methods, devices, computer devices and computer programs | |
| CN113391885A (en) | Distributed transaction processing system | |
| JPH032939A (en) | Data control method | |
| CN112100188B (en) | Concurrency control method and system based on machine learning | |
| EP3692452B1 (en) | Consistency across key value stores with shared journals | |
| WO2001013202A2 (en) | Concurrent commit lock | |
| CN109710629A (en) | Data access method, device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 250100 No. 1036 Tidal Road, Jinan High-tech Zone, Shandong Province, S01 Building, Tidal Science Park Applicant after: Inspur cloud Information Technology Co.,Ltd. Address before: 250100 No. 1036 Tidal Road, Jinan High-tech Zone, Shandong Province, S01 Building, Tidal Science Park Applicant before: Tidal Cloud Information Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |