CN112148930B

CN112148930B - Method, system and medium for graph database system transaction processing based on RTM

Info

Publication number: CN112148930B
Application number: CN202011041143.9A
Authority: CN
Inventors: 陈榕; 李嘉麒; 陈海波; 臧斌宇; 管海兵
Original assignee: Shanghai Jiaotong University
Current assignee: Shanghai Jiaotong University
Priority date: 2020-09-28
Filing date: 2020-09-28
Publication date: 2023-01-06
Anticipated expiration: 2040-09-28
Also published as: CN112148930A

Abstract

The invention discloses a graph database system transaction processing method based on RTM, which comprises the following steps: s1: using RTM to submit the transaction; s2: judging whether the transaction submission is successful, and finishing the transaction processing when the judgment result is successful; and when the judgment result is unsuccessful, the step S3 is carried out: s3: judging the number of attempts, and when the number of attempts is more than or equal to the specified number, performing transaction processing by using a rollback method to finish the processing of the specified transaction; when the number of attempts is less than the specified number, the process returns to step S1. In addition, the invention discloses the system for transaction processing of the graph database system based on RTM and a computer readable storage medium storing a computer program. Compared with the large amount of expenditure and defects existing in the traditional locking mechanism, the RTM is used for completing most transactions, and the graph database has better graph database performance and expandability and improves the throughput of a system.

Description

Method, system and medium for graph database system transaction processing based on RTM

Technical Field

The invention relates to the field of transaction processing of graph database systems, in particular to an optimized transaction processing method based on RTM.

Background

Transaction processing is one of the important basic functions of modern graph database systems, and the performance of transaction processing directly determines the response speed and throughput rate of the whole graph database system. Most of the existing graph database system transactions are based on the traditional locking mechanism. However, the complexity of the graph data in reality is high, and due to the flexibility of the expression mode of the graph data, an association relationship may exist between any data nodes. Given the number of points n, the maximum number of associations, i.e., edges, of a single type can be n ² And under a multi-type relation scene, the incidence relation is richer.

In such a complex data context, it is difficult for the locking mechanism to achieve its intended effect. Coarse-grained locks, due to the high associativity of graph data, will cause their concurrent execution to be very conflicting, since they do not themselves serve as much partitioning data. The coarse-grained lock is more difficult to implement, and easily causes problems such as deadlock which is difficult to find. In addition, due to the power-law distribution (power-law) characteristic of graph data, the probability of collision is actually small when no hot node is involved. Most nodes in the graph data are not hot nodes, which results in a small portion of the locks having serious contention conflicts and a large portion of the locks having little effect. Therefore, in the context of graph databases, conventional lock mechanisms suffer from the disadvantages of poor performance and low scalability.

Transactional Memory (Transactional Memory) can solve the above problems well. Transactional memory assumes that most transactions do not conflict with each other when executed concurrently, and that the commit of a transaction is cancelled only when actually there is a conflict, which assumes the power-law distribution characteristics of graph data. By transacting different parallel processes and replacing a locking mechanism with a transactional operation, the method can obviously reduce programming complexity, avoid the defects brought by the locking mechanism and enjoy the benefits brought by the transactional characteristics. RTM (Restricted Transactional Memory) provides CPU instruction set level support to allow hardware Transactional Memory related programming to be easily performed by a user.

Although RTM is capable of handling concurrent programs more efficiently than traditional lock mechanisms, it also has some functional drawbacks due to hardware limitations. For example, some system events (e.g., interrupts, system calls, etc.) may unconditionally abort an executing transaction; in addition, the RTM maintained read and write set is also limited, and if a RTM code block reads and writes too much data, the execution of the block of code is suspended and the system state is rolled back to before the RTM code block is executed. Even if everything is normal, RTM techniques do not themselves guarantee that the involved transactions will be committed, so there is still a need to provide a rollback handler. By combining with the existing lock-based transaction processing method, the transaction can be guaranteed to be successfully submitted finally.

Therefore, how to combine the RTM technology with the conventional method, to take advantage of the performance and scalability of RTM with fewer conflicts, to avoid the inherent defects of the lock mechanism on the graph database, and to ensure that the transaction can be finally processed by the conventional stabilization method becomes a technical problem to be solved by those skilled in the art.

Disclosure of Invention

In view of the drawbacks of the prior art, the present invention aims to provide an optimized method for transaction processing of a RTM-based graph database system.

In order to achieve the above purpose, the invention provides the following technical scheme:

in a first aspect, the present invention provides a method for transaction processing in a graph database system based on RTM, which includes the steps of:

step S1: performing RTM transaction submission;

step S2: judging whether the transaction submission is successful, and finishing the transaction processing when the judgment result is successful; and when the judgment result is unsuccessful, the step S3 is carried out:

and step S3: judging the number of times of attempts of transaction submission, and when the number of times of attempts is more than or equal to the specified number of times, performing transaction processing by using a rollback method to complete the processing of the specified transaction; when the number of attempts is less than the specified number, the process returns to step S1..

In the technical scheme of the invention, considering that because a great amount of edges exist in graph structure data as associated information, the problems that deadlock and deadlock positions are easy to occur in a traditional concurrent transaction processing program based on a lock mechanism and are difficult to check are solved, so that the performance and expandability of application are greatly limited, the inventor processes a graph database system by adopting an RTM (remote transfer molding) technology, so that the problems in the prior art are solved, the method can complete most of transactions by using a hardware transaction memory in an RTM (remote transfer molding) in a scene with less data conflict among the transactions, and compared with the large amount of expenses and defects existing in the traditional lock mechanism, the method realizes better graph database performance and expandability and increases the throughput of the system.

Preferably, before step S1, the method further comprises the steps of:

step S0: the RTM code block parameters are initialized.

Preferably, in the step S3, the threshold of the number of attempts is set according to the time proportion of the fast and slow method adopted by the transaction processing and the stability degree of the running environment.

Preferably, in the step S2, before determining whether the transaction commit is successful, the processing module further performs the following steps:

inquiring whether a rollback program is currently executed or not, and when the rollback program is executed, self-blocking to wait for the rollback program to be executed;

when the rollback program is not executed, obtaining a rollback lock, putting the rollback lock into a read-write set, performing transaction processing, judging whether the read-write set is changed, and when the read-write set is judged to be changed, judging that the transaction submission fails; and (4) directly entering step S3, and entering step S3 after transaction submission is carried out when the read-write set is judged not to be changed.

Preferably, in the step S3, when the rollback method is used, it is determined whether the rollback lock is held by a thread, and if not, the rollback lock is obtained, the transaction is performed, and then the rollback lock is released; if the thread is held by other threads, the above steps are performed after the rollback lock is released.

In the above scheme, whether the value of the rollback lock is null is checked to determine whether other rollback programs are currently executing, so as to ensure that no RTM code block starts to be executed during the execution of the rollback handler, and simultaneously ensure that a plurality of rollback handlers do not execute simultaneously to generate a conflict.

When a variable is newly created in the RTM code block and a rollback lock is assigned to the code block, the external shared variable is automatically placed in the RTM read set by reading the external shared variable. According to different RTM management mechanisms, the comparison check is carried out on the values in the read-write set and external variables in the running process or in the submission process, if a conflict occurs, an RTM conflict exception is generated, and therefore the RTM execution is finished.

In each embodiment, the maximum number of allowed abnormal failures of the RTM method (i.e., the determination as to whether the number of attempts is equal to or greater than the predetermined number in step S3) may be defined according to the specific situation of each embodiment, and when the number of consecutive failures exceeds the threshold value of the predetermined number, it is determined that the RTM method cannot successfully execute the transaction, and the rollback processing procedure is used instead. The reasonable threshold value can reduce the number of rollback processing times caused by system accidental factors and improve the throughput of the system. The specific setting can be set according to the time ratio of the two fast and slow methods adopted and the stability degree of the operating environment, such as 20.

Considering that RTM itself does not guarantee that the contained transaction will eventually be successfully committed, in some embodiments, the most stable method may be used as a rollback handler to guarantee the successful commit rate of the whole program, for example: conventional lock mechanism transactions are used as the rollback handler, although some other method of stably performing transactions may be used instead as long as the speed is within an acceptable range.

Preferably, the retraction lock is a coarse-grained lock, and the granularity value is table-scale.

In a second aspect, the present invention provides a system for graph database system transaction processing based on RTM, the system comprising:

a storage module that stores data;

and a processing module that performs the above-described method.

Preferably, the processing module is implemented by using an instruction set embedded in the processor, for example, a hardware transactional memory function embedded in the processor thereof issued by Intel may be used.

In a third aspect, the present invention provides a computer-readable storage medium storing a computer program, which when executed by a processor, implements the method for transaction processing in an RTM-based graph database system.

Compared with the prior art, the invention has the following beneficial effects:

1. the invention uses RTM to process most affairs of graph database, to avoid the defects of lock mechanism, such as graph database performance problem caused by priority inversion, deadlock, livelock, and high cost of lock mechanism. Because the RTM replaces a locking mechanism realized by software, under the graph database scene with less data conflict, the performance which is better and faster than that of the locking mechanism can be obtained, the throughput rate of the whole graph database system is improved, and the delay of transaction execution is greatly reduced.

2. The invention combines the new RTM transaction processing method with the traditional lock-based transaction processing method, thereby solving the defect that the RTM can not ensure that the transaction can be submitted certainly, and simultaneously obtaining the advantages of the RTM. In the worst case, the new bonding method will perform comparable to the conventional method.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a general logic flow diagram of a method for transaction processing based on a RTM graph database system in accordance with one embodiment of the present invention;

FIG. 2 schematically illustrates a portion of the logic flow of an RTM in one embodiment of a method for RTM-based graph database system transaction processing according to the present invention;

fig. 3 schematically shows part of the logic of a rollback procedure in an embodiment of a RTM-based graph database system transaction method according to the present invention.

Detailed Description

The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications can be made by persons skilled in the art without departing from the spirit of the invention. All falling within the scope of the present invention.

Example 1

It should be noted that, in the embodiment, the RTM is an Intel, but it should be noted that all processors supporting the RTM instruction set can be used in the technical solution of the present invention.

FIG. 1 is a general logic flow diagram of a method for transaction processing based on a RTM-based graph database system in accordance with the present invention.

As shown in fig. 1, in the present embodiment, the method includes the following steps:

step S1: using RTM to submit the transaction;

and step S3: judging the number of times of attempts of transaction submission, and when the number of times of attempts is greater than or equal to the specified number of times, performing transaction processing by using a rollback method to complete the processing of the specified transaction; when the number of attempts is less than the specified number, the process returns to step S1..

When the transaction processing is submitted in step S1, an RTM is used for the transaction processing, and as to the flow of this step, see fig. 2.

As shown in fig. 2, when the RTM is used for transaction processing, the following steps are performed:

s1.1: reading and returning the unlocking by the program and judging whether the unlocking is occupied currently, and calling the program if the unlocking is not occupied_ xbegin() The method enters the RTM code block and enters S1.2, if the fallback lock is already occupied, then the _ xabort () method is called to show that this RTM call is interrupted, and S2.1 is entered.

S1.2: the rollback lock is stored in the RTM code block so that it is placed in the read-write set.

S1.3: specific transactional code portion execution is performed.

S1.4: and calling a _ xend () method to attempt to submit, wherein the RTM automatically compares and judges whether all contents in the read-write set including the rollback lock are modified, and if the contents in the read-write set including the rollback lock are modified, the RTM operation is forcibly interrupted and the transaction is considered to fail. If not, a commit step can be entered.

It should be noted that, in the RTM code area execution part, if the number and size of values to be saved therein exceed the maximum value of the read-write set, or any type of system interrupt occurs, the RTM operation will be forcibly interrupted, and is regarded as a transaction failure. Therefore, it is necessary to ensure that the data involved in the RTM block all need to be read and written, that is, the amount of data involved in this transaction that needs to be read and written is reduced as much as possible, and in addition, the operation amount of the RTM code area is reduced once, which can also significantly reduce the occurrence probability of system interruption during execution. If necessary, one large RTM operation can be split into several small RTM operations for execution.

The steps of using a rollback handler to perform transactions are shown in fig. 3:

s2.1: the program reads back the unlock and determines whether it is currently occupied by other threads.

S2.2: if the backspacing lock is not occupied currently, entering step 2.3; if the rollback lock is currently occupied, it blocks itself until the rollback lock is released.

S2.3: the program acquires and occupies the rollback lock and then proceeds with the specific transaction code section.

S2.4: the transaction is committed and the rollback lock is released.

To ensure low complexity of the code and stability of the rollback handler, the rollback handler is typically a transactional method based on a coarse-grained lock implementation. In most cases, the rollback handler should not be invoked. The large number of call rollback handlers will roll the performance of the entire system back to the traditional coarse-grained lock scheme. In addition, the method should ensure that under the condition of normal operation of the system, the transaction can be stably submitted depending on a rollback processing program after multiple execution attempts of the RTM method fail.

In addition, it should be noted that _ xbegin (), _ xabort (), and _ xend () are built-in APIs (application programming interfaces) in Intel new processors, and the calculation and processing procedures thereof are known to those skilled in the art, and therefore, will not be described herein again.

Example 2

In this embodiment, a system for transaction processing of a RTM-based graph database system is provided, the system comprising:

a storage module storing data;

and a processing module that performs the method of the steps of:

step S0: initializing RTM code block parameters;

step S1: using RTM to submit the transaction;

and step S3: judging the number of times of attempts of transaction submission, and when the number of times of attempts is greater than or equal to the specified number of times, performing transaction processing by using a rollback method to complete the processing of the specified transaction; when the number of attempts is less than the specified number, the process returns to step S1, where the threshold of the number of attempts is set according to the time ratio of the fast and slow methods adopted by the transaction processing and the stability degree of the operating environment, for example, in this embodiment, the value may be set to 20..

And, in the step S2, when determining whether the transaction submission is successful, the processing module further executes the following steps:

when the rollback program is not executed, obtaining a rollback lock, putting the rollback lock into a read-write set, performing transaction processing, judging whether the read-write set is changed, and when the read-write set is judged to be changed, judging that the transaction submission fails; and (4) directly entering the step (S3), and entering the step (S3) after transaction submission is carried out when the read-write set is judged not to be changed.

In addition, in the step S3, when the rollback method is used, it is determined whether the rollback lock is held by a thread, and if not, the rollback lock is retrieved, a transaction is performed, and the rollback lock is released; if the thread is held by other threads, the above steps are performed after the rollback lock is released.

In this embodiment, the processing module is implemented by an instruction set embedded in the processor.

In summary, the RTM-based graph database system transaction optimization method provided by the present invention can avoid the defects of both RTM and lock mechanisms to the greatest extent, while retaining the advantages of both methods. In a graph database scenario with less data collision, the associative method can achieve higher throughput and better performance than the traditional method.

The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims

1. A method for graph database system transaction processing based on RTM, comprising the steps of:

step S1: performing RTM transaction submission;

step S2: judging whether the transaction submission is successful or not by using the RTM, and finishing the transaction processing when the judgment result is successful; when the judgment result is unsuccessful, the step S3 is carried out;

and step S3: judging the number of times of attempts of transaction submission, and when the number of times of attempts is more than or equal to the specified number of times, performing transaction processing by using a rollback method to complete the processing of the specified transaction; when the number of attempts is less than the designated number, returning to the step S1;

before the step S1, the method also comprises the following steps:

step S0: initializing RTM code block parameters;

in step S3, a threshold of the number of attempts is set according to a time ratio of a fast and slow method adopted for the transaction processing and a stability degree of the operating environment;

in step S2, before determining whether the transaction submission is successful, the processing module further performs the following steps:

when the rollback program is not executed, acquiring the rollback lock, placing the rollback lock into a read-write set, performing transaction processing, judging whether the read-write set is changed, and when the read-write set is judged to be changed, considering that the transaction submission fails, and directly entering the step S3; when the read-write set is judged not to be changed, the step S3 is entered after the transaction is submitted;

when RTM is used for transaction processing, the following steps are executed:

s1.1: reading the back lock by the program and judging whether the back lock is occupied currently, if the back lock is not occupied, calling an _ xbegin () method by the program to enter an RTM code block and enter S1.2, and if the back lock is occupied, calling a _ xabort () method to interrupt the RTM calling and enter S2.1;

s1.2: a rollback lock is stored in the RTM code block, so that the RTM code block is placed into a read-write set;

s1.3: performing specific transaction processing code part execution;

s1.4: calling a _ xend () method to try to submit, wherein at the moment, RTM can automatically compare and judge whether all contents in a read-write set including a rollback lock are modified, if so, the RTM operation is forcibly interrupted and the transaction is considered to fail; if not, then can enter the submitting step;

the step S2 further includes, S2.1: reading and taking out the unlocking by the program, and judging whether the current is occupied by other threads; s2.2: if the backspacing lock is not occupied currently, entering step 2.3; if the rollback lock is currently occupied, self-blocking until the rollback lock is released; s2.3: the program acquires and occupies a rollback lock, and then a specific transaction processing code part is carried out; s2.4: the transaction is committed and the rollback lock is released.

2. The method according to claim 1, wherein in step S3, when using a rollback method, it is determined whether the rollback lock is held by another thread, and if not, the rollback lock is retrieved, the transaction is performed, and then the rollback lock is released; if the thread is held by other threads, the above steps are performed after the rollback lock is released.

3. The method of claim 1, wherein the fallback lock is a coarse grain lock and the grain value is on the scale of the table.

4. A system for RTM-based graph database system transaction processing, the system comprising:

a storage module that stores data;

and a processing module performing the method of any one of claims 1-3.

5. The system for graph database system transactions according to claim 4, wherein said processing module is implemented using an instruction set embedded in a processor.

6. A computer-readable storage medium storing a computer program, wherein the computer program, when being processed and executed, implements the method for RTM-based graph database system transaction processing according to any one of claims 1 to 3.