US9317549B2 - Constraint-based consistency with snapshot isolation - Google Patents

Constraint-based consistency with snapshot isolation Download PDF

Info

Publication number
US9317549B2
US9317549B2 US13/926,186 US201313926186A US9317549B2 US 9317549 B2 US9317549 B2 US 9317549B2 US 201313926186 A US201313926186 A US 201313926186A US 9317549 B2 US9317549 B2 US 9317549B2
Authority
US
United States
Prior art keywords
constraints
transaction
transactions
constraint
current transaction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US13/926,186
Other languages
English (en)
Other versions
US20140379658A1 (en
Inventor
David R. Cheriton
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
OptumSoft Inc
Original Assignee
OptumSoft Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by OptumSoft Inc filed Critical OptumSoft Inc
Priority to US13/926,186 priority Critical patent/US9317549B2/en
Priority to CN201810044573.2A priority patent/CN108108232B/zh
Priority to CN201410220268.6A priority patent/CN104252382B/zh
Publication of US20140379658A1 publication Critical patent/US20140379658A1/en
Assigned to OPTUMSOFT, INC. reassignment OPTUMSOFT, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHERITON, DAVID R.
Priority to US15/067,737 priority patent/US9805088B2/en
Application granted granted Critical
Publication of US9317549B2 publication Critical patent/US9317549B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2365—Ensuring data consistency and integrity
    • G06F17/30371—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G06F16/2315—Optimistic concurrency control
    • G06F16/2322—Optimistic concurrency control using timestamps
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2379—Updates performed during online database operations; commit processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Definitions

  • This invention relates to transaction processing.
  • Atomic transactions are often used to simplify concurrent and fault-tolerant programming.
  • a transaction is atomic if it is indivisible, such that an attempt to perform the transaction can have only two possible outcomes: 1) either all parts of the transaction occur (transaction commits), or 2) no parts of the transaction occur (transaction aborts).
  • transaction commits either all parts of the transaction occur
  • transaction aborts no parts of the transaction occur
  • a transaction is a transfer of funds from one account to another account, it is highly desirable for this transaction to be atomic, to avoid the possibility of a credit being applied to one account without a corresponding debit in the other account (or vice versa). Similar advantages accrue to atomic transactions in more general programming situations.
  • the first is normally implemented by locking to prevent concurrent updates as well as an undo log, to be able to undo changes in the case of transaction abort. It can also be implemented optimistically, instead of using locks, by aborting the transaction if another transaction writes data that this transaction is writing (a so-called write-write conflict) or writes data this transaction is reading (a so-called read-write conflict).
  • in-place update has been favored because of the benefits of maintaining the disk layout, given that most transactional data has been disk-based until recently.
  • read-locking is a significant overhead, given the preponderance of reads over writes in many applications.
  • read-locking data means updates to the locked data are delayed, which really means it ends up being potentially inconsistent with the real world, if the corresponding real world values change during the time it is locked. In some sense, this means its focus on achieving internal consistency can lead to inconsistency with the external environment.
  • the optimistic form of in-place update can suffer from excessive abort rate, given the prevalence of read-write conflicts.
  • Shadow copy approach With the move to in-memory databases, the shadow copy approach becomes more attractive than before.
  • an updating transaction makes a copy of the data to be updated, makes the modification and then atomically updates the root reference (or pointer) to this data to refer to the new (previously shadow) copy.
  • a transaction can execute from snapshots (i.e. immutable copies) of even the data that it is just reading.
  • snapshot isolation This approach provides most of the properties of conventional serialized transactions with the additional significant benefit of not suffering from the read-write conflicts of the in-place update transactions, which can either incur significant locking overhead or else increase the abort rate in the case of an optimistic implementation.
  • the shadow copy can be attractive in a distributed implementation because there it is often necessary to duplicate data at a process performing the transaction, thereby providing a copy or snapshot of the data to provide efficient local access.
  • the copy overhead of SI is effectively eliminated by the access savings this copy provides for local processing, or conversely the creation of the local copy effectively has already paid the cost of a snapshot as required by the SI transaction mechanism.
  • This local copy can also reduce a process's exposure to the failure and restart of a process containing the primary copy of the state because it can continue to operate with its local snapshot.
  • SI does not provide sequential consistency (or serializability as it is described in the database world) in the sense that all executions produce the same results as if executed in some sequential order.
  • the compromising behavior arises because of the so-called write skew problem.
  • This problem can be illustrated by considering a simple assert constraint that a transactional application is expected to maintain. For example, consider the assert constraint: 39> b+c; where separate transactions Tb and Tc can update b and c, respectively. If b and c are initially 10, Tb could update b to 20, viewing a snapshot of c as 10, whereas Tc could update c to 20, similarly viewing a snapshot of b as 10. These two transactions can both commit concurrently in the SI model because there is no write-write conflict, yet doing so causes the constraint to be violated.
  • the present approach is based on identifying the constraints that need to be satisfied for correct application behavior. With these constraints identified, commit processing for a transaction can then refer to the constraints and to concurrent updates to see if committing the current transaction causes a problem with the constraints. If there is a conflict with the constraints, the transaction aborts. If there is no conflict with the constraints, the transaction commits.
  • the present approach provides superior semantics because it addresses the write skew problem of conventional snapshot isolation.
  • the present approach can provide greater efficiency (e.g., reduced transaction overhead and/or fewer unnecessary transaction aborts). For example, many apparent write-write conflicts can be resolved without aborting a transaction if the requirements on the data are expressed as constraints that can be checked as part of commit processing.
  • FIG. 1 shows an illustrative time line of several transactions.
  • FIG. 2 shows an example of updating the snapshot of the current transaction according to concurrent updates to provide merged snapshots.
  • FIG. 3 shows an example of updating a snapshot according to operations performed by the current transaction.
  • FIG. 4A provides an illustrative example of transactions, assert constraints, and update constraints.
  • FIG. 4B is a diagram relating to the example of FIG. 4A .
  • FIG. 5 shows an example of performing transaction processing in rounds.
  • the infrastructure can ensure that a transaction that violates such a constraint is aborted.
  • each application transaction is realized as a sequence of instructions that read and write the application state, i.e., its data model.
  • each application transaction executes independently and concurrently except for its interaction through this data model.
  • This application processing makes certain assumptions about the data model for its correct execution. These assumptions can be captured in terms of constraints, statements that are to be true of the data model during the execution of an application transaction. Thus, if an application is correct procedurally relative to these constraints, and the data model constraints are true during the execution of each transaction, the application executes correctly.
  • the input source code includes a specification of the constraints.
  • corresponding constraint checking code can readily be generated automatically. If the input source code does not include an explicit specification of its constraints, automatic generation of corresponding constraint check code may be possible via automated program analysis. However, in practice, it is likely that the user will need to provide the constraint checking code for input source code that does not have explicit constraints.
  • the conventional solution effectively materializes each constraint as a variable that is updated by any transaction that updates a variable in this constraint. For example, using the earlier example, there would be a variable, say, with the constructed name “constraint ⁇ (39>b+c)” that is incremented as part of updating b or c. Consequently, any two concurrent transactions affecting the same constraint would have a write-write conflict and at least one of them would be aborted by the SI transaction mechanism. Thus, it is feasible for the transaction mechanism to automatically ensure these constraints hold during each transaction by this construction.
  • the present approach thus provides a method of committing transactions that allows greater concurrency and fewer aborts based on the actual semantics of the constraints (as opposed to declaring write-write conflicts in all cases).
  • the basic idea is to have knowledge of these constraints available at commit time, to identify any conflicts at the time, attempt to resolve these conflicts, and only abort if these conflicts cannot be resolved, assuming the transaction would not be aborted for other reasons, such as a resource limitation.
  • a concurrent transaction is one that committed to the data model after the start time of the current transaction and before the target commit time of this transaction.
  • a concurrent update is one performed by such a concurrent transaction.
  • FIG. 1 provides some examples to illustrate this terminology.
  • T0 is the current transaction, shown on a time line.
  • Transactions T1 and T2 are concurrent with T0, while transactions T3 and T4 are not concurrent with T0.
  • the definition of concurrent is relative to which transaction is designated as the current transaction. For example, if T2 is regarded as being the current transaction, then T0 is not concurrent with T2.
  • current transaction T0 has a start time t0 s and a current target commit time t0 e .
  • transactions T1, T2, T3, and T4 have corresponding start times t1 s , t2 s , t3 s and t4 s and have corresponding end times t1 e , t2 e , t3 e , and t4 e , (at which the transactions either commit or abort).
  • T1 and T2 are concurrent with current transaction T0 because t0 s ⁇ t1 e ⁇ t0 e and t0 s ⁇ t2 e ⁇ t0 e .
  • t1 e and t2 e are intermediate times after the current start time (tc s ) and before the current target commit time (tc e )
  • An exemplary transaction processing method includes the following steps:
  • a commit processing check that automatically determines whether the current transaction can be committed consistently with the set of constraints from at least a) the snapshot of the current transaction and b) changes to the data model made by the concurrent transactions.
  • steps 4 and 5 Automatically aborting the current transaction if it cannot be made consistently with the set of constraints.
  • steps 4 and 5 rely on the results of the commit processing check of step 3.
  • An “assert” constraint checks that a given condition is true and if not, forces the abort of the transaction.
  • each assert constraint is checked against other concurrent updates, aborting the current transaction if the assert constraint is violated using the current values, i.e. after these updates, and those updated in the current transaction.
  • the read set for an assert constraint is the set of variables that need to be read to evaluate the constraint expression, i.e. determine if it is true. If the current transaction has written at least one variable (X) that is specified as part of the constraint, the checking then needs to determine if any concurrent transaction has changed any of the other variables associated with the constraint. This could be regarded as a read-write conflict check. If one or more such variables have been modified by a concurrent transaction, the constraint check needs to re-evaluate the constraint with those newly committed values as well as its updated but not committed value of X to verify that the constraint still holds.
  • FIG. 2 shows an example of this.
  • the concurrent update of transaction T1 is applied to the initial snapshot S0 to generate a first merged snapshot Sm1 (which could replace the initial snapshot S0).
  • the concurrent update of transaction T2 is applied to merged snapshot Sm1 to generate a second merged snapshot Sm2 (which could replace the earlier merged snapshot Sm1).
  • the commit processing check for current transaction T0 can refer to snapshot Sm2 and thereby include the effects of the concurrent updates from transactions T1 and T2.
  • the commit processing check can include re-evaluation of assert constraints using the merged snapshot.
  • FIG. 2 shows separate merged snapshots for clarity of illustration.
  • a single merged snapshot is generated at the time of commit processing that includes the effects of all concurrent updates to variables in the read set determined from the constraints as above.
  • such merged snapshots account for committed changes due to concurrent transactions by updating an earlier snapshot, while not exposing the current transaction to uncommitted changes by concurrent transactions.
  • FIG. 3 shows a variation on this theme, where current transaction T0 updates its snapshot according to operations it performs to provide an updated snapshot for the commit processing check.
  • operations O1, O2, and O3 are applied in sequence to initial snapshot S0 by current transaction T0 to provide updated snapshots S1, S2, and S3 respectively.
  • the commit processing check can refer to the updated snapshot S3 to determine whether or not to commit current transaction T0. If T0 is aborted, the changes made in updated snapshots S1, S2, and S3 never get committed to the main data model, thereby preserving the desired property of atomic transactions.
  • the example of FIG. 3 shows separate updated snapshots for clarity of illustration. Alternatively, a single updated snapshot is generated at the time of commit processing that includes the effects of all operations of the current transaction.
  • the concurrent updates can be determined from the redo log, as is conventional practice.
  • concurrent updates can be determined by comparing the current transactions snapshot to the current committed data model state, with the differences, excluding those caused by updates in the current transaction, indicating the concurrent updates.
  • concurrent updates can be determined by comparing the current committed data model state to an unmodified snapshot of the committed state as of the start time of current transaction with the differences indicating the concurrent updates.
  • concurrent potentially conflicting updates can be detected by materializing and updating a variable per constraint, as described earlier, but using the indication of a write-write conflict on such a constraint variable to trigger constraint resolution processing as above rather than immediately aborting the current transaction.
  • the transaction mechanism can select this approach of re-evaluating constraints for some constraints while using the conventional “constraint variable” approach for others, for instance, if a constraint was deemed to expensive or difficult for the commit processing to perform.
  • most constraints are relatively simple.
  • an embodiment that only applies this constraint analysis approach to simple constraints and resorts to the conventional construct of a “constraint variable” otherwise should provide significant performance improvement over purely using the conventional approach while minimizing the development complexity.
  • constraints are “update constraints” in which the value of some attribute is determined by the constraint and thus updated when the parameter values of the constraint are changed.
  • the implementation causes the determined attribute, namely a1 in this case, to be updated when an argument to the constraint is changed, namely a0 in this case.
  • FIGS. 4A-B provide a simple example of the difference between assert constraints and update constraints.
  • Transaction T1 is a transfer of funds from account 1 to account 2
  • transaction T2 is an interest accrual in account 1
  • transaction T3 is an interest accrual in account 2.
  • Assert constraints C1 and C2 are to prevent overdrawing accounts 1 and 2 respectively.
  • FIG. 4B shows Venn diagrams of constraints (left side) and data model members (right side) for the example of FIG. 4A .
  • 402 is the set of all constraints
  • 404 is the set of all constraints that relate to current transaction T1
  • 406 is the subset of constraints 404 that are update constraints.
  • 412 is the set of all data model members
  • 414 is the set of all data model members affected by the current transaction T1
  • 416 is the set of data model members affected by T1 because of its associated update constraints. It is convenient to refer to data model members that are affected by update constraints of the current transaction as “target members”.
  • the commit processing check can include: 1) determining, for the current transaction, target members of the data model that would be updated as a result of update constraints by committing the current transaction; and 2) determining whether committing the current transaction and updating the target members according to the update constraints is consistent with the set of constraints.
  • the determined attribute can be automatically updated and added to the write set before, or as part of, committing the transaction.
  • the transaction, on commit automatically ensures this constraint is maintained.
  • the transaction is aborted if there exists an unresolvable write-write conflict, even if one or more of the writes were caused by such a constraint resolution.
  • assert and update constraints are specified such that an update constraint cannot cause an assert constraint to be violated. This situation can be realized by having an assert constraint that precludes such a violation. For instance, if variable a1 is determined by an update constraint as in the earlier example and a1 is constrained to be less than 100 by an assert constraint a1 ⁇ 100, this assert constraint is replaced by the assert constraint a0 ⁇ 99, so the update constraint never causes an assert violation.
  • a transaction is committed independent of conflicting writes to a determined attribute by the associated update constraint when the update constraint is (re)evaluated at commit time using the merged snapshot. This is warranted because the determined attribute is determined by the constraint and the merged snapshot, not directly by the actions of any transaction.
  • the constraints specified for the data model are translated into a collection of single-trigger constraints, e.g., as part of compilation and instantiation of the data model.
  • the imperative procedure associated with each triggered constraint is invoked, and any writes performed by these constraints are tracked as part of the transaction.
  • the concurrent updates are logically applied to the current snapshot, which means these updates can also cause triggering of update constraints, but all these constraints are then evaluated with the currently committed state plus that to be committed as part of the current transaction, ensuring the constraint holds in the final (to-be-committed) state.
  • a write by a constraint-handling procedure can trigger other constraints, which can trigger further writes. It is the responsibility of the application programmer in developing the constraint specification to ensure that this sequence eventually completes. In practice, the sequence is normally short, such as one or two levels of triggering.
  • the data model instantiation can be executed as a separate “sysdb” process from the processes executing code to perform the actions of the transactions, the latter processes referred to as “agents”.
  • agents When such an agent completes the transaction execution, it communicates this fact to the sysdb process which then handles the commit processing. If the entire data model is maintained by a single sysdb process, it can perform the commit processing of all the constraints as a local action, thus making it efficient compared to a design requiring distributed communication.
  • the sysdb process logs the changes of the transaction to the data model (i.e. a redo or write-ahead log) and has not actually applied these changes to the data model instantiation. During the initial prepare-to-commit, it then applies these logged changes to the data model instantiation causing the constraints to generate additional writes. These additional writes are captured by a logging mechanism that detects changes to the data model for persisting these changes to secondary storage and/or communicating these changes to other copies of the data model, such as a backup copy provided by a separate process (provided to allow fast recovery in the case of failure). Applying these logged changes can also invoke code to check any affected assert constraints. If any of these assert constraints fail, the transaction can then be aborted.
  • the data model i.e. a redo or write-ahead log
  • the state in this sysdb process during the application of these logged changes represents the merged snapshot up to this point in the log. If an assert constraint is affected by multiple changes in the log, the last such change provides the fully merged snapshot state relative to said assert constraint. Thus, the assert constraint is always evaluated effectively on the final merged snapshot of the state for the current transaction.
  • the transaction state is committed in the conventional way, typically flushing the updates out to a redo log with an indication of transaction commit. If the transaction is forced to abort during commit processing, the changes to the sysdb state can be undone from an undo log, as is well-established in the art.
  • the commit processing attempts to resolve the write-write conflict, as described below, and otherwise aborts the transaction if it cannot resolve the write-write conflict.
  • metadata is available at commit time regarding the attributes in the data model that have been updated by this transaction, describing the category, type and or semantics of these attributes. Then, on a write-write conflict, the transaction commit mechanism examines the associated attributes, attribute values and the associated metadata to determine if there is a possible resolution.
  • the metadata may indicate that attribute ai, on which there is a write-write conflict between the current transaction and a concurrent transaction, is a counter.
  • the commit processing can determine the increment to use from the current transaction.
  • the commit processing for the current transaction can then apply this increment to the value of ai already committed by the concurrent transaction to determine a correct final value for ai.
  • both the previous committed write and the current transaction are incrementing the counter, so the write-write conflict need not abort the current transaction provided it commits with a value that preserves both increments.
  • the commit processing for the current transaction applies the current transaction increment to the most recently committed concurrent update.
  • the commit processing can determine if the concurrent write changes the entry to the default value, indicating it was effectively deleting the entry from the map, whereas the current transaction is writing the same entry to a non-null value.
  • the value written by the current transaction can override that of the committed value, given the earlier committing transaction effectively deleted an entry and the current one added an entry with the same key, but a different value.
  • the key for the entry is a generated or allocated value so that the current transaction can simply select a different key for an entry that does not conflict with concurrent updates, and thereby resolve the write-write conflict.
  • Another case is a concurrent transaction writing an element to the tail of the same queue as the current transaction does. This case can be resolved by simply appending the current transaction's element after the element(s) enqueued by the concurrent transaction(s).
  • an attribute may have a write conflict resolution procedure associated with it that can do one of the processing sequences indicated above or alternatively some application-specific processing that resolves the write-write conflict.
  • the procedure may determine that a write-write conflict arose as a result of the concurrent transaction allocating some shared resource yet this concurrent transaction represents a lower priority process or user.
  • the resource can be reclaimed from this lower priority process, possibly running a compensating transaction as a result, and this write-write conflict is then removed as a reason to not allow the current transaction to commit.
  • a constraint may cause writes that conflict with a concurrent transaction yet can be resolved by the commit processing based on metadata associated with the affected attribute.
  • a counter can be handled as above with a suitable constraint, i.e. the counter is constrained to be equal to the number of some particular event.
  • an attribute containing the average value of those in a collection can be affected and thus written by two concurrent transactions that update values in this collection. However, by recomputing the average in the current transaction with the previously committed values, the revised value eliminates the need for its original write-write conflict to cause an abort of the current transaction.
  • an application data model is commonly comprised of both raw data input and (internally) derived data.
  • the raw data input is normally sourced from specific input devices, so does not itself lead to write-write conflicts.
  • the raw data input from sensor 1 does not overwrite that raw data input from sensor 2.
  • the derivative data can often be specified in terms of equality constraints (or similar constructs) that determine the values of the derived data.
  • two concurrent updates of raw data e.g., from sensors 1 and 2 can cause write-write conflicts on the derived data, such as the sum above, and these write-write conflicts can be resolved by allowing the recomputation of the constraint based on the merged snapshot.
  • write-write conflicts can be resolved in many, if not all cases, in an application, significantly reducing the abort rate, and thereby providing better performance than a conventional realization of fully serializable execution.
  • an update constraint can be designated as being asynchronous by tagging the determined attribute.
  • the commit processing on the original sysdb process initiates a separate “compensating” transaction to execute on the sysdb process maintaining a1, rather than treating it as part of the current transaction. Consequently, the current transaction does not need to be expanded to include this update. Nor does it need to wait for this update to a1 to be completed.
  • a key issue for performance is careful partitioning of the data model across multiple sysdb processes when this is required, and specifying constraints such that when there are cross-partition constraints, they are specified as asynchronous if feasible.
  • the determined collection may be sufficiently close to use. For example, if the application logic is iterating over the determined collection to compute some statistics, having an approximately accurate collection may be adequate for the purposes of this statistics computation.
  • a constraint can be specified as approximate by indicating that one or more attributes can use their previous value.
  • the assert constraint of the earlier example can be specified as: 39 >b ′prev+ c; indicating that a transaction updating c can simply use the snapshotted value of b and does not need to check for a concurrent write to attribute b.
  • This relaxed constraint eliminates the need to identify the read-write set associated with this constraint and re-evaluate the constraint.
  • the definition of the previous value, as specified by the ′prev suffix, is the value of the attribute at the start-time of the current transaction. Thus, it may be the current value as well if this attribute was not updated concurrently with the current transaction.
  • Application logic can use the relaxed semantics by effectively being able to assume that an assert constraint is approximately true. In some cases, this approximation can be tightly estimated. For example, considering a bank overdraft situation, if the amount someone can withdraw within a given time interval is limited to say D dollars, the assertion of the account not being overdrawn is correct within D dollars, assuming transactions complete in a relatively short period of time compared to one day.
  • an application can also know the time interval between the snapshot value of an attribute (i.e. taken at the start time), and the current time. This known interval can also allow an application to bound inaccuracy of the assert constraint expression.
  • the constraint-based consistency of this invention can be provided with greater efficiency than if all constraints are specified without allowing these relaxed semantics. This is particularly true for the case of constraints that can span data model partition boundaries, because then, the relaxed semantics can avoid performing a distributed transaction with its associated round-trip communication message costs and delays.
  • an alternative structure is to tie transaction commit of every agent to a system-wide round completion, indicated by a (round) clock.
  • a round is synchronized across the application such that an agent does not start processing round i until it has received all the updates relevant to it from other agents executing their round i ⁇ 1. Normally, this is accomplished by not allowing any agent to proceed to round i until all agents have indicated their completion of round i ⁇ 1.
  • each new round message from the clock indicates the start of a new transaction by the agent, also acting to ensure it has received the state updates from the previous round.
  • the agent then completes the round processing and responds to the clock, to effectively commit the transaction. If an agent has no update to perform during a given round, it effectively commits a null transaction. This incurs relatively low overhead.
  • FIG. 5 provides a schematic illustration of transactions organized into rounds.
  • round 1 starts at time t1 s and ends at time t1 e
  • round 2 starts at time t2 s and ends at time t2 e
  • round 3 starts at time t3 s and ends at time t3 e
  • Agents a, b, c, and d are expected to execute one transaction each round (which may be a null transaction).
  • the transactions are labeled T1a for the transaction by agent a in round 1, T3d for the transaction by agent d in round 3, etc.
  • a new round usually does not begin until after all agents have completed the previous round (i.e., by committing or aborting their respective transactions).
  • Each agent logically starts its transaction at the beginning of the round. Its actual execution start may be delayed by process scheduling constraints, as shown on FIG. 5 .
  • each agent's transaction logically completes, i.e. commits, at the end of the round. Its actual execution may complete earlier because it has less to do than the processing available during the round time. Exceptions to this general scheme are possible.
  • transaction T1b on FIG. 5 spans two rounds. Thus, we regard transactions as being organized into rounds if most transactions take place within a single round, with relatively few transactions taking more than one round.
  • individual agent (trans)actions can be aborted.
  • the agent whose actions were aborted receives an indication to that effect, either immediately or at the start of the next round. It is an application-level decision on how the abort of an agent's actions is handled. For instance, in some cases, the agent can just retry the transaction in the following round or take an alternative action. Alternatively, the completion of the current round can be held up until the agent has successfully retried the same transaction. In some cases, this is equivalent to, and can in fact be implemented as, the agent failing and restarting.
  • the snapshot behavior can be implemented by blocking updates on receipt at each participating agent once a new round has started, releasing these updates to be provided to the application agent only once the round completes.
  • an abort an abort message is sent to each agent, which discards these blocked and queued updates, ensuring these agents do not see the results of round in which the abort occurred. If only the updates generated by the aborting process are discarded, only this aborting process needs to retry its processing for the current round. Otherwise, all processes revert to their state as of the start of the round and redo the processing for the current round.
  • update messages can be generated as part of reverting the state of the aborting process to the start of the round that effectively reverts the state to undo the changes generating by this aborting process, with the subsequent retry of this processing generating update messages that provide the result of the retry processing.
  • the transaction mechanism can rely on the new messages generated by the aborting and retrying process that provide updates to the state to provide the correct net state result in the blocking and queued messages of each receiving process before these messages are unblocked and processed in the agent for the next round. For instance, if the first aborted processing sets a variable “foo” to 3 and the subsequent retry sets the variable “foo” to 4, the blocked and queued messages contain one setting “foo” to 3 followed by one setting “foo” to 4, so the net result in the state of the agent is the correct value, namely 4 in this case.
  • the failure of a process during a round can be handled by restarting the process, having it recover its state as of the start of the round and redo the processing of the round. In this case, the failure is transparent to the transaction round processing except possibly for delaying the completion of the round.
  • the round mechanism could proceed if a specified process has indicated completion of processing within one of the last 3 rounds.
  • Such a process can encounter a concurrent update by another process that committed its transaction in an intervening round.
  • transaction T1b on FIG. 5 would need to consider concurrent updates from transactions T1a, T1c and T1d.
  • the processing of a specific process can be allowed to execute outside of the round mechanism, based on the changes that such a process makes to the state having no impact on the application correctness. For instance, a temperature sensor process may update the temperature periodically with small differences between consecutive values. The application logic can be designed to operate correctly even if this transpires concurrently with the round-based transactions, i.e. the temperature appears to change in the middle of a round. This can be another source of concurrent updates within a round-based computation.
  • a process can indicate whether the abort of a transaction needs to be retried in the current round or can slip to the next round.
  • This indication can be provided in the action of an agent process registering with the round clock to indicate it is participating in the rounds, i.e. as a parameter in the round registration process.
  • an embodiment can allow each application process to select different optimizations in handling aborts and failures with the goal of minimizing the impact on application performance of these events while still ensuring application correctness.
  • the ′next suffix is used to designate that the value is updated in the next round.
  • the ′prev suffix may refer to a concurrently updated value when the current transaction can span multiple rounds. That is, the value could have been updated during one of the intervening rounds.
  • the round-based approach reduces the cost of the transactions both in terms of messages to initiate and commit transactions, but also the need to specify ′prev constraint values, or conversely, the cost of failing to do so.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
US13/926,186 2013-06-25 2013-06-25 Constraint-based consistency with snapshot isolation Active 2034-03-07 US9317549B2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US13/926,186 US9317549B2 (en) 2013-06-25 2013-06-25 Constraint-based consistency with snapshot isolation
CN201810044573.2A CN108108232B (zh) 2013-06-25 2014-05-23 用于确保与数据模型有关的并发事务的一致性的系统和方法
CN201410220268.6A CN104252382B (zh) 2013-06-25 2014-05-23 具有快照隔离的基于约束的一致性
US15/067,737 US9805088B2 (en) 2013-06-25 2016-03-11 Constraint-based consistency with snapshot isolation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/926,186 US9317549B2 (en) 2013-06-25 2013-06-25 Constraint-based consistency with snapshot isolation

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/067,737 Continuation US9805088B2 (en) 2013-06-25 2016-03-11 Constraint-based consistency with snapshot isolation

Publications (2)

Publication Number Publication Date
US20140379658A1 US20140379658A1 (en) 2014-12-25
US9317549B2 true US9317549B2 (en) 2016-04-19

Family

ID=52111796

Family Applications (2)

Application Number Title Priority Date Filing Date
US13/926,186 Active 2034-03-07 US9317549B2 (en) 2013-06-25 2013-06-25 Constraint-based consistency with snapshot isolation
US15/067,737 Active US9805088B2 (en) 2013-06-25 2016-03-11 Constraint-based consistency with snapshot isolation

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/067,737 Active US9805088B2 (en) 2013-06-25 2016-03-11 Constraint-based consistency with snapshot isolation

Country Status (2)

Country Link
US (2) US9317549B2 (zh)
CN (2) CN104252382B (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180173778A1 (en) * 2016-12-16 2018-06-21 Linkedin Corporation Database uniqueness constraints

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6394032B2 (ja) * 2014-03-31 2018-09-26 富士通株式会社 処理管理プログラム、処理管理装置及び処理管理方法
US10795881B2 (en) 2015-12-18 2020-10-06 Sap Se Table replication in a database environment
US10572510B2 (en) 2015-12-21 2020-02-25 Sap Se Distributed database transaction protocol
CN106371770B (zh) * 2016-09-05 2019-07-12 杭州华为数字技术有限公司 数据写入方法及装置
US10394663B2 (en) 2016-12-16 2019-08-27 Red Hat, Inc. Low impact snapshot database protection in a micro-service environment
US10678775B2 (en) * 2016-12-20 2020-06-09 International Business Machines Corporation Determining integrity of database workload transactions
US10977227B2 (en) * 2017-06-06 2021-04-13 Sap Se Dynamic snapshot isolation protocol selection
CN107609048B (zh) * 2017-08-18 2020-09-04 中国人民大学 一种可延迟唯一约束的方法
CN110059074A (zh) * 2019-03-18 2019-07-26 华迪计算机集团有限公司 一种用于内存数据库的数据处理方法及系统
EP3628093B1 (en) * 2019-04-30 2021-07-07 Advanced New Technologies Co., Ltd. Method and device for avoiding double-spending problem in read-write set-model-based blockchain technology
CN112231071B (zh) 2020-05-20 2021-06-18 腾讯科技（深圳）有限公司 事务处理方法、装置、计算机设备及存储介质
CN114428774B (zh) * 2022-04-02 2022-07-12 北京奥星贝斯科技有限公司 数据库的约束关系检查方法及装置

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6275863B1 (en) * 1999-01-25 2001-08-14 International Business Machines Corp. System and method for programming and executing long running transactions
US6618851B1 (en) * 1999-08-31 2003-09-09 Autodesk, Inc. Method and apparatus for state-reversion
US20050021567A1 (en) * 2003-06-30 2005-01-27 Holenstein Paul J. Method for ensuring referential integrity in multi-threaded replication engines
US20060219772A1 (en) * 2005-04-01 2006-10-05 Microsoft Corporation Relaxed currency constraints
US7653665B1 (en) * 2004-09-13 2010-01-26 Microsoft Corporation Systems and methods for avoiding database anomalies when maintaining constraints and indexes in presence of snapshot isolation
US20100114841A1 (en) * 2008-10-31 2010-05-06 Gravic, Inc. Referential Integrity, Consistency, and Completeness Loading of Databases
US20110153566A1 (en) * 2009-12-18 2011-06-23 Microsoft Corporation Optimistic serializable snapshot isolation
US20110191297A1 (en) * 2001-06-05 2011-08-04 Kenneth Beck Clustered filesystem with data volume snapshot maintenance
US20110246725A1 (en) * 2010-03-31 2011-10-06 Moir Mark S System and Method for Committing Results of a Software Transaction Using a Hardware Transaction
US20120136839A1 (en) 2010-11-30 2012-05-31 Peter Eberlein User-Driven Conflict Resolution Of Concurrent Updates In Snapshot Isolation
US20120150802A1 (en) * 2009-08-25 2012-06-14 Peter Popov Database replication
US20120166407A1 (en) 2010-12-28 2012-06-28 Juchang Lee Distributed Transaction Management Using Two-Phase Commit Optimization
US20120259863A1 (en) 2011-04-11 2012-10-11 Bodwin James M Low Level Object Version Tracking Using Non-Volatile Memory Write Generations
US8676851B1 (en) * 2012-08-30 2014-03-18 Google Inc. Executing transactions in distributed storage systems
US20140337303A1 (en) * 2013-05-07 2014-11-13 Red Hat, Inc Bandwidth optimized two-phase commit protocol for distributed transactions

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7606838B2 (en) * 2006-02-22 2009-10-20 Microsoft Corporation Distributed conflict resolution for replicated databases
US7899997B2 (en) * 2008-03-12 2011-03-01 International Business Machines Corporation Systems and methods for implementing key-based transactional memory conflict detection
US9158519B2 (en) * 2008-05-21 2015-10-13 Optumsoft, Inc. Dynamic collection attribute-based computer programming language methods
US8166481B2 (en) * 2008-10-20 2012-04-24 Microsoft Corporation Transaction processing in transactional memory
US8762333B2 (en) 2009-07-08 2014-06-24 Pivotal Software, Inc. Apparatus and method for read optimized bulk data storage
US8935205B2 (en) * 2011-11-16 2015-01-13 Sap Ag System and method of performing snapshot isolation in distributed databases
CN102629221B (zh) * 2012-02-28 2014-11-19 华为技术有限公司 用于分布式共享存储的任务同步方法、装置及系统

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6275863B1 (en) * 1999-01-25 2001-08-14 International Business Machines Corp. System and method for programming and executing long running transactions
US6618851B1 (en) * 1999-08-31 2003-09-09 Autodesk, Inc. Method and apparatus for state-reversion
US20110191297A1 (en) * 2001-06-05 2011-08-04 Kenneth Beck Clustered filesystem with data volume snapshot maintenance
US20050021567A1 (en) * 2003-06-30 2005-01-27 Holenstein Paul J. Method for ensuring referential integrity in multi-threaded replication engines
US7653665B1 (en) * 2004-09-13 2010-01-26 Microsoft Corporation Systems and methods for avoiding database anomalies when maintaining constraints and indexes in presence of snapshot isolation
US20060219772A1 (en) * 2005-04-01 2006-10-05 Microsoft Corporation Relaxed currency constraints
US20100114841A1 (en) * 2008-10-31 2010-05-06 Gravic, Inc. Referential Integrity, Consistency, and Completeness Loading of Databases
US20120150802A1 (en) * 2009-08-25 2012-06-14 Peter Popov Database replication
US20110153566A1 (en) * 2009-12-18 2011-06-23 Microsoft Corporation Optimistic serializable snapshot isolation
US20110246725A1 (en) * 2010-03-31 2011-10-06 Moir Mark S System and Method for Committing Results of a Software Transaction Using a Hardware Transaction
US20120136839A1 (en) 2010-11-30 2012-05-31 Peter Eberlein User-Driven Conflict Resolution Of Concurrent Updates In Snapshot Isolation
US20120166407A1 (en) 2010-12-28 2012-06-28 Juchang Lee Distributed Transaction Management Using Two-Phase Commit Optimization
US20120259863A1 (en) 2011-04-11 2012-10-11 Bodwin James M Low Level Object Version Tracking Using Non-Volatile Memory Write Generations
US8676851B1 (en) * 2012-08-30 2014-03-18 Google Inc. Executing transactions in distributed storage systems
US20140337303A1 (en) * 2013-05-07 2014-11-13 Red Hat, Inc Bandwidth optimized two-phase commit protocol for distributed transactions

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180173778A1 (en) * 2016-12-16 2018-06-21 Linkedin Corporation Database uniqueness constraints

Also Published As

Publication number Publication date
CN108108232B (zh) 2021-10-22
US9805088B2 (en) 2017-10-31
US20160292212A1 (en) 2016-10-06
CN104252382A (zh) 2014-12-31
US20140379658A1 (en) 2014-12-25
CN104252382B (zh) 2018-02-09
CN108108232A (zh) 2018-06-01

Similar Documents

Publication Publication Date Title
US9805088B2 (en) Constraint-based consistency with snapshot isolation
US11386065B2 (en) Database concurrency control through hash-bucket latching
EP0457473B1 (en) Method for accessing shared data
JP5660693B2 (ja) ハイブリッドｏｌｔｐ及びｏｌａｐ高性能データベースシステム
Wu et al. Transaction healing: Scaling optimistic concurrency control on multicores
Levy et al. A theory of relaxed atomicity
Fekete et al. Consistency models for replicated data
US10394605B2 (en) Mutable chronologies for accommodation of randomly occurring event delays
US5524239A (en) Replay recovery protocol for real-time database management systems
US5497487A (en) Merge, commit recovery protocol for real-time database management systems
Martin et al. Toward formalizing recovery of (advanced) transactions
JPH0628199A (ja) 並行処理の同期方法
Özsu et al. Introduction to Transaction Management
Faleiro High Performance Multi-core Transaction Processing via Deterministic Execution
Niles et al. Exploiting parallelism of distributed nested transactions
Elmagarmid et al. Reservable transactions: An approach for reliable multidatabase transaction management
Wojciechowski et al. Having your cake and eating it too: Combining strong and eventual consistency
CN118260049A (zh) 一种基于Flink流式处理的可靠事务性交易方法
Niles Jr Improving performance of highly-programmable concurrent applications by leveraging parallel nesting and weaker isolation levels
Beynon-Davies Transaction Management
Ibrahim et al. Mathematical Model and Algorithms for some Type of Concurrency Control Problems in Database Applications
Lingam Freezing as a correctness measure for Multiversion Timestamp Ordering protocol
Humm Scheduling in a generalized transaction/thread model
Sharif-Askary Reservable Transactions: An Approach for Reliable Multidatabase Transaction Management
Signer Introduction to Databases

Legal Events

Date Code Title Description
AS Assignment

Owner name: OPTUMSOFT, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CHERITON, DAVID R.;REEL/FRAME:037582/0699

Effective date: 20160125

STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8