Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/23—Updating
  • G06F16/2308—Concurrency control
  • G06F16/2336—Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
  • G06F16/2343—Locking methods, e.g. distributed locking or locking implementation details

Definitions

• the present invention relates generally to database management systems and transaction processing systems that utilize a lock manager for protecting database resources from simultaneous incompatible uses, and more particularly to a lock manager that allows greater concurrent use of resources than the lock managers in traditional transaction processing systems while diminishing the "ACID" properties of transactions only with respect to isolation between concurrent transactions.
• the present invention also relates to a database management system and method for utilizing and combining data, some of which is entirely reliable and some of which is less reliable, unknown or missing, using a methodology herein called annotated nullable logic for comparing and combining annotated data and truth values in a manner that preserves reliability category information associated with the data and truth values being compared and/or combined.
• the present invention is directed at the management of transactions in database management systems so as to enable greater concurrency, and therefore more efficient transaction execution, than is allowed by DBMS's requiring strict adherence to the traditional "ACID" properties of transactions. More specifically, the present invention is directed at solving the "serializability" problems introduced by long lived transactions (LLT's).
• LLT's long lived transactions
• the present invention may also be used in persistent programming languages as well as to concurrency control services for object resource brokerage systems. For simplicity, the present invention will be described with respect to DBMS's.
• ACID Atomicity, Consistency, Isolation and Durability.
• Atomicity means that either the results of the transaction (i.e., changes to the database) are all properly reflected in the database, or none of them are.
• a transaction is said to commit or abort.
• a transaction commits all changes made to the database by the transaction are durably stored, leaving the database in a consistent state.
• a transaction aborts any changes made to the database by the transaction are backed out, once again leaving the database in a consistent state.
• Consistency means that each transaction commits only legal results to the database. Thus, a transaction must bring the database from one consistent state to another consistent state.
• Isolation means that the events within a transaction must be hidden from other transactions running concurrently. Concurrent transactions must not interfere with each other. They execute as if they had the database to themselves.
• Durability means that once a transaction has been completed and has committed its results to the database, the system must guarantee that these results survive any subsequent malfunctions.
• atomicity is defined to mean “concurrency atomicity,” meaning that no transaction can observe any partial results of an atomic transaction.
• This document and the present invention take the opposite approach.
• atomicity is defined to mean that a transaction's commitment must be atomic. That is, once -3- some work is committed to the DBMS (as opposed to committed to the parent of a subtransaction), the transaction in question cannot continue to perform work that may or may not be committed at some later point in time.
• This notion of atomicity excludes such things as open nested transactions, but does not exclude partial rollbacks, the use of persistent savepoints, or other mechanisms that can be used by application programmers to control the behavior of the system's recovery mechanisms, since the final outcome of the transaction is still abort or commit.
• Failure atomicity implies that all effects of incomplete transactions must be undone in the case of failure. Failure atomicity may be undesirable for long lasting transactions (LLT's). For example, a designer who experiences a power failure just as he is about to commit a week's worth of work is unlikely to consider failure atomicity to be a valuable property of the design database. The obvious solution for this situation (as well as many other LLT's) is for LLT's to have persistent savepoints, which would make it possible to recover an incomplete transaction to the last savepoint taken before a crash. It is noted that removing failure atomicity does not require removing commitment atomicity.
• the atomicity and durability properties of transactions are required for system failure recovery, the isolation property is required for concurrency control, and the consistency property is needed for both failure recovery and concurrency control.
• LLT's seem to be the only class of transactions where the ACLD properties cause significant problems. LLT's can be complex queries that last for minutes or hours, data mining queries that last for hours or days, or concurrent engineering transactions, controlled by humans and lasting from minutes to months. Full application of the ACID properties in a DBMS that supports LLT's can effectively prevent multiple users from simultaneously using the system. Long term locking of system resources by an LLT can prevent other users or transactions from being able to perform useful work.
• isolation is the only ACID property that it is desirable to compromise so as to reduce the impact of LLT's on system performance. More specifically, it would be desirable to compromise isolation in a controlled manner so as to give rise to as little inconsistency in the database as possible.
• null data there are situations in which some of the data values used by a transaction are missing or completely unknown. Missing information is sometimes called null data.
• DBMS database management system
• null data requires extensions to the framework used for predicate evaluation during query execution. In particular, when a predicate is evaluated against a null, the result of the evaluation cannot be true or false. Thus, to allow or support evaluations that involve nulls, one must introduce a value called "maybe” or "unknown” as a truth value, or introduce Boolean nulls. Since predicates can be combined by means of the operators AND, OR and NOT to form arbitrarily complex predicates, rules are needed to specify the results of evaluating complex predicates. This is usually done by means of an appropriate form of algebra.
• null data when there are no nulls involved, evaluation of complex predicates is performed according to Boolean algebra.
• prior art DBMS's that support null data support only one kind of null data and evaluate complex predicates having one or more null data values using three valued logic.
• the present invention was developed, in part, based on the premise that it would be desirable in many applications to have more than one kind of null, where the different types of nulls represent the different reasons why information is missing (e.g., "unknown,” “divide by zero,” “undefined,” “prohibited,” and so on). Although it is possible to allow multiple kinds of null data and still use three valued logic, using a three valued logic process to evaluate truth expressions having multiple kinds of null data effectively erases all information about the kinds of null data involved.
• LLT long lived transaction
• the transaction is likely to have different phases and the values of the data on which write locks are held may have corresponding levels of reliability.
• data values for which an LLT hold write locks will be least reliable at the beginning phases of the LLT and more reliable at the later phases of the LLT.
• the prior art treats all such data uniformly as either unreliable data or null data.
• the present invention is a database management system (DBMS) that has been modified to provide improved concurrent usage of database objects, particularly when the system is executing long lived transactions.
• a subset of the transactions access database objects using parameterized read and parameterized write access modes.
• Each transaction using a parameterized write mode of access for a database object specifies a write access mode and a write access mode parameter, where the write access mode parameter indicates a reliability classification that indicates the reliability of the write locked data.
• Each transaction using a parameterized read mode of access for a database object specifies a read access mode, and a read access mode parameter, where the read access mode parameter indicates one or more reliability classifications that are acceptable to the transaction.
• the DBMS Whenever a transaction requests access to a database object, the DBMS generates a corresponding lock request for the object. If the access request is a parameterized conditional access request, a corresponding parameterized lock request is generated.
• a lock manager processes each lock request by checking to see if any previously granted lock is conflicting or potentially conflicting with the requested lock.
• Two lock requests are unconditionally conflicting if their resource range overlaps and the access modes of the two requests are incompatible.
• Two requests are conditionally conflicting if analysis of their read/write parameters is necessary to determine whether a conflict exists. If the lock request being processed is unconditionally conflicting with any outstanding, previously granted lock, the lock request is put on a queue of pending requests. If the lock request is not unconditionally conflicting with any outstanding, previously granted locks, but is conditionally conflicting with an outstanding, previously granted lock, the conditional conflict is resolved by determining whether the write parameters for the write lock in question are a subset of the read parameters for the read lock in question. If so, then there is no conflict. If not, then the requested lock is in conflict with the outstanding previously granted lock. In none of the outstanding, previously granted locks is in conflict with the requested lock, the requested lock is granted. Otherwise it is put on a queue of pending lock requests.
• Another aspect of the present invention is a data processing method suitable for use in a database management system (DBMS) or other data processing system or product that stores and processes information.
• DBMS database management system
• a subset of the stored information is stored as annotated values, each annotated value having a stored data value and an associated data reliability value. That is, the data value is annotated with a data reliability value.
• the data reliability value associated with each datum is a member of a domain that includes at least three distinct data reliability values corresponding to at least three distinct data reliability classifications.
• one of the data reliability classifications represents fully reliable data and at least two of the distinct data reliability classifications represent two distinct classifications of less than fully reliable data.
• the domain of reliability categories might include the following six reliability categories: fully reliable, good, medium, poor, unknown and missing.
• a query executer or other data processing procedure executes queries (or other data processing tasks) requesting access to information stored in the database.
• the query executer includes annotated nullable logic for evaluating expressions containing at least one annotated value as an operand.
• the annotated nullable logic includes logic (i.e., instructions) for combining annotated values, for comparing annotated values so as to generate annotated truth values, and for combining annotated truth values in accordance with a predefined set of rules.
• the annotated nullable logic generates result values in a manner that preserves the relevant data reliability values associated with the annotated data and/or truth values that have been combined and/or compared.
• the data reliability values associated with the annotated values are each encoded as a reliability category bitmap having a multiplicity of bits.
• Each bit of the bitmap corresponds to a respective data reliability classification. Since fully reliable data does not need to be annotated, or can be annotated with a bitmap whose bits are all set to zero, the number of bits in the annotation bitmap is typically one less than the number of distinct data reliability classifications. -8-
• the annotated nullable logic includes means for combining first and second annotated values (k, A) and (m, B) using a predefined operator Op to generate an annotated result value (r, C):
• the bitmap for data reliability value C is generated by selecting one operand's annotation bitmap (e.g., in the case of the OR operation, the annotation bitmap for operand having the strongest truth value is selected).
• Fig. 1 is a block diagram of a database management system implementing the present invention.
• Fig. 2 depicts a set of lock management data structures for keeping track of granted and pending resource lock requests.
• Fig. 3 depicts an alternate set of lock management data structures for keeping track of granted and pending resource lock requests.
• Fig. 4 is a flow chart of the object locking method of the present invention.
• Fig. 5 A depicts the data structure of a meta-datum and its schema
• Fig. 5B depicts some examples of meta-data values.
• DBMS database management system
• Transaction operation and management are handled by a transaction manager 102 and a lock manager 104, both of which are software procedures executed by the system's data processor(s) 106.
• the transaction manager maintains a transaction table 108, sometimes implemented as a tree structure, for keeping track of the identity and status of all pending transactions.
• the lock manager 104 maintains a lock table 110, usually implemented using a hash table and various linked lists and/or tree structures (which will be discussed in more detail below).
• the lock table 110 keeps track of all locks that have been requested on resources in a database 112.
• the database 112 stores all of the data that is accessible to transactions executed by the DBMS 100.
• the DBMS 100 will typically also include additional memory resources 114, one or more system busses 116 for interconnecting the various parts of the system, and a network interface 118 or other communications interface for handling communications with client workstations 120.
• Conflicting transactions are two or more transactions that access the same resource and at least one of them will, at least potentially, update the resource in question.
• the results generated by at least one of the conflicting transactions may depend on the order in which the transactions are performed.
• a "data lock” is a mechanism for assigning a transaction certain rights to a database object, such as a table, a page, or a datum or record in a database table.
• a first transaction may lock a particular object so as to ensure that no other transaction accesses the data in that data until the first transaction commits or aborts.
• the prior art includes many types of data locking mechanisms. -10-
• “Overlapping resources” are database objects whose address range is the same or at least partially overlapping.
• An "access mode” refers to the way in which a transaction or application accesses a data resource.
• the traditional access modes (browse, read, update, write and exclusive) are described below.
• each unique read access parameter value is associated with a distinct read access mode
• each unique write access parameter value is associated with a distinct write access mode.
• a hash function 150 is used to convert a resource identifier into a hash table address in a fixed size hash table 152.
• the resource identifier that is hashed by function 150 may include a resource type or level indicator (e.g., indicating whether the resource to be locked is a database, table, page or tuple) and the starting address of the resource.
• Each addressable slot 154 of the hash table includes either a null value if there are no locked resources corresponding to that slot's hash table address, or a pointer to a list of lock control blocks (LCB's) 160 if there is at least one locked resource whose hash value corresponds to that slot's hash table address.
• LLB's lock control blocks
• the lock manager will allocate (i.e., generate and store) one lock control block (LCB) 160 for each lockable data item that is actually locked, and will allocate (i.e., generate and store) one lock request block (LRB) 162 for each lock held by a transaction.
• LCB lock control block
• LRB lock request block
• Each LCB 160 preferably includes: a lock ID 170, which will typically include a copy of the resource identifier for a locked resource; -11- a mode indicator 171 indicating the most restrictive access mode (e.g., browse, read, parameterized read, write, parameterized write, or exclusive) of all the locks granted on the database resource represented by this LCB; a read parameters indicator 172, preferably in the form of a bitmap, representing the logical OR of the read parameters being used by the parameterized read locks (if any) outstanding on the locked resource;
• a pending request list pointer 175 to a list (also called a queue) of LRB's 162 for pending (i.e., not yet granted) resource requests represented by this LCB;
• the read and write parameters represented by fields 172 and 173 in LCB 160 represent an extension by the present invention to the conventional access modes used by DBMS's, and are discussed in more detail below.
• Each distinct value of a defined set of read/write parameter domain represents a corresponding data reliability classification or category.
• a parameter domain having eight parameter values would allow for the definition of up to eight different data reliability classifications.
• the parameters may be used to indicate properties of a database object other than "reliability," such as the type or identity of the application which holds a write lock on the object, or other information that can be used by applications to determine if they are willing to read the data despite the presence of a write lock on it.
• Each LRB 162, representing one granted or pending lock request preferably includes:
• a mode indicator 181 indicating the access mode (e.g., browse, read, parameterized read, write, parameterized write, or exclusive) in which the resource is being accessed or being requested by a particular transaction
• a transaction identifier 184 which identifies the transaction that requested or holds the lock corresponding to this LRB
• a lockset pointer 185 that is used to form a linked list of all the LRB's owned by transaction identified by the transaction ID 184; and
• Typical sizes for the read and write parameter fields in the LCB's and the access mode parameter field in the LRB's are one to two bytes, supporting eight to sixteen distinct parameter values.
• Fig. 3 depicts an alternate data structure implementation 110' of the lock table.
• the lock control blocks (LCB's) 190 each contains a granted request bitmap 194 and a pending request bitmap 195 in place of the granted and pending request list headers 174 and 175.
• Each of the request bitmaps preferably contains a bitmap of sufficient size (e.g., 64 or 128 bits) to represent the maximum number of concurrent transactions supported by the system. Every active transaction is assigned to one of the bitmapped transaction identifiers.
• the granted request bitmap 194 contains a set bit for each active transaction that has been granted a lock on the resource represented by the LCB 190.
• the pending request bitmap 195 contains a set bit for each active transaction that has a pending lock request (also called an access request) on the resource represented by the LCB 190.
• the read and write parameter bitmaps 172 and 173, as in the Fig. 2 implementation, represent all the read and write parameterized access modes that have been granted on the database resource represented by the LCB 190.
• the read and write parameters in the LCB in question are efficiently updated by logically ANDing the read/write parameters of the granted lock request with the read/write parameter bitmaps previously stored in the LCB.
• the read and write parameters in the corresponding LCB are immediately updated. However, this would require ORing the read parameters of all remaining -13- read lock holders (if a read lock is released) and ORing the write parameters of all remaining write lock holders (if a write lock is release). Since this would put an undue delay on transaction commit processing, in a preferred embodiment the lock is released without updating the read and write parameter bitmaps in the LCB.
• the read and write parameter bit maps can be cleared, and if there are no pending lock requests, then the LCB can be eliminated altogether. Otherwise, the read and write parameter bitmaps in the LCB are updated with respect to lock releases only when a potential read/write or write/read conflict is detected, in order to figure out whether to grant the requested lock. Alternately, LCB read/write parameter bitmap updating can be handled as a background task, similar to garbage collection.
• a transaction history is defined to be serializable if it is equivalent to a serial transaction history. This definition only makes sense if we also define what it means for two histories to be equivalent and what it means for a history to be serial. Equivalence can be defined in more than one way. Two histories are "conflict equivalent” if:
• Two operations are defined to conflict if they do not commute. That is, if the results of executing one before the other is in general not equivalent to executing them in the reverse order. It is common to model transactions as consisting of read and write operations only, and the conflict relation for those two operations is that two operations conflict if at least one of them is a write operation.
• the basic lock compatibility matrix is:
• Table 1 indicates that two transactions can read the same data item, but a transaction that is performing write operations must have exclusive access to it.
• a transaction history H is serial if for every pair of transactions T, and T j , either all operations of T, come before all operations of T. in H, or all operations of T j come before all operations of T, in H.
• a transaction history is serial if it does not have any concurrency. Each transaction executes to completion before the next one starts. If transactions are atomic, durable and consistent, then a serial transaction history will be correct. It follows that a concurrent execution of transactions that is conflict equivalent to a serial one, must necessarily be correct, too.
• a transaction history that is conflict equivalent to a serial history is called conflict serializable, and the corresponding correctness criterion is called conflict serializability.
• a serialization graph (SG) for a transaction history H is denoted SG(H).
• This is a directed graph whose nodes are the committed transactions of H, and it has an edge between all pairs of nodes representing transactions that have issued conflicting operations. The direction of the edges are in accordance with the sequence of the conflicting operations.
• An edge from T, to T j in SG(H) means that T, as issued an operation that conflicts with and precedes some operation issued by T .
• T, and T j are involved in a cycle in SG(H)
• T comes both before and after T j in H, in which case H cannot be equivalent with any serial history.
• the fundamental theorem of serializability says that a history H is serializable if and only if its serialization graph SG(H) is acyclic.
• Two phase locking operates according to the following rules: 1 ) a transaction may not perform an operation on a data item unless it holds a lock corresponding to the operation (e.g., a read or write operation) in question on that data item; 2) a lock request from a transaction must be delayed or rejected by the transaction scheduler if another transaction holds a conflicting lock on the data item in question; and 3) a transaction may not acquire a new lock if it has released any of its old ones.
• a transaction may not perform an operation on a data item unless it holds a lock corresponding to the operation (e.g., a read or write operation) in question on that data item; 2) a lock request from a transaction must be delayed or rejected by the transaction scheduler if another transaction holds a conflicting lock on the data item in question; and 3) a transaction may not acquire a new lock if it has released any of its old ones.
• the first two rules prevent transactions from directly interfering with each other.
• the third rule called the two phase locking rule, prevents cycles in the serialization graph.
• the intuitive explanation of the two phase locking rule is as follows. When a transaction acquires a lock, that may establish an incoming edge to its node in the serialization graph. An outgoing edge from a transaction's node in the serialization graph can only be established if that transaction has released a lock. Thus, in order to create a cycle in the serialization graph, some transaction must first release a lock and then later acquire a lock. Since this is prohibited by the two phase locking rule, transaction schedulers that obey the two phase locking rule ensure acyclicity of serialization graph's, and therefore ensure serializable histories.
• the solution for maintaining a strict transaction history is to add the restriction that all write locks acquired by a transaction must be held until the transaction commits.
• a rigorous transaction history is maintained by requiring that all read and write locks acquired by a transaction be held until the transaction commits. With these added restrictions, two phase locking is called “strict two phase locking,” or strict 2PL.
• a nasty problem with transaction schedulers that use the two phase locking rule is that transactions can get involved in deadlocks.
• transactions sometimes have to wait for locks. Such waiting is caused by another transaction holding a conflicting lock, and the waiting transaction cannot make any progress until the other transaction releases its lock. If two transactions are waiting for each other, neither can make progress until the other one releases its lock. As long as neither of them releases its lock, the two transactions are deadlocked. More generally, deadlocks can involve more than two transactions that are waiting for each other in a cyclic way.
• isolation levels i.e., levels of isolation between transactions
• the present invention assumes the use of a lock manager for implementing isolation control.
• Isolation levels supported by commercial systems, ordered from least to most restrictive, include:
• UR uncommitted read
• This isolation level allows an application to read both committed and uncommitted data. As a result, the data read by an application (i.e., transaction) may be inconsistent with other data read by the application.
• Applications using the UR isolation level do not acquire any ordinary read locks, but will typically hold a browse lock at some high level in the resource hierarchy. Applications using the UR isolation level must still use write locks to protect write operations. There are two types of situations where UR isolation is most typically used: (1) when the application does not need an exact answer, and (2) when the application (i.e., the person who wrote the application) knows that the data to be read is not being updated by anyone else.
• CR transmitted read
• the committed read isolation level allows an application to read only committed data.
• CR isolation can be implemented using "zero duration" read locks. That is, if an application wants to read a database object, if suffices for the DBMS to check that a read lock could have been granted. If the read lock could have been granted, the object is read, but a read lock is not acquired.
• CR isolation is much less expensive than cursor stability (CS) isolation (described below) in terms of CPU cycles. Unless an application needs the additional semantics offered by cursor stability isolation (and many applications do not), committed read isolation is the better alternative.
• CS cursor stability
• Cursor stability isolation allows an application to read only committed data and guarantees that a row will not change as long as a cursor is positioned to it.
• CS isolation causes locks to be kept until the cursor moves to the next lockable object.
• CS isolation is useful for an application that fetches a row from a database table (using a cursor to point to the row) and then performs a database manipulation based on the current row's data values before fetching another row.
• An application should have at least CS isolation for cursors that are to be used to point to rows of data that are to be updated or deleted by UPDATE or DELETE operations. -17-
• RR isolation allows an application to read only committed data and guarantees that read data will not change until the transaction terminates (i.e., a read that is repeated will return the original row, unchanged). RR isolation will not prevent the so-called phantom row phenomenon. That is, when a cursor is reopened, a row not present the previous time may appear. Read locks covering data items retrieved by an application using RR isolation must be kept until the transaction commits or aborts.
• TC isolation is also known as serializable isolation.
• TC isolation allows an application to read only committed data and guarantees that the transaction has a consistent view of the database, as if no other transactions were active. Transactions using TC isolation may be part of a transaction history that is not serializable where other transactions use lower levels of isolation. On the other hand, if all transactions in a history use TC isolation, that history will be serializable. All read locks acquired by a transaction using TC isolation must be kept until the transaction commits or aborts.
• QC isolation allows an application to read only committed data, and guarantees that all data accessed in a single query is consistent.
• Implementing QC isolation by means of data locking is straightforward: all read locks must be kept until the query is completed (i.e., until the cursor is closed). Since cursors are defined using queries, QC isolation guarantees that all rows in a cursor's answer set are consistent. QC isolation is also valuable when performing statistical analysis, or when comparing or otherwise using together data values from different cursor row occurrences. QC isolation is weaker than RR isolation, but stronger than CS isolation.
• Isolation level CS or weaker, is often not suitable for any query that uses aggregation and for any application that processes a cursor where the answer set must be consistent. While using RR or TC isolation solves this problem, RR and TC actually provide more isolation than is needed for most such applications.
• the QC isolation level is the lowest isolation that provides the necessary and sufficient isolation for such queries.
• read and write There are two basic access modes: read and write.
• read mode When a database object is accessed in read mode, the agent in question can perform only read operations on that object. Two or more transactions may access a given object concurrently, provided they all use read mode.
• write mode When a database object is accessed in write mode, the transaction in question can perform both read and write operations on that object. More specifically, write mode enables reading, deleting, inserting and modifying objects. If an object is accessed in write mode by one transaction, no other transaction can access that object in either read or write mode.
• many DBMS's support browse, upgrade and exclusive access modes.
• Browse mode enables a transaction to read an object even if some other transaction is currently accessing it in write mode.
• transactions have no guarantee of a consistent view of the database, since there is a risk that they will read uncommitted data.
• the use of browse mode is often denoted as read through mode or dirty read mode, and is used with isolation level UR. Even an application using the UR isolation level needs to inform others of its presence, and it does so by accessing resources in browse mode.
• Upgrade mode is similar to read mode, with the added semantics that the transaction in question may at any time request an upgrade to write mode. That is, it may upgrade its access mode.
• a first transaction accesses an object in upgrade mode, no other transaction can access the same object in write mode, or upgrade mode, until the first transaction commits or aborts.
• Exclusive mode is used by a transaction to prohibit any other transactions from accessing the same object, irrespective of access mode.
• Exclusive mode is used when even browsers must be denied access to an object, such as when a table is to removed from a relational database.
• Asterisks (*) in Table 2 indicate compatibility. For example, read mode (R) access by a first transaction is compatible with browse (B), read (R) or upgrade (U) mode in a second transaction.
• Write mode (W) is compatible only with browse mode.
• Exclusive mode (X) is not compatible with any other access mode.
• the five access modes are typically implemented using locks. It should be noted that it is usually necessary to support a number of different lock granularities. Typical lock granularities are tuple (i.e., database table row), object, page, table, class, file, tablespace, and database. Unless it is required that all transactions use the same lock granularity, the DBMS must be able to coordinate concurrent transactions that request locks at different levels in the resource hierarchy. The typical solution is once a transaction requests locks at some resource level, it will not request additional locks on lower levels of the same resource (because other concurrent transactions may acquire other locks on portions of that resource that are compatible with the first lock put on the resource), but it may request locks at higher levels. It is possible for a single transaction to use different lock granularities for different statements, but this is not significant for the discussion at hand. -20-
• intent to request read (IR) locks indicate an intent to request read locks at some lower level
• intent to request write (IW) locks indicate an intent to request write locks at some lower level
• RIW which provides read access to the entire resource in question (e.g., a table) while also enabling the transaction to request update and write locks at some lower level (such as at the page or tuple level).
• an IW lock on a table enables its holder to request R, U and W locks (not just W locks) on tuples or pages with the table.
• An IW lock on a resource will be promoted to an RIW lock if the transaction holding the IW lock requests an R lock on the same resource.
• Two transactions with overlapping IW locks are considered to be compatible because the potential conflicts will be resolved at some lower level in the resource hierarchy. For example, two transactions may need to update various tuples in a relational table. The both acquire IW locks on that table (and probably also on some higher level resources, such as file or tablespace, as well as the database), and then R, U or W locks on individual tuples. As long as the two transactions do not access the same tuple, then there will no conflict. Should they happen to access the same tuple, then there will be a conflict, and the possibility of deadlock cannot be completely excluded. However, the potential deadlocks caused by overlapping IW requests are, in general, no worse than the potential deadlocks associated with other resource locking situations.
• RIW row/column is identical to the intersection of the R and IW rows/columns.
• browse (B) locks and exclusive (X) locks are not used at the lowest resource levels.
• X exclusive
• B locks at the lowest levels would, at least partially, defeat the purpose of using isolation level UR in the first place.
• a transaction using the UR isolation level may request a browse lock at the table level (and all levels above the table level), and then proceed without requesting any locks on pages or tuples.
• the transaction may request some sort of low-cost, short duration locks (known as latches) on pages or tuples to ensure atomicity of individual read operations.
• the present invention is based on the premise that serializability is an unsuitable correctness criterion for some types of applications, such as concurrent engineering, typified by CAD and CASE applications.
• the present invention uses a new correctness criterion, herein called conditional conflict serializability (CCSR) which is a weaker kind of serializability based on a weaker notion of conflict between transactions.
• CCSR conditional conflict serializability
• write-read and read-write -22- conflicts are made conditional. This is achieved by using "parameterized” read and write modes, and corresponding parameterized read and write locks.
• R(A) and W(B) denote parameterized read and write modes, respectively, where A and B denote subsets of some parameter domain D
• R(A) and W(B) are compatible if and only if B is a subset of A: B ⁇ A.
• the parameter domain D contains modes ul, u2, through u5, R(ul, u2) is compatible with W(ul) because ul c (ul, u2).
• a transaction history is defined to be "conditional conflict serializable” if and only if it is conditional conflict equivalent to a serial transaction history.
• a history H is serializable if and only if its serialization graph is acyclic.
• a generalized version of this theorem applies to CCSR.
• a conditional conflict serialization graph (CCSG) is defined in the same way as a regular serialization graph, provided the term "conflicting operations" is understood to mean conditionally conflicting operations.
• CCSR conditional conflict serializable
• CCSG conditional conflict serializable
• the parameterized access modes of the present invention make it possible for database users (i.e., typically, application programs) to receive information about the reliability of uncommitted data from the parameters the writer of the data has attached to the write lock on the data. More generally, the parameterized access modes of the present invention (A) enables application programmers to customize the notion of conflict between transactions, and (B) enables applications to communicate to each other the quality of uncommitted data. -23-
• the parameter domain D can be user defined, or defined differently in different database systems.
• the "data model" used can be simple or complex, and thus the number of parameters in domain D can be small or large, depending on the needs of the application programs.
• R(ul) and W(ul) are compatible, R(ul) and W(u2) are not compatible, and R(u2) and W(u2, u3) are not compatible.
• ⁇ on-parameterized read and write modes can still be denoted as R and W, but can be thought of as R(0) and W(*), where 0 denotes the empty set and * denotes an arbitrary superset of D.
• R(0) and W(B) are compatible if and only if B s A, R(0) is incompatible with all write modes and W(*) is incompatible with all read modes.
• W(D) denotes the read mode that is compatible with every write mode W(B) except W.
• Parameterized lock modes increase the amount of memory used by the lock manager to record each lock, and also increase the amount of computation required to resolve lock requests.
• the present invention does not force users (i.e., applications) to quantify uncertainty, but rather allows them to classify it. That is, users can denote unreliable data as belonging to one or more of a predefined set of reliability categories. Each such category is represented by a reliability classification indicator, that is by a parameter in domain D.
• the write locked data may initially be very unreliable (denoted by parameter ul), and then may be progressively upgraded to higher and higher levels of reliability (denoted by parameters, u2, u3, and so on) as the transaction progresses.
• parameterized read modes may be treated as a request for data filtering, such that objects that are locked in incompatible write modes are skipped by parameterized reads.
• -25- applications using parameterized read access requests may include an additional filter/wait flag to indicate if the read request is to be serviced by filtering out objects locked in incompatible write modes or is to wait for them to become available.
• the "uncommitted dependency problem,” the "dirty read” problem and the “temporary update problem” are three names for the same thing: the use of uncommitted data is unreliable because it is subject to further change, due either to transaction abort or further modification by the application.
• the real problem is that an application can retrieve uncommitted data "assuming” that it is reliable, and then go ahead and do something based on this assumption.
• the parameterized access mode method of the present invention explicitly informs applications when uncommitted data is encountered and also delivers information about the reliability of the uncommitted data.
• the application is free to handle such a situation in whatever way it sees fit: it may continuing as if nothing special happened, it may invoke some special procedure, it may perform a full or partial rollback, or do something else. Since no application will be deceived into believing that uncommitted data can be fully trusted, the uncommitted dependency problem is eliminated when using the present invention.
• the inco ⁇ ect summary problem and the inconsistent analysis problem are two versions of another problem: an application may retrieve multiple data items and use them as input to some calculation, such as finding a sum or average, but interference from a concurrent transaction may cause the calculated value to be inco ⁇ ect.
• This problem occurs when an updating application involves at least two data items: one that has already been retrieved by the analyzing application and one that has not yet been retrieved.
• One possible solution to this problem is to use an unparameterized read, which is incompatible with all write modes. However, this solution is undesirable or impossible in some situations.
• each transaction using a parameterized write mode of access for a database object specifies a write access mode, and a write access mode parameter, where the write access mode parameter indicates a reliability classification to other transactions that may request read access to the database object.
• Each transaction using a parameterized read mode of access for a database object specifies a read access mode, and a read access mode parameter, where the read access mode parameter indicates one or more reliability classifications that are acceptable to the transaction.
• the DBMS Whenever a transaction requests access to a specified database object, the DBMS generates a co ⁇ esponding lock request for the object (step 220). If the access request is a parameterized conditional access request, a co ⁇ esponding parameterized lock request is generated.
• the system's lock manager processes each lock request by searching the lock table (steps 222, 224) for any co ⁇ esponding previously generated locks. Next, it checks to see if any previously granted lock is conflicting or potentially conflicting with the requested lock (steps 226, 230, 232). Two lock requests are unconditionally conflicting if their resource ranges overlap and the access modes of the two requests are incompatible (step 226). For example, the blank positions in Table 4 represent unconditionally conflicting access requests. In terms of the data structures shown in Figs. 2 and 3, the access mode of the cu ⁇ ent request is compared with the most restrictive access mode previously granted for each of the overlapping resources for -27- which any locks have been granted.
• the cu ⁇ ent access request is for an upgrade (U) mode or a write (W) mode, and there is already a granted lock for an overlapping resource that has a U or W mode
• the cu ⁇ ent access request is unconditionally conflicting with the previously granted lock. If the lock request being processed is unconditionally conflicting with any outstanding, previously granted lock (step 226), the lock request is put on a queue of pending requests (step 228).
• Two requests are conditionally conflicting if analysis of their read/write parameters is necessary to determine whether a conflict exists (step 230).
• the read/write parameter comparison performed for each pair of potentially conflicting requests are those shown in Table 4 (step 232). That is, the conditional conflict is resolved by determining whether the write parameters for the write lock in question are a subset of the read parameters for the read lock in question. If so, then there is no conflict. If not, then the requested lock is in conflict with the outstanding previously granted lock. If none of the outstanding, previously granted locks is in conflict with the requested lock, the requested lock is granted (step 234). Otherwise it is put on a queue of pending lock requests (step 228).
• the definitions of compatible read and write modes in CCSR can be generalized.
• the R(A) and W(B) read and write modes are defined to be compatible if and only if A is equal to B; 3) the R(A) and W(B) read and write modes are defined to be compatible if and only if the intersection of A and B is empty; 4) the R(A) and W(B) read and write modes are defined to be compatible if and only if the intersection of A and B is non-empty; or -28-
• the R(A) and W(B) read and write modes are defined to be compatible if and only if f(A,B) yields a True result, where f(A,B) is any suitably defined truth-valued function.
• the access mode compatibility matrix for alternate embodiment number 5 which is actually a generalized representation of all the embodiments, is shown in Table 5.
• * indicates unconditional compatibility
• blank table entries indicate unconditional incompatibility
• formulas such as f(B2,Al) and f(Bl,A2) indicate conditional compatibility (i.e., the modes are compatible if the function f() evaluates to True).
• parameterized lock modes increase the amount of memory used by the lock manager to record each lock, and also increase the amount of computation required to resolve lock requests.
• a "data lock” is a mechanism for assigning a transaction certain rights to a database object, such as a table, a page, or a datum or record in a database table.
• a first transaction may lock a particular object so as to ensure that no other transaction accesses the data in that data -29- until the first transaction commits or aborts.
• the prior art includes many types of data locking mechanisms.
• An "access mode” refers to the way in which a transaction or application accesses a data resource.
• the traditional access modes are browse, read, update, write and exclusive.
• Other access modes include parameterized read and write access modes. These are described above. For the purposes of this document, it is sufficient to know that when a transaction accesses data using a parameterized write access mode, the value of the associated write access parameter may be suitable for interpretation as a data reliability classification or category for the data values covered by the write lock held on that data.
• the DBMS provides a read/write parameter domain having eight parameter values (each represented by a co ⁇ esponding bit of an eight-bit parameter field), that system would allow for the definition of up to eight different data reliability classifications for data covered by parameterized write locks (i.e., there might be additional data reliability classifications for data not covered by parameterized write locks).
• the present invention annotates both data and logical values that are less than fully reliable. Instead of replacing these values with a special "unknown value,” they are annotated with a meta-value representing a reliability classification.
• Meta-data provides information about the data that is being manipulated.
• Fig. 5A represents one possible schema for representing the reliability of a meta-data value.
• a meta-datum 350 is encoded as a data value 352 and a meta-value 354.
• the schema 360 for the meta-datum which would typically be stored as part of the schema for a database table or object, includes: a data type 362 for the data value 352, and a meta-value type 364 specified using a bitmap representation.
• the meta-value bitmap has five defined bit fields.
• the first two bits of the meta-value are used to indicate cases of missing data, while the third through fifth bits are used to indicate degrees of reliability of data that is not missing. More specifically, the first bit is set when a datum's value is not applicable, the second bit is set when a datum's value is unknown, the third bit is set when the reliability of a datum's value is poor, the fourth bit is set when the reliability of a datum's value is medium, and the fifth bit is set when the reliability of a datum's value is good.
• Fig. 5B represents some examples of meta-data values. Note that a data value that is 100% reliable is indicated by a zero meta-value.
• the meta-value associated with a meta-datum will be encoded so as to represent a data reliability classification, a data uncertainty classification, or the like.
• the meta-value need not be represented as a bit map. However, if the number of distinct data reliability classifications is small (e.g., less than or equal to the number of bits in a memory word), the bit map representation may be convenient and inexpensive to implement. -31-
• a data reliability parameter may be associated with a single datum, with a database tuple, or with a larger set of data.
• the present invention uses a form of computational logic herein called annotated nullable logic (ANL) for combining meta-data values.
• NNL annotated nullable logic
• the memory 114 of the system which will typically include volatile memory (e.g., high speed random access memory) as well as non-volatile memory (e.g., disk storage), will typically contain: an operating system 130 for handling platform dependent tasks as well as typical operating system functions such as memory management, file system management, input/output tasks, and the like; • application programs 132, which use the DBMS to store and retrieve data, perform transactions and resource management;
• volatile memory e.g., high speed random access memory
• non-volatile memory e.g., disk storage
• an operating system 130 for handling platform dependent tasks as well as typical operating system functions such as memory management, file system management, input/output tasks, and the like
• application programs 132 which use the DBMS to store and retrieve data, perform transactions and resource management
• a query parser 134 which parses and compiles queries received from application programs and users into execution plans
• a query executer 136 which performs database access operations, including query predicate processing; and other programs not directly relevant to the present discussion.
• the annotated nullable logic techniques of the present invention are primarily embodied in the query executer 136, and possibly also in the application programs 132, depending on the division of computational duties between the application programs 132 and the query engine 134/136 of the DBMS.
• Fig. 1 shows the query executer 136 as including an ANL module 138 for evaluating ANL expressions, also called meta-data expressions.
• the ANL evaluation logic in the query executer 136 (and in the application program 132) may be implemented as in-line code within the query executer, instead of as a separate module or function.
• the result is another non-annotated value. More generally, when two values (also called operands) of the same reliability classification are combined, a result with the same reliability classification results.
• a mathematical data combining operation is performed on a non-annotated value and an annotated value, the non-annotated value is explicitly or implicitly converted into an annotated value (with a reliable data meta-value) and then the two values are combined.
• the present invention does not require special handling for unknown and missing values.
• the result is unknown. For instance, if both the "unknown" and “medium” reliability classification bits for -33- a result are set, the result is unknown because the "unknown" classification is dominant over the "medium” classification.
• execution of an SQL query may require the expression x ⁇ 4 to be evaluated for all the rows in a table (assuming the table has a column called x). For each row of the table, this expression will evaluate to either True or False. Alternatively, if the DBMS in question supports null values, the expression may sometimes (i.e., for some rows of the table) evaluate to a third logical value: unknown.
• Table 7 contains some examples of comparisons of meta-data values (operands) using annotated nullable logic.
• the truth value part of the result generated by each comparison is equal to True, False or N/A (indicating that the truth value, if any, is meaningless).
• the meta-value for each comparison result is generated by performing a bitwise union (i.e., a bitwise logical OR'ing) of the meta- values of the operands.
• a truth value generated by a meta-data comparison is annotated with -34- two or more reliability classifications, in most application environments it is only the least reliable classification that is significant.
• the meta-value for the result is generated by performing a bitwise union (i.e., a bitwise logical OR'ing) of the meta-values of the operands.
• a bitwise union i.e., a bitwise logical OR'ing
• the meta-value for the result is generated by selecting one of the operands' meta-values (e.g., in the case of the OR operation, the meta-value for the operand having the strongest truth value is selected).
• Table 9 sets forth the rules that govern the combining of truth values in the prefe ⁇ ed embodiment.
• the annotation bitmap for data reliability value C is generated by forming a bitwise union of the bitmaps for operands A and B.
• the bitmap for data reliability value C is generated by selecting one operand's annotation bitmap (e.g., in the case of the OR operation, the annotation bitmap for operand having the strongest truth value is selected).
• Strength of truth values f ⁇ qf ⁇ u ⁇ qt ⁇ t. 1, m each mean a truth value that may be True, False, Quasi True, Quasi False, or unknown, with the following relative strength of truth: l ⁇ m ⁇ True.
• N is a non-empty set of meta-values that indicate missing information (as opposed to unreliable information)
• M is a non-empty set of meta-values that indicate unreliable information (as opposed to missing information)
• the ANL could instead generate a result meta-value representing the lowest reliability category associated with the operands. This would require the reliability categories to be ordered with regard to reliability to enable a determination of the lowest applicable reliability category.
• the present invention could be used even in systems having just two levels of data reliability: reliable and unreliable, or reliable and missing.
• meta data values would be annotated with a data reliability value that could be as short as a single bit (e.g., equal to 0 for reliable data and 1 for unreliable or missing data).
• a data reliability value that could be as short as a single bit (e.g., equal to 0 for reliable data and 1 for unreliable or missing data).
• the advantages of the present invention over systems using three valued logic are most readily appreciated when at least three distinct levels or categories of data reliability are used.
• the present invention may be used in conjunction with a database management system that utilizes nested databases.
• a database management system that utilizes nested databases.
• the nested database mechanisms disclosed in the Doctoral thesis by Ole J ⁇ rgen Anfindsen, "Apotram - an Application-Oriented Transaction Model," Chapter 5, “Nested Databases,” UNLK Center for Technology at Kjeller, Department of Informatics, University of Oslo, Norway, Research Report 215, 1997 could be used in conjunction with the present invention, and the aforementioned Doctoral thesis is hereby incorporated by reference as background information.
• the present invention may be implemented in DBMS's, transaction processing monitors, persistent programming languages, and concu ⁇ ency control services for object resource brokerage systems. It may be distributed either in the form of such systems, or as a computer program product that is distributed on computer readable media such as magnetic data storage -38- media (i.e., disks or tapes), CD-ROM, or the like.
• a computer program product embodiment of the present invention may be distributed electronically, over the Internet or other communications network, as a computer data signal embodied in a carrier wave.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Data Mining & Analysis (AREA)
• Databases & Information Systems (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Applications Claiming Priority (5)

Publications (1)

Family

ID=26685121

Family Applications (1)

Country Status (5)

Families Citing this family (34)

Family Cites Families (3)

• 1999
  • 1999-01-25 JP JP53822699A patent/JP2001523367A/ja active Pending
  • 1999-01-25 WO PCT/NO1999/000018 patent/WO1999038095A1/en not_active Application Discontinuation
  • 1999-01-25 EP EP99932435A patent/EP0990214A2/de not_active Withdrawn
  • 1999-01-25 AU AU32789/99A patent/AU763133B2/en not_active Ceased
  • 1999-01-25 CA CA002285019A patent/CA2285019A1/en not_active Abandoned
• 2003
  • 2003-07-28 AU AU2003227293A patent/AU2003227293A1/en not_active Abandoned

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events