Connect public, paid and private patent data with Google Patents Public Datasets

Caching of distributed dynamic sql statements in a multiple node rdbms.

Download PDF

Info

Publication number
US20020026448A1
US20020026448A1 US09162165 US16216598A US20020026448A1 US 20020026448 A1 US20020026448 A1 US 20020026448A1 US 09162165 US09162165 US 09162165 US 16216598 A US16216598 A US 16216598A US 20020026448 A1 US20020026448 A1 US 20020026448A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
node
sql
cache
database
variation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US09162165
Other versions
US6353833B1 (en )
Inventor
Paul Bird
Michael Snowbell
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/30286Information retrieval; Database structures therefor ; File system structures therefor in structured data stores
    • G06F17/30386Retrieval requests
    • G06F17/30424Query processing
    • G06F17/30427Query translation
    • G06F17/30433Access plan code generation and invalidation; reuse of access plans
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/30286Information retrieval; Database structures therefor ; File system structures therefor in structured data stores
    • G06F17/30386Retrieval requests
    • G06F17/30424Query processing
    • G06F17/30477Query execution
    • G06F17/3048Database cache management
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99938Concurrency, e.g. lock management in shared database
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99951File or database maintenance
    • Y10S707/99952Coherency, e.g. same view to multiple users

Abstract

A method and system for caching dynamic SQL statements and their sections at different nodes in a multiple node Relational Database Management System (RDBMS) while maintaining cache and application integrity across all nodes. Each node in the RDBMS having a global cache shared by all applications on that node. One of the nodes in the RDBMS being a catalogue node which controls access to a database. The catalogue node maintaining a list of participating nodes that have requested access to the database controlled by the catalogue node. The list of participating nodes being used to notify each participating node of any changes to the database that may affect the validity of a dynamic SQL statement stored in the global cache of the node.

Description

    FIELD OF THE INVENTION
  • [0001]
    This invention relates to relational database management systems (RDBMS) and more particularly to a method for caching dynamic SQL statements (and their sections) at different nodes in a multiple node database while maintaining cache and application integrity across all nodes.
  • BACKGROUND OF THE INVENTION
  • [0002]
    Structured Query Language (SQL) is the database access language most commonly used to access relational databases (such as the DB2 product sold by IBM Canada Ltd.) in an open, heterogeneous environment. Although this disclosure refers to the DB2 relational database product sold by IBM, individuals skilled in the art will recognize that the caching of database access statements is applicable to any relational database management system (RDBMS).
  • [0003]
    Within this specification including the claims, the following terms will be used:
  • [0004]
    Access Plan An access plan is the method(s) chosen by the SQL Compiler to satisfy an application request as stated in the form of an SQL statement.
  • [0005]
    Agent A process used by a RDBMS to provide services for an application request.
  • [0006]
    Node A node is a physical entity (eg. a processor and memory) that is used to process application requests to the RDBMS and contains some or all of the database. A serial version of the RDBMS contains at most 1 node, while a parallel version of the RDBMS can contain 1 or more nodes.
  • [0007]
    Package A package is associated with an application and contains the information required by the RDBMS for all SQL statements defined in that application. The information in a package consists of a collection of sections and the compilation environment settings (e.g. compile or binding options) used to compile any static SQL statements; some of the same settings are also used as the default environment for any dynamic SQL statements compiled by the application during execution.
  • [0008]
    Section A section contains all the information required by the RDBMS to execute the chosen access plan for an SQL statement. A section is the compiled version of the access plan chosen by the SQL compiler.
  • [0009]
    Section Entry A section entry contains information about a specific section as well as the SQL statement corresponding to that section.
  • [0010]
    There are two basic types of SQL statements, static and dynamic. In using static SQL the user embeds SQL requests for data in an application program. An SQL precompiler removes these statements from the application program and replaces them with function calls whose parameters indicate a specific section entry for the package corresponding to the current source file. The removed SQL statement is then sent to DB2 for compilation. Compiling (also known as preparing) a SQL statement is the process by which the DB2 SQL compiler chooses and builds an access plan to efficiently resolve the SQL statement. The access plan is saved in its executable format, a section, in the system catalogues.
  • [0011]
    The parsing of the statement and building of the access plan can be relatively long and complicated. Compilation of static SQL improves run time performance by building the access plan before the application is executed.
  • [0012]
    Dynamic SQL is generally used for ad hoc SQL requests. For example, in a database used to track sales of individual products, a dynamic SQL query may be invoked to list the top ten individual products sold, by sales region. Depending upon the nature of a dynamic SQL request, the time required to parse it and create an access plan to satisfy the user request can be significant. Further, if the dynamic SQL request is repeated later in the application by the same agent or perhaps by a different agent, a new access plan must be seated in each instance. Thus, the creation of an identical access plan may often have to be repeated, thereby impacting performance of the application.
  • [0013]
    A dynamic SQL request originates on the node running an application and may require data from one or more other nodes (the remote nodes). In such a scenario, the prior art solution is to generate the section to obtain the data from the remote node(s) and send the section to the remote node(s) for execution. The inventors are not aware of any solution proposed to date which considers allowing caching of the same statement across multiple nodes, thus not requiring the section to be executed to be shipped. Maintaining multiple iterations on multiple nodes result in less communications traffic and faster response times for remote parts of a dynamic SQL request due to faster startup times.
  • SUMMARY OF THE INVENTION
  • [0014]
    In accordance with the present invention there is provided a database system comprising:
  • [0015]
    a) a plurality of nodes;
  • [0016]
    b) a plurality of databases stored on the nodes; and
  • [0017]
    c) a global cache stored on at least one node, the global cache being accessible to a plurality of applications in the database system.
  • [0018]
    In accordance with the present invention there is also provided a method of maintaining fiche and application integrity across a relational database system comprising a plurality of nodes and a plurality of databases shared on the nodes, the method comprising the steps of: (a) providing, for at least some of the databases, a global cache including information to enable agents to access the databases, (b) originating a request through a coordinating node, the coordinating node sending information on the variation to be executed to remote nodes; (c) having each remote node receive the request and check the global cache on the respective remote node for a current variation; and (d) if the a remote node does not have the current variation, the remote node requesting the coordinating node to send the current version of the variation.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0019]
    Reference will now be made, by way of example, to the accompanying drawings which show a preferred embodiment of the present invention and in which:
  • [0020]
    [0020]FIG. 1 is a conceptual diagram of a Relational Database Management System;
  • [0021]
    [0021]FIG. 2 is a conceptual diagram of a node of the Relational Database Management System of FIG. 1;
  • [0022]
    [0022]FIG. 3 is a conceptual diagram of the global cache;
  • [0023]
    [0023]FIG. 4 is a conceptual diagram of the statement portion of the dynamic cache portion of the global mesh;
  • [0024]
    [0024]FIG. 5 is a conceptual diagram of the dependency portion of the dynamic cache portion of the global cache; and
  • [0025]
    [0025]FIG. 6 is a conceptual diagram of a catalogue node, the participating nodes list and the participating nodes.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
  • [0026]
    [0026]FIG. 1 is a conceptual diagram of a Relational Database Management system (RDBMS) 10. The RDBMS of 10 of FIG. 1 illustrates only one of many configurations available in an RDBMS and is intended only to show a networked system that may utilize the present invention. In the configuration as shown, RDBMS 10 contains a plurality of nodes (12, 14, 16, 18). Each node (12, 14, 16, 18) may contain a partition or all of a database. In the example of FIG. 1, the RDBMS has two databases, although any number of databases can be provided, the content of which is partitioned among the plurality of nodes. The first of these databases is divided into a partition 20 and a partition 21. The catalogue 22 for this first database is stored on catalogue node 18. The catalogue 22 contains tables in which resides all the “meta-data” such as the structure, tables packages and functions relating to the first database. The second database is divided into partitions 23, 24 and 25. Note that in the illustrated example, the partition 24 of the second database also contains the catalogue tables for the second database. Thus, a catalogue node 18 may contain data as well as catalogue tables. Catalogue tables are per database and can be located at different nodes for different database. This means that the term “catalogue node” is relative per database, i.e. the catalogue could reside on any node.
  • [0027]
    [0027]FIG. 2 illustrates one scenario of the activity that may occur on any node of the system and, by way of example, the node 16 of the RDBMS system 10 of FIG. 1 is shown. Node 16 may contain any number of applications 30, each accessing data from the database(s) to which the node is connected. Each application 30 has a respective SQL work area 32. Also contained within node 16 is a global cache 40 in accordance with the present invention which is shared by all applications on all the nodes.
  • [0028]
    The Global Cache
  • [0029]
    The global cache 40 acts as a repository for package and section information for static SQL and statement and section information for dynamic SQL as detailed below.
  • [0030]
    In the preferred embodiment, the global cache is kept at the database level, and is accessible to all agents for a database at a particular node of the database (physical or logical). It is allocated from the same memory set as other database-level entities, such as the lock list or buffer pool; it is created when the database is initialized, and it remains active until the database is shut down. The global cache acts like a “public” library for all the agents using the database at a given node. Agents simply copy the package information and the modifiable portions of the sections (e.g. buffers and flags.)
  • [0031]
    Referring to FIG. 3, the global cache 40, consists of two logical areas; the static SQL cache 42 and the dynamic SQL cache 44. The static cache 42 contains the information for packages 46, section entries 48, and static SQL sections 50. Thc dynamic SQL cache 44 contains the information and sections for dynamic SQL statements.
  • [0032]
    The dynamic SQL portion 44 of the global cache 40 is subdivided into two portions: the statement portion 60 (FIG. 4) and the dependency portion 58 (FIG. 5). The statement portion 60 contains SQL statement entries 62 which contain the text of the cached dynamic SQL statements 64 as well as thc different sections compiled for each SQL statement. The statement portion 60 of the dynamic cache 44 is used to support application requests to prepare the dynamic SQL statement 64 and obtain an executable section. The dependency portion 58 of the dynamic cache 44 contains entries for all the objects upon which the cached dynamic SQL sections are dependent. This information is used Lo support cached object invalidation due to data definition language statements and other database activities.
  • [0033]
    The primary structure of the dynamic SQL portion 44 of the global cache 40 is based upon the unique text for a given SQL statement 64, which is stored within SQL statement entry 62. A SQL statement entry 62 contains characteristic information about the SQL statement 64 that is determined once the first occurence of the statement text 64 is compiled. This information is invariant since thc statement text 64 is constant and these characteristics are inherent in the statement text 64 itself, independent of the environment in which it was compiled.
  • [0034]
    A section generated for the exact identical SQL statement text 64 is stored within the compilation environment 66. The compilation environment 66 contains information on the environmentL used to generate or compile the section. The compilation environment 66 contains all information not already contained in the catalogue tables, that influence or control an access plan created by the SQL compiler, and thus the section.
  • [0035]
    Below each compilation envionment 66 are stored individual units known as “variations” 68. A variation 68 represents a unique section for the statement text 64, where the uniqueness is specified by the compilation environment 66 used to generate the section. The compilation environment 66 encompasses all those elements that affect the nature and result of the section generated by the SQL compiler for a dynamic SQL statement (e.g., special registers, relevant package compilation defaults, use of default qualifiers, use of function path, etc.). Note that since privileges do not affect the actual section generated, only whether it is allowed to be executed, the prerequistite privileges for a dynamic SQL statement 64 are irrelevant to the compilation environment 66. In the preferred embodiment, privileges are not part of the compilation environmnt 66.
  • [0036]
    A variation 68 is simply the representation of the fact that two applications issuing the same dynamic SQL statement 64 in the same context with the exact same compilation environment 66 should generate the exact same section, although the results of execution may differ due to actions within the individual units of work. For example, two applications using the same section for SELECT C1 FROM T1 may return different results if the first application has inserted some rows into T1 but not committed its changes. The first application will get the row returned by the section, but the second application may not.
  • [0037]
    In addition to the identifying compilation environment, each variation 68 under a statement entry 62 and compilation environment 66 also contains: the required privileges list of privileges needed to execute the section, the dependency list for the section and the section generated for the specified compilation environment. The dependency list refers to those objects and entities in the catalogue tables required, either directly or indirectly, by the section for a variation 68. The dependency list is also used to determine whether the section is no longer valid when an object on the dependency list is dropped; if the variation 68 is in use at the time, the drop request is rejected. When a section is no longer valid, the variation 68 becomes an invalid variation 70 and must be regenerated. The dependency list functions in much the same manner as the package dependencies that are recorded for static SQL statements in the SYSCAT.PACKAGEDEP catalogue table of DB2. For static SQL the SYSPLANDEP table is scanned to locate packages dependent upon the affected object and the packages are invalidated. Details of the dependency list and its use to invalidate dynamic SQL sections is discussed below in the description of FIG. 5. The major exception is that the loss of privileges does not result in any variation 68 being marked as an invalid variation U since, as noted previosuly, privileges do not affect the contents of the section. Since a variation 68 represents a dynamic SQL statement, and dynamic SQL always reflects the current environment, other actions such as issuing thc RUNSTATS command against a table or creating a new User Defined Function (UDF) can cause a variation to be marked invalid.
  • [0038]
    Each dynamic SQL statement entry 62 may have one or more compilation environments 66, but each compilation environment 66 may contain only one valid variation 68. An invalid variation 70 will be flushed from the cache during cache space management. For example, an ALTER TABLE statement can cause a variation 68 to become an invalid variation 70 in the dynamic cache 44 and will eventually be flushed from the dynamic cache 44.
  • [0039]
    Dynamic SQL always reflects the current environment, i.e. the section for a dynamic SQL statement always represents the choices the SQL compiler would make given the most current environment. This is why a dynamic SQL section is invalidated when a new index is added or statistics are updated. The sections are fine in the sense that they will work, but the environment has changed and the sections may not reflect new choices that thc compiler might make.
  • [0040]
    Referring now to FIG. 5, the basic unit of the dependency portion 58 of the dynamic SQL cache 44 is the object 72. An object 72 represents a database object upon which a dynamic SQL variation 68 is dependent. Each object 72 is associated with one or more variations 68. Each object 7 in the dependency portion 58 of the global cache 40 is distinct and only one occurrence of the object 72 may appear at any one time. To facilitate access, each object 72 is associated with a specific object type anchor point 74 by hashing the qualified object name 76 to a unique anchor point 74. Once associated with an anchor point 74, the object 72 is inserted into the list of objects 78 off that anchor point 74 in alphabetical order based on the object name 76 followed by the qualifier. Anchor points 74 are type specific (i.e. an alias anchor point only holds aliases) and only objects 72 of the type represented by the anchor point 74 are to be found off that and/or point 74. In the present embodiment, objects of types: view, alias, index, User Defined Functions (UDF), table, structured types, summary tables, hierarchiess and pagesize are supported.
  • [0041]
    An essential characteristic of a dynamic SQL statement is that it reflects the current database environment and the dynamic SQL cache 44 does not alter this characteristic. If an environment change affects a dynamic SQL section, the variation 68 for that section will become an invalid variation 70. When invalidating a dynamic SQL variation 68 due to a change in a dependent object, the name of the affected object 76 is hashed to obtain the appropriate anchor point 74 for the same object type. The list of objects 78 is then scanned to find a matching object 72. Once a matching object 72 is found, the list of dependant variations below that object is scanned and an attempt is made to invalidate them.
  • [0042]
    Referring now to FIG. 6, when a section needs to be executed at a remote node (24, 28), a request to execute the same variation as the current variation is sent to that node by a coordinator node 92. The coordinator node is responsible for coordinating all database requests made by an application on that node. When the remote node (24, 28) receives the request, the list of shipped variations contained within the global cache 40 is searched for a matching entry from the coordinator node 92. If none is found, then the remote node (24, 28) issues a request to the coordinator node 92 to send the specified variation and section to it. A long with the section, there is also sent the dependency information and required privileges information. When this information is received at the remote node (24, 28), it is inserted into the cache 40 at that node (24, 28) as a complete entry (i.e. as if it had originated at the remote node (24, 28); all information is in the cache including privileges and dependencies). At this point, the normal actions taken for invalidating events will process the transhipped variation just like any other in the remote nodes (24, 28) cache 40.
  • [0043]
    The Participating Nodes List
  • [0044]
    In order to minimize network traffic and to allow for the ignoring of nodes that are either down or have not connected to a particular database, a list of participating nodes 94 for a database 20 is, built and maintained on the catalogue node 22. A participating node is a node which is active, conencted to the database 20 and has registered in the participating nodes list 92 at the database catalogue node 22.
  • [0045]
    The list of participating nodes 94 is used to define which nodes must be informed when a communication bulletin (described hereinbelow) is to be broadcast for a database 20. The information is broadcast to all nodes in the list of participating nodes 20 other than the current one (the catalogue node 22) by the agent wishing to send the information. Before any broadcast, the broadcasting agent musy acquire the latch for thc list of participating nodes 92 to prevent any new nodes from being added. Processing for the cache 40 at the catalogue node 22 is handled by the broadcasting agent.
  • [0046]
    Every time a node becomes active, a REGISTER request is sent to the catalogue node 22 to update the list of participating nodes 94 by adding an entry for this newly active node; the response to this request will contain the current values of all global relevancy counters (described hereinbelow) for the database 20. When a node becomes inactive, an UNREGISTER request is sent to the catalogue node 22 to remove the node from the participating nodes list 92.
  • [0047]
    When an action occurs that could invalidate any variation, an event broadcast is sent from the catalogue node 22 to all active nodes using all of the nodes in the participating nodes list 94 to ensure that all affected variations at all nodes arc properly invalidated. If a node is not registered in the participating nodes list 94, then it has no global cache 40 and thus does not need to be informed of changes until it registers.
  • [0048]
    All information broadcasts for a database 20 must be done from the catalogue node 22 and use the list of participating nodes 94. If an error occurs during the broadcast, the transaction fails and is rolled back
  • [0049]
    Dynamic SQL in a Parallel Environment
  • [0050]
    Dynamic SQL statements in a parallel environment require that the identical section be executed on all participating nodes. To ensure this behaviour, the coordinator agent (the agent dealing directly with the application) will place the needed identification for the variation to be executed in the request sent to cache remote node, where a remote node is a node working for the application that is not the coordinator node. The remote nodes will request that this variation be loaded into their SQL Work Area (32 of FIG. 2, which will be in the application's common memory in most cases). The node's dynamic SQL cache 40 is then chocked for this variation, if it is not there, a request is made to the coordinating node for a copy of the variation from that node which is inserted into the requesting node's dynamic SQL cachce 40 upon receipt. Once loaded into the SQL Work Area (32 of FIG. 2), this section will be used by all agents executing in parallel for this application at this node. If the variation is not found at the coordinating node (or is not valid), an error will be returned to the requesting agent.
  • [0051]
    The shipped ID sent by thc coordinating agent with the request to execute consist of the originating node, the statement ID and variation ID at the node of origin, and the insertion time of the variation into the global SQL cache at the node of origin as well as the specific subsection to be executed. A subsection is a logical sub-component of a section. Requests are made to different agents specifying different subsections. The insertion time is required to ensure the identical variation is used at alI nodes as the one specified by the coordinator node in its request. No check on the validity of the variation is performed as it is assumed to be still protected by the coordinator node; however, the status of the variation inserted in to the local node's global SQL cache will be identical to its value at the node of origin (e.g. an invalid variation will be inserted as invalid and marked as such).
  • [0052]
    Since the specified variation is from another node, it may or may not exist at the current node and if it does exist at the current node, it will have a different variation ID. To help map from shipped IDs to valid variation IDs at a node, a list of shipped IDs and the associated variation ID at this node is linked from a control block (not shown) within the statement portion 60 of the global cache 40, to help agents find the desired variation quickly. Once the variation ID for this node is known, the variation is handled exactly like any other variation.
  • [0053]
    If a slipped ID is not found in the list, the variation is requested from the node of origin and inserted into the dynamic SQL cache of the requesting node. When the variation is requested, the following information will be sent to the requesting node in order to allow complete support of the shipped variation in the cache for the requesting node:
  • [0054]
    Statement Length
  • [0055]
    Statement Text
  • [0056]
    Statement Information
  • [0057]
    Compilation Environment
  • [0058]
    Required Privileges List
  • [0059]
    Dependency List
  • [0060]
    Section Data
  • [0061]
    Relevancy Counters
  • [0062]
    In order to ensure that database changes are recognized and communicated to all agents working on a database in a simple yet effective manner, a number of relevancy counters are used. Relevancy counters are a simple method of determining whether past decisions or cached information are still relevant in the current environment. When an object is cached or a privilege is checked, the value of the appropriate relevancy counter is stored at that time. When the cached object or decision point is next revisited, the stored value is compared to the current value. If they are the same, then the old object or decision is still valid, else it must be revisited. Relevancy counters are only meaningful to coordinator agents. There are two types of relevancy counters; those based on global, shared database information, and those focussed on application/connection information.
  • [0063]
    The following counters are global across the database and are maintained at ale catalogue node.
  • [0064]
    Database Privileges Counter
  • [0065]
    Reflects revocation actions against the SYSDBAUTH catalogue table.
  • [0066]
    Package Privileges Counter
  • [0067]
    Reflect revocation against the SYSPLANAUTH catalogue table.
  • [0068]
    User Privileges Counter
  • [0069]
    Reflects revocation actions against the SYSUSERAUTH catalogue table.
  • [0070]
    Statistics Iteration Counter
  • [0071]
    Reflects changes to statistical information either through an Update Statistics statement or a call to RUNSTATS
  • [0072]
    Any change to the value of one of these counters must be broadcast to all nodes of the database in order to ensure database and application relevancy.
  • [0073]
    The following counters are unique to a specific application and are only referenced or updated by the coordinator agent. They are used to determine if a new compilation environment exists for the application or whether any SQL statements being compiled are possibly dependent on uncommitted Data Definition Language (DDL) performed by the application in this Unit of Work.
  • [0074]
    Compilation Ennvironment Counter
  • [0075]
    Reflects changes in the application compilation environment for dynamic SQL that affect the compilation environment used to select variations. An example would he if the application issued a SET CURRENT SCHEMA statement which changes the default qualifier to be used for unqualified object references in any future dynamic SQL statements issued by the application.
  • [0076]
    DDL in UOW Counter
  • [0077]
    Reflects the sentence of DD1, actions by the application within the current unit of work.
  • [0078]
    These counters are kept with other application information that is shared by all agents working for that application. These counters are used to determine if a new variation is required during a unit of work due to DDL or SRT statements issued by the applications since the beginning of current unit of work.
  • [0079]
    Invalidation of a Variation
  • [0080]
    A variation for a dynamic SQL statement can become invalid for a number of reasons, some of which relate to a change in the compilation environment in which the variation was created. Since dynamic SQL must reflect all changes in the compilation environment, as these changes occur a variation may become invalid and be eligible to be flushed from the dynamic SQL cache.
  • [0081]
    All invalidating events are broadcast from the catalogue node to participating nodes once the invalidation event has occurred: i.e. the invalidation event is broadcast as part of the processing for an invalidation event.
  • [0082]
    Inter-node Communication Messages
  • [0083]
    There are a number of cache synchronization messages used to pass information between global SQL caches on different nodes. Synchronization messages are used to inform the cache at a node that a change has been made to the database environment and action needs to be taken to ensure the cache at this node remains relevant to the current environment. Upon receipt of a synchronization message that affects cached dynamic SQL statements, the receiving node will search the depedency portion 58 of the dynamic SQL cache 44 to determine which variations, if any, are affected by the message contents; these variations are invalidated. Upon receipt of a message affecting a cached package, the receiving node will perform the indicated action on the specified package.
  • [0084]
    The format of a synchronization message is,
  • [0085]
    Node of Origin
  • [0086]
    Application ID
  • [0087]
    Originating Agent ID
  • [0088]
    Event Class
  • [0089]
    Update package privilege counters
  • [0090]
    Flush package from cache (Exclusive Lock and remove from cache
  • [0091]
    Invalidate package in cache (Exclusive Lock and set package status to INVALID)
  • [0092]
    Inoperate package in cache (Exclusive Lock and set package status to INOPERATE)
  • [0093]
    Invalidate dynamic SQL—Hard (Exclusive access on each variation affected and invalidate the variation)
  • [0094]
    Invalidate dynamic SQL—Soft (Invalidate the variation)
  • [0095]
    Update user privilege counters
  • [0096]
    Update db auth privilege counters
  • [0097]
    Temporary tablespace DDL—Soft (Invalidate the variation)
  • [0098]
    Temporary tablespace DDL—Hard (Exclusive access on each variation affected and invalidate the variation)
  • [0099]
    Object Type
  • [0100]
    Object Schema
  • [0101]
    Object Name
  • [0102]
    Object Usage
  • [0103]
    Object Column Information (if applicable)
  • [0104]
    Counter information (if applicable)
  • [0105]
    A synchronization message which is invalidating a package or cached dynamic SQL statements requires that the cache at the receiving node guarantee that all relevant cached packages or variations are locked exclusively for the requesting application. Once the appropriate action has been taken for a synchronization message, the receiving node(s) must respond to the requesting node with a confirmation message with a success or failure indicator.
  • [0106]
    Since all DDL actions take place at the catalogue node, the catalogue node will be the source of all synchronization messages. All messages will be sent to all participating nodes except the current one (the catalogue node) as the cache at the current node will be manipulated directly. If a communication failure occurs during a transmission to a participating node, the sending function fails and the transaction is rolled back. Any other errors will also halt processing and cause the transaction to be rolled back.

Claims (11)

We claim
1. A database system comprising:
a) a plurality of nodes;
b) a plurality of databases stored on the nodes; and
c) a global cache stored on at least one node, the global cache being accessible to a plurality of applications in the database system.
2. A database system as claimed in claim 1, wherein the global cache comprising a static portion and a dynamic portion.
3. A database system as claimed in claim 2, wherein the dynamic portion of the global cache includes a plurality of variations.
4. A database system as claimed in claim 3, which includes a catalogue node, wherein the database system includes, in the catalogue node, a list comprising a list of participating nodes that are active and connected to the database system.
5. A database system as claimed in claim 4, wherein the static cache of the global cache comprises:
a) one or more packages, and one or more section entries within each of said packages; and
b) wherein the dynamic portion of the global cache comprises a statement portion and a dependency portion.
6. A method of maintaining cache and application integrity across a database system comprising a plurality of nodes and a plurality of databases stored on the nodes, the method comprising providing a global cache on one node including variations for access to data in a database, and the global cache being accessible to a plurality of applications on the database system; when a request is to be made of a database, first checking the global cache for a variation matching the current request and executing said variation if present.
7. A method of maintaining cache and application integrity across a relational database system comprising a plurality of nodes and a plurality of databases shared on the nodes, the method comprising the steps of:
(1) providing, for at least some of the databases, a global cache including information to enable agents to access the databases;
(2) originating a request through a coordinating node, the coordinating node sending information on the variation to be executed to remote nodes;
(3) having each remote node receive the request and check the global cache on the respective remote node for a current variation; and
(4) if the a remote node does not have the current variation, the remote node requesting the coordinating node to send the current version of the variation.
8. The method as claimed in claim 7, the method including providing a catalogue node and maintaining, on the catalogue node, a list of participating nodes that are active and connected to the database system, and broadcasting a communication bulletin from the catalogue node to the nodes on the list of participating nodes when a change occurs to an object in the database that may invalidate a variation.
9. The method as claimed in claim 8, the method including:
(1) the catalogue node receiving a REGISTER request when a node becomes active and updating a list of participating nodes by adding an entry for the newly entered node, and
(2) the catalogue node receiving an UNREGISTER request when a node becomes inactive and removing that node from the participating nodes list.
10. The method as claimed in claim 9, the method including broadcasting a communication bulletin from the catalogue node to nodes on the list of participating nodes to ensure that all affected variations and all nodes are properly invalidated.
11. The method as claimed in claim 10, the method composing the steps of:
(a) verifying if the section requested to be executed is valid and executing such section or,
(b) if such section requested to be executed is invalid, requesting a valid section from the coordinating node and executing such section upon receipt of the same from the coordinating node.
US09162165 1998-09-29 1998-09-29 Caching of distributed dynamic SQL statements in a multiple node RDBMS Active US6353833B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US09162165 US6353833B1 (en) 1998-09-29 1998-09-29 Caching of distributed dynamic SQL statements in a multiple node RDBMS

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US09162165 US6353833B1 (en) 1998-09-29 1998-09-29 Caching of distributed dynamic SQL statements in a multiple node RDBMS

Publications (2)

Publication Number Publication Date
US20020026448A1 true true US20020026448A1 (en) 2002-02-28
US6353833B1 US6353833B1 (en) 2002-03-05

Family

ID=22584435

Family Applications (1)

Application Number Title Priority Date Filing Date
US09162165 Active US6353833B1 (en) 1998-09-29 1998-09-29 Caching of distributed dynamic SQL statements in a multiple node RDBMS

Country Status (1)

Country Link
US (1) US6353833B1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030204504A1 (en) * 2002-04-29 2003-10-30 Stuy Michael A.F. Access concurrency for cached authorization information in relational database systems
US20050262103A1 (en) * 1997-12-31 2005-11-24 International Business Machines Corporation Low overhead methods and apparatus shared access storage devices
US7222117B1 (en) 2003-11-14 2007-05-22 Advent Software, Inc. Segmented global area database
US20100293140A1 (en) * 2009-05-18 2010-11-18 Shuhei Nishiyama Distributed database system by sharing or replicating the meta information on memory caches
US20120221768A1 (en) * 2011-02-28 2012-08-30 Bagal Prasad V Universal cache management system
US8886671B1 (en) 2013-08-14 2014-11-11 Advent Software, Inc. Multi-tenant in-memory database (MUTED) system and method

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3510042B2 (en) 1996-04-26 2004-03-22 日立ソフトウエアエンジニアリング株式会社 Database management method and system
US7739308B2 (en) * 2000-09-08 2010-06-15 Oracle International Corporation Techniques for automatically provisioning a database over a wide area network
US7240324B2 (en) * 2001-02-28 2007-07-03 Hewlett-Packard Development Company, L.P. Event-based scheduling method and system for workflow activities
CA2365731C (en) * 2001-12-20 2008-09-23 Ibm Canada Limited-Ibm Canada Limitee Support for concurrent active sql statements having shared sql context information
US7305410B2 (en) * 2002-12-26 2007-12-04 Rocket Software, Inc. Low-latency method to replace SQL insert for bulk data transfer to relational database
CA2433377A1 (en) * 2003-06-25 2004-12-25 Ibm Canada Limited - Ibm Canada Limitee Computing frequent value statistics in a partitioned relational database
US7899797B2 (en) * 2003-12-04 2011-03-01 International Business Machines Corporation Package resolution mechanism for database systems
US7421562B2 (en) * 2004-03-01 2008-09-02 Sybase, Inc. Database system providing methodology for extended memory support
US8126873B2 (en) * 2007-04-13 2012-02-28 International Business Machines Corporation Portable and iterative re-usable suboptimization of database queries
US9715438B2 (en) * 2007-06-29 2017-07-25 International Business Machines Corporation Static execution of statements in a program
US7831569B2 (en) * 2007-10-10 2010-11-09 International Business Machines Corporation Preserving a query plan cache
US8458217B1 (en) 2009-08-24 2013-06-04 Advent Software, Inc. Instantly built information space (IBIS)
US8769350B1 (en) 2011-09-20 2014-07-01 Advent Software, Inc. Multi-writer in-memory non-copying database (MIND) system and method
US8332349B1 (en) 2012-01-06 2012-12-11 Advent Software, Inc. Asynchronous acid event-driven data processing using audit trail tools for transaction systems

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5010478A (en) * 1986-04-11 1991-04-23 Deran Roger L Entity-attribute value database system with inverse attribute for selectively relating two different entities
US5136707A (en) * 1988-10-28 1992-08-04 At&T Bell Laboratories Reliable database administration arrangement
US5349678A (en) 1991-08-21 1994-09-20 Norand Corporation Versatile RF data capture system
JP3140621B2 (en) * 1993-09-28 2001-03-05 株式会社日立製作所 Distributed File System
US5544345A (en) * 1993-11-08 1996-08-06 International Business Machines Corporation Coherence controls for store-multiple shared data coordinated by cache directory entries in a shared electronic storage
US5848241A (en) * 1996-01-11 1998-12-08 Openframe Corporation Ltd. Resource sharing facility functions as a controller for secondary storage device and is accessible to all computers via inter system links
US5897634A (en) * 1997-05-09 1999-04-27 International Business Machines Corporation Optimized caching of SQL data in an object server system
US5974129A (en) * 1997-05-21 1999-10-26 Lucent Technologies Inc. Distributed virtual cache method for use in a database query control system
US5924096A (en) * 1997-10-15 1999-07-13 Novell, Inc. Distributed database using indexed into tags to tracks events according to type, update cache, create virtual update log on demand
JPH11120048A (en) * 1997-10-20 1999-04-30 Fujitsu Ltd Device and method for data caching of clinet-server decentralized system and medium where data caching program is recorded
US6073129A (en) * 1997-12-29 2000-06-06 Bull Hn Information Systems Inc. Method and apparatus for improving the performance of a database management system through a central cache mechanism
US6115703A (en) * 1998-05-11 2000-09-05 International Business Machines Corporation Two-level caching system for prepared SQL statements in a relational database management system

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050262103A1 (en) * 1997-12-31 2005-11-24 International Business Machines Corporation Low overhead methods and apparatus shared access storage devices
US7299232B2 (en) * 1997-12-31 2007-11-20 International Business Machines Corporation Low overhead methods and apparatus shared access storage devices
US20030204504A1 (en) * 2002-04-29 2003-10-30 Stuy Michael A.F. Access concurrency for cached authorization information in relational database systems
US7970787B2 (en) * 2002-04-29 2011-06-28 International Business Machines Corporation Access concurrency for cached authorization information in relational database systems
US7222117B1 (en) 2003-11-14 2007-05-22 Advent Software, Inc. Segmented global area database
US20100293140A1 (en) * 2009-05-18 2010-11-18 Shuhei Nishiyama Distributed database system by sharing or replicating the meta information on memory caches
US8140498B2 (en) * 2009-05-18 2012-03-20 Shuhei Nishiyama Distributed database system by sharing or replicating the meta information on memory caches
US20120221768A1 (en) * 2011-02-28 2012-08-30 Bagal Prasad V Universal cache management system
US9703706B2 (en) * 2011-02-28 2017-07-11 Oracle International Corporation Universal cache management system
US8886671B1 (en) 2013-08-14 2014-11-11 Advent Software, Inc. Multi-tenant in-memory database (MUTED) system and method

Also Published As

Publication number Publication date Type
US6353833B1 (en) 2002-03-05 grant

Similar Documents

Publication Publication Date Title
Cahill et al. Serializable isolation for snapshot databases
US5890166A (en) Versioned-database management system in which tasks are associated with promote groups which comprise a set of parts whose changes are to be promoted
US5680602A (en) Trigger generation in an active database management system
US6304873B1 (en) System and method for performing database operations and for skipping over tuples locked in an incompatible mode
US5333316A (en) Locking and row by row modification of a database stored in a single master table and multiple virtual tables of a plurality of concurrent users
US6219675B1 (en) Distribution of a centralized database
US6078955A (en) Method for controlling a computer system including a plurality of computers and a network processed as a user resource
US6604102B2 (en) System and method for performing database operations on a continuous stream of tuples
US7739223B2 (en) Mapping architecture for arbitrary data models
US6684203B1 (en) Using global temporary tables to transform queries
US5592661A (en) Detection of independent changes via change identifiers in a versioned database management system
US6058389A (en) Apparatus and method for message queuing in a database system
US6631374B1 (en) System and method for providing fine-grained temporal database access
US5758355A (en) Synchronization of server database with client database using distribution tables
US6073129A (en) Method and apparatus for improving the performance of a database management system through a central cache mechanism
US6453313B1 (en) Database management system and method for dequeuing rows published to a database table
US6205451B1 (en) Method and apparatus for incremental refresh of summary tables in a database system
US7272591B1 (en) Method and system for updating value correlation optimizations
US6256712B1 (en) Scaleable method for maintaining and making consistent updates to caches
US7310653B2 (en) Method, system, and product for maintaining software objects during database upgrade
US6098075A (en) Deferred referential integrity checking based on determining whether row at-a-time referential integrity checking would yield the same results as deferred integrity checking
US20040139116A1 (en) Time in databases and applications of databases
US6728719B1 (en) Method and mechanism for dependency tracking for unique constraints
US5920857A (en) Efficient optimistic concurrency control and lazy queries for B-trees and other database structures
US6446136B1 (en) System and method for dynamic correlation of events

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BIRD, PAUL M.;SNOWBELL, MICHAEL J.;REEL/FRAME:009715/0412

Effective date: 19981029

FPAY Fee payment

Year of fee payment: 4

FPAY Fee payment

Year of fee payment: 8

REMI Maintenance fee reminder mailed
FPAY Fee payment

Year of fee payment: 12

SULP Surcharge for late payment

Year of fee payment: 11