JP2008507008A - Efficient extraction of XML content stored in a LOB - Google Patents

Abstract

  A method and system is provided for extracting valid free-standing fragments for nodes in an XML document stored in a database management system. The XML index is used to identify the location where the XML fragment data corresponding to the node is located. The top of the node is identified and examined for any information necessary for proper interpretation of the fragment. If the superior node contains such necessary information, this information is patched to the XML fragment to ensure that the fragment is a valid freestanding XML fragment.

Description

FIELD OF THE INVENTION The present invention relates to information management, and more particularly to extracting valid freestanding XML fragments identified from stored XML data by an XPath path expression. It is.

Background In recent years, database systems have been developed that allow storage and querying of extensible markup language data ("XML data"). There are many evolving standards for XML queries, but they all include some variation of the X path. XPath is a language that describes how to locate and process items in an XML document by using an addressing syntax based on the document's logical structure or path through the hierarchy. The part of the XML document identified by the X path “path expression” is the part existing at the end of any path that matches the path expression in the structure of the XML document.

  XML documents managed by a relational database server are typically stored in some form of LOB (Large Object) data type as unstructured serialized data. For example, an XML document may be stored in an unstructured storage device such as CLOB (character LOB) or BLOB (binary LOB), or the document may be OR (object-relational structure using XML schema). May be stored as

  Regardless of how the XML document is stored to satisfy many X-path queries, a method is needed to identify and extract fragments of the stored XML document that match the X-path path expression.

  Unfortunately, even database systems with built-in support for storing XML data are not usually optimized for handling path-based queries, and the query performance of database systems is often flawed. In the specific case where an XML schema definition will be available, the structure and data types used in the XML instance document can be used to optimize the X-path query. However, if the XML schema definition is not available and the document being searched does not follow any schema, there is no efficient technique for path-based queries.

  To improve the performance of querying documents when XML schema definitions are not available, special mechanisms such as full scans of all documents or text keyword-based indexes may be used. However, these mechanisms do not meet the need for an efficient method for quickly identifying and extracting stored XML document fragments that match an X-path path expression.

  Even if a method for quickly identifying the location of a fragment of stored XML data is available, there still remains a need for a method that efficiently extracts fragments from the identified location. The fragment that exists at the identified location may not be a valid free-standing XML document. For example, a namespace prefix used within a fragment may be declared outside that fragment, and thus a fragment retrieved from an identified location will not have all the necessary declarations.

  Based on the above, there is clearly a need for a system and method for identifying and extracting valid freestanding XML fragments that are consistent with an X-path path expression.

  The approaches described in this section are approaches that can be pursued, but not necessarily approaches that have been previously considered or pursued. Accordingly, unless otherwise indicated, none of the approaches described in this section should be assumed to qualify as prior art solely because they are included in this section.

  The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like reference numerals refer to like elements.

DETAILED DESCRIPTION In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent that the invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.

Exemplary XML Document For illustrative purposes, examples are given below in connection with the following two XML documents:

As described above, po1. xml and po2. xml is just two examples of XML documents. The techniques described herein are not limited to XML documents having any particular type, structure or content. Examples are given below of how such documents can be indexed and accessed according to various embodiments of the present invention.

XML Index US Patent Application Serial No. 10 / 884,311 entitled “INDEX FOR ACCESSING XML DATA” filed on July 2, 2004 (hereinafter “XML Index”). Application ") describes various examples of indexes that can be used to efficiently access XML documents managed by a relational database server based on X-path queries. Such an index is referred to herein as an XML index.

  The XML index described in the XML index application can be used to process X-path queries regardless of the format and data structure used to store the actual XML data ("base structure"). For example, the actual XML data can be CLOB (character LOB that stores the actual XML text), OR (object-relational structured format in the presence of the XML schema), or BLOB (any binary format of XML data). Any form, such as binary LOB to store, can exist in the database or in structures outside the database.

  In accordance with one embodiment, the XML index is a domain index that improves the performance of queries that include X path based predicates and / or X path based fragment extraction. For example, an XML index can be built on XML schema-based columns stored as CLOB or structured storage and XML-type columns without schemas. In one embodiment, the XML index is a logical index that results from the cooperative use of a path index, a value index, and an order index.

  The path index brings this mechanism to the lookup node based on a simple (navigation) path expression. A value index provides a lookup based on value equality or range. There could be multiple secondary value indexes, one per data type. The order index associates hierarchical ordering information with the indexed nodes. The order index is used to determine parent-child relationships, upper-lower relationships, and sibling relationships between XML nodes.

  When a user submits a query with an X path (as a predicate or fragment identifier), the X path statement is broken down into an SQL query that accesses the XML index table. The generated query typically performs a set of path, value, and ordered constraints, and merges the results appropriately.

  For purposes of explanation, the techniques described herein are described in the context where an XML index is used to index an XML document as described in an XML index application. However, the techniques described herein are not limited to any particular index structure or mechanism, and identify and extract freestanding XML fragments that are valid regardless of which method of query is used. Can be used to do.

PATH Table According to one embodiment, a logical XML index includes a set of PATH tables and secondary indexes. As described above, each indexed XML document may include a number of indexed nodes. The PATH table contains one row per indexed node. For each indexed node, a row in the PATH table for the node contains various information associated with the node.

According to one embodiment, the information contained in the PATH table includes (1) PATHID (path ID) indicating the path to the node, (2) “location data” for positioning the fragment data for the node in the base structure, And (3) “hierarchical data” indicating the position of the node in the structural hierarchy of the XML document including the node. Optionally, the PATH table may also include value information for those nodes associated with the value. Each of these types of information is described in more detail below.

Path The structure of an XML document establishes a parent-child relationship between nodes in the XML document. The “path” of a node in the XML document reflects a series of parent-child links starting from the “root” node and reaching a particular node. For example, po2. The path to the “User” node in xml is / PurchaseOrder / Actions / Action / User. This is because the “User” node is a child of the “Action” node, the “Action” node is a child of the “Actions” node, and the “Actions” node is a child of the “PurchaseOrder” node.

  The set of XML documents that the XML index indexes is referred to herein as “indexed XML documents”. According to one embodiment, an XML index may be built on all of the paths in all of the indexed XML documents, or on a part of the paths in the indexed XML documents. Techniques for specifying which paths are indexed are described below. The set of paths that are indexed by a particular XML index are referred to herein as “indexed XML paths”.

PATHID
In accordance with one embodiment, each indexed XML path is assigned a unique path identifier (“PATHID”). For example, po1. xml and po2. Paths present in xml may be assigned a PATHID as illustrated in the table below.

  Various techniques can be used to identify a route and assign a PATHID to the route. For example, the user may explicitly enumerate the routes and specify the corresponding PATHID for the routes thus identified. Alternatively, the database server may parse each XML document as it is added to the indexed set of XML documents. During the parsing operation, the database server identifies any paths that have not already been assigned a PATHID and automatically assigns a new PATHID to those paths. The mapping of the PATHID to the route may be stored in the database in various ways. According to one embodiment, the mapping of the PATHID to the path is stored as metadata separated from the XML index itself.

  According to one embodiment, the same access structure is used for XML documents that conform to different schemas. Since indexed XML documents can follow different schemas, each XML document will typically contain only a portion of the path that is assigned a PATHID.

Position data The position data associated with a node is (1) where the XML document containing the node exists in the base structure, and (2) where the XML fragment corresponding to the node is located in the stored XML document. Indicate. Accordingly, the nature of the position data will vary from implementation to implementation based on the nature of the base structure. The location information is typically added to the PATH table when the XML document is parsed.

  For purposes of explanation, it is assumed that (1) the base structure is a table in a relational database and (2) each indexed XML document is stored in a corresponding row of the base table. In such a context, the position data about the node is, for example, (1) the identifier (“RID”) of the row in the base table in which the XML document including the node is stored, and (2) the fragment data corresponding to the node. Locators that provide fast access in stored XML documents.

  A locator is conceptually information that “points” to the original document and is typically used to retrieve fragment data starting from that point. The locator depends on the actual storage used for the XML document and can vary from one storage CLOB, OR or BLOB format. For example, the locator of a node in an XML document stored in the CLOB could be the starting character offset in the CLOB where the node begins. Further, the byte length of the node may be stored as part of the locator. Together, this information provides a starting and ending position within the stored XML document and can be used to efficiently extract XML fragments. For example, a locator can match a specified X-path query by extracting data, by starting at the character offset specified by the locator, and by reading the data for the number of bytes indicated by the locator May be used to search for XML fragments containing

  However, locators can be more complex than character or byte offsets. For example, the locator could include a specific flag. As another example, if a shredded XML document is stored in a relational table, the locator could include an appropriate table and / or row identifier, etc.

The row of the PATH table for the hierarchical data node also includes information indicating where the node exists in the hierarchical structure of the XML document including the node. Such hierarchical information is referred to as “OrderKey” of a node in this specification.

  According to one embodiment, hierarchical order information is represented using a Dewey-type value. Specifically, in one embodiment, the node's OrderKey is created by adding a value to the node's immediate parent's OrderKey, where the added value is the parent node's parent node's parent node's OrderKey. Indicates the position within the child.

  For example, assume that a particular node D is a child of node C, node C itself is a child of node B, and node B is a child of node A. Further, the node D is OrderKey 1.2.2.3. Suppose we have The last “3” in OrderKey indicates that node D is the third child of its parent node C. Similarly, 4 indicates that node C is the fourth child of node B. 2 indicates that node B is the second child of node A. The leading 1 indicates that node A is the root node (that is, has no parent).

As described above, the child OrderKey can be easily created by adding a value corresponding to the number of children to the parent OrderKey. Similarly, the parent OrderKey is easily derived from the child OrderKey by removing the last number in the child OrderKey.

  According to one embodiment, the composite number represented by each OrderKey is converted to a value comparable to a byte, so a mathematical comparison between two OrderKeys is the structural hierarchy of the XML document at the node to which OrderKey corresponds. Indicates the relative position within.

  For example, the node associated with OrderKey 1.2.7.7 precedes the node associated with OrderKey 1.3.1 in the hierarchical structure of the XML document. Thus, the database server uses a conversion mechanism that converts OrderKey 1.2.7.7 to a first value and OrderKey 1.3.1 to a second value, where the first value is the second value. Is less than the value of. By comparing the second value with the first value, the database server can easily determine that the node associated with the first value precedes the node associated with the second value. Various conversion techniques may be used to achieve this result, and the invention is not limited to any particular conversion technique.

Value information Some nodes in the indexed document may be attribute nodes or nodes corresponding to simple elements. As used herein, a “simple element” is an element that does not have any attributes or child elements, and its value is a single text string. For example, in “po1.xml”, the “Reference” element is a simple element with a single text value of “SBELL-200210091233601PDT”.

  According to one embodiment, for attribute nodes and simple elements, the rows of the PATH table also store the actual values of the attributes and simple elements. Such a value may be stored, for example, in a “value column” of the PATH table. A secondary “value index”, described in more detail below, is built on the sequence of values.

PATH Table Example According to one embodiment, the PATH table includes columns defined as specified in the following table.

As explained above, PATHID is an identifier assigned to a node and uniquely represents a fully expanded path to the node. ORDER_KEY is a system representation of the Dewey ordered number associated with the node. According to one embodiment, the internal representation of OrderKey also preserves document ordering.

  The VALUE column stores effective text values for simple element (ie, no child elements) nodes and attribute nodes. According to one embodiment, adjacent text nodes are merged by concatenation. As described in the XML Index application, a mechanism is provided to allow the user to customize the effective text values stored in the VALUE column by specifying options during indexing, eg mixed text, margins, Movements such as case sensitivity can be customized. The user can store VALUE columns in any number of formats, including bounded RAW columns or BLOBs. If the user selects a bounded storage device, any overflow during indexing will be flagged as an error.

  The following table has (1) the above-mentioned columns, and (2) po1. xml and po2. FIG. 4 is an example of a PATH table filled with input for xml. FIG. Specifically, each row of the PATH table is po1. xml or po2. Corresponds to xml indexed nodes. In this example, po1. xml and po2. It is assumed that xml is stored in rows R1 and R2 of the base table, respectively.

  In this example, the rowid (row id) column stores a unique identifier for each row of the PATH table. Depending on the database system in which the PATH table is created, the rowid column may be an implicit column. For example, the disk location of a row may be used as a unique identifier for that row. As described in more detail below, the secondary order and value index uses the PATH table rowid value to locate a row in the PATH table.

  In the example shown above, the node's PATHID, ORDER_KEY and VALUE are all contained in a single table. In an alternative embodiment, separate tables may be used to map PATHID, ORDER_KEY and VALUE information to corresponding location data (eg, base tables RID and LOCATOR).

  In the example shown above, the information in the “RID” and “LOCATOR” columns of the PATH table is used to identify the location where the indexed node is stored. In this example, each row in the base table corresponds to an indexed XML document. Each row in the base table uses CLOB to store the associated XML document. The RID column in the PATH table identifies the row in the base table where the XML document is stored as a CLOB, and the LOCATOR column is the character offset into the CLOB where the indexed node begins and the node's Stores the character length.

  For example, the sample XML document po1. xml and po1. xml is stored in unstructured serialized form in the base table rows R1 and R2 as a CLOB data structure. The node identified by rowid “1” in the PATH table is located in row R1 of the base table and starts with the stored CLOB character 1 and has a length of 350 characters. As another example, the node identified by rowid “9” is located in row R2 of the base table and starts at character 72 and has a length of 36 characters. This line of the PATH table is po2. Corresponds to the first <Action> node in xml.

<Action>
<User> ZLOTKEY </ User>
</ Action>
The example shown in the embedded PATH table above illustrates an embodiment where locator information is not stored for simple elements and attribute nodes. In other embodiments, locator information could be stored and maintained for all nodes that contain simple elements. Furthermore, the example shown in the filled PATH table illustrates an embodiment where the LOCATOR column stores both offset and length information. In an alternative embodiment, only offset information may be stored. Alternatively, as described above, other types of locator information may be stored in the LOCATOR column. The techniques described herein do not rely on any particular type of location data.

The secondary index PATH table contains the information needed to locate XML documents and / or XML fragments that satisfy a wide range of queries. However, even without a secondary access structure, using a PATH table to satisfy such a query often requires a full scan of the PATH table. Thus, according to one embodiment, various secondary indexes are created by the database server to (1) perform path lookups and / or (2) accelerate queries that identify order-based relationships. According to one embodiment, the following secondary index is created in the PATH table:

・ PATHID_INDEX on (PATHID, RID)
-ORDERKEY_INDEX on (RID, ORDER_KEY)
・ VALUE INDEXES
・ PARENT_ORDERKEY_INDEX on (RID, SYS_DEWEY_PARENT (ORDER_KEY))

PATHID_INDEX
PATHID_INDEX is constructed in the PATHID and RID columns of the PATH table. Thus, the input to PATHID_INDEX is of the form (key value, rowid), where the key value is a composite value representing a particular PATHID / RID combination, and rowid identifies a particular row in the PATH table.

  When (1) a row in the base table and (2) a node's PATHID is known, PATHID_INDEX may be used for that node to quickly locate the row in the PATH table. For example, based on the key value “3.R1”, the PATHID_INDEX may be traversed to find an input associated with the key value “3.R1”. Assuming that the PATH table is filled as shown above, the index entry will have a rowid value of 3. A rowid value of 3 refers to the third row of the PATH table, which is the row for the node associated with PATHID3 and RID R1.

ORDERKEY_INDEX
ORDERKEY_INDEX is constructed in the RID and ORDER_KEY columns of the PATH table. Thus, the input to ORDERKEY_INDEX is in the form (key value, rowid), where the key value is a composite value representing a particular RID / ORDER_KEY combination, and rowid identifies a particular row in the PATH table.

  When (1) a row in the base table and (2) an ORDERKEY of a node are known, ORDERKEY_INDEX may be used for that node to quickly locate the row in the PATH table. For example, based on the key value “R1.′1.2 ′”, ORDERKEY_INDEX may be traversed to find an input associated with the key value “R1.’1.2 ′”. Assuming that the PATH table is filled as shown above, the index entry will have a rowid value of 3. A rowid value of 3 refers to the third row of the PATH table, which is the row for the node associated with ORDERKEY 1.2 and RID R1.

Value Index Queries based on value lookups are accelerated by an index built on the VALUE column of the PATH table, just as queries based on path lookups can be accelerated using PATHID_INDEX. Can do. However, the VALUE column of the PATH table can hold values for various data types. Thus, according to one embodiment, a separate value index is constructed for each data type stored in the VALUE column. Thus, in an implementation where the VALUE column holds a string, number and timestamp, the following value (secondary) index is also created:

STRING_INDEX on SYS_XMLVALUE_TO_STRING (value)
NUMBER_INDEX on SYS_XMLVALUE_TO_NUMBER (value)
TIMESTAMP_INDEX on SYS_XMLVALUE_TO_TIMESTAMP (value)
These value indexes are used to perform data type based comparisons (equalities and ranges). For example, the NUMBER value index is used to handle number-based comparisons within the user X path. For example, the input to NUMBER_INDEX may be of the form (number, rowid), where rowid refers to the row in the PATH table for the node associated with the value of “number”. Similarly, the input in STRING_INDEX may have the form (string, rowid) and the input in TIMESTAMP_INDEX may have the form (timestamp, rowid).

The format of the values in the PATH table may not correspond to the specific format of the data type. Thus, when using a value index, the database server may invoke a conversion function to convert the value bytes from the stored format to the specified data type. In addition, the database server applies any necessary variations as described below. According to one embodiment, the conversion function operates on both RAW and BLOB values and returns null if conversion is not possible.

  By default, a value index is created when an XML index is created. However, the user can suppress the creation of one or more value indexes based on knowledge of the query workload. For example, if all X-path predicates involve only string comparisons, NUMBER and timestamp value indexes can be avoided.

PARENT_ORDERKEY_INDEX
According to one embodiment, the set of secondary indexes built on the PATH table includes PARENT_ORDERKEY_INDEX. Similar to the ORDER_KEY index, PARENT_ORDERKEY_INDEX is built on the RID and ORDER_KEY columns of the PATH table. As a result, the index entry for PARENT_ORDERKEY_INDEX has the form (key value, rowid), where the key value is a composite value corresponding to a particular RID / ORDER_KEY combination. However, unlike the ORDER_KEY index, the rowid at the PARENT_ORDERKEY_INDEX input does not point to a PATH table row with a particular RID / ORDER_KEY combination. Rather, the rowid of each PARENT_ORDERKEY_INDEX input points to the PATH table row of the node that is the immediate parent of the node associated with the RID / ORDER_KEY combination.

  For example, in the embedded PATH table illustrated above, the RID / ORDER_KEY combination “R1.’1.2 ′” corresponds to the node in row 3 of the PATH table. The immediate parent of the node in row 3 of the PATH table is the node represented by row 1 of the PATH table. As a result, the PARENT_ORDERKEY_INDEX entry associated with the key value of “R1.’1.2 ′” will have a rowid pointing to row 1 of the PATH table (ie, rowid = 1).

Using an XML index to process an X-path query As mentioned above, an XML index is an X-path based query by incorporating the required parts of an XML document, ie tags, values and nesting information, into the PATH, VALUE and ORDER indexes. As well as improving the performance of fragment extraction. The PATH index is used to index tags and provides a mechanism to identify fragments based on simple path expressions. The VALUE index allows XML values to be indexed. The ORDER index associates hierarchical ordering information with indexed nodes and is used to determine parent-child relationships, upper-lower relationships and sibling relationships between XML nodes.

  When a user submits a query with an X path, the X path expression can be decomposed into an SQL query that accesses the XML index table. The generated query typically performs a set of path, value and order constrained lookups and merges the results appropriately.

  In particular, a co-pending US patent application serial number entitled “EFFICIENT QUERY PROCESSING OF XML DATA USING XML INDEX” filed on September 16, 2004 entitled “EFFICIENT QUERY PROCESSING OF XML DATA USING XML INDEX” No. 10 / 944,170 (hereinafter “query processing” application) for executing an “index-enabled” query that uses an XML index to identify XML data corresponding to a specified path Various embodiments of the method are described. In particular, query processing applications describe techniques for evaluating X path operators using XML indexes.

  More specifically, the query processing application (1) breaks down generic path expressions into simpler components such as simple paths, predicates and structural joins, and (2) indexed path components. To generate an SQL query against a table of XML indexes that may involve representing a structure join using an SQL predicate with a Dewey order key of (3) and for fragment extraction using a locator pointing to the original data Describe the technology.

  The index-enabled query is generated based on the path expression and accesses the XML index PATH table. A path expression of a path-based query or a fragment thereof is combined with a template. Each template is associated with a rule. When the specified path fragment is in a format that matches the template, the corresponding rule is used to generate the SQL for the index-enabled query. This process is described in detail in the query processing application.

Processing an extract () operator using an XML index One X-path operator that can be evaluated using techniques described in a query processing application is the extract () operator. The result of the X path extract () operator is an XML type that contains an XML fragment of the XML document that satisfies the specified X path expression.

  As described in the query processing application, the extract () operator can be rewritten as an SQL query in the XML index table. For example, the extract () operator for an X path query at the / PurchaseOrder / Actions node may be translated into an SQL query as follows:

Where: B1 = pathid ('/ PurchaseOrder / Actions') (pathid () is an internal function used to find the PATHID associated with the path) and po_tab is the stored XML document It is a base table that contains.

  The SYS_XMLINDEX_MKXML () operator constructs an XML type image based on the index column values. In one embodiment, this lookup may be implemented using the SYS_XMLINDEX_GETFRAG () operator. Given a row identifier and a locator, the SYS_XMLINDEX_GETFRAG () operator constructs an XML type image consisting of XML fragments corresponding to the row identifier and locator.

  XMLAGG () is an operator that concatenates fragments generated by the SYS_XMLINDEX_MKXML () operator. Using the example above, for each row containing the node '/ PurchaseOrder / Actions', the fragments are retrieved from the base table and aggregated into a single XML type image.

  For example, using the embedded PATH table above,

The output of

Will be. In one embodiment, the output returned is a single long string created by concatenating the above results, including a start tag and an end tag.

  The techniques described herein are used to implement a SYS_XMLINDEX_GETFRAG () operator that obtains the actual text fragment corresponding to a node.

Efficient Extraction Process The process 200 shown in FIG. 2 illustrates the steps of one technique for extracting XML fragments according to an embodiment of the invention. As shown, in step 210, a node is first identified. Any technique, such as the techniques described in XML indexing and query processing applications, can be used to identify nodes that match a path expression.

  Next, in step 215, the node is examined to determine whether it is a simple element or a complex element. As mentioned above, a simple element is an element that has no children or attributes, and its value is a single text value. A composite element is an element that has attributes or has child elements.

  If the node is a simple element, the fragment stored can be constructed using the information stored in the XML index without examining the original XML document, as shown by step 220. If the node is a complex element, the original XML document stored in the base table is examined to extract the fragments, as indicated by step 230, and the extracted fragments are optionally needed for proper interpretation. Patched. Each process is described in more detail below.

  The embodiment of the process shown in FIG. 2 utilizes the information stored in the XML index to construct fragments without examining the original XML document, but treats simple and complex elements differently. It is not a requirement. Fragments that match either simple element or complex element types can be extracted from the stored XML data.

Simple Element Fragment When a stored XML document is indexed with an XML index, the value of the simple element is in the VALUE column of the PATH table. Thus, XML fragments for simple elements can be constructed without examining the base table that stores the original XML document. The fragment is constructed by adding the appropriate start and end tags to the value obtained from the VALUE column of the PATH table for the identified node.

For example, the node '/ PurchaseOrder / Reference' is the XML document po1. xml
And po2. A simple element in xml. The PATHID of the expression '/ PurchaseOrder / Reference' is first determined. In this example, PATHID is “2”. The PATH table is consulted to determine whether any node corresponds to this PATHID (step 210). In this example, nodes with rowids “2” and “7” match PATHID = 2. The process of FIG. 2 is performed for each matching node.

  In step 215, when there is no locator information for both node 2 and node 7 and the VALUE column contains a simple text string, each is a simple element by examining the LOCATOR and VALUE columns for these rows Can be judged. For each of these simple element nodes, the process proceeds to step 220. In step 220, a fragment for a node can be constructed by creating a string that includes a start tag, a value, and an end tag. The start tag is created by extracting the last component of the path (in this example “Reference”) associated with this PATHID. The VALUE corresponding to this node in the PATH table is placed in the fragment after the start tag. For example, the VALUE component of the fragment for node 2 is “SBELL-200210091233601PDT”. The tag of the conclusion consisting of the letter “/” and the component string determined above (eg, “Reference”) completes the string of fragments. By following this process, the fragment of node 2 is determined to be “<Reference> SBELL-200210091233601PDT </ Reference>”. This is the original XML document po1. Matches xml fragment.

  Queries that extract only attributes can be treated like simple elements. However, elements containing attributes are treated as composite elements described in more detail below.

  Since the system can add namespaces and generated prefixes, simple elements do not require patching for proper interpretation, and the process proceeds to step 290 for simple elements.

Extracting complex elements using an XML index In the case of a complex element node, the fragment must be parsed from the base table that stores the XML document associated with the complex element. As described above, each row in the PATH table corresponds to a node in the XML document, for finding the node in the XML document stored in the base table and the RID of the row in the base table containing the original XML document. Including locator.

  For example, an X path extract () at node / PurchaseOrder / Reference / Actions should result in an aggregated fragment.

  However, unlike the simple elements described above, these fragments are extracted from the stored XML document. For example, the path expression “/ PurchaseOrder / Reference / Actions” corresponds to PATHID3. From the PATH table, nodes with rowids 3 and 8 match this PATHID. The VALUE column in these rows is free, and the LOCATOR column provides offset and length information for extracting fragments. Accordingly, in step 215, it is determined that each of these nodes corresponds to a composite element, and the process proceeds to step 230.

  In step 230, the fragment text corresponding to the node is located and read. For example, for node 3, the RID column indicates that the stored XML data is located in row R1 of the base table, and the LOCATOR field indicates that the fragment starts at character 64 and has a length of 56. Thus, the fragment text corresponding to node 3 can be created by extracting characters 64-120 from the CLOB in row R1 of the base table containing “po1.xml”. The XML fragment corresponding to node 8 can similarly be created by extracting characters 63-152 from the CLOB in row R2 of the base table containing “po2.xml”.

  In these examples, the extracted XML fragment is valid by chance. In many cases, however, XML fragments extracted using these methods may not be self-supporting. For example, the extracted fragment may contain or use references that are not defined within the fragment. The methods described herein allow for "patching" fragments created using the techniques described above to ensure that the resulting fragments are valid and free standing.

Prefixes and Namespaces Because element names in XML are not fixed, name collisions can occur when two different documents use the same name representing two different types of elements. One standard way to avoid name collisions is to use a prefix with the name.

  For example, Table 1 and Table 2 illustrate XML documents that both use the “Table” element.

  If both of these two XML documents are stored in the database, in some cases there may be an element name collision. This is because both documents contain <table> elements with different contents and definitions. One standard way to resolve and prevent these types of conflicts is through the use of namespace prefixes. As an example, Tables 1A and 2A below illustrate how the XML documents in Tables 1 and 2 can be modified to avoid element name collisions.

  As shown in Tables 1A and 2A, element name collisions are no longer a problem. This is because the two documents use different names for the <table> element (ie, <h: table> and <f: table>). By using a prefix, two different types of <table> elements are possible.

  The prefix typically refers to an XML document that carries information about the element. Tables 1B and 2B show how prefixes can be defined to refer to specific namespaces.

  Instead of using just the prefix, an xmlns attribute was added to the <table> tag to give the element prefix a qualified name associated with the namespace. Typically, namespace attributes are placed in the element's start tag using the following syntax:

xmlns: namespace-prefix = "namespace"
As shown by Table 1B and Table 2B, a boilerplate resource identifier (URI) can be used, but the namespace itself can be defined using an Internet address. Multiple namespace prefixes can be declared as single element attributes.

  When a namespace is defined as an attribute in an element start tag, all child elements with the same prefix are associated with the same namespace. In addition, a default namespace can be used for elements as shown in Tables 1C and 2C. When the default namespace is used, the prefix need not be used on all child elements. The default namespace declaration applies to element names that do not have all prefixes in the scope.

  The prefix provides the namespace prefix part of the qualified name and must be associated with the namespace reference in the namespace declaration. The prefix serves only as a placeholder for the namespace name. Namespace names, not prefixes, are used in constructing names that extend beyond the containing document. Prefix and namespace declarations may apply to attributes and elements.

  The scope of namespace declarations that declare prefixes extends from the beginning of the start tag that appears to the end of the corresponding end tag, eliminating the scope of any internal declarations that use the same prefix name. Such namespace declarations apply to the names of all elements and attributes within the scope that match the prefix specified in the declaration.

  The namespace prefix must have been declared in the attribute of the namespace declaration in the start tag or ancestor element of the element in which the prefix is used. This constraint can be problematic if the namespace declaration attribute is not brought directly into the XML document but via the default attribute declared in the external entity.

  This is particularly problematic in the context of fragment extraction. Not only is declaration in the external document problematic, but the extracted XML fragment may use the prefix declared in earlier sections of the document from which the fragment is extracted. In addition, since the extracted fragment does not have a direct reference to any namespace, fragments that are valid on it may be extracted, but the extracted fragment is within the scope of the superordinate element In that case, the higher-level default namespace declaration should be used.

  The technique described herein solves this problem by building a list of namespace declarations from the desired node and all its ancestors. This list is constructed by querying the PATH table. This list is then spliced to the fragments created in step 230 to obtain a complete and valid freestanding XML fragment.

Handling Namespace Declarations in Fragment Extraction As mentioned above, when the X-path extract () operator is evaluated on a simple element, the desired fragment can be constructed using only the PATH table. When the composite element is extracted, the fragment is read from the original data using position information from the PATH table. However, when a prefix is used in the extracted XML fragment, the extracted fragment must also describe the prefix. In addition, any default namespace declarations used in the superior element of the extracted node must be considered.

  For example, consider the exemplary XML document “po3.xml” in Table 3.

  If the X-path query “extract (/ po: purchaseOrder / po: lineItem / myns: SomeOtherTag)” is evaluated using only the process described above, the resulting fragment returned by this query will be the lines 101-104 in Table 3 Will consist of: However, this XML fragment references the namespace prefix “po”, which is not defined anywhere in the fragment that is extracted according to the locator information (ie, lines 101-104). Instead, this prefix is declared and mapped to the namespace “po.xsd” in line 1 of Table 1.

  Declaration “xmlns: po = ″ po. xsd ″ ”needs to be spliced to the fragments that are created in step 230 in order for the properly interpreted fragments, ie“ self-supporting ”fragments.

  In one embodiment, the namespace declaration can be maintained within the locator itself. However, this information will be present at all levels. In the preferred embodiment, the declaration information is constructed using information stored in the PATH table. In this example, an SQL query is used to identify all superior nodes of the node being extracted, and namespace declarations are collected from the superior nodes. In addition, the techniques described herein provide names that are accurate, that is, in reverse order, with deeper declarations taking precedence over shallower declarations to comply with the XML namespace scoping rules described previously. Solve the space declaration.

As indicated by step 240 in FIG. 2, the top of the node is identified. If an XML index is used, this is a simple query because the superior information is stored using OrderKey. In step 250, information required for proper interpretation of the XML fragment is retrieved for each identified superior. If there is any declaration or other information retrieved from the top that is necessary for proper interpretation of the fragment, in step 280 this information is patched to the fragment. For example, namespace declarations for any prefixes that are used but not defined in the fragment are retrieved from the nearest ancestor node and patched to the fragment created in step 230.

  For example, the following SQL query could be used to gather namespace declarations and go through all the superior nodes to solve them correctly. (: B1 = RID of the considered document, B2 = OrderKey of the extracted node)

  As shown, the outer subquery selects all namespace declarations in a given document. For each such declaration, the exists () subquery determines whether the declaration exists in the ancestor element.

  To accurately explain the scoping rules, the declarations that exist in the superordinate element also exist in the subordinates, and the declarations should be ignored because the subordinates take precedence over the parent declaration. Furthermore, a declaration that exists in a parent element takes precedence over a declaration in a grandchild element. A list of declarations that need to be added to the fragment is created in step 250 by considering each ancestor in the proper order and explaining the scoping rules. To account for scoping rules, the top nodes are considered from the closest to the farthest. As each declaration is found in the ancestor, it is ignored if the declaration was already considered as part of the fragment itself or in an earlier ancestor node. Otherwise, the declaration is added to the string that is patched to the fragment.

For example, consider the following X-path query for the nodes in Table 3.
extract ('/ po: purchaseOrder / po: lineItem / myns: SomeOtherTag')
The fragments extracted from Table 3 in step 230 are as follows.

The prefix “po” is not defined in this fragment.
When the top of this fragment is considered in step 250, the following list of definitions is created.

  After splicing to fragments in the list of definitions in step 280, the resulting fragment is as follows:

  The declaration xmlns: ps2 = “po2.xsd” is not required to make this exemplary fragment into a free-standing fragment, but what it contains does not invalidate the fragment or the meaning of the fragment Never change. In an alternative embodiment, before a declaration is patched to a fragment, the declaration is examined to determine if the node being extracted requires a declaration.

  The free-standing fragment created in step 280, containing all the information necessary for proper interpretation, is then returned in step 290.

  Although the techniques described herein have been described in the context of namespace declarations and prefixes, the techniques can be used in other situations. For example, the presence of an entity or macro reference complicates the free standing nature of the fragment as well. As with namespaces, any entity reference must first be added with a DTD (data type definition) declaration, so fragments identified by a CLOB offset cannot be easily drained.

Hardware Overview FIG. 1 is a block diagram that illustrates a computer system 100 upon which an embodiment of the invention may be implemented. Computer system 100 includes a bus 102 or other communication mechanism for communicating information, and a processor 104 coupled with bus 102 for processing information. Computer system 100 also includes a main memory 106, such as random access memory (RAM) or other dynamic storage device, coupled to bus 102 for storing information and instructions executed by processor 104. Main memory 106 may also be used to store temporary variables or other intermediate information during execution of instructions executed by processor 104. Computer system 100 further includes a read only memory (ROM) 108 or other static storage device coupled to bus 102 for storing static information and instructions for processor 104. A storage device 110, such as a magnetic disk or optical disk, is provided and coupled to the bus 102 for storing information and instructions.

Computer system 100 may be coupled via bus 102 to a display 112 such as a cathode ray tube (CRT) for displaying information to a computer user. An input device 114 containing alphanumeric characters and other keys is coupled to the bus 102 for communicating information and command selections to the processor 104. Another type of user input device is a cursor control device 116, such as a mouse, trackball or cursor direction key, for communicating direction information and command selections to the processor 104 and for controlling cursor movement on the display 112. . The input device typically has two degrees of freedom in two axes, a first axis (eg, x) and a second axis (eg, y) so that the device is positioned in a plane. Can be specified.

  The invention is related to the use of computer system 100 for implementing the techniques described herein. In accordance with one embodiment of the invention, the techniques are performed by computer system 100 in response to processor 104 executing one or more sequences of one or more instructions contained in main memory 106. Such instructions may be read into main memory 106 from another machine-readable medium such as storage device 110. Execution of the sequence of instructions contained in main memory 106 causes processor 104 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions to implement the present invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.

  The term “machine-readable medium” as used herein refers to any medium that participates in providing data that causes a machine to operation in a specific fashion. In an embodiment implemented using computer system 100, various machine-readable media are involved, for example, in providing instructions to processor 104 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks such as storage device 110. Volatile media includes dynamic memory, such as main memory 106. Transmission media includes coaxial cables, such as the wire containing bus 102, copper wire, and optical fiber. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

  Common forms of machine readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes or other magnetic media, CD-ROMs, other optical media, punch cards, paper tapes, hole patterns Other physical media include RAM, PROM and EPROM, flash EPROM, other memory chips or cartridges, carrier waves described below, or other media that can be read by a computer.

  Various forms of machine-readable media may be involved in carrying one or more sequences of one or more instructions to processor 104 for execution. For example, the instructions may initially be carried on a remote computer magnetic disk. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 100 can receive the data along the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector can receive the data carried in the form of an infrared signal and a suitable circuit can place the data on the bus 102. Bus 102 carries data to main memory 106, from which processor 104 retrieves and executes instructions. The instructions received by main memory 106 may optionally be stored on storage device 110 either before or after execution by processor 104.

Computer system 100 also includes a communication interface 118 coupled to bus 102. Communication interface 118 provides a two-way data communication coupling to network link 120 that is connected to local network 122. For example, communication interface 118 may be an integrated services digital network (ISDN) card or modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 118 may be a LAN card to provide a data communication connection to a compatible local area network (LAN). A wireless link may also be implemented. In any such implementation, communication interface 118 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

  Network link 120 typically provides data communication through one or more networks to other data devices. For example, the network link 120 may provide a connection to the host computer 124 via the local network 122 or may provide a connection to data equipment operated by an Internet service provider (ISP) 126. ISP 126 then provides data communication services through a worldwide packet data communication network now commonly referred to as the “Internet” 128. Local network 122 and Internet 128 both use electrical, electromagnetic or optical signals that carry digital data streams. Signals over various networks, and signals through communication interface 118 along network 120 that carry digital data to and from computer system 100 are exemplary forms of carriers that carry information.

  Computer system 100 can send messages over network, network link 120 and communication interface 118 and can receive data including program code. In the Internet example, server 130 may communicate the requested code for the application program via Internet 128, ISP 126, local network 122 and communication interface 118.

  The received code may be executed by the processor 104 when received and / or stored in the storage device 110 or other non-volatile storage for later execution. In this manner, computer system 100 can obtain application code in the form of a carrier wave.

  In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Accordingly, what is the invention and what is intended by the applicant to be the invention is, in a specific form issued by such claims, a patent issued from this application. Claimed exclusively in the claim set, including any subsequent modifications. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Accordingly, no limitation, element, property, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.

1 is a block diagram of a system in which the techniques described herein may be implemented. FIG. 6 is a flowchart illustrating steps for efficiently providing a free-standing XML fragment in response to a request.

Claims (26)

A method for providing a self-supporting XML fragment for a node in an XML document managed by a database management system comprising:
Receiving a request for an XML fragment comprising a computer-implemented step, wherein the request includes an XML path expression, the method further comprising:
A computer-implemented step of identifying a node in the XML document managed by the database management system that matches the XML path expression;
A computer-implemented step of extracting a first XML fragment corresponding to the identified node;
A computer-implemented step of identifying a superior node for the identified node;
For each identified superior node, it is determined whether the superior node contains information necessary for proper interpretation of the first XML fragment, and if the superior node contains the necessary information, A computer-implemented step of inserting two XML fragments into the first XML fragment;
Providing a first XML fragment in response to the request.   The database management system includes an index that indexes an XML document stored in the database management system, and identifying a node in the XML document includes using the index to identify the node. The method according to 1.   The method of claim 2, wherein the index comprises a path, value, and order index. Extracting the first XML fragment comprises:
Determining the location of stored XML data corresponding to the identified node;
2. The method of claim 1, comprising reading XML data from the determined location.   5. The method of claim 4, wherein determining the location of stored XML data corresponding to the identified node comprises reading location information from an index that indexes XML documents stored in the database management system. . Extracting the first XML fragment comprises:
The method of claim 2, comprising constructing an XML fragment using information in the index.   The method of claim 3, wherein identifying an upper node includes using an order index.   The method of claim 1, wherein the information required for proper interpretation of the first XML fragment is a namespace declaration.   9. The method of claim 8, wherein determining whether a superior node contains information necessary for proper interpretation comprises determining whether a namespace declaration has been declared at a previously considered superior node. .   The method of claim 8, wherein determining whether an ancestor node contains information necessary for proper interpretation comprises determining whether a namespace declaration has been declared in the first XML fragment.   The method according to claim 1, wherein the step of determining whether an upper node includes information necessary for proper interpretation is executed for each upper node in order from the closest upper node to a root upper node.   The information required for proper interpretation of the first XML fragment is a namespace declaration, and if the namespace declaration in the higher node matches the namespace declaration in the higher node already considered, the namespace declaration is properly interpreted. The method of claim 11, wherein it is determined that it is not required.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 1.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 2.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 3.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 4.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 5.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 6.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 7.   9. A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 8.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 9.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 10.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 11.   A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 12. A computer-implemented method for receiving a request for an XML fragment, wherein the request includes an XML path expression, the method further comprising:
Including a computer-implemented step of using an index to identify nodes in the database management system that match the XML path expression;
The node exists in the XML document managed by the database management system,
The XML document is stored in one or more base structures managed by a database management system, the method further comprising:
A computer-implemented step of determining whether the node is for a simple element;
If the node is for a simple element,
Constructing an XML fragment for a node based on information contained in an index without accessing one or more base structures;
Providing the XML fragment in response to the request.   26. A computer readable medium carrying one or more sequences of instructions that, when executed by one or more processors, cause one or more processors to perform the method of claim 25.

Images