JP3492247B2 - XML data search system - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an XML data retrieval system for storing and retrieving a large amount of data described in XML in a relational database, and more particularly to all XML data independent of the structure of an XML document. XML for enabling storage and for stored XML data
The present invention relates to an XML data search system capable of executing a query following a tree structure at high speed.

[0002]

2. Description of the Related Art The techniques currently used to store XML data can be broadly classified into two types: File storage: A method of storing an XML document in the file format. This method is intended to use the whole or part of the original XML file as it is, and therefore stores the XML document in the file format as it is. However, this alone makes it difficult to find the target file when the number of files increases, so it is necessary to prepare an index for searching the target file.

Table storage: A method for mapping and storing XML in a relational database table. In this method, the XML document is regarded as structured data and is stored in the database to perform high-speed search. Therefore, in this method, each element is mapped and stored in each column of the table of the relational database. In order to map XML data to a table, a mapping rule is required as to how each element of XML is mapped to each column of the table. This mapping rule needs to be specified by the user in advance.

[0004]

The biggest problem in storing XML data is that its data structure is not uniquely determined. In particular, in XML data without DTD (document type declaration), it is not known where and what tag appears, and the data structure is completely unknown. D
Even XML data with TD does not have a unique data structure because it is allowed to repeat tags, select tags, and recursively declare tags in DTD. Note that such data is called semi-structured data. When attempting to store XML data whose data structure is not fixed, the design of the storage schema becomes a problem. For example, sample XML data [X having the [DTD] shown in FIG.
ML data] is stored in the database as a table. This sample XML data is 2
It is data of a book catalog including information on books.

FIG. 9 is a diagram showing how the XML data is stored in a table. In the table of FIG. 9, one tuple corresponds to the information for one book, and all the tags that may appear in the XML data are taken in columns. At first glance, it looks as if the sample data was stored without problems. However, although there is no limit on the number of authors in the definition written in the sample data DTD,
In the table of FIG. 9, the space for storing the authors is prepared for only two persons at the maximum. If there are more authors in the XML data, can the data not be stored?
Even if it is stored, some information will be lost. As described above, the table storage cannot store the repeated tag described in the XML DTD. This is because it is necessary to specify the elements to be stored in advance as columns in the table storage, and therefore it is not possible to represent the repeating elements whose maximum number has not been determined. Also, you cannot store tags that are recursively defined for the same reason. In addition, XM
If there is no DTD in the L data and it is not known what kind of tag will appear, the structure of the table cannot be determined and it cannot be handled at all.

On the other hand, in the file storage, since XML data is stored in the file format as it is, there is no XML data that cannot be stored regardless of whether it is XML data without DTD or semi-structured XML data. However, it is not possible to retrieve only the information desired by the user from a large amount of stored data, so an index for retrieval is required. The structure of the index can be variously designed according to the purpose, and in a simple case, a set of a tag name and a character string is used as a key to search for an XML document in which the character string is surrounded by the tag and appears. There is something that comes. However, such a simple index cannot perform a search considering the hierarchical structure of tags. It may be possible to devise an index so that it has information on the hierarchical structure of tags, but the following still remains a problem.

(1) Since the index does not have all the information of the XML tree structure, it is impossible to search using all the information of the XML data. (2) Since the index is not optimized for tracing a tree structure, the search speed is slow when such a search is performed. As described above, in XML data whose data structure is not uniquely defined, how to use X without DTD
There is a problem of how to store ML data or semi-structured XML data, and how to execute a complicated inquiry that follows a tree structure to the stored XML data at high speed. The present invention has been made in view of the above-mentioned circumstances, and an object of the present invention is to store XML data whose data structure is not uniquely determined in a database, and to perform complicated trimming at high speed. It is to provide a data retrieval system .

[0008]

FIG. 1 is a diagram showing the basic configuration of the present invention. As shown in the figure, the system of the present invention is a system for searching data described in XML represented by a tree structure in which elements are intermediate nodes, element values and attribute values are leaf nodes, and tags are links. In the above, storage means 1 for storing XML data is provided, and an intermediate node table 2 for storing at least information of intermediate nodes and a link table 3 for storing information of links are provided in a relational database of the storage means 1. , And a leaf node table 4 for storing information of leaf nodes. Then, the XML data represented by the XML tree structure is divided in units of nodes, and the nodes 2 and 4 are associated with the link information and stored. In XML,
Since the optimal storage structure for storing is different between the intermediate node forming the tree structure and the leaf node that has the value of the element, store them in separate dedicated tables optimized as described above. Is desirable. in this way,
By storing the leaf node, which is a node for holding values, and the intermediate node, which is a node for holding tree structure information, in separate tables, it is possible to save the storage space for storing values. . A link for holding connection information between the nodes also needs to be stored and held in the link table 3. Further, the attribute table 5 for storing the attribute information may be separately provided. Further, the full path information from the root of each node is described in the intermediate node table 2 by ID, and the correspondence table between the ID for the path and the character string is stored in the path I.
By having the D table 6 separately, it is possible to save the storage space and speed up the search. Similarly,
The tag name of the link table 3 and the attribute name of the attribute node table are described by IDs, and a correspondence table of these label IDs and character strings is separately provided as the label ID table 7 to save storage space and speed up character string retrieval. Can be realized. Further, by adding information on the order in which each child element appears in the link table 3 and information on the order in which each element value appears in the element in the leaf node table,
The original XML document can be restored.

In the present invention, since the XML tree structure is stored in the storage means 1 as it is, XML data without DTD and semi-structured XML data can also be stored. Further, since all XML tree structures are stored in the database, all the information of the tree structure can be used for searching. However, with this alone, when an inquiry is made, it takes time to rejoin the tree structure of the XML data that is divided and stored for each node, and the execution time of the inquiry becomes slow. Therefore, in the present invention, X is added to the above tables 2 to 7.
The index 8 is set in consideration of the inquiry pattern for the ML data. As a result, it becomes possible to execute a complex query that follows the XML tree structure at high speed. To retrieve the XML data, for example, XML
Make inquiries in the data search language. As a result, the query processing means 9 checks the syntax of the query statement, generates a syntax tree for the query, and generates an optimum execution plan. This execution plan is described by a function set for tree structure search. According to this execution plan, an inquiry that follows the tree structure is executed using the index 8 and the requested search result is output.

The present invention can also be configured as follows. (1) Check XML syntax rules by applying relational database constraint functionality to tables. (2) In the link table, add information on the order of appearance in sibling elements that have the same label for each element,
It is possible to execute an inquiry that specifies the order of appearance of each label. (3) By waiting for not only the information on both nodes of the link but also the information on the tag name in the link table, an inquiry that follows the link by specifying the tag name is executed at high speed. (4) by the intermediate node to the destination attribute node in the attribute table rather than link, the attribute condition to reduce a table search count when executing the query following the tree structure, fast query Enable execution. (5) When constructing an index in B ⁺ -tree for performing a search by path ID in the intermediate node table at high speed, duplication of key values is eliminated by using a key value as a set of path ID and node ID. (6) When constructing an index in B ⁺ -tree for performing a high-speed search by the document ID of the intermediate node table, by using a key value as a set of the document ID and the node ID, duplication of the key value is eliminated.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below. (1) System Configuration FIG. 2 is a diagram showing a system configuration of an embodiment of the present invention. As shown in the figure, the system of this embodiment is roughly divided into an XML data storage unit 11 and an XML data storage unit 1.
1, an XML data insertion module 12 for inserting XML data, and an inquiry processing engine unit 13 for processing an inquiry to the stored XML data. The XML data is the XML data insertion module 12
Is inserted into the XML data storage unit 11. XM
The L data insertion module 12 is composed of an XML parser 12a and a loader 12b. The XML parser 12a parses the input XML data, and a tree structure of the XML data is stored in the XML data storage unit 11 in units of nodes. Disassemble. Further, the loader 12b inserts the tree structure decomposed into node units into the table of the XML data storage unit 11.

FIG. 3 is a flow chart showing a storage process of the XML data. In this embodiment, the XML data storage process is performed as follows. First, step S1
At, the XML file is read. In step S2, the XML parser parses the input file. If the analysis is successful, the process proceeds to step S3, and the XML parser outputs the XML tree structure node information and link information as an intermediate format in a file. If the parsing is not successful, an error is output as a syntax parsing failure and the process ends. In step S4, the generated intermediate format file is read, and in step S5, the read XML data is inserted into each table of the relational database by the loader, and the process ends. If the insertion is not successful, an error is output as a data insertion failure and the process ends.

The inquiry for the stored XML data is made in the XML data inquiry language, and the inquiry is processed by the inquiry processing engine 13.
The query processing engine 13 includes a query language parser 13a, a query optimization engine 13b, and a tree structure search API (application programming interface) 13c. The query language parser 13a checks the syntax of the input query text and generates a syntax tree for the query. The query optimization engine 13b creates an optimal execution plan based on the syntax tree. This execution plan is for tree structure search A
It is described by the function set of PI13c. For tree structure search A
The PI 13c is an interface with the XML data storage unit 11 and is a set of functions for performing a basic search on the XML tree structure.

Next, the configuration of each part in the above system will be described in more detail. (1) Table Configuration First, the configuration of the table stored in the XML data storage unit 11 will be described. Although there are several methods of expressing XML data in a tree structure, the present embodiment assumes the tree structure expression shown in FIG. FIG. 4 shows the XML data shown in FIG. 8 in a tree structure. In this tree structure representation, the round intermediate nodes represent elements, and the parent-child relationship of the nodes represents the inclusion relationship of the elements.

The number in the circle of the node is the node ID
Is represented. The link (branch) connecting the nodes represents a tag, and the character string written next to the link represents the tag name. The triangular leaf node represents the value of the element, and the square leaf node is the attribute (Atrrib) attached to the tag.
ute). Only these two leaf nodes have a value. If only the node information is stored in the database table when the nodes are divided and stored in the database, the connection between the nodes in the tree structure, that is, the link information is lost. Therefore, for the link information, a dedicated table for storing the link information is prepared. Also, regarding the nodes, since the optimum storage structure is different between the intermediate node, the leaf node of the element value, and the leaf node of the attribute, it is necessary to store them in separate tables.

The following six tables are used in total in this embodiment. Intermediate node table This is a table that stores information about intermediate nodes. In addition to the node ID (id), the column has a document ID (docid) of a document including the node and a full path ID (pathid) from the root to the node. Link table This is a table that stores links between nodes. Node ID (id), link label (tag name) ID (labe
lid), the node ID of the child node (child), the order of appearance of the child node in all sibling nodes (tord: total order), the order of appearance of the child node in sibling nodes having the same label (p
ord: partial order) as a column. As described above, in the link table, the label (tag name) ID (l
By adding abelid), it becomes possible to execute a query that follows a link by specifying a tag name at high speed.

Leaf node table This is a table for storing leaf nodes of element values. The node ID of the intermediate node that corresponds to that element (i
In addition to d), it has the value of the element and the order in which the value appears in the element as a column. Thus, by providing the leaf node table for holding values separately from the intermediate node table, it is possible to save the space for storing the values.

Attribute node table This is an attribute attached to a tag (for example, <boo in FIG. 8).
This is a table that stores years in k year = "1995">. In addition to the node ID (id) of the intermediate node corresponding to the element including the tag, the ID (labelid) of the attribute name and the attribute value (Attvalue) are waited as columns. In addition, using the constraint function of the relational database for the attribute table, (id, labeli
By applying the constraint that the set of d) is unique, it is possible to check the syntax rule regarding the attribute of XML that "the same attribute name must not appear in the same tag". Further, in the tree structure representation assumed in this embodiment, the XML tag corresponds to a tree structure link, and therefore the attribute attached to the XML tag should be attached to the link as it should be. However, in FIG. 4, the attribute is attached to the node below it, not to the link. This is to reduce the number of table references at the time of search. That is, it is possible to reduce the number of table searches when executing a query that follows a tree structure with an attribute as a condition, and to speed up the query.

Path ID Table This is a correspondence table of path IDs and path character strings. Having the path string separately instead of directly writing it to the intermediate node table saves space, but when a search involving path name string matching is performed, the search target This is because there is less and the search speed can be increased. Label ID Table This is a correspondence table of label IDs and label character strings. In this way, the tag name of the link table and the attribute name of the attribute node table are described by ID, and the correspondence table of the ID of this label and the character string is separately provided as a label ID table, so that the same as the path ID table is stored. You can save space and speed up searches.

Further, as described above, in the link table, the appearance order (tord: tota
(l order) information is added, and the order in which each element value appears in the element in the leaf node table.
By adding the information of (order), the original XML document can be restored from the XML data decomposed into node units stored in the XML data storage unit 11. For example, "Today was <weather> Fine </ weather>.
I went to Places> Department Store </ Place>. It is also possible to restore sentences separated by tags such as "." In addition, by adding the information of the appearance order (pord: partial order) in sibling nodes that have the same label of each element in the link table, queries that specify the appearance order of each label can be executed at high speed. Is possible.

As an example, FIGS. 5 and 6 show how the sample XML data of FIG. 8 (tree structure representation of FIG. 4) is stored in the above table group. FIG. 5 is a diagram showing an example of the intermediate node table and the link table. In the intermediate node table, for example, the id (= 5) in the first line indicates the node marked with "5" in FIG. 4, and the document ID (docid) of the document containing that node is 1. . Also, the full path ID (pathid) from the root to that node is 1.
And the path corresponding to this ID is "bib.book.publi
sher.name ". In addition, in the link table, for example, the id (= 4) in the first line indicates the node marked" 4 "in FIG. 4, and its label is 5, and this labe
The label corresponding to the lid is "name". Also, tord and pord, which indicate the order of appearance, are "0" and "0" respectively,
The child node is a node marked "5" in FIG.

FIG. 6 is a diagram showing an example of a leaf node table, an attribute node table, a path ID table, and a label ID table. In the leaf node table, for example, the id (= 5) in the first row indicates the node marked "5" in FIG. 4, its order is "0", and the value of that leaf node.
(value) is "Addison-Wesley". In the attribute node table, for example, the id (= 3) in the first row indicates the node marked "3" in FIG. 4, and its labelid is 3
(Corresponding to "year"), its attribute value (attvalue) is "19"
95 ". The path ID table and the label ID table respectively include pathid and labe in the above tables.
The path character string and the label character string corresponding to the lid are stored. For example, the character string corresponding to pathid = "1" is "bib.book.publisher.name" as described above, and
For example, the character string corresponding to labelid = "1" is "bib".

(2) Structure of Index In this embodiment, the tree-structured nodes that were originally connected are divided into individual nodes and stored in the relational database table as described above. For this reason, when an inquiry is made that follows the tree structure, a join operation is performed to reconnect the links of the part that is followed by the inquiry. Since the speed of this join operation greatly affects the overall search speed, it is necessary to effectively extend the index so that the join operation can be performed at high speed. Also, when an inquiry is made, what is specified as a search condition is an element value, an attribute, a path, an appearance order, or the like. It is also necessary to prepare an index for those searches as well because they need to be searched at high speed.

FIG. 7 shows a list of indexes created in the tables shown in FIGS. 5 and 6. This index is extended by B ⁺ -tree, and an index whose key is a set of multiple attributes can also be used for searching with a partial set of attributes from the beginning of the set. The index in the intermediate node table whose key is (pathid, id) is used when searching all the nodes corresponding to a certain path. The key to this index may seem ok at the seemingly pathid alone. However, if the key is only pathid, there will be too many entries with the same key value and the B ⁺ -tree index will not work. The key value is the path ID (p
By making the combination of ath id) and the ID (id) of the node, it is possible to eliminate the duplication of the key value, and to search the B ⁺ -tree at high speed. The same applies to the index (docid, id) in the index that is set up in the intermediate node table.
By using the combination of the document ID (docid) and the node ID (id), it is possible to eliminate the duplication of the key value and to search the B ⁺ -tree at high speed.

(3) Execution of Inquiries As described above, inquiries regarding the stored XML data are made in the inquiry language of the XML data, for example. A search language XQL is one of the search languages for XML data. An inquiry sentence in XQL will be briefly described with an example.

SELECT result: <$ book.title> FROM book: bib.book WHERE $ book.author.lastname = "Darwen"; The meaning of this inquiry is "bib.book.author.lastname is
For a bib.book that is Darwen, bib.book.title
Is to be obtained as a search result ”.

As shown above, the inquiry sentence is large and divided into three parts: SELECT, FROM, and WHERE.
In the SELECT part, specify the projection of the element you want to get as a search result. In the FROM part, the element to be searched is specified. In the WHERE part, a selection of search conditions is specified. As described above, the inquiry as described above is made for the inquiry processing engine 13
Is processed in. The query processing engine 13 checks the syntax of the query statement as described above and generates a syntax tree for the query. Then, based on the syntax tree,
Generate an optimal execution plan. This execution plan is described by a function set for tree structure search.

Next, how the inquiry processing for the XML data is performed will be described. Here, as an example of the case where the sample XML data of FIG. 8 is stored in the XML data storage unit 11 and is inserted into the tables shown in FIGS.
A description will be given of a case in which an inquiry is made asking for the title of the book that is "en". The table search in this case is performed as follows. The following processings 1 to 10 are executed by the tree structure search function.

1. The leaf node table is searched to obtain the node ID (= 16) of the node whose value is "Darwen". 2. Search the path ID table to find the path "bib.book.auth.
Get the path ID (= 4) of "or.lastname." 3. Set the intermediate node table to node I obtained in 1. above.
It searches by D (= 16) and confirms that the obtained path ID (= 4) matches the path ID (= 4) obtained in 2. above. 4. Search the label ID table and label "lastname"
Label ID (= 8) of 5. Search the link table to find the node ID (= 16) obtained in 1 above and the label ID obtained in 4 above.
The node ID (= 15) of the parent node is obtained from (= 8). 6. Search the label ID table and label "author"
To obtain the label ID (= 7). 7. Search the link table to find the node ID (= 15) obtained in 5 above and the label ID obtained in 6 above.
The node ID (= 9) of the parent node is obtained from (= 7). 8. The label ID table is searched to obtain the label ID (= 6) of the label "title". 9. Search the link table to find the node ID (= 9) obtained in 7 above and the label ID (= 9) obtained in 8 above.
From 6), the node ID (= 12) of the child node is obtained. 10. Search the leaf node table, and from the node ID (= 12) obtained in 9. above, find the value of that node ("Foundat
ion for Object / Relational Database "). The search result obtained as described above is output via the query processing engine 13 and presented to the user.

[0030]

As described above, according to the present invention, the relational database has an intermediate node table for storing information of intermediate nodes, a link table for storing information of links, and information of leaf nodes. A table such as a leaf node table for storing the tree structure is provided, the XML tree structure is decomposed into nodes and links, and each node and link information are stored in the above table in association with each other. Since a query following is executed and XML data is searched, a complex query for XML data whose data structure is not uniquely determined can be executed at high speed. In addition, since the XML tree structure is stored in the storage means as it is, XML data without DTD and semi-structured XML data can also be stored.
Furthermore, since all XML tree structures are stored in the database, all information of the tree structure can be used for searching.

[Brief description of drawings]

FIG. 1 is a basic configuration diagram of the present invention.

FIG. 2 is a diagram showing a configuration example of a system according to an embodiment of the present invention.

FIG. 3 is a diagram showing a storage processing flow in the system of the embodiment of the present invention.

FIG. 4 is a diagram showing an example of a tree structure representation of XML data.

FIG. 5 is a diagram (1) showing an example of a table configuration according to the embodiment of the present invention.

FIG. 6 is a diagram (2) showing an example of a table configuration according to the embodiment of the present invention.

FIG. 7 is a diagram showing an index list of an example of the present invention.

FIG. 8 is a diagram showing an example of XML data.

9 is a diagram showing how the XML data of FIG. 8 is stored in a table.

[Explanation of symbols]

1 XML data storage storage means 2 Intermediate node table 3 link table 4 leaf node table 5 attribute table 6-pass ID table 7 Label ID table 8 indexes 9 Inquiry processing means 11 XML data storage 12 XML data insertion module 12a XML parser 12b loader 13 Inquiry processing engine section 13a Query language parser 13b Query optimization engine 13c For tree structure search

Continuation of the front page (56) References Somura Shimura, Masatoshi Yoshikawa, general-purpose storage and retrieval of XML documents using object-like relationships, Proc. Of IPSJ 58th (the first half of 1999) 3), March 9, 1999, pp.265-266, Takashi Tajima, Data models and operational languages for semi-structured data, Journal of Information Processing Society of Japan, February 15, 1999, Volume 40, SI G3 (TOD1), pp. 152-170 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06F 17/30 G06F 12/00 JISST file (JOIS)

Claims (3)

(57) [Claims] 1. A system for retrieving data described in XML represented by a tree structure in which an element is an intermediate node, element values and attribute values are leaf nodes, and tags are links, the system comprising: An intermediate node table for storing at least intermediate node information in a relational database of the storage means, the storage means storing XML data;
A link table for storing link information and a leaf node table for storing leaf node information are provided, and an intermediate node table is quickly searched by a node ID.
Depending on the index to do and the document ID of the table
Index for fast search and path
Prepare an index for high-speed search and search the linked table from parent node to child node at high speed.
And the parent node from the child node
Prepare an index for quick search , and store the value of that node from the node ID in the leaf node table.
Find an index to get and a node with a certain value
Prepares an index for doing so , decomposes the XML tree structure into nodes and links, and stores each node and link information in association with each other in the table,
An XML data search system , which refers to the table and executes a query that follows a tree structure using the index to search XML data. 2. The relational database is provided with a path ID table, which is a correspondence table of path character strings and path IDs , and a label ID table, which is a correspondence table of label character strings and label IDs. Bus ID corresponding to the path character string in the table
Prepare an index to search for
Search the label ID corresponding to the label character string
An XML data search system according to claim 1, wherein an index for performing the above is prepared, and a query for tracing a tree structure is executed using the index. 3. The link table adds information about the order in which each child element appears in that element, and the leaf node table adds information about the order in which each element value appears in that element, Based on the above information, the original X
Claim, characterized in that to enable restoration of the ML document 1
Alternatively, the XML data search system according to claim 2 .