KR20050116089A - Fixed-rdbms model based indexing technology for massive xml data searching with different schema - Google Patents

Abstract

The present invention is a system for storing XML documents in fixed database tables to efficiently manage large amounts of XML data of various structures and to execute various queries. In more detail, when parsing an XML document using Java SAX PARSER, each element of the XML document includes element depth, element sequence, element attribute, and name. After extracting the name, parent element, start position, and end position, each schema information and element information file is generated to store these information in the database. The generated text file is stored in schema table and element table and becomes DB data that can process XML query of various conditions. In addition, direct access to specific elements is possible, and improved search performance can be provided for XML documents of various structures.

Description

Fixed-RDBMS Model Based Indexing Technology for Massive XML Data Searching with Different Schema}

The present invention relates to a method for storing and managing an XML document in a fixed database table so as to efficiently manage a large amount of XML data having various structures and to execute various queries, and software programmed to perform the same. In more detail, after parsing an XML document, each element, attribute, and value of each element depth, element sequence, element attribute, element name, and parent information are parsed. (parent element), start position (end position), end position (end position) method of extracting, and how to create a text file and stored in two fixed database table and execution program.

Recently, due to the development of the Internet and the rapid increase in the amount of information, research on the storage and retrieval of XML (eXtensible Markup Language), which is the standard of next-generation web documents, is being actively conducted for efficient information exchange on the Internet. In addition, many standards and tools for creating, modifying, and manipulating data in these XML formats are on the rise. In such a situation, optimizing the data storage of large-scale XML documents having different structures and extracting the necessary information from them easily and quickly becomes a very important issue.

Several XML indexes have been proposed to support efficient path retrieval for XML documents. They aim to support efficient path retrieval for one large XML document or multiple XML documents with the same structure. Therefore, when using these indexes to find the desired path from several XML documents with different structures, we must construct an index for each XML document and inspect all indexes. In addition, when the search path is represented by an ancestor-descendant relationship, all nodes of the constructed indexes must be visited to find the desired path, and thus the search performance is drastically degraded. In order to compensate for this, an index has been proposed. However, when an ancestor-progeny relationship appears in the middle of a path other than the root, performance has a disadvantage.

On the other hand, unified indexes that can search XML documents with different structures were also proposed. In the field of information retrieval, they were extended by reflecting the structural characteristics of XML in RDBMS-based inverse index. In addition, all node information in the index is stored in a table in the database so that it can be stably managed. By using the efficient query processing functions provided by the database system for path search, it prevents additional implementation for path search and supports efficient search. It was. However, as the number of documents among the proposed indexes increases, the data in the index to be compared / searched increases, so that the search performance is deteriorated. As the number of documents with different structures increases, performance decreases. This feature makes it difficult to apply in situations that require path retrieval for large amounts of XML documents.

Looking at the file system-based storage method that saves individual XML documents in a specific folder in the form of a file, the additional work on the entire document is done in units of files, so it is relatively fast to save and extract, while parsing XML every time an XML document is extracted. This is inefficient because of the need for the document system and the number of folders due to the limitation of the file system, and it is impossible to implement support and access control for simultaneous user access, and to search with the need to implement all functions separately for XML processing. Additional DBMS needs to be used. Looking at the integrated RDBMS-based storage method for storing XML documents in the form of BLOB or CLOB in the RDBMS, while storing and extracting documents is fast, it takes a lot of time to modify or retrieve the documents and requires processing for XML documents. Each time you need to load the entire document into memory, you need a lot of memory and time, and all the functions for XML processing must be implemented separately.

The partitioned RDBMS-based storage method that divides each element of an XML document into units of the table and stores them in a field of the table can be used to perform partial modification or retrieval of the document quickly. It takes a lot of time because you need to access the table, and it is very slow because it takes several tables to join the document extraction, and it is difficult to expand because the design of the DB table needs to be changed or added whenever the document format is changed or added. Therefore, all functions for XML processing must be implemented separately.

Therefore, in order to solve the above-mentioned problems, that is, the disadvantage of retrieval performance as the number of documents having different structures increases, or the problems arising in the filesystem-based storage method, the schema information and element information of the XML document can be solved. It is designed to be extracted regardless of the complexity or size of the document. The fixed database and element information is stored regardless of the addition or change of XML document structure. Improve efficiency In addition, by providing a method of executing user queries in SQL, it is possible to make the best use of the existing mature database technology and improve the improved search performance.

Hereinafter, described in detail by the accompanying drawings as follows. 1 is a configuration diagram of an XML document storage system. Referring to FIG. 1, an XML document composed of String Data is read and an XML document is parsed through an XML SAX Parser. As a result of parsing the XML document, each element is assigned a start and end number with data created in a DOM tree structure. Then, various kinds of information are extracted from the XML of the tree structure. Depth information, order information, attribute information, and contents of each extracted element are stored in a fixed table of the RDBMS.

FIG. 2 is an example XML document for describing FIGS. 3 to 7.

FIG. 3 is a diagram illustrating a method of determining start and end numbers of elements after parsing an XML document of FIG. 2. As shown in Figure 1, after parsing an XML document, a DOM Tree structure is created. Here's how to assign start and end numbers to each element. Assign the starting number 0 to the books element (root) that starts first. Assign the start number of the title element, which is a child of the books element, to 1. Assign the start number of the 'Data Model', the string value of the title element, to 2 and the end number to 3. Assign the end number of the title element, which is the parent of the string value 'Data Model', to 4. Assign the starting number of the section element, which is a child of the books element, which is a sibling of the title element, to 5. Assign the start number of the title element, which is a child of the section element, to 6. Assign 'Syntax For Data Model', the string value of the title element, to 7 and to 8. Assign the end number of the title element, which is the parent of the string value 'Syntax For Data Model', to 9. Assign the end number of the section element, which is the parent of the title element, to 10. Assigns 11 the start number of the section element, which is a child of the books element, the third sibling of the title and section elements. Assign the starting number of the title element, which is a child of the section element, to 12. Assign 'XML', the string value of the title element, to 13 and to 14. Assign the end number of the title element, which is the parent of the string value 'XML', to 15. Assign the start number of the section element, which is a child of the section element and which is a sibling element of the title element, to 16. Assign the starting number of the title element, which is a child of the section element, to 17. Assign the starting number of 'Basic Syntax', the string value of the title element, to 18 and the ending number to 19. Assign the end number of the title element that is the parent of the string value 'Basic Syntax' to 20. Assign the end number of the section element, which is the parent of the title element, to 21. It assigns 22 the start number of the section element, which is a child of the section element, the third sibling of the title and section elements. Assign the starting number of the title element, which is a child of the section element, to 23. Assign 'XML and Semistructured Data', the string value of the title element, to 24 and to 25. Assign the end number of the title element, which is the parent of the string value 'XML and Semistructured Data', to 26. Assign the end number of the section element, which is the parent of the title element, to 27. Assign the end number of the section element, which is the parent of the section element, to 28. Assign the end number of the book element (root), which is the parent of the section element, to 29.

FIG. 4 is a flowchart expressing schema structure information extraction and file generation of an XML document by executing XML SAX. When you run Sax, first run (1). (1) initializes the current position to 1 and the level (Depthid) to 29999. The document number is increased by one to the final document number of the DB. (2) reads each element sequentially. (3) compares the syntax of the elements read and processes each function according to the start ('<>') and end ('</>'). (4) is the start of the element, which increases the current level (Depthid) by one. If the current level (Depthid) + current element name + parent level (Depthid) + parent element name exists in the element sequence hash, the element sequence of the current level (Depthid) branches to the next step without change, otherwise the current level (Depthid) After increasing the element sequence of 1, add the element sequence to the element sequence hash with the current level + current element name + parent level (Depthid) + parent element name. Current level, element sequence, attribute sequence ('0'), element name, parent level (current level-1), parent element sequence (parent level + parent element name + grandparent level + grandparent element in element sequence hash) Retrieve name by key), and store parent property sequence ('0'). (5) checks whether an element is present or not and executes (6) if present, and reads the next element (2). (6) is to store the attribute sequence of an element and increases the current level (Depthid) by one. If the current level (Depthid) + current attribute name + parent level (Depthid) + parent element name exists in the attribute sequence hash, the attribute sequence of the current level (Depthid) branches to the next step without change, otherwise the current level (Depthid) After increasing the attribute sequence of 1, add the attribute sequence to the attribute sequence hash with the current level + current attribute name + parent level (Depthid) + parent element name. Current level, element sequence ('0'), attribute sequence, element name, parent level (current level-1), parent element sequence (parent level + parent element name + grandparent level + grandparent element in element sequence hash) Retrieve name by key), and store parent property sequence ('0'). Decreases the current level by 1. (7) is the end of the element, which decreases the current level (Depthid) by one. (8) is the case where the end of the XML document is met, and the execution of XML SAX is terminated.

FIG. 5 shows the contents of a text file of the generated schema structure information after executing the XML document of FIG. 2 in FIG. 4. The columns in each row are separated by commas. The information in the first column is the element depth, the second column is the sibling order of the elements under the same parent, the third column is the attribute order within the same element, the fourth column is the parent element depth, and the fifth column is the The sibling order of the parent element, the sixth column shows the attribute order within the parent element, and the seventh column shows the element name. Each row is schema information of an element having a different structure.

6 is a flowchart illustrating the extraction of each element path information and file generation in XML by executing XML SAX. When you run Sax, first run (1). (1) initializes the current position to 1 and the level (Depthid) to 29999. The document number is increased by one to the final document number of the DB. (2) reads each element sequentially. (3) compares the syntax of the elements read and processes each function according to the start ('<>'), end ('</>'), and values (characters). (4) is the case where the element starts, and increases the current position by one and increases the current level (Depthid) by one. The level hash stores the current level (Depthid) and the current level (Depthid) + current element name. The start position hash stores the current level (Depthid) + current element name + parent level (Depthid) + parent element name and start position (current position), respectively. (5) checks whether an element is present or not and executes (6) if present, and reads the next element (2). (6) is to save the attribute value of an element. It increases the current position by 1 and increases the current level (Depthid) by 1. The level hash stores the current level (Depthid) and the current level (Depthid) + current property name. The start position hash stores the current level (Depthid) + current attribute name + parent level (Depthid) + parent element name and start position (current position). End position is the current position increased by one. Enter the current document number, current level, element sequence ('0'), and attribute sequence (Depthid + current attribute name + parent level (Depthid) + parent element name in the attribute sequence hash of FIG. 4) in the element information file. Search), start position, end position and attribute value. Decreases the current level by one. (7) is the end of the element. After increasing the current position by 1, the current position is stored at the end position. In the element information file, the current document number, current level, and element sequence (search for the current level (Depthid) + current element name + parent level (Depthid) + parent element name in the element sequence hash of FIG. 4 by key), and attribute sequence (' 0 '), start position (search current level (Depthid) + current element name + parent level (Depthid) + parent element name with key), end position, and attribute value. Decreases the current level by one. (8) is the case of encountering an element value. After increasing the current position by 1, save the current position at the start position. Again, increase the current position by 1 and save the current position at the end position. In the element information file, the current document number, current level, and element sequence (search for the current level (Depthid) + current element name + parent level (Depthid) + parent element name in the element sequence hash of FIG. 4 by key), and attribute sequence (' 0 '), start position, end position, element value are saved. (9) is the case where the end of the XML document is met, and the execution of XML SAX is terminated.

FIG. 7 illustrates the text file contents of the generated element path information after executing the XML document of FIG. 2. The columns in each row are separated by commas. The information in the first column is the XML document id, the second column is the element depth, the third column is the sibling order of the elements under the same parent, the fourth column is the attribute order within the same element, and the fifth column is the The start position of the element, the sixth column the end position of the element, and the seventh column the element contents. Each row is an element path and content information on an XML document.

FIG. 8 is a table structure diagram for storing text files FIGS. 5 and 7 generated in FIGS. 4 and 6 in a database, respectively. X table is a fixed table structure that stores the schema structure information of the XML document (FIG. 5), and Y table stores path information and values (FIG. 7) of each element. The following describes specific attribute information of X table. (However, if the depth information and the element name are the same under the same parent, it is regarded as the same schema structure and only the first element information is stored in the Y table.)

A field stores depth information of each element.

B field stores order information between sibling elements under the same parent.

The C field stores the order information of the attributes in each element.

D Field stores the name of each element.

E Field stores the depth information of each element's parent element.

The F field stores sibling element order information of the parent element of each element.

G field stores the order information of the attributes in the parent element of each element.

The following describes the specific property information of the Y table.

I Field stores the Document ID.

J field stores depth information of each element.

The K field stores the order information between sibling elements under the same parent.

The L field stores the order information of the attributes in each element.

The M field stores the start number of each element.

The L field stores the end number of each element.

O Field stores the contents of each element.

As described above, in the present invention, data structure information and data of XML documents having various hierarchical structures can be stored and managed in two types of fixed relational database tables to quickly search a large XML document. It can cope with efficient management of physical storage location and addition and change of XML document. In addition, even if the format of the XML document is changed, it provides the convenience that does not require logical change of the table property, and it is possible to process various search conditions with only two tables, thereby increasing the practical use of the work. File information generated when parsing an XML document can be easily stored in an existing mature database and can be applied to various RDBMS systems.

  1 is a configuration diagram of an XML document storage system

  2 shows an example XML document.

  3 is a start number of elements after parsing the XML document of FIG.

 Drawing showing how to extract start position and end position

  Figure 4 extracts schema structure information of a document from XML by executing XML SAX and

Flowchart showing file creation

  5 is a schema information file generated after executing the XML document of FIG.

Drawing shown

  6 illustrates the extraction of respective element path information from XML by executing XML SAX;

 Flowchart showing file creation

  FIG. 7 illustrates an element information file generated after executing the XML document of FIG.

Drawing shown

  8 is a table structure for storing the text file of FIGS. 5 and 7 in a database.

drawing

Claims (2)

To store an XML document in a database table, XML schema information (element depth, element order, element attribute order, element name, parent element depth, parent element order, parent element attribute order) and element information (document id, element depth, Process of creating element order, element attribute order, element start position, element end position, element value) into text file. In order to save the text file created in the relational database, the same schema management table as each file attribute (element depth, element order, element attribute order, element name, parent element depth, parent element order, parent element attribute order) ) And the process of creating an element management table (document id, element depth, element order, element attribute order, element start position, element end position, element value), and storing the text file created in claim 1 in each table ( A method for managing database storage of XML documents characterized by a process of inserting and software programmed to perform the same.