KR20040000194A - A Method of Extracting a Document Type Definition - Google Patents

Abstract

PURPOSE: A method for extracting document structure information is provided to extract DTD(Document Type Definition) from XML(eXtensible Markup Language) which is not provided DTD, to generate DTD in a short time and efficiently. CONSTITUTION: A root tree node is generated(S1). A current path and insert pointer is generated, and initiates the current path and insert pointer(S2). An element is extracted by an XML Tokenizer at a tree constructing step. Tags in the XML document are analyzed, and extract start tag, end tag and PCDATA token(S3). Existence of same named element is checked(S4). In case that the same named tree node is existed, the new node is added(S5-1). S3 or S5 is executed for all elements(S6).

Description

A method of extracting a document type definition

The present invention relates to a DTD extraction method, and more particularly, to a method for extracting a DTD (Document Type Definition) from an unstructured XML document.

Extensible Markup Language (XML), which is being standardized by the W3C organization, is becoming a major standard for representing data on the World Wide Web. XML's proposal started with the goal of "any type of data can be freely exchanged and manipulated if it is expressed only in standard open form." XML with this flexibility is emerging as a standard for exchanging documents and querying the Web, combining next-generation applications such as e-commerce and intelligent web search. XML data is a form of semistructured data. XML data is a collection of hierarchically structured elements, each of which is a basic simple type (atomic) or a collection of other elements. Furthermore, each element in XML documents has a 'tag' that can represent the semantics of the data rather than just how the element is represented, as in HTML. Thus, like semi-structured data, XML data is hierarchically structured and self-describing. But XML differs from semi-structured data in the concept of document type definition (DTD), which is accompanied by free choice. The DTD contained in the XML document is responsible for specifying the internal structure of the document. In essence, the DTD marks each element with a regular expression pattern that the subelements of that element must follow. The DTD is an industry standard that allows not only the free exchange of electronic documents, but also the extraction, understanding and analysis of the contents of the documents in accordance with the DTD. In addition, the DTD provides a basic means of defining the structure of XML data. As such, DTDs play a very important role in query formation, optimization, and processing of XML documents as well as the efficient storage of XML data. For example, the DTD speeds up processing by allowing you to retrieve only the appropriate portion of the data when processing an XML data query. The basic idea is to use information about the data structure represented by the DTD to cut out elements that are unlikely to satisfy the path expression of the query. Despite the importance of these DTDs, DTDs are not required to be provided in XML documents. As a result, many XML data often do not have a DTD. However, even if no DTD is given, the schema information is already included in the form of 'tag' in the actual XML data. This is because XML data follows a semistructured data model.

The most essential technique for extracting product information from an informal document is to express the information in the document in a hierarchical structure and to extract the information. In order to extract information from unstructured data sources, active research is being conducted in the field of data mining, and in particular, information extraction from HTML documents is actively conducted through the use of wrappers. However, satisfactory information extraction has not been made yet.

Similar studies have been conducted by other research groups, but have the limitation that they cannot utilize the overall DTD grammar or interface with other program modules. Also, in order to extract the common structure of XML documents, they first adopt a two-stage algorithm that expresses the structure of each XML document using tree or graph and integrates them.

The present invention can be used as an interface of other program modules to solve the above problems, extract the DTD from the XML document is not provided DTD, efficient DTD extraction method that can generate DTD faster and more efficiently The purpose is to provide.

Another object of the present invention is to provide a DTD extraction method that is implemented as an independent module but can be utilized in other modules through an interface.

It is still another object of the present invention to provide a DTD extraction method using an advanced one-step algorithm that eliminates the step of transforming the structure of each XML document into an intermediate tree or graph in the conventional two-step DTD extraction algorithm.

The DTD extraction method according to the present invention for achieving these objects is characterized by representing the XML information input to create a single DTD for multiple XML data in a hierarchical data structure (Tree).

A specific feature of the DTD extraction method according to the present invention is a method for extracting a DTD from an Extensible Markup Language (XML) document that does not provide a Document Type Definition (DTD): a root tree node of an XML document. Generating a first process; Generating a current path and an insert pointer indicating a location to be searched and inserted in the tree and initializing the pointer to point to the root node; Extracting each element of the XML document; A fourth step of searching for a tree node having the same name as the delivered element with respect to the extracted element; If there is a tree node with the same name as the element, it updates the attributes of the tree node based on the element level, and if it does not exist, it indicates the current path and A fifth process of generating a tree node based on setting information of an insertion pointer and setting information of a start pointer; A sixth step of repeating steps 3 to 5 for all components of the XML document such that there are no unprocessed elements; A seventh step of extracting and merging a common node among two subtrees set in an OR relationship with respect to all subtrees; An eighth step of extracting the DTD grammar from the unified tree in which no subtree exists anymore is included.

1 is a system configuration diagram of an unstructured document collector;

2 is a tree representation of a DTD operator expressing each operator of an XML DTD using attributes of a tree and a tree node;

3 is a flowchart illustrating a process of a DTD extraction method according to the present invention;

4 is an exemplary diagram of a task of simplifying a tree in a tree modification and supplementation step

Hereinafter, a DTD extraction method according to the present invention will be described with reference to the accompanying drawings.

1 is a block diagram showing a schematic configuration of a system for processing information of a document when receiving informal product information from a shopping mall server connected through the Internet.

An XML document collector module 1 for receiving XML documents from each shopping mall, an extraction and grouping module 2 for extracting and grouping XML information from the collected data, a DTD extractor module 3, and XML mapping ) And a data storage module 4. Dual XML information extraction and grouping module (2) and DTD extractor module (3) is the core of the present invention, serves to extract a DTD from an unstructured XML document.

In the present invention, to extract a DTD from an unstructured XML document, two data structures, a tree node and an original tree node table, are used to represent an XML document as a tree.

The tree consists of nodes that represent each element of the XML document. Such a tree node consists of attributes that indicate the relationships each element can have in an XML document, namely, Mandatory, Optional, OR, and Repetition. In addition, a tree node requires a field for indicating a relationship between nodes in addition to an attribute. Table 1 shows the important fields and roles of the tree node.

Attribute name Role Remarks ele_name XML element name of the node Common attributes of tree nodes isMandatory Mandatory and Optional Representation of the Node isRepeat Repetition expression among relationships of the node isChecked Check the repetition of the node.Visit the node during tree construction isNewCreated Whether the node is a newly created tree node refnodeID Link information of the original tree node of the node General Treenode Attributes chain Repetitive expression of a certain pattern of a tree nodeSaves path information of the next tree node Attributes for Iterating Trees linked Iterative representation of element patternStorage path information of previous tree node isTemp Indicates that this node is a temporary node Temporary Node Related Attributes isOR Indicates that temporary node is in OR relationship isOptional Express that the temporary node is optional

As can be seen from the table above, the isChecked property to determine whether the element is processed and whether the tree node is newly created while processing the paragraph surrounded by the parent element of the XML document being processed. Has an isNewCreated attribute to indicate whether It also has an isTemp attribute, which is used to represent a temporary node.

In the process of constructing the XML document as a tree, several temporary nodes are used to express the order relationship, repetition, and optional relationship between elements included in each XML document. Because of this method, there may be a plurality of tree nodes for one element on the tree and each such tree node may have child nodes. In order to solve this problem of redundancy, the present invention is designed such that each tree node stores a path to an original tree node representing a structure of the corresponding tree node on the tree under the name refnodeID.

Also, by managing the list of original tree nodes as a table, when the new node is added to the tree or when updating the isChecked property of the tree node, the path information of the original tree node is extracted from the original tree node table, and the current path is inserted. It can be used to set the position of a pointer (insertPointer).

2 illustrates each operator of the XML DTD by using the above-described tree node and attributes of the tree node. Using the tree nodes and data structures described above, the DTD extractor will represent a nested structure in an XML document as a tree.

This tree structure can be divided into three stages: initialization, tree construction, tree modification, and supplementation. 3 is a flowchart illustrating a DTD extraction process according to the present invention.

In the initialization step, first, a root tree node that is the parent of all XML elements is generated (S1).

After generating a current path and an insertion point indicating a location to be searched and inserted in the tree, the path is initialized to point to a root node (S2).

In the tree construction phase, each element of the XML document is extracted by the XML Tokenizer. The extraction method analyzes a tag in an XML document. Analysis is to extract the token corresponding to the start tag (end tag), end (end tag) and PCDATA (S3).

It is searched whether a tree node having the same name as the delivered element exists for the extracted element (S4). If there is no tree node of the name, a new node is added (S5-1) or an attribute of an existing node is changed (S5-2). First, the search is performed in the tree based on the name of a given element. Based on the current path, insertPointer setting information, and start pointer setting information indicating the location to be added, a new tree node is created and added to the tree, or existing tree node properties Steps S3 to S5 for updating the attributes are performed for all elements (S6).

The tree constructed by the tree building phase will be in a form of significant complexity. In the tree modification and supplementation phase, a simplified tree is constructed to improve the readability of the DTD to be extracted. In the tree modification and supplementation phase, the simplified operation is performed on nodes whose isOR attribute is True among temporary nodes in the constructed tree. That is, a common node is extracted and merged among two subtrees set in an OR relationship.

For example, consider processing the following xml document:

<test>

<a> .... </a>

<b> .... </ b>

<c> .... </ c>

</ test>

<test>

<b> .... </ b>

<c> .... </ c>

</ test>

4 shows the simplification of the nodes whose isOR attribute is True in this tree modification and enhancement phase. The above example XML document will build the first tree through the tree construction phase. Here, there are common parts (b, c) in the two subtrees set in the OR relationship, but the tree is constructed by separating them into one (S7). The tree modifier traverses each element of the table by inputting the constructed original tree node table, extracts common nodes among two subtrees constructed in OR relationship among the child nodes of each element, and then further simplified tree It will be built (S8).

The DTD grammar is extracted from the integration tree formed through this process. Using the extracted DTD, the XML document is mapped and stored in the commercial database. In this way, the XML file is read and processed only once from the beginning to the end. Compared to the prior art, which requires reading twice, the 1-Path method according to the present invention increases the efficiency (speed) of the program because the I / O time is so small.

As described above, the DTD extraction method according to the present invention compensates for the shortcomings of the two-stage algorithm for extracting information from the XML document and expresses the structure of the XML document as a tree in a one-stage algorithm. It makes it possible to extract a single DTD for XML data.

Claims (1)

In a method of extracting document structure information (DTD) from an XML (Extensible Markup Language) document that does not provide document type definition (DTD) provided from a shopping mall via the Internet: A first step of generating a root tree node of an XML document; Generating a current path and an insert pointer indicating a location to be searched and inserted in the tree and initializing the pointer to point to the root node; Extracting each element of the XML document; A fourth step of searching for a tree node having the same name as the delivered element with respect to the extracted element; If there is a tree node with the same name as the element, it updates the attributes of the tree node based on the element level, and if it does not exist, it indicates the current path and A fifth process of generating a tree node based on setting information of an insertion pointer and setting information of a start pointer; A sixth step of repeating steps 3 to 5 for all components of the XML document such that there are no unprocessed elements; A seventh step of extracting and merging a common node among two subtrees set in an OR relationship with respect to all subtrees; And an eighth step of extracting the DTD grammar from the unified tree in a state where a subtree no longer exists.