JP2002189740A - Data conversion system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data conversion system which converts an HTML(hypertext markup language) document into data attached a meaning. SOLUTION: The system is equipped with a data conversion part 2 which converts the document described in HTML into data attached a meaning and the data conversion part has a function of hierarchically structuring the HTML document according to a fixed hierarchy rule and replacing the hierarchically structured data with data attached a meaning according to a conversion rule corresponding to desired data to be extracted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for converting an HTML document into meaningful data.

[0002]

2. Description of the Related Art For example, sentences on a web page are HT
This is an HTML document described in ML. Such HT
The ML document describes information such as the structure and design of a sentence expressed on the page. In other words, where on the page, what characters, in what size, in what colors,
It decides whether to display.

[0003]

The computer can recognize the arrangement of characters from the HTML document, but cannot distinguish the meaning of a word represented by the characters. For this reason, it is difficult to extract specific data by specifying a meaning, for example, extracting only a person's name from the HTML document, for example, many web applications. Therefore, it has been difficult to integrate and use data on different web applications. That is, it is almost impossible to extract common items from different web applications or to collectively collect data of all applications. An object of the present invention is to provide a data conversion system for converting an HTML document into meaningful data.

[0004]

Means for Solving the Problems The first invention is an HTML document.
A data conversion unit for converting the document described in the above into meaningful data, the data conversion unit forms a hierarchical structure of the HTML document based on a certain layering rule,
It is characterized in that it has a function of reassembling the hierarchically structured data into meaningful data based on a conversion rule according to the data to be extracted.

According to a second aspect of the present invention, a step of reading an HTML document, a step of correcting tags that are not hierarchically structured among the tags of the read HTML document, and a step of hierarchically structuring the HTML document, and reading a conversion rule It is characterized in that it comprises a step and a step of converting the hierarchically structured data into data meaningful based on the conversion rule.

The third invention is characterized in that the conversion rule is a correspondence table between a path for specifying the position of data to be extracted in the hierarchically structured data and the meaning of the data.

A fourth invention comprises a conversion rule generation unit,
When an HTML document, a template, and a specific signal for specifying the location of data to be extracted in the HTML document are input to the conversion rule generation unit, the conversion rule generation unit converts the HTML document into a hierarchical structure, HTML
A path of data to be extracted in the hierarchically structured data is generated from the location specifying signal of the data to be extracted and the hierarchically structured data, and this path is applied to the variable portion of the template to perform the conversion. The feature is that rules are generated.

The fifth invention is characterized in that the HTML document is a document on a web site. In the present invention, a hierarchical structure means that a pair of tags surrounding data is closed properly, and the unit of each data is clear. It also means that an unclosed tag is not included between a pair of tags. In this way, by forming a hierarchical structure, it is easy to specify the position of each data.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiment shown in FIG. 1 comprises an HTML document input unit 1, a data conversion unit 2, a storage unit 3 storing conversion rules, and a data output unit 4 for outputting meaningful data. , Conversion rule generation unit 20 for generating a conversion rule
This is a system comprising: The HTML document is data expressed in HTML, such as a web page.

For example, when a business daily report is displayed on a web page as shown in FIG. 2, the information is represented by H shown in FIG.
Expressed in TML. HTML is data that specifies the appearance of the screen in FIG. For example, “work daily report” is displayed in the upper frame 5. Below that, the text 6 is displayed, and in it, four date / time columns 7a, 7b, 7
c, 7d, and business contents columns 8a, 8b, 8c, 8 for each day
Displaying d can be expressed in HTML. The HTML shown in FIG. 3 expresses that a character string “business logbook” is displayed in the frame 5, and “November 27, 2000” is displayed in the date column 7a in order from the top.
It has been decided that the word "date" and the word "create employee list" be displayed in the work content column 8a below it. In addition, the column 7b below that indicates “November 2, 2000
It has been decided that the text "8 days" and the text "Add Mr. Yamada's information to the employee list" will be displayed in the column 8b below it.

[0011] However, the text 6 is a business diary, "November 27, 2000" is a character string representing the date, and the "creation of employee list" under the text is "20".
The computer cannot determine from the HTML document that it is the business content of November 27, 2000. In other words, the meaning of such a character string can be determined by the user.
That is, it cannot be determined from the TML document. When such an HTML document is input to the HTML document input unit 1, the data conversion unit 2 converts the HTML document based on a conversion rule described later and outputs the converted HTML data as meaningful data. Output from section 4.

On the other hand, the meaningful data output from the data output unit 4 of this system is a business daily report, and the body of the text includes the date and the corresponding business content of the day. It is data that can be understood to be described. In other words, from the meaningful data, it is possible to know what kind of business was performed on a certain specific day.

Next, the data conversion unit 2 operates as shown in FIG.
A procedure for converting an ML document into meaningful data is as follows.
This will be described with reference to the flowchart of FIG. In step 1, an HTML document is input to the data conversion unit 2. In step 2, the data conversion unit 2 recognizes the position and type of the tag in the input HTML document. In step 3,
The data conversion unit 2 recognizes a tag having a hierarchical structure and a tag having no hierarchical structure in the tags.
Here, being hierarchically structured means that a pair of tags surrounding data is closed properly, and the unit of each data is clear. It also means that an unclosed tag is not included between a pair of tags.

Conversely, a tag that is not hierarchically structured is a tag whose end tag does not correspond to the start tag. For example, even if the end tag is omitted or the end tag is Is not appropriate. The pair of tags is, for example, a start tag <he
ad> and end tag </ head>. That is, the end tag is represented by adding “/” to the start tag.
In step 4, tags that are not hierarchically structured as described above are corrected. That is, a missing end tag is added or the position of the tag is moved. The modification of this tag is performed in accordance with a preset hierarchy rule.

In step 5, by modifying the tags as described above, the hierarchical structure of the HTML document is completed, and the data is stored. Specifically, the hierarchically structured data is data in which data surrounded by a pair of tags are grouped together and described so that the hierarchy of individual data in each group is clear.

In step 6, the hierarchical structure of the hierarchically structured data is recognized. Recognition of the hierarchical structure refers to specifying the hierarchical level of each data from the position of the tag of the hierarchically structured data based on the previously set hierarchical rules. In FIG. 5, the positions of the tags are shifted so that the hierarchical structure of the data can be easily understood. Here, as the position of the tag changes from left to right,
This indicates that the hierarchy of the data enclosed by the tags is lower. In addition, a pair of tags is connected by a line so that the relationship between the tags can be easily understood. Then, it indicates that the data enclosed by the outer tag is a higher layer than the data enclosed by the inner tag among the pair of tags.

In FIG. 5, tags corresponding to frame 5 on the display screen of FIG. 2 are <head> and </ head>, and tags surrounding the “business daily report” are <title> and </ title. >. Since the tags <title> and </ head> are closed while the tags <head> and </ head> are closed, <title> is lower in the hierarchy than <head>. Understand. Also, <head> and <body> are at the same level, <hr> is under <body>, and <br> is under that.
> Further, each date and the business content of that day are surrounded by <hr> and </ hr>, respectively, and these four sets of <hr> and </ hr> are all on the same level. In FIG. 5, <br> and </br> are tags indicating a line feed.

After storing the hierarchically structured data in the storage unit 3, in step 7, the conversion rules and templates are read. The conversion rule is a rule for converting the data to meaningful data. This conversion rule can be stored in the storage unit 3 in advance, or generated by the conversion rule generation unit 20 and
You can read it from there. in short,
In step 7, the data conversion unit 2 reads a conversion rule for converting the previously input HTML document into meaningful data.

The conversion rule is a correspondence table in which the position of each data (character string) in the hierarchically structured data is associated with the meaning of each data. Therefore,
This conversion rule is a rule created for each data having a different hierarchical structure. That is, if the display configuration of the screen shown in FIG. 2 is different, the conversion rule is also different. Conversely, if the screen configuration is the same, the same conversion rule can be applied. For example, if the display method of displaying the date and time and displaying the business content of the day under it does not change, the same conversion rule can be used even if the specific date and business content are different. On the other hand, a template determines how to represent converted data. This template must be input by the user to the conversion rule generation unit 20 in the process of generating the conversion rule. Therefore, this step 7
Thus, the data conversion unit 2 can read the template together with the conversion rule.

Here, the conversion rule generated by the conversion rule generation unit 20 will be described, but a method of generating the conversion rule will be described later in detail. FIG. 6 shows the conversion rules of this embodiment. This conversion rule is <re
Under <report> 11a below ports> 9a, the data (character string) specified by the path 12c is described as <date
> 12a and </ date> 12b. Below that, the data (character string) specified by the path 13c is stored as <content> 13a and </ content> 13
b.

The paths 12c and 13c are information indicating the position of a specific character string in the hierarchical data shown in FIG. However, <for select = ”in FIG.
"> 10a is called a" for statement ", but" for "means that it is repeated, so" for select = """
Is the path “html / body / h” displayed between “” and “”.
This means that the character string at the position "r" is extracted. That is, it means that the process of extracting the character string at the position of the path “html / body / hr” is repeated by the number of the paths. In FIG. 5, "html
There are four “/ body / hr” corresponding to each date. Therefore, here, the same processing is repeated four times. On the other hand, “.” Between <date> 12a and </ date> 12b in FIG. 6, and <content> 13a and </ content> 1
3b, there is a "."
This means that the path displayed between “” and “” in the sentence is substituted. For example, in the case of FIG. 5, the path 12c is “html / body / hr”, and the path 13c is “html / body / hr”.
/ body / hr / br ". The following describes this case.

In step 8, the data conversion unit 2 extracts a character string specified by the read paths 12c and 13c of the conversion rule from the hierarchically structured data shown in FIG. First, a character string specified by the path 12c is searched and extracted. The path 12c is “html / body / hr”, which is <hr> under <html>, <hr> under <body>.
Indicates the position of the character string enclosed by>. Therefore, in FIG. 5, the character string specified by the path “html / body / hr”, “2
November 27, 2000 "is extracted. This operation is repeated by the data conversion unit 2 for the same number of passes, that is, four times. As a result, as a character string specified by the path 12c, "November 27, 2000" or "1.
The four dates "January 28" are extracted.

Next, a character string specified by the path 13c is extracted. <Body> specified by path 13c
The process of extracting the character string enclosed by <br> under <hr> is repeated for the number of passes 13c in FIG. 5, as in the case of pass 12c. As a result, as the character strings designated by the path 13c, four character strings corresponding to the business contents of each day, such as "creation of employee list" and "in the employee list ..." are extracted.

The character string extracted as described above is applied to the variable portion of the template in step 9 and output in step 10. The template is a display format of an output result as meaningful data, and is shown in FIG. In FIG. 7, the underlined portion is called a constant portion, and the portions □ 14 and 15 are called a variable portion. Then, a character string at a location specified by the path of the conversion rule in FIG. 6 is applied to this variable part and output. The output result is
As shown in FIG. 8, the display is such that the date is inserted between <date> and </ date>, and the business content is inserted between <content> and </ content> below. Note that, also here, the portion surrounded by <for> and </ for> is repeatedly displayed four times as one set.

According to the above procedure, meaningful data is output. For example, a character string “November 27, 2000” is given “date”, that is, a date. In this way, by converting an HTML document having no meaning, it is possible to add meaning and unify the format after conversion. As a result, necessary data can be extracted from various HTML documents by specifying the meaning.

Further, since the converted data can be unified into the same format, all the data can be made into a database as meaningful data and managed in a unified manner. By converting data originally described in HTML into, for example, XML with meaning and unifying the data, the data can be commonly used in different systems.

Next, the procedure for generating the conversion rule is as follows.
This will be described with reference to the flowchart of FIG. This conversion rule generation is performed by the conversion rule generation unit 20 in FIG. First, in step 101, an HTML document is input. The input HTML document has the same data as that shown in FIG. Hereinafter, Step 102 to Step 1
05 generates hierarchically structured data. These steps are exactly the same as steps 1 to 5 in FIG. 2 described above, and a description thereof will be omitted here.

At step 106, the hierarchical structure of the hierarchically structured data is recognized. In step 107, FIG.
Enter the template shown in. The template entered here is created and entered by the operator.
Note that, in the flowchart of FIG. 7, steps manually performed are surrounded by double lines. When the template is input, the conversion rule generation unit 20
Recognize the position and number of variable parts. In the case of the template shown in FIG. 7, it is confirmed that there are two variable parts, the variable part 14 and the variable part 15.

At step 109, the operator enters
Specify the character string that you want to extract on the screen of. For example, when it is desired to apply a date between <date> and </ date>, a specific date “November 27, 2000
"Day" and input it to the variable section 14 of the template. Similarly, “create employee list” is input to the variable section 15. In step 110, the conversion rule generation unit sets “200” which is a character string designated by the operator.
"November 27, 0" is retrieved from the hierarchically structured data.

Then, in step 111, a path for "November 27, 2000" is generated. As is clear from FIG. 5, the character string "November 27, 2000"
It is under <hr> under <body> under <html>. Therefore, the path “html / body / hr” is generated. Step 11
At 2, it is determined whether there are a plurality of generated paths.
The fact that multiple paths are generated means that the character string "2000
In this case, the process proceeds to step 113. In step 113, multiple passes are presented to the operator,
Let them choose what they want to extract.

In practice, instead of displaying a plurality of paths, a plurality of portions corresponding to the above-mentioned character strings are indicated by blinking or the like on the screen of FIG. . In step 114, the path is specified by the operator specifying a necessary part. If there is only one generated character string in step 112, the process proceeds from step 112 to step 115. At step 115, it is determined whether or not all character strings corresponding to the variable portions of the template have been searched. In the process of the above steps, since the search for the character string “create an employee list” has not been completed, the process returns to step 110.

Similarly, paths are generated for all character strings input by the operator. When all the paths have been generated, the process proceeds to step 116, and the path corresponding to the variable part of the template is applied. Thus, the conversion rule is completed, and in step 117, the conversion rule generation unit 20
Stores the completed conversion rule. This conversion rule is shown in FIG. Note that the conversion rule generated as described above can be applied as it is if the display configuration of the screen is the same. In the case of a business daily report as in this embodiment, data conversion can be performed in exactly the same manner even if dates and business contents are different.

Further, as described above, the operator simply inputs the template and the character string that is present at the position of the data to be extracted, and the conversion rule generation unit automatically generates a path so that the conversion rule is generated. Generated. Therefore, if the conversion rule generation unit 20 of the present invention is used, the burden on the operator in generating the conversion rule is reduced.
However, the operator may generate a path while viewing the hierarchically structured data, create a conversion rule, and input the conversion rule to the data conversion unit 2. That is, a conversion rule created by an operator may be manually input without using the conversion rule generation unit 20. Further, a conversion rule generated by using another conversion rule generation system may be input to the data conversion unit 2.

[0034]

According to the first to fifth aspects of the present invention, meaningful data can be extracted from an HTML document in which it is difficult to extract specific data by designating the meaning. Also, since the converted data can be unified into the same format, all the data is converted into a database as meaningful data,
It can be centrally managed. By converting data described in HTML into, for example, XML with meaning and unifying the data, the data can be commonly used in different systems. According to the fourth aspect, the conversion rule can be generated semi-automatically. Further, according to the fifth aspect, by converting an HTML document displayed on various websites into meaningful data, it becomes possible to centrally manage the data.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an overall configuration of an embodiment.

FIG. 2 is a diagram illustrating a screen according to an embodiment.

FIG. 3 is data representing the screen of FIG. 2 in HTML.

FIG. 4 is a flowchart illustrating a procedure of data conversion according to the embodiment.

FIG. 5 shows hierarchically structured data in the embodiment.

FIG. 6 is a diagram illustrating a conversion rule according to the embodiment.

FIG. 7 is a diagram showing a template according to the embodiment.

FIG. 8 is a diagram showing meaningful data of the embodiment.

FIG. 9 is a flowchart illustrating a procedure for generating a conversion rule according to the embodiment;

[Explanation of symbols]

 Reference Signs List 1 HTML document input unit 2 Data conversion unit 3 Storage unit 4 Data output unit 20 Conversion rule generation unit

Claims (5)

[Claims] 1. A data conversion unit for converting a document described in HTML into meaningful data, wherein the data conversion unit forms a hierarchical structure of the HTML document on the basis of a certain layering rule, Based on the conversion rules according to the data you want to extract,
A data conversion system that has the function of translating into meaningful data. 2. A step of reading an HTML document, a step of correcting a tag that is not hierarchically structured among the tags of the read HTML document to hierarchically structure the HTML document, a step of reading a conversion rule, Converting the structured data into data meaningful based on the conversion rule. 3. The data conversion system according to claim 1, wherein the conversion rule is a correspondence table between a path for specifying the position of data to be extracted in the hierarchically structured data and the meaning of the data. 4. A conversion rule generation unit, wherein the conversion rule generation unit includes an HTML document, a template,
When a specific signal for specifying the location of the data to be extracted in the TML document is input, the conversion rule generation unit sets the H
The TML document is hierarchically structured, and a path of the data to be extracted in the hierarchically structured data is generated from the location specifying signal of the data to be extracted in the HTML and the hierarchically structured data. The data conversion system according to any one of claims 1 to 3, wherein the conversion rule is generated by applying the conversion rule to a variable part of the template. 5. The data conversion system according to claim 1, wherein the HTML document is a document on a web site.