JP4046000B2 - Structured document extraction method, apparatus and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to extracting method and apparatus and program of the structured document, in particular, from the structured document such as HTML, to specify and extracting a desired portion, reused in another structured on the document, or, after the text or the like is changed to include a method and apparatus and a program extraction of the structured document to extract the corresponding portion from the document.
[0002]
[Prior art]
There is a need to extract a specific designated portion from a structured document such as HTML that is updated daily. For example, if it is possible to collect and extract only the part of interest from a website that the user visits every day, it is possible to easily list only necessary information without scrolling the page. In the extraction, even if the text included in the designated portion is changed, it is required to extract the latest information after specifying the original designated portion from the changed document.
[0003]
Two conventional technologies for realizing this will be described below.
[0004]
One is a WWW information extraction system. In order to provide a clue to the designated part of a structured document such as HTML, the text at the start and end points is stored. Extraction of a designated portion from a document is performed by specifying a start portion and an end portion in HTML from a stored text and extracting a portion surrounded by both (see, for example, Patent Document 1).
[0005]
The other is a method that is the basis of the method proposed by WebMethods and the method proposed by Iocchi (for example, see Non-Patent Document 1 and Non-Patent Document 2).
[0006]
In order to designate a part of a structured document such as HTML, the document is first converted into a syntax tree as shown in FIG. 8, and the designated part is stored as a position in the syntax tree. Extraction of a designated portion from a document is performed by first converting the document into a syntax tree, specifying a saved position in the syntax tree, and extracting a corresponding portion.
[0007]
[Patent Document 1]
Japanese Patent No. 2867986
[Non-Patent Document 1]
http://www.w3.org/TR/NOTE-widl.html
[0009]
[Non-Patent Document 2]
Luca Iocchi, “The Web-OEM approach to Web information extraction,” Journal of Network and Computer Applications, Vol.22, pp259-269, 1999
[0010]
[Problems to be solved by the invention]
However, in the WWW information extraction system, a part of a structured document such as HTML is specified using the text at the start and end points as a clue, and the document that can change the text itself is shown in FIG. As such, there is a problem that it cannot be applied.
[0011]
In addition, in this method, as shown in FIG. 10, when a designated part is viewed as a syntax tree, it is generally difficult to reuse this part in another structured document because a partial tree is not constructed. is there.
[0012]
Also, in the method based on the position in the syntax tree of the structured document proposed by WebMethods or Iocchi, even if there is no effect on the screen display, there is some change in this syntax tree itself (the sibling node's When there is addition or deletion), there is a problem that an intended designated portion cannot be extracted.
[0013]
The present invention has been made in view of the above points, and does not depend on the start and end points of the specified part, and can reliably extract the specified part even when there is a change in the syntax tree as described above. and to provide an extraction method and apparatus and program of the document.
[0014]
[Means for Solving the Problems]
FIG. 1 is a diagram for explaining the principle of the present invention.
[0015]
The present invention (claim 1) includes an HTML document, the method of extracting the structured document syntax tree to extract part of the modified structured document,
The structured document is read from the structured document storage unit storing the structured document, and the structured document is displayed on the display unit (step 1).
Specifying the part by the sequence the coordinates of the display area of the node corresponding to the portion designated by the user from the displayed structured document, and coordinate information elements of the display area of the set of nodes higher than the node (step 2), the designated the coordinate information stored in the position information storage means (step 3),
The updated structured document is read from the structured document storage means and displayed on the display means (step 4).
Read coordinate information of a display area of a series of nodes accumulated in the position information storage means (step 5),
In the syntax tree of a structured document, for a plurality of paths consisting of a root node and a series of nodes lower than the root node, the coordinates of the display area of each node constituting each path and the display of the read series of nodes The node having the smallest coordinate distance in the region is selected (step 6). The process is repeated by the number of coordinate information stored in the position information storage means (step 7), and finally the selected node is extracted (step 8) Output (step 9).
[0018]
Further, according to the present invention (Claim 2) , in the syntax tree of the structured document, relative coordinates obtained by normalizing the display area of the child node with the width and height of the display area of the parent node in adjacent nodes of the syntax tree. Is the coordinate information of the display area.
[0019]
FIG. 2 is a principle configuration diagram of the present invention.
[0020]
The present invention (claim 3) includes an HTML document, an extraction device structured document syntax tree to extract part of the modified structured document,
A document reading unit 110 that reads a structured document from a structured document storage unit in which the structured document is stored;
Document display means 120 for displaying the read structured document;
By an array having the coordinates of the display area of the node corresponding to the part designated by the user from the structured document displayed on the document display means 120 and the coordinate information of the display area of a series of nodes higher than the node as elements. a part specifying unit 130 for specifying the portion Te,
Position information storage means 140 for storing designated coordinate information in the position information storage means 300;
An updated document reading unit 210 for reading the updated structured document from the structured document storage unit;
An updated document display means 220 for displaying the read structured document on the display means;
A position information reading means 240 for reading coordinate information of a display area of a series of nodes accumulated in the position information storage means 300;
In the syntax tree of the displayed structured document, for a plurality of paths consisting of a root node and a series of nodes lower than the node, the coordinates of the display area of each node constituting each path and the position information reading means 240 The process of selecting the node having the smallest coordinate distance in the display area of the series of nodes read in step S is repeated by the number of coordinate information stored in the position information storage means 300 to finally extract the selected node. Partial extraction means 230;
And a partial information output unit 250 that outputs the node extracted by the partial extraction unit 230 .
[0023]
According to the present invention (Claim 4) , in the partial extraction means 230 ,
In the syntax tree of the structured document, in adjacent nodes of the syntax tree, means for using the relative coordinates obtained by normalizing the display area of the child node by the width and height of the display area of the parent node as the coordinate information of the display area Including.
[0024]
The present invention (Claim 5) is a structured document extraction program that causes a computer to function as each means constituting the structured document extraction apparatus according to any one of Claims 3 to 4.
[0028]
As described above, the present invention specifies a part of a structured document such as HTML, and in the extraction system, in the syntax tree of the document, the coordinate information of the display area of the node corresponding to the part indicated by the user, The part is specified by an array having the coordinate information of the display area of a series of higher-order nodes as an element, and the root node and lower-order series of nodes in the syntax tree of the updated structured document For the plurality of paths consisting of, the lowest node of the path that most closely matches the coordinate information of the display area of each node that constitutes each path and the coordinate information of the display area of the specified series of nodes described above Thus, the extracted portion is specified by using the coordinates of the actual display area as a clue.
[0029]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0030]
FIG. 3 shows a configuration of the structured document partial designation / extraction apparatus according to the embodiment of the present invention. The apparatus shown in FIG. 1 includes a designation unit 100, an extraction unit 200, and a position information storage unit 300. In this embodiment, the partial designation / extraction device is described. However, the partial designation / extraction device may be used.
The designation unit 100 includes a document reading unit 110, a document display unit 120, a partial designation unit 130, and a position information storage unit 140.
[0031]
The operation in the above configuration will be described below.
[0032]
FIG. 4 is a flowchart of the operation for designating a part of a structured document according to an embodiment of the present invention. FIG. 5 shows an example of the part designation according to an embodiment of the present invention.
[0033]
Step 101) The document reading unit 110 reads a structured document stored in storage means such as a database and displays it on the document display unit 120.
[0034]
Step 102) The user designates a part to be selected from the structured document on the browser displayed on the document display unit 120.
[0035]
Step 103) The part designating unit 130 detects the display area of the node corresponding to the part designated by the user. As a detection method, for example, Java (registered trademark) Script ("Java (registered trademark) Script The Definitive Guide, Fourth Edition" David Flanagan O'reilly & Associates, Inc. USA, Dec. 15,2001) is used. And
[0036]
Step 104) Next, the part designating unit 130 calculates the display position of the detected node.
[0037]
Step 105) Furthermore, the part designation | designated part 130 detects the display area (coordinate information) of the parent node of the detected node according to a syntax tree.
[0038]
Steps 104 and 105 are repeated up to the highest node. Note that “Java (registered trademark) Script” provides a method for searching for nodes according to a syntax tree.
[0039]
Step 106) Next, the position information storage unit 140 stores the calculated display positions of all the nodes in the position information storage unit 300.
[0040]
Note that the amount of information stored in the position information storage unit 300 and the overall calculation amount can be reduced by specifying the coordinates of the display area of the node to be stored in advance by the tag name corresponding to the node. it can.
[0041]
The coordinate information stored in the position information storage unit 300 may be relative coordinates normalized by the coordinates of the display area of the parent node. For example, in the syntax tree, the display area of the parent node is the upper left and lower right. Using the coordinates of (left0, top0, right0, bottom0), if the display area of the child node is similarly (left1, top1, right1, bottom1), the relative coordinates are
(left1 / (right0-left0), top1 / (bottom0-top0), right1 / (right0-left0), bottom1 /
(bottom0-top0))
And so on.
[0042]
Next, the extraction unit 200 of the structured document partial designation / extraction apparatus will be described.
[0043]
The extraction unit 200 includes a document reading unit 210, a document display unit 220, a partial extraction unit 230, a position information reading unit 240, and a partial information output unit 250.
[0044]
The operation in the above configuration will be described below.
[0045]
FIG. 6 is a flowchart of the extraction operation of the designated portion in one embodiment of the present invention, and FIG. 7 shows an example of extraction in one embodiment of the present invention.
[0046]
Step 201) The updated structured document to be processed is read from the storage means such as a database by the document reading unit 210 and displayed on the browser screen of the document display unit 220.
[0047]
Step 202) The partial extraction unit 230 detects the display area of the highest node displayed. Specifically, the “html node” shown in FIG. 7 is selected.
[0048]
Step 203) The partial extraction unit 230 detects the display area of the child node of the node according to the syntax tree.
[0049]
Step 204) The position information reading unit 240 reads the position information from the position information storage unit 300 and passes it to the partial extraction unit 230. Thereby, the partial extraction unit 230 selects a child node that best matches the display position of the stored position information.
[0050]
Note that there are usually a plurality of the above child nodes and the most recently assigned nodes with the specified tag name. However, the coordinate information stored in the position information storage unit 300 from these nodes by the following procedure is used. Select the best matching node.
[0051]
Here, the stored coordinate information is (left2, top2, right2, bottom2), and the coordinate information of the display area of the i-th node among the plurality of nodes is (left (i), top (i), right (i ), bottom (i)). The node to be selected is i that minimizes the distance | left2-left (i) | + | top2-top (i) | + | right2-right (i) | + | bottom2-bottom (i) | The node corresponding to. As the coordinate information used here, the relative coordinates defined above can be used. Furthermore, the coordinates of the center of gravity of the rectangle corresponding to the area, and the difference in width and height | (left2 + right2) / 2− (left (i) + right (i)) / 2 |
+ | (Top2 + bottom2) / 2− (top (i) + bottom (i)) / 2 |
＋ ｜ （right2-left2） − (right (i) -left (i)) ｜
+ | (Bottom2-top2)-(bottom (i)-top (i)) |
If the distance between the two is compared, it is possible to surely extract the designated part even if the position is shifted due to the update of the document.
[0052]
Step 205) The same operation is performed on the child node of the node selected as described above (or the node having the specified tag name that is closest). The above operation is repeated as many times as the number of coordinate information stored in the position information storage unit 300. The finally selected node is extracted and output from the partial information output unit 250.
[0053]
Further, the processing by the configuration of the above-described structured document partial designation unit 100 and extraction unit 200 is constructed as a program, installed in a computer used as the partial designation device and extraction device, and executed by a control means such as a CPU. Is also possible. It is also possible to distribute the constructed program via a network.
[0054]
The constructed program is stored in a hard disk device connected to a computer used as a partial designation device or extraction device, a portable storage medium such as a flexible disk or CD-ROM, and installed in the computer at the time of execution. It is also possible to execute.
[0055]
The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.
[0056]
【The invention's effect】
As described above, according to the present invention, the extracted portion of the structured document is specified by using the coordinate information of the actual display area as a clue. Therefore, the syntax tree does not depend on the start and end portions of the designated portion. Even if there is a change, the specified part can be extracted reliably.
[0057]
[Brief description of the drawings]
FIG. 1 is a diagram for explaining the principle of the present invention.
FIG. 2 is a principle configuration diagram of the present invention.
FIG. 3 is a configuration diagram of a structured document partial designation / extraction apparatus according to an embodiment of the present invention;
FIG. 4 is a flowchart of a partial designation operation according to an embodiment of the present invention.
FIG. 5 is an example of partial designation in one embodiment of the present invention.
FIG. 6 is a flowchart of an operation for extracting a designated portion according to the embodiment of the present invention.
FIG. 7 is an example of extraction in one embodiment of the present invention.
FIG. 8 is an example of a syntax tree corresponding to an HTML source.
FIG. 9 is an example in which the text at the start / end points is partially specified and extracted using a clue.
FIG. 10 is an example in which a corresponding part of HTML is extracted using a text instructing start and end as a clue.
[Explanation of symbols]
100 Finger tough 110 document reading unit, document reading unit 120 document display means, the document display unit 130 portion specifying unit, portion designation unit 140 position information holding means, the position information holding section 200 extractor, the extraction unit 210 updates the document reading unit, Document reading unit 220 Update document display unit, document display unit 230 Partial extraction unit, partial extraction unit 240 Position information reading unit, position information reading unit 250 Partial information output unit, partial information output unit 300 Position information storage unit, position information storage unit

Claims (5)

Including HTML documents, in the extraction method of the structured document syntax tree to extract part of the modified structured document,
Reading the structured document from the structured document storage means storing the structured document, and displaying the structured document on the display means;
The coordinates of the display area of the node corresponding to the portion designated by the user from the displayed the structured document, the portion by the sequence of coordinate information of the display area of the set of nodes of the upper and elements from the node specified, accumulates the designated the coordinate information in the position information storage means,
Read the updated structured document from the structured document storage means, display it on the display means,
Read coordinate information of a display area of a series of nodes accumulated in the position information storage means,
In the syntax tree of the structured document, for a plurality of paths including a root node and a series of nodes lower than the node, the coordinates of the display area of each node constituting each path and the read series of nodes The process of selecting the node with the smallest coordinate distance in the display area is repeated by the number of coordinate information stored in the position information storage means, and finally selected nodes are extracted and output. To extract structured documents. In the syntax tree of the structured document, relative coordinates obtained by normalizing the display area of the child node with the width and height of the display area of the parent node in the adjacent nodes of the syntax tree are used as the coordinate information of the display area. Item 3. A method for extracting a structured document according to Item 1 . Including HTML documents, an extraction device structured document syntax tree to extract part of the modified structured document,
A document reading means for reading the structured document from the structured document storage means in which the structured document is stored;
Display means for displaying the read structured document;
The coordinates of the display area of the node corresponding to the portion designated by the user from the displayed the structured document, the portion by the sequence of coordinate information of the display area of the set of nodes of the upper and elements from the node A part designation means to designate;
Position information storage means for storing designated coordinate information in the position information storage means;
An updated document reading means for reading an updated structured document from the structured document storage means;
Updated document display means for displaying the read structured document on a display means;
Position information reading means for reading coordinate information of a display area of a series of nodes accumulated in the position information storage means;
In the syntax tree of the displayed structured document, for the plurality of paths including a root node and a series of nodes lower than the node, the coordinates of the display area of each node constituting each path and the position information reading The process of selecting the node with the smallest coordinate distance in the display area of the series of nodes read by the means is repeated as many times as the number of coordinate information stored in the position information storage means, and finally the selected node is Partial extracting means for extracting;
Partial information output means for outputting the node extracted by the partial extraction means;
An apparatus for extracting a structured document, comprising: The partial extraction means includes
In the structured document syntax tree, in adjacent nodes of the syntax tree, relative coordinates obtained by normalizing the display area of the child node with the width and height of the display area of the parent node are used as coordinate information of the display area. The structured document extracting apparatus according to claim 3 , comprising: As each means which comprises the extraction apparatus of the structured document of any one of Claim 3 thru | or 4 A structured document extraction program that allows a computer to function.

Images