JPH10260979A - Information collecting method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently collect information by comparing an explanation text that corresponds to a link destination described on a hyperlink document with a narrowing condition and holding the link destination that coincides with the narrowing condition as an access prearranged node. SOLUTION: A start condition of search and an end condition of search are set (step S1), a narrowing condition of a link destination is set (step S2), a hyperlink document is acquired by accessing an access destination that is designated with the start condition and an access prearranged node, and the acquired hyperlink document is stored as a search result (step S3). An explanation text that corresponds to a link destination which is described on the hyperlink document is compared with a narrowing condition, and a link destination that coincides with the narrowing condition is held as an access prearranged node (step S4). The above processing is performed of entire hyperlink document to be accessed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information collecting method and apparatus, and more particularly to an information collecting method and apparatus for automatically collecting documents following a link destination of a hyperlink document. More specifically, the present invention relates to an information collection method and apparatus for supporting information collection when hyperlink documents are distributed and stored on servers distributed on a network.

[0002]

2. Description of the Related Art A conventional information collecting apparatus will be described.
FIG. 8 shows a configuration of a conventional information collecting apparatus. The conventional information collecting device includes a search condition setting unit C1, a search unit C2, and a search result display unit C3. The search unit C2 includes a hyperlink document access unit C21, a scheduled access node extraction unit C22, and an end condition determination unit C23.

FIG. 9 shows the operation of a conventional information collecting apparatus. Step 10) First, the initial access node and the maximum node depth are set as search conditions by the search condition setting unit C1. Step 11) As an initial condition, an initial access node is set as a current node.

Step 12) The current node is accessed by the hyperlink document access section C21, a hyperlink document is obtained, and stored as a search result.
Step 13) The termination condition determination unit C23 checks whether the current node is at the maximum depth. If the current node is at the maximum depth, the process proceeds to the next step 15. If it is not at the maximum depth, the process proceeds to step S14.

[0005] Step 14) The scheduled access node extracting unit C22 stores the link destination described in the hyperlink document as the scheduled access node. Step 1
5) The end condition determining unit C23 checks whether there is an unprocessed access scheduled node, and if so, sets it as the current node. If there is an unprocessed access scheduled node, the process proceeds to step S12.

Step 16) The search result is displayed on the search result display section C3. Through the above processing, all hyperlinked documents from the initial access node to the maximum node depth can be obtained.

[0007]

However, in the above-mentioned conventional apparatus, since all linked hyperlink documents are accessed, if the designated maximum node depth is deep, a large amount of documents will be obtained. . If there are n link destinations per document and the maximum node depth is designated as m, then n is the mth power, and when the search depth is increased,
You will have access to a huge amount of documents. For example, n
= 5, m = 5, 3125 documents are accessed in the fifth hierarchy.

However, the information desired by the user is often a very small part of these documents. For this reason,
A large amount of labor is required for narrowing down the collected documents by performing a search process and manually searching for necessary documents. In addition, there is a problem that if a large number of documents distributed and arranged on the network are accessed, a heavy load is imposed on the network.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and provides a document which is highly likely to be desired by a user by accessing only link destinations satisfying a narrowing condition, instead of accessing all link destinations. It is an object of the present invention to provide an information collecting method and apparatus capable of efficiently collecting information by limiting access to the information.

[0010]

FIG. 1 is a diagram for explaining the principle of the present invention. The present invention sets an initial condition and an end condition of a search in an information collection method for supporting information collection when hyperlink documents are distributed and stored on servers distributed on a network (step 1). Setting the conditions for narrowing down the link destination (step 2), accessing the access destination and the scheduled access node specified in the initial condition, acquiring the hyperlink document,
The acquired hyperlink document is saved as a search result (step 3), the explanatory text corresponding to the link destination described in the hyperlink document is compared with the narrowing conditions, and the link destination that matches the narrowing condition is accessed. The process of storing as a scheduled node (step 4) is performed for all hyperlink documents to be accessed.

Further, according to the present invention, a keyword specified as a condition for narrowing down a link destination is developed based on a thesaurus dictionary representing a relation between word concepts, set as a collation target word, and described in a hyperlink document. The link destination in which the collation target word is included in the description text corresponding to the link destination is set as the scheduled access node.

FIG. 2 is a diagram showing the principle of the present invention. The present invention relates to an information collecting apparatus for automatically collecting documents following a link destination of a hyperlink document, wherein a search condition setting means 1 for setting an initial condition and an end condition of a search, and a condition for narrowing down a link destination are set. Narrowing condition setting means 2 to perform
An access candidate narrowing means 3 which determines a description text corresponding to a link destination described in the hyperlink document based on a narrowing condition and sets a link destination meeting the narrowing condition as a scheduled access node; and an access destination specified by the initial condition. And a hyperlink document access unit 4 for accessing the access scheduled node to acquire the hyperlink document and storing the hyperlink document as a search result, and a hyperlink document access unit 4 based on the initial condition and the end condition. A search management means 5 for repeatedly activating the candidate narrowing means 3.

Further, in the present invention, the access candidate narrowing means 3 further includes a thesaurus dictionary representing the relationship between the concepts of the words. And a narrowing-down means for setting a link destination in which the description target word is included in the explanation text corresponding to the link destination in which the hyperlink document is described as an access scheduled node.

As described above, in the information collecting method and apparatus according to the present invention, the explanation text corresponding to the link destination described in the accessed hyperlink document is compared with the narrowing condition set by the narrowing condition setting means. Thus, by narrowing down the matching link destinations as scheduled access nodes, it is possible to avoid unnecessary access to documents and efficiently collect documents that may contain information desired by the user. .

[0015]

FIG. 3 shows the configuration of an information collecting apparatus according to the present invention. The information collecting apparatus in FIG. 1 includes a search condition setting unit 1, a narrowing condition setting unit 2, an access candidate narrowing unit 3, a hyperlink document access unit 4, a search management unit 5, an access management table 6, an acquired document storage unit 7, a process, and the like. Request receiving unit 8, search result display unit 9, target word storage unit 1
0, and a thesaurus dictionary 11, and the information collecting apparatus is connected to a server group 13 via a network 12.

The access management table 6 is a table for storing an identifier of a document to be accessed (hereinafter, referred to as an access scheduled node) and the like, and used for managing access to the document. In the access management table 6, items such as a scheduled access node, a processing result, and a depth of the node are defined. The identifier of the document determined to be accessed is described in the scheduled access node, and “processed” or “unprocessed” is described in the processing result depending on whether or not the document is acquired. Describes the number of links from the start document to the start document, with 0 as the start document.

The acquired document storage unit 7 stores the document acquired by the hyperlink document access unit 4.
The processing request receiving unit 8 is an input / output interface with the user, has an information input / output function, is connected to a terminal via a network, and receives a search condition setting request from the user.
The search condition setting unit 1, the search condition setting unit 2, the search management unit 5, and the search result display unit 9 are respectively accepted for a search condition setting request, a search request, and a search result output request.

The search result display section 9 outputs a document stored in the acquired document storage section 7 in response to a request from the processing request receiving section 8. The matching target word accumulating unit 10 allows the access condition narrowing unit 3 to refer to the narrowing condition setting unit 2
The collation target word set by is saved. The thesaurus dictionary 11 is a dictionary that stores information indicating relationships between concepts represented by words, and is used as knowledge for narrowing down.

The search condition setting unit 1 receives a request for a search condition via the processing request receiving unit 8, sets a start document as an initial condition of the search condition, and sets a maximum node depth as an end condition. The narrow-down condition setting unit 2 receives a narrow-down condition setting request via the processing request receiving unit 8 and sets a narrow-down condition. As a method of specifying a narrowing condition, various specifying methods such as a method of specifying a keyword, a method of specifying a conditional expression, and a method of specifying a collation pattern can be applied.

Upon receiving the search request via the processing request receiving unit 8, the search management unit 5 refers to and updates the access management table 6 and, until the end condition specified by the search condition is reached, the access candidate narrowing unit. 3 and the hyperlink document access unit 4 are repeatedly activated to perform a search.
The access candidate narrowing-down unit 3 extracts a description text corresponding to the link destination from the hyperlink document, and determines a link destination in which the verification target word stored in the verification target word storage unit 10 is included in the description text, as an access scheduled node. And registers it in the access management table 6 via the search management unit 5.

The hyperlink document access unit 4 receives information on a node to be accessed from the search management unit 5,
The server group 13 is accessed via the network 12 to acquire a corresponding hyperlink document and store it in the acquired document storage unit 7. FIG. 4 is a flowchart showing a series of operations of the information collection method of the present invention. Step 101) The search condition setting unit 1 sets a start document as an initial condition of the search condition, and sets a maximum node depth as an end condition.

Step 102) The access candidate narrowing unit 3 registers the start document in the access management table 6 as a scheduled access node. Step 103) The narrowing-down condition setting unit 2 receives the narrowing-down condition from the processing request receiving unit 8, and the keyword is set as the narrowing-down condition. Step 104) The narrowing-down condition setting unit 2 sets the thesaurus dictionary 1 of the keyword specified by the narrowing-down condition.
1 is obtained, and the refinement condition setting unit 5 extracts the synonyms, upper terms, and lower terms, and sets them as matching target words.
It is registered in the collation target word storage unit 10.

Step 105) The hyperlink document access unit 4 accesses the scheduled access node registered in the access management table 6 via the search management unit 5, acquires the hyperlink document, and acquires the acquired document storage unit 7.
As a search result, and the <processing result> of the node to be accessed in the access management table 6 is set to “processed” via the search management unit 5.

Step 106) The search management unit 5 checks whether or not the depth of the node of the acquired document has reached the maximum node depth. If not, the process proceeds to step 107. Step 107) The search management unit 5 extracts the description text corresponding to the link destination from the hyperlink document.

Step 108) The access candidate narrowing unit 3 checks whether or not the matching target word exists in the explanation text. If there is, the access management table 6 sets the link corresponding to the explanation text as an access scheduled node. Register to Step 109) The search management unit 5 checks whether there is an “unprocessed” access scheduled node in the access management table, and if so, proceeds to step 105.

Step 110) If no search result output request is received in the search result display section 9,
The search result stored in the obtained document storage unit 7 is output.

[0027]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 5 shows an example of a hyperlink document according to an embodiment of the present invention. [H00] of the hyperlink document shown in FIG.
[01], [h0002], etc. are identifiers given to the hyperlink document. Document shown here (hereafter, hyperlink document is omitted here and simply written as document)
Is simplified for explanation, and may be a document including a table, an image, and the like. In the case of a hyperlink document, a tag for designating a display format or the like and a tag for indicating a link destination are defined, and an explanatory text corresponding to the link destination can be extracted based on such information. Further, a configuration may be adopted in which natural language analysis processing is performed to extract explanation text that cannot be extracted only with tag information.

In the document [h0001], the document [h001]
For the link to [1], “computer” corresponds to the description text, and for the link to the document [h0012], “mail order” corresponds to the description text. Similarly,
"Company", "Art", and "Media" are each [h00
13], [h0014], and [h0015] are explanatory texts of links. When the link destination and the corresponding text are represented in the form of (<link destination>, <description text>), the document [h0001] contains (h0011, computer) (h0012, mail order) as link information. , Company) (h0014, art) (h0015, media).

FIG. 6 shows an example of a thesaurus dictionary according to an embodiment of the present invention. The thesaurus dictionary 11 shown in FIG. 1 is a dictionary that stores information indicating relationships between concepts represented by words, and is used as knowledge for narrowing down. In the example shown in the figure, a double-headed arrow indicates a synonym, and a one-way arrow indicates an upper / lower relationship. For example, “sales”, “shopping”, and “shopping” are synonyms, and the lower terms of these words are “mail order” and “retail”. From a “retail” perspective, the broader term is “sales,”
They are "department stores" and "specialty stores".

Here, as an example of specifying a keyword as a narrowing-down condition, it is expanded into a collation target word with reference to the thesaurus dictionary 11, and a link destination is determined by whether or not the collation target word is included in the description text. The following describes an example of a case in which the search is narrowed down. In the present embodiment, in order to make the reference in the access candidate narrowing unit 3 easy, the narrowing condition setting unit 2 sets the synonym of the designated keyword,
Higher-order words and lower-order words are set as matching target words, and the matching storage unit 10
To hold.

FIG. 7 shows an example of an access management table according to an embodiment of the present invention. 11A shows the state of the table before document access, FIG. 10B shows the state of the table after [h0001] document acquisition, and FIG. 10C shows the state of [h001].
2] Shows the state of the table after document acquisition. Hereinafter, the case where the document [h0001] is set as the start document and "2" is set as the maximum node depth will be described with reference to the flowchart of FIG.

Step 101) A request for setting a search condition is received from the processing request receiving unit 8, a start document is set as an initial condition of the search condition, and a maximum node depth is set as an end condition. That is, the document [h0001] is set as the start document, and “2” is set as the maximum node depth. Step 102) The start document specified as the initial condition of the search is registered in the access management table 6 as a scheduled access node. The specified start document may be plural. Here, a case where the document [h0001] in FIG. 5 is designated as the start document will be described. For the start document [h0001], as shown in FIG.
“H0001” is described in the item of <access scheduled node>, “unprocessed” is described in the item of <processing result>, and “0” is described in the item of <node depth>.

Step 103) Refinement condition setting unit 2
, A narrowing condition setting request from the processing request receiving unit 8 is received, and a keyword is designated as the narrowing condition. Here, it is assumed that “shopping” and “furniture” are designated as keywords. Step 104) The narrowing-down condition setting unit 2 refers to the thesaurus dictionary 11, extracts synonyms, high-order words, and low-order words of the keyword specified by the narrowing-down condition, sets them as matching target words, and uses these as matching target word storage units 10. Save to When the thesaurus dictionary of FIG. 6 is used, the collation target words are determined as follows.

The terms to be collated with “shopping” include synonyms “sale”, “shopping”,
“Shopping” is extracted, and “mail order” and “retail” are extracted as lower words, and “department store” and “specialty store” are extracted as lower words of “retail”. For "furniture",
As a broader term, “product” is used as a broader term, and “desk”,
"Chairs" are extracted.

That is, the words to be collated are “sale”, “shopping”, “shopping”, “mail order”, “retail”, “department store”, “specialty store”, “product”, “desk”, “chair”. And these words are stored in the matching target word storage unit 10. Step 105) In the node to be accessed registered in the access management table 6, access is made to the node whose <processing result> is "unprocessed" via the network 12,
Get a hyperlink document. Then, the acquired hyperlink document is stored in the acquired document storage unit 7 as a search result, and the <processing result> of the access scheduled node is stored.
As “processed”. Here, for simplicity of explanation,
Although the link destination is represented only by the identifier of the hyperlink document, it also includes information indicating the location of the hyperlink document as the link destination, that is, information on which server exists, and which server should be accessed. Can be determined.

Step 106) It is checked whether the node depth of the acquired document has reached the maximum node depth. By checking <node depth> in the access management table 6,
You can check if the maximum node depth has been reached. If the maximum node depth has been reached, step 10
7. Skip step 108 and proceed to step 109.

Step 107) Extract an explanatory text corresponding to the link destination from the obtained hyperlink document. In the case of document [h0001], (h0011,
(Computer), (h0012, mail order), (h001
3, companies), (h0014, art), (h0015,
Media) is extracted.

Step 108) It is checked whether or not the collation target word exists in the explanation text. If there is, the link destination corresponding to the explanation text is registered in the access management table 6 as a scheduled access node. Document [h00
01], only “mail order” exists as (h0012, mail order). Therefore, the document [h0012] is registered in the access management table 6. At this time, "unprocessed" is described in <processing result>, and <node depth> describes a value obtained by adding 1 to the depth of the parent node. Document [h0
001] is 0, so the document [h0012] <node depth> is 1. As a result, the contents of the access management table 6 are as shown in FIG.

Step 109) Check if there is an “unprocessed” access scheduled node in the access management table 6, and if there is, go to step 105 and execute “unprocessed”.
If there is no scheduled access node, step 110
Move to. Step 110) Upon receiving the search result output request from the processing request receiving unit 8, the search result display unit 9
Outputs the search result stored in the acquired document storage unit 7 to the processing request receiving unit 8.

According to the processing described above, in the case of the hyperlink document shown in FIG. 5, the processing from step 105 to step 110 is as follows. (1) The document [h designated as the starting document of the initial condition
0001] is obtained (step 105). (2) Since <node depth> of the document [h0001] is 0 and has not reached the maximum node depth 2, the process proceeds to the next step 107 (step 106).

(3) From the document [h0001], the explanation text (h0011, computer), (h0012,
(Mail order), (h0013, company), (h0014, art), (h0015, media) are extracted (step 107).

(4) If the word “mail order” to be checked is (h00
12, [mail order], so that [h0012] is registered in the access management table 6 as a scheduled access node. (5) Since [h0012] has not been processed, step 10
The process proceeds to step 5 (step 109). (6) Document [h0012] is acquired (step 1)
05).

(7) Since <node depth> of the document [h0012] is 1 and has not reached the maximum node depth of 2, the process proceeds to the next step 107 (step 10).
6). (8) From the document [h0012], the description text (h0
121, a department store), (h0122, shopping mall), (h0123, food), (h0124, computer), (h0125, book) are extracted (step 107).

(9) The matching target word "department store" is (h01
21, department store), and the matching target word "shopping"
Is included in the character string “shopping mall” of (h0122, shopping mall), so [h012
1] and [h0122] are registered in the access management table 6 as scheduled access nodes. Here, [h01
21] and [h0122] have not been processed, so step 1
05 (step 109).

(10) [h0121] and [h012]
Since 2] has not been processed, the process proceeds to step 105 (step 109). (11) The document [h0121] is obtained (step 105). (12) Since <node depth> of the document [h0121] is “2” and has reached the maximum node depth “2”, steps 107 and 108 are skipped.
The process proceeds to step 109 (step 106).

(13) Since [h0122] remains as "unprocessed", the flow shifts to step 105 (step 109). (14) The document [h0122] is obtained (step 105). (15) <node depth> of document [h0122]
Since it is “2” and has reached the maximum node depth “2”, steps 107 and 108 are skipped,
The process proceeds to step 109 (step 106).

(16) Since there is no access scheduled node remaining as unprocessed, the process proceeds to step 110 (step 109). (17) When a search result output request is received, the documents [h0001] and [h001] stored in the obtained document storage unit 7
2], [h0121], and [h0122] are output.

In this way, access is made only to the narrowed link destination. As in the example above, for example,
If "shopping" and "furniture" are designated by the motive of "furnishing furniture", access to documents other than those that may include information desired by the user is suppressed. Therefore, it is possible to efficiently collect desired information in a reasonable time even if the maximum node depth is increased.

The present invention is not limited to the above embodiment, but can be variously modified and applied within the scope of the claims.

[0050]

As described above, according to the information collection method and apparatus of the present invention, since the link destination is narrowed down by the narrowing-down condition, the amount of documents to be accessed can be greatly reduced, and the possibility that desired information is included. High documents can be easily obtained. Therefore, by using the apparatus according to the present invention, information can be collected efficiently.

A quantitative comparison with the conventional apparatus is as follows. In the conventional device, all the link destinations are accessed to obtain a document. Therefore, if one document has n link destinations and access is performed up to the m-th hierarchy, the number of documents in the m-th hierarchy is n to the m-th power, and when the search depth is increased, an enormous amount of You will have access to the document.

On the other hand, in the information collecting apparatus of the present invention,
In order to narrow down the link destination according to the link narrowing condition,
The number of documents to be accessed can be reduced. Specifically, if one document is narrowed down to 1 / k, the number of documents accessed in the m-th hierarchy is reduced to 1 / (k to the m-th power).
For example, if the screen is narrowed down to 1/3, the number is reduced to 1/243 at the fifth hierarchical level as compared with the case where no narrowing is performed.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a configuration diagram of an information collection device of the present invention.

FIG. 4 is a flowchart of the operation of the information collection device of the present invention.

FIG. 5 is an example of a hyperlink document according to an embodiment of the present invention.

FIG. 6 is an example of a thesaurus dictionary according to an embodiment of the present invention.

FIG. 7 is an example of an access management table of the present invention.

FIG. 8 is a configuration diagram of a conventional information collection device.

FIG. 9 is a flowchart of a process performed by a conventional device.

[Description of Signs] 1 search condition setting means, search condition setting unit 2 narrowing condition setting means, narrowing condition setting unit 3 access candidate narrowing means, access candidate narrowing unit 4 hyperlink document accessing means, hyperlink document accessing unit 5 search management Means, search management unit 6 access management table 7 acquired document storage unit 8 processing request reception unit 9 search result display unit 10 collation target word storage unit 11 thesaurus dictionary 12 network 13 server group

Claims (4)

[Claims] 1. An information collection method for supporting information collection in a case where hyperlink documents are distributed and stored on servers distributed on a network, comprising: setting an initial condition and an end condition of a search; Setting the narrowing-down condition, accessing the access destination and the scheduled access node specified in the initial condition, acquiring the hyperlink document, holding the hyperlink document as a search result, and Comparing the explanatory text corresponding to the described link destination with the narrowing-down condition, and holding a link destination that matches the narrowing-down condition as a scheduled access node for all hyperlink documents to be accessed. Information collection method to be used. 2. A keyword specified as a condition for narrowing down a link destination is developed based on a thesaurus dictionary representing a relationship between word concepts, and set as a matching target word, and a link destination described in the hyperlink document is set. 2. The information collection method according to claim 1, wherein a link destination including the collation target word in the description text corresponding to the target text is set as an access scheduled node. 3. An information collecting apparatus for automatically collecting documents following a link destination of a hyperlinked document, comprising: a search condition setting means for setting an initial condition and an end condition of a search; and setting a condition for narrowing down a link destination. Access candidate narrowing means for determining a description text corresponding to a link destination described in a hyperlink document based on the narrowing condition, and setting a link destination that meets the condition as an access scheduled node; A hyperlink document access unit for accessing the access destination designated by the condition and the access scheduled node to obtain a hyperlink document, and storing the hyperlink document as a search result, based on the initial condition and the end condition Repeating the hyperlink document access means and the access candidate narrowing means. Information collection apparatus characterized by having a search management means for start. 4. A method according to claim 1, further comprising a thesaurus dictionary representing a relationship between word concepts, wherein said access candidate narrowing unit expands a keyword specified as said narrowing condition based on said thesaurus dictionary and sets it as a matching target word. 4. The information collecting apparatus according to claim 3, further comprising: a narrowing-down condition setting unit that performs the search, and a narrowing-down unit that sets a link destination including the matching target word in an explanatory text corresponding to the link destination described in the hyperlink document as an access scheduled node. apparatus.