JP2003150636A - Document retrieval device, document retrieval method, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a document retrieval device capable of obtaining an appropriate retrieved result for a retrieval request while hardly generating problems of retrieval omissions and excessive retrieval compared to the conventional case. SOLUTION: The document retrieval device obtains an appearing set of the same character string as a notation of a word extracted from a retrieval condition for a plurality of documents composed of the character strings expressed in a natural language, simultaneously obtains the appearing set in which the same word as the word extracted from the retrieval condition appears, and obtains an appearing degree for each word on the basis of the appearing sets. By obtaining a sufficiency degree on the basis of the appearing degree obtained for all the words extracted from the retrieval condition and selecting and outputting the document of the retrieved result corresponding to the sufficiency degree, the appropriate retrieved result for the retrieval condition is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document retrieval device for retrieving a desired document from digitized and accumulated document information,
Regarding the document search method, program, and recording medium, in detail, for each word in the search condition and each document in the document group,
Executes the functions of a document search device, a document search method, and a document search device that select a document based on the degree of sufficiency calculated from the degree of occurrence of a character string of a word and the degree of appearance of a word in a document And a computer-readable recording medium recording the program.

[0002]

2. Description of the Related Art As a technique for inputting a document and searching for a similar document, there are the following known techniques. Japanese Patent Laid-Open No. 11-73
The "similar document retrieval system and recording medium used therefor" of Japanese Patent No. 422 publication extracts a word group and its frequency information from all document texts, updates them, and stores them in an index. It is possible to obtain and obtain useful search results for users considering the inclusion relation between documents of important concepts by referring and processing the limited information, and the characteristics of the input document in calculating the similarity between documents. We provide an efficient similar document search system that can omit redundant calculation considering search results.

Further, the "document retrieval device" of Japanese Patent Laid-Open No. 11-73429 discloses that for any retrieval condition character string,
Even if the user inputs a search condition that includes a character string (unknown word) that is not registered in the dictionary, it guarantees a fast full-text search with no omissions in the search. In addition, by using the accurate frequency information of the character string, it is possible to calculate the similarity between the document and the search condition and perform the document ranking without lowering the accuracy. It can be used for accurate search.

[0004]

For a search condition or a document expressed in natural language text, the degree to which each document in the document group satisfies the search condition (hereinafter referred to as "satisfaction level") is determined. A document search system has been proposed in which a search result is a document group ranked in descending order of the degree of satisfaction or a high document group. Examples of document retrieval methods using such a degree of sufficiency can be found in the above-mentioned JP-A Nos. 11-73422 and 11-73429. In the following, using these examples, the satisfaction degree R (Q, Q) from the request sentence or document representing the search condition (hereinafter referred to as search condition Q) and the document to be searched (hereinafter referred to as document D).
A conventional method for calculating D) will be described.

Method 1: A word group W is extracted from the search condition Q, and for each word w that is an element of this word group W, the total number of times when the same character string as the description of that word appears in the document D A certain appearance frequency SF (w, D) is obtained, and the degree of sufficiency R increases as the sum ΣSF (w, D) of them is larger.
It is assumed that (Q, D) is large.

Method 2: A word group W is extracted from the search condition Q, and each word w that is an element of the word group W appears as a word in the document D. The appearance frequency WF (w, w,
D) is obtained, and it is assumed that the satisfaction degree R (Q, D) is larger as the sum ΣWF (w, D) thereof is larger.

These methods will be described using the following examples. It is assumed that the two documents (D1, D2) have D1: knowledge and deduction D2: Tokyo, respectively, and the two search conditions (Q1, Q2) are respectively Q1: machine knowledge Q2: Kyoto.

A word group is extracted from each of these documents (D1 and D2) and search conditions (Q1 and Q2) as follows (each constituent word is separated by "/" as an example). D1: Knowledge / and / deduction D2: Tokyo / Miyako Q1: Machine / No / Knowledge Q2: Kyoto

With respect to each word w in the word group extracted from each of the above search conditions (Q1, Q2), the appearance frequency SF (w,
D) and WF (w, D) are calculated as follows.

For search condition Q1: SF (machine, D1) = 0, SF (of, D1) = 0, SF
(Intelligence, D1) = 1 WF (machine, D1) = 0, WF (of, D1) = 0, WF
(Knowledge, D1) = 0 SF (machine, D2) = 0, SF (of, D2) = 0, SF
(Intelligence, D2) = 0 WF (machine, D2) = 0, WF (of, D2) = 0, WF
(Intelligence, D2) = 0

For the search condition Q2: SF (Kyoto, D1) = 0 WF (Kyoto, D1) = 0 SF (Kyoto, D2) = 1 WF (Kyoto, D2) = 0

Here, the satisfaction degree R (Q, D) is defined as a sum ΣSF of appearance frequencies or a sum ΣWF of appearance frequencies, and a document whose satisfaction degree R (Q, D) is larger than 0 is set as a search result. In this case, in the above example, the search result for the search condition Q1 is: Method 1: Document D1 Method 2: None, and the search result for the search condition Q2 is: Method 1: Document D2 Method 2: None. Become.

That is, in the calculation method 1 of the degree of sufficiency based on the frequency of appearance of the character string of the word notation in the document, the document D1 (knowledge and deduction) can be searched from the search condition Q1 (machine knowledge), There is no problem of omission of search. However, the search condition Q2
Document D2 (Tokyo) was retrieved from (Kyoto),
The problem of excessive search comes out.

On the other hand, in the sufficiency calculation method 2 based on the frequency of appearance of words as words in the document, the search condition Q1
Since the document D1 (knowledge and deduction) is not retrieved from (machine knowledge), there is a problem of omission of retrieval. However, the search condition Q2
Document D2 (Tokyo) is not searched from (Kyoto), and the problem of excessive search does not occur.

Therefore, according to the conventional method 1 or 2, the problem of missing search or excessive search is likely to occur, and it is often impossible to obtain a suitable search result for the search condition.

The present invention has been made in view of the above-mentioned circumstances, and it is less likely to cause problems such as omission of search and excessive search as compared with the prior art, and it is possible to obtain an appropriate search result for a search condition. An object of the present invention is to provide a document retrieval device, a document retrieval method, a program for executing the function of the document retrieval device, and a computer-readable recording medium recording the program.

[0017]

In order to solve the above-mentioned problems, the document search device according to claim 1 of the present invention is expressed in natural language and an input unit for inputting a request such as a search condition. A document group storage unit that stores a plurality of documents each including a character string, a word extraction unit that extracts a word from the search condition and the document, and the same notation as the word extracted from the search condition for the document A character string search unit for obtaining the appearance set of the character string of, and for the words extracted from the document,
A word search unit for obtaining an appearance set in which the same word as the word extracted from the search condition appears, and for each word extracted from the search condition, obtained from each of the character string search unit and the word search unit. An appearance degree estimation unit that obtains the appearance degree based on the appearance set, a satisfaction degree calculation unit that obtains the satisfaction degree based on the appearance degree obtained for all the words extracted from the search condition, and the document group storage An output unit that selects and outputs a document as a search result according to the degree of sufficiency obtained for all the documents stored in the unit.
According to claim 2 of the present invention, in the document search device according to claim 1, for each element of the appearance set of the character string search unit, which is obtained for each word extracted from the search condition, the element Is present in the appearance set of the word search unit, and otherwise is set as a default value, and the sum of all elements is set as the appearance degree. Further, a third aspect of the present invention is characterized in that, in the document search apparatus according to the first or second aspect, a sum of the appearance degrees obtained for each word extracted from the search conditions is set as the satisfaction degree. And Further, according to the document search method of claim 4 of the present invention, for each word existing in the search condition,
Find the appearance set that appears as a character string and the appearance set that appears as a word in each document in the document group, find the degree of occurrence for a word from these occurrence sets, and search based on the degree of satisfaction of the document obtained from this occurrence degree It is characterized by selecting a document as a result. The program of claim 5 of the present invention is
It is a program for causing a computer to function as the document search device according to claim 1. A recording medium according to claim 6 of the present invention is a computer-readable recording medium recording the document search program according to claim 5.

Therefore, according to the above configuration, for each word in the search condition and each document in the document group, the character string or the frequency of occurrence of the word itself in the document is not used, but the character of the word in the document is used. From the appearance set appearing as a column and the appearance set appearing as a word, the degree of occurrence of the word in the document is obtained, and the document of the search result is selected based on the degree of sufficiency obtained from the degree of occurrence. Problems such as omission of search and excessive search are unlikely to occur, and it is possible to obtain appropriate search results for the search conditions.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION A document retrieval apparatus according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a functional block diagram of a document search device according to an embodiment of the present invention. In FIG. 1, the document search apparatus according to the present exemplary embodiment includes a control unit 10, an input unit 20, a document group storage unit 30, and a word extraction unit 4.
0, a character string search unit 50, a word search unit 60, an appearance degree estimation unit 70, a satisfaction degree calculation unit 80, and an output unit 90.

The control unit 10 controls the entire process from the input of search conditions to the output of search results. The input unit 20 inputs a search condition sentence, a document, or the like from the user and temporarily stores them. The document group storage unit 30 stores a plurality of search target documents each of which is a text described in natural language. Also,
The processing can be speeded up by creating indexing for character string search or word search in the document group in advance. This index stores a character string or word in association with a position where they appear in a document (position of a leading character).

The word extraction unit 40 extracts a word from the search condition input by the input unit 20 or the document in the document group storage unit 30. For this word extraction, there are a method of dividing into words by using breaks of character types and a method of performing morphological analysis using a word dictionary. When an index is added to the document group storage unit 30, the words need not be extracted from the document.

The control unit 10 calls the character string search unit 50 for each word extracted from the search conditions by the word extraction unit 40 for the documents stored in the document group storage unit 30,
Find the appearance set of a character string. At the same time, the word search unit 60 is called for each word extracted from the search condition, and the appearance set of words is obtained. This character string search unit 5
If 0 and the word retrieval unit 60 are not called separately, but they are called simultaneously, the speed can be increased. next,
The control unit 10 calls the appearance degree estimation unit 70 for each word to obtain the appearance degree, and calls the satisfaction degree calculation unit 80 to obtain the degree of document satisfaction with respect to the search condition. The sufficiency of this document is calculated for all the documents in the document group storage unit 30. Subsequently, the control unit 10 calls the output unit 90 and outputs the search result in which the documents are arranged according to the degree of sufficiency.

The character string search unit 50 checks, for one word extracted from the search condition, whether the same character string as the character string of the word appears in the document. When the character string appears in the document, the appearance set is obtained with the start position (offset from the head of the document, starting from 1) where the character string appears as an element. When it does not appear, the appearance set is an empty set. If the document group storage unit 30 stores indexing for character string search, this processing can be speeded up.

The word search unit 60 checks whether or not one word extracted from the search condition appears as a word in the document. At this time, the document is the word extraction unit 4
With 0, it is divided into words using the character type and morphological analysis. When it appears in the document, the appearance set is obtained with the start position (offset from the beginning of the document, starting from 1) where the character string of the word appears as an element. When it does not appear, the appearance set is an empty set. If the document group storage unit 30 stores indexing for word search,
This processing can be speeded up.

The appearance degree estimation unit 70 calculates the appearance degree for one word from the appearance set of character strings and the appearance set of words obtained for each word extracted from the search condition as follows. The appearance set S of the character strings in the document D for one word w is S = {s (1), s (2) ,. ． , S (n)}. Here, s (i) is a character string start position when the word w appears in the document D as the i-th character string. Also, a set W of appearances of words in the document D for one word w is W = {w (1), w (2) ,. ． , W (m)}. Here, w (i) is the word start position when the word w appears as the i-th word in the document D. Also,
Generally, m <n. The degree of appearance E (w, D) of the word w in the document D is defined as follows.

When the appearance set S of the character string is an empty set, E
(W, D) = 0. If the appearance set S is not an empty set, the following unit appearance u (s (i)) is obtained for each s (i), (i = 1, ..., N). If s (i) is an element of the word occurrence set W, then u (s (i)) = 1, otherwise u (s (i)) = e, where 0 <e <1 (where e Is a default value (for example, 0.1).)
Then, the appearance degree E (w, D) of the word w in the document D is defined as the sum of the unit appearance degrees u (s (i)) as follows. E (w, D) = Σu (s (i))

The sufficiency calculating unit 80 calculates the sufficiency of the document from the appearances of all the words extracted from the search conditions. For example, the sufficiency R (Q, D) of the document D is calculated as the sum of the appearances E (w, D) calculated for all the words w extracted from the search condition Q by the following formula. R (Q, D) = ΣE (w, D)

The output unit 90 selects a document as a search result according to the degree of sufficiency calculated for each document and outputs it to a display device, a printer, or the like. This selection is a set of documents with non-zero sufficiency or a set of documents arranged in descending order of sufficiency. At the time of display, a fixed number of documents may be discontinued from the retrieved document set, or a document whose satisfaction level is a certain value or more may be selected.

FIG. 2 is a flow chart for explaining the procedure of the document search process of this embodiment. For explanation here,
It is assumed that the document group storage unit 30 stores the document D1 and the document D2 D1: knowledge and deduction D2: Tokyo.

First, the control unit 10 inputs a search condition from a searcher using the input unit 20 (step S01). This search condition is temporarily stored in a search condition buffer or the like. Hereinafter, for the sake of explanation, the search condition Q is “knowledge of machine”.

Next, the control unit 10 uses the word extraction unit 40 to extract a word from the search condition Q (step S02).
Here, the search condition is divided into words by using the breaks of the character types, and the words are extracted. The search condition Q is divided into words such as machine / no / knowledge and three words {machine, no, knowledge}
Is obtained.

Next, the control unit 10 performs the following process for each word w for each document D stored in the document group storage unit 30 to obtain the appearance degree of the word w.

One document is retrieved from the document storage unit 30 (step S03). This document is called D. First, one of the words extracted from the search condition by the word extracting unit 40 is extracted (step S04). This word is called w.

The character string retrieving unit 50 obtains an appearance set of the word w that appears as a character string in the document D (step S05). For example, from the beginning to the end of the document D, the character string is collated one by one with the character string of the word w, and when both character strings match, the character string start position (offset from the beginning, 1 counting) appears in the set. Register as an element of. For example, {1} is obtained as an appearance set from the word w = “knowledge” and the document D1 = “knowledge and deduction”. Word w =
An empty set {} is obtained from "knowledge" and document D2 = "Tokyo".

Next, the word search unit 60 finds the appearance set of the word w that appears as words in the document D (step S06). For example, the word extraction unit 40 is used to divide the document D into words by character type and morphological analysis. This divides the document D into words like knowledge / and / deduction. From the beginning to the end of the divided document D, one character is compared with the word w, and when both character strings match and the word boundaries are before and after that, the character string start position (from the beginning The offset and 1 count are registered in the appearance set. For example, the word w =
From "knowledge" and document D1 = "knowledge and deduction", an empty set {} is obtained as an appearance set. Similarly, the word w =
The empty set {} is also obtained from "knowledge" and document D2 = "Tokyo".

Next, the appearance degree estimation unit 70 is used to obtain the appearance degree of the word w in the document D (step S07). From the character string search unit 50 described above, the document D for the word w
The appearance set S of the character string in is obtained. This is expressed as follows. S = {s (1), s (2) ,. ． , S (n)} where s (i) is the character string start position when the word w appears in the document D as the i-th character string. Further, the word search unit 60 described above obtains the word appearance set W in the document D for the word w. This is expressed as follows. W = {w (1), w (2) ,. ． , W (m)} where w (i) is the word start position when the word w appears i-th as a word in the document D. From these appearance sets, the appearance degree E (w, D) of the word w in the document D is calculated as the sum of the unit appearance degrees u (s (i)) shown below.

When the appearance set S of the character string is an empty set, E
(W, D) = 0. If the appearance set S is not an empty set, the following unit appearance u (s (i)) is obtained for each s (i), (i = 1, ..., N). If s (i) is an element of the word occurrence set W, then u (s (i)) = 1, otherwise u (s (i)) = e, where 0 <e <1 (where e Is a default value (for example, 0.1).)

From the above, the degree of appearance E (intelligence, D1) = 0.1
Is obtained. The appearance degree E (w, D) of the document D with respect to the word w obtained here is temporarily stored.

The above steps S04 to S07 are executed for all the words extracted from the search condition Q (step S08). As a result, the appearance degree for the document D1 is E (machine, D1) = 0, E (of, D1) = 0, E (know,
D1) = 0.1

When the appearance degrees are calculated for all the words extracted from the search condition Q, the satisfaction degree R (Q, D) of the document D with respect to the search condition Q is obtained from them (step S
09). For example, if the satisfaction degree R (Q, D) is the sum of appearance degrees E (w, D) for all the words extracted from the search condition Q, R (Q, D1) = E (machine, D1) + E (of, D1) + E (intelligence, D1) = 0 + 0 + 0.1 = 0.1. The satisfaction degree R (Q, D) of the document D with respect to the search condition Q obtained here is temporarily stored in association with the document D (step S07). Steps S03 to S above
Up to 09 are executed for all the documents in the document group storage unit 30 (step S10).

Finally, the control unit 10 calls the output unit 90, selects a document as a search result according to the degree of sufficiency of each document, and outputs it to a display device or a printer (step S).
11). For example, if a search result is a set of documents with a non-zero degree of sufficiency, {D1} will be the search result. In addition, documents (or their identifiers) may be arranged in descending order of satisfaction level and used as search results. In this case, {D1, D2} is the search result. When the number of search results is large, only a certain number may be output from the one having a high degree of sufficiency, or a search result having a degree of sufficiency of a certain value or more may be output.

As described above, when the embodiment is configured, the document group D1 and the document D2 are searched for in D1: knowledge and deduction D2: Tokyo search condition Q1 and search condition Q2 Q1: machine knowledge Q2: Kyoto. As a result, R (Q1, D1) becomes a non-zero value and becomes the search result. Further, R (Q2, D2) is also non-zero, but if there are other documents that include "Kyoto" as a word in the document group, the degree of sufficiency is lower than those documents. Therefore, if a certain number of documents are selected in descending order of sufficiency and used as the search result, the document D2 is less likely to be the search result than in the conventional case.

Furthermore, the present invention is not limited to the above-mentioned embodiments. Each function that constitutes the above-described embodiment is programmed, and a CD is prepared in advance.
-Write to a recording medium such as a ROM and write this CD-R
The object of the present invention can also be achieved by mounting the OM on a computer equipped with a medium driving device such as a CD-ROM drive, storing these programs in a memory or a storage device of the computer, and executing the programs. Needless to say. In this case, the program itself read from the recording medium realizes the functions of the above-described embodiments, and the program and the recording medium recording the program also constitute the present invention.

The recording medium is a semiconductor medium (eg, ROM, non-volatile memory card, etc.), optical medium (eg, DVD, MO, MD, CD-R, etc.), magnetic medium (eg, magnetic tape, flexible disk). Etc.).

Further, not only the functions of the above-described embodiment are realized by executing the loaded program, but the operating system or the like performs some or all of the actual processing based on the instructions of the program, The case where the functions of the above-described embodiments are realized by the processing is also included.

Further, when the above-mentioned program is stored in a storage device such as a magnetic disk of a server computer and is distributed in a format such as download from a user's computer connected by a communication network, the storage device of this server computer is also stored. It is included in the recording medium of the present invention.

[0047]

As described above, according to the present invention, for each word in the search condition and each document in the document group, the appearance frequency itself of the character string or the word in the document is not used, but From the appearance set that appears as a character string in a document and the appearance set that appears as a word, determine the degree of occurrence of that word in that document, and select the document that is the search result based on the degree of sufficiency obtained from that degree of occurrence. As a result, both problems of omission of search and excessive search are less likely to occur,
An appropriate search result can be obtained for the search condition.

[Brief description of drawings]

FIG. 1 is a functional configuration diagram of a document search device according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a document search processing procedure according to the embodiment.

[Explanation of symbols]

10 ... Control unit, 20 ... Input unit, 30 ... Document group storage unit, 4
0 ... Word extraction unit, 50 ... Character string search unit, 60 ... Word search unit, 70 ... Appearance estimation unit, 80 ... Satisfaction degree calculation unit, 90 ...
Output section.

Claims (6)

[Claims] 1. An input unit for inputting a request for search conditions and the like, a document group storage unit for storing a plurality of documents composed of character strings expressed in natural language, a word from the search conditions and the document. With respect to the words extracted from the document, a word extraction unit to extract, a character string search unit to obtain an appearance set of the same character string as the notation of the word extracted from the search condition for the document, A word search unit for obtaining an appearance set in which the same word as the word extracted from the search condition appears, and for each word extracted from the search condition, obtained from each of the character string search unit and the word search unit. An appearance degree estimation unit that obtains the appearance degree based on the appearance set, a satisfaction degree calculation unit that obtains the satisfaction degree based on the appearance degree obtained for all the words extracted from the search condition, and the document group storage Everything remembered in the club And a output unit that selects and outputs a document as a search result according to the satisfaction level obtained for the document. 2. The document search device according to claim 1, wherein for each element of the appearance set of the character string search unit, which is obtained for each word extracted from the search conditions, the element is the word search unit. 1 if present in the appearance set
If not, the document retrieval apparatus is characterized in that, as a default value, the sum of all elements is the appearance degree. 3. The document search device according to claim 1, wherein the sum of the appearance degrees obtained for each word extracted from the search conditions is set as the satisfaction degree. . 4. For each word existing in a search condition or the like, an appearance set that appears as a character string and an appearance set that appears as a word in each document in the document group are obtained, and the appearance degree for the word is calculated from these appearance sets. A document search method characterized by selecting a document as a search result based on the degree of sufficiency of the document obtained from the appearance degree. 5. A computer as claimed in claim 1, 2 or 3
A program for functioning as the document search device described in 1. 6. A computer-readable recording medium in which the document search program according to claim 5 is recorded.