JP2000222432A - Document retrieval device, document retrieval method and recording medium recording document retrieval program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a document retrieval device and a document retrieval method capable of shortening the time to be required for retrieval without reducing the retrieval accuracy of document, and to provide a recording medium on which a document retrieval program is recorded. SOLUTION: In this document retrieval method for extracting a retrieving word only when the extraction of the retrieving word from a retrieval request sentence is necessary, whether plural retrieving words are included in the retrieval request sentence or not is judged, and when the retrieval request sentence is not a sentence constituted of plural retrieving words as the result of judgment, whether the extraction of a retrieving word is necessary or not is judged by inquiring the retrieval request sentence and a word index 105. When the extraction of the retrieving word from the retrieval request sentence is unnecessary as the result of judgment, a document including the retrieval request sentence is retrieved from plural documents without extracting the retrieving word in the retrieval request sentence.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document search technique, and more particularly, to a document search apparatus and method for retrieving documents matching a condition specified by a user from a large number of documents.
The present invention relates to a recording medium on which a document search program is recorded.

[0002]

2. Description of the Related Art In the prior art, as a document search service realized on the Internet or the like, a free sentence input method which allows a free text to be input in a search request sentence is mainly used. According to such a free sentence input method, a user using the document search service does not need to memorize complicated logical expressions, and can specify search conditions extremely easily, thereby improving convenience.

The main technology for realizing the free sentence input method is a morphological analysis technology. Morphological analysis is a process of dividing a natural sentence into words and assigning parts of speech. In the free sentence input method, a search word used as a search condition is extracted from a search request sentence freely input by a user using this technique. The morphological analysis process is a process of decomposing a document consisting of a plurality of words into words, and this process requires a long calculation time as compared with other series of processes in the search. Hereinafter, the morphological analysis processing will be described in more detail.

[0004] For morphological analysis, two dictionaries, a word dictionary and a concatenation dictionary, are used. The word dictionary is composed of a word character string, a part of speech, and a word cost, and the concatenation dictionary is composed of a concatenation possibility and a concatenation cost of a part of speech. Here, the word cost is a numerical value indicating the likelihood of occurrence of the word, and indicates that the smaller the value is, the more likely the word is to occur. The connection cost is a numerical value indicating the ease of connection of parts of speech corresponding to two consecutive words. Using these dictionaries, processing based on an algorithm called the minimum cost method is performed. That is, the following processing is performed.

(1) First, a character string that matches a character string of an entry in a word dictionary is extracted as a word from a character string starting from each position of a sentence. (2) Next, referring to the concatenation dictionary, a list of concatenable words is left. By these processes, a plurality of analysis results are obtained in the form of a lattice structure having words as nodes. From the results, the sum of the word cost and the concatenation cost is calculated for each analysis result, and the path having the minimum cost is selected as the optimal solution.

Here, when performing a morphological analysis on a search request sentence consisting of n characters, the number of character strings that can be selected from n characters and up to n consecutive characters depends on the number of consecutive characters as follows. Become. (1) The number of character strings formed by selecting one continuous character from n characters is n. (2) The number of character strings formed by selecting two consecutive characters from n characters is n-1... (I) n The number of character strings formed by selecting i consecutive characters from characters is n−i + 1... (N) The number of character strings formed by selecting n consecutive n characters from n characters is 1 in each case. Can be considered.

From this sum, from the search request sentence of n characters,

[0008]

(Equation 1)

[0009] A possible word is considered. That is, in order to perform a morphological analysis on a search request sentence consisting of n characters, a word dictionary is looked up this number of times and whether the character string is a word,
If it is a word, it is necessary to obtain a cost, which is the probability of occurrence of the word. Therefore, morphological analysis is
The comparison between the word dictionary and the search request sentence requires a calculation amount proportional to the square of the length n of the character string of the search request sentence.

As described above, the morphological analysis requires a large calculation time as compared with a search process because the process of selecting one optimal solution based on cost calculation from a plurality of analysis result options. I do. Therefore, it is necessary to shorten the time required for morphological analysis in order to speed up the search, and techniques for performing morphological analysis at high speed have been studied. For example, Japanese morphological analysis by Fujitsu Laboratories Ltd. System Breakfast, JUM of Kyoto University
AN and Nara Institute of Technology ChaSen. However, no matter how the processing efficiency is increased, the morphological analysis requires a calculation amount that is essentially proportional to the square of the length n of the search character string.

Further, in a document search, a method of morphologically analyzing a document to be searched, registering words in the document in a word index, and inquiring the input word and the word index is generally used. Has been taken. Since the words in the document to be searched are registered in the word index, the fact that a certain character string is included in the word index means that the character string is a word included in the search target sentence. This method is adopted because the conditions are sufficient and sufficient.

Here, when the search request sentence is a sentence composed of a plurality of search words, such as a free sentence input type search system, it is not possible to query the sentence and the word index. The request sentence is subjected to morphological analysis, and a query is performed between each extracted search word and the word index.
For this reason, each time a search request sentence is input, morphological analysis is performed.

[0013]

However, according to the above-mentioned prior art, morphological analysis requires a large amount of calculation time as compared with other processes in document retrieval, but it does not require a large amount of calculation time. Since it was executed every time input,
There is a problem that the time required for morphological analysis extends the search time in document search.

The present invention has been made in view of the above points, and does not perform morphological analysis for every search request, but determines whether or not morphological analysis is necessary. An object of the present invention is to provide a document search device, a method, and a recording medium on which a document search program is recorded, which can reduce the time required for search without lowering search accuracy by performing only morphological analysis. .

[0015]

In order to achieve the above object, the present invention is configured as follows. The present invention is a document search device for searching a document including a search word extracted from a search request sentence as needed from a plurality of documents using a word index, and extracts a search word from the search request sentence. Search term extraction determining means for determining whether or not it is necessary;
As a result of the determination, when it is not necessary to extract the search term, means for searching a document including the search request sentence from the plurality of documents without extracting the search term from the search request sentence. Have. According to the present invention, it is first determined whether or not it is necessary to extract a search word. Therefore, it is not necessary to perform a morphological analysis on all search request sentences and extract a search word as in the related art. In this case, the calculation time required for the document can be reduced, so that the document can be searched in a short time.

[0016] In the above-mentioned configuration, the search term extraction judging means judges whether or not the search request sentence is a sentence composed of a plurality of search words based on a length of the search request sentence. If the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words, it is determined whether or not it is necessary to extract a search word by referring to the search request sentence and the word index. It may have a second determination unit.

According to the present invention, if it is found that the word is not composed of a plurality of search words, the query with the word index is performed as it is. If the query succeeds, the document can be extracted as it is. Therefore, the search time can be reduced. Furthermore, even if the search is not successful, if the morphological analysis is performed as usual, the search accuracy does not decrease.

Further, in the above configuration, the search term extraction determining means determines whether or not the search request sentence is a sentence composed of a plurality of search words based on a character type constituting the search request sentence. Determining means, if the result of the determination indicates that the search request sentence is not a sentence composed of a plurality of search words, whether or not it is necessary to extract a search word by referring to the search request sentence and the word index And a second determination unit that determines

According to the present invention, if it is found that the word is not composed of a plurality of search words, the query with the word index is performed as it is. If the query succeeds, the document can be extracted as it is. Therefore, the search time can be reduced. Furthermore, even if the inquiry is not successful, the morphological analysis is performed as usual, and the search accuracy does not decrease.

Further, in order to achieve the above object, the following may be performed. The present invention is a document search method for searching a document including a search word extracted from a search request sentence as needed from a plurality of documents using a word index, and extracting a search word from the search request sentence. A search word extraction determining step of determining whether it is necessary or not, and as a result of the determination, if it is not necessary to extract the search word, the search request sentence is not extracted from the search request sentence; Retrieving a document including the following from the plurality of documents.

The search term extraction determining step includes a first determination step of determining whether the search request sentence is a sentence composed of a plurality of search words based on a length of the search request sentence. If the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words, a second determination is made as to whether it is necessary to extract a search word by referring to the search request sentence and the word index. And the search word extraction determining step determines whether the search request sentence is a sentence composed of a plurality of search words based on a character type constituting the search request sentence. A first determining step, and if the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words,
There may be provided a second determining step of determining whether or not it is necessary to extract a search word by referring to the search request sentence and the word index.

The present invention also provides a document search program for causing a computer to execute a document search process for searching a document including a search word extracted from a search request sentence from a plurality of documents using a word index as necessary. A recording medium for recording, a search term extraction determination procedure for determining whether or not it is necessary to extract a search term from the search request sentence; and, as a result of the determination, when it is not necessary to extract the search term. Recording a document search program on a computer, the program further comprising: causing a computer to execute a step of searching a document including the search request sentence from the plurality of documents without extracting a search term from the search request sentence. It can also be a medium.

Further, in the above configuration, the search word extraction determining step includes a first determination for determining whether the search request sentence is a sentence composed of a plurality of search words based on a length of the search request sentence. And if the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words, it is determined whether or not it is necessary to extract a search word by referring to the search request sentence and the word index. The search request extraction determination step may be a sentence in which the search request sentence is composed of a plurality of search words by a character type constituting the search request sentence. A first determination procedure for determining whether or not the search request sentence is not a sentence composed of a plurality of search words; Need to be extracted It may have a second determining step determines whether.

In a document search, when a search request sentence composed of a plurality of words is input, it is necessary to extract a search term constituting the sentence from the search request sentence. Requires a morphological analysis process that requires a long calculation time, but according to the present invention described above, the necessity of the search word extraction is determined, and only when it is determined that the search word extraction process is necessary, the morphological analysis is performed. Execute As described above, the speed of the information search can be improved by reducing the morphological analysis processing requiring a long calculation processing time in the document search.

According to the present invention, a process for determining the necessity of extracting a search word is required. The time required for this process is sufficiently smaller than the time required for morphological analysis. It does not extend. In addition, the accuracy of the search result is exactly the same as that obtained by performing the morphological analysis.

[0026]

FIG. 1 is a block diagram showing a configuration of a document search apparatus according to an embodiment of the present invention. As shown in the figure, the present document search device includes a search request sentence input unit 1
01, search term extraction execution determination unit 102, search term extraction unit 10
3. Consists of a word index inquiry unit 104, a word index 105, and a document output unit 106. The search request sentence input unit 101 has a function of receiving a search request sentence input by a user and transmitting the search request sentence to the search term extraction execution determination unit 102. As the search request sentence, only a single search word such as “NTT” can be specified.
For example, a sentence including a plurality of search words such as "about DN service" can be specified. Search term extraction execution determination unit 1
02 has a function of determining whether or not it is necessary to perform a morphological analysis on an input search request sentence.
No. 3 has a function of extracting a search word by performing a morphological analysis on a search request sentence when it is determined that morphological analysis is necessary. The word index inquiry unit 104 has a function of inquiring a search request sentence or a search word and a word index stored in the word index 105, and the word index 105 performs an inquiry of the search request sentence or the search word. It has a function to perform. Further, the document output unit 106 has a function of outputting the documents successfully queried by the word index query unit 104 in an appropriate order based on the word frequency information.

For the search request sentence input in the search request sentence 101, the search term extraction execution determination unit 102 first determines whether or not to perform morphological analysis. As a result of the determination, if morphological analysis is necessary, morphological analysis is performed in the search word extracting unit 103, and an inquiry with the word index 105 is performed in the word index inquiry unit 104. If it is determined that morphological analysis is not necessary, morphological analysis is not performed. The above determination has the following three criteria.

That is, the judgment based on the criterion 1 is for making an inquiry using the search request sentence and the word index without performing morphological analysis, and the judgment based on the criterion 2 is performed when the number of characters of the search request sentence is more than a certain number. The sentence is determined to be a sentence composed of a plurality of search words, and the determination based on the criterion 3 is performed when a plurality of types of characters are included in the search request sentence.
The search request sentence is determined to be a sentence composed of a plurality of search words. This criterion 3 is, specifically,
Determine if there are any punctuation, periods, commas, quotes, double quotes, whitespace, or other symbols. How each criterion is used to make the determination will be described in the following description of the operation.

The operation of the document retrieval apparatus will be described in more detail with reference to the flowchart shown in FIG. First, in step 201, a search request sentence is input from the search request sentence input unit 101. The input search request sentence is passed to the search term extraction execution determination unit 102, and a reference 2
And criterion 3 are determined. As a result of the judgment, when the judgment criteria 2 and 3 are satisfied, that is, when it is judged that the search request sentence is a sentence composed of a plurality of search words, the search word extraction unit 103 performs The search term included in the search request sentence is extracted by the analysis,
In step 206, the word index query unit 104 makes a query with the word index 105.

If the result of the determination in step 202 is that the search criterion does not correspond to the criterion, that is, if it is determined that the search request sentence is composed of a single search word, then in step 203, the search request sentence and the word index are queried. I do. At this time, if the search request sentence is a single search word and the query succeeds, the document ID and the word frequency information are extracted from the word index in step 204. If the collation is not successful in step 203, the cause of the failure is
It cannot be determined whether (1) the search word is not included in the word index, or (2) whether the search request sentence is composed of a plurality of words. Therefore, here, the morphological analysis is performed in step 205 in the same manner as in the case where the judgment criterion is satisfied, the retrieved search words are extracted, and in step 206, the query with the word index is performed. If the query is successful, the word index query unit 104 determines in step 207 that the word index 105
Then, a document ID and word frequency information are extracted from the document ID, and are passed to the document output unit 106. If the query fails, the document containing the search term is not included in the word index 105, and thus a signal indicating that there is no document corresponding to the search request sentence is passed to the output unit 106. After extracting the document ID and the word frequency information from the word index 105 in step 204, the result is similarly transferred to the document output unit 106.

When the document output unit 106 receives the document ID and the word frequency information from the word index inquiry unit 104, it calculates based on the word frequency information and outputs the documents in an appropriate order. When the document output unit 106 receives a signal indicating that there is no document corresponding to the search request statement,
The display indicates that there is no document containing the search term. Here, the operation and effect of each standard described above will be described.

The criterion 1 is a criterion for determining whether or not it is necessary to extract a search word using a word index. In this determination processing, if the query of the word index and the search request sentence is successful, At that time, it is determined that the search request sentence is composed of a single search word, and a document including the search word can be obtained from the word index.

The criterion 2 and the criterion 3 are criteria for judging whether or not it is necessary to extract a search term from the nature of the search request sentence. If these criteria are met, the search request sentence is very often composed of a plurality of search terms.
The probability of this determination being erroneous is extremely small, and a search word extraction process is extremely rarely performed for a search request composed of one search word. Further, even if an erroneous determination is made, the same processing as in the conventional search of performing a morphological analysis and performing a query with a word index is performed, so that the search accuracy is not affected at all.

Therefore, by using the criteria 1, 2, and 3 as shown in the above-described flowchart, when the search request sentence is composed of only one word, the query processing of the search request sentence and the word index is not performed. A document can be retrieved by only one inquiry process. Here, the calculation time required for a single query process with a word index is
It is about the same as the calculation time required for one comparison process between a character string and a word dictionary in morphological analysis, and is proportional to the length n of the search request. Further, it takes a calculation time proportional to n to judge whether or not the morphological analysis is necessary in the criteria 2 and 3. Here, the number of times of the comparison process between the character string and the word dictionary in the morphological analysis is proportional to the square of n. Therefore, by not performing morphological analysis, the length of the search request sentence n
The search time can be shortened in inverse proportion to.

As described above, when it is determined that there is no need to extract a search term from a search request sentence, that is, when the query of the search request sentence and the word index is successful,
The search request sentence can be searched without performing morphological analysis.
With the above-described means, the search accuracy remains unchanged from the case where morphological analysis is performed, and only the search time can be reduced.

Next, the operation of the document search apparatus will be described more specifically. FIG. 3 is a diagram illustrating a specific example of the word index 105. As shown in the figure, the word index is a table in which a word and an ID of an appearing document are paired. At the time of retrieval, an inquiry is made between the search word and the word in the word index, and a list of IDs of documents containing the word is obtained from the matching line. For example, for the word index in FIG.
Given the search term "service",
The query with the line where the word column is "service" succeeds and I
Documents having D numbers of 3, 25, etc. are obtained as search results.
Only words are registered in the word index. Therefore, when the search request sentence is composed of a plurality of search words as in “NTT ISDN service”, the query of the search request sentence and the word index fails. In such a case, individual search terms are extracted, and each search term and word index are queried.

Next, with reference to the flowchart of FIG. 2, a query between the search request sentence and the word index will be described with reference to the example of FIG. The case where the search request sentence is composed of a plurality of search words will be described separately. (1) When the search request sentence is composed of a single search word When the search request sentence is “service”, it is determined that there is only one word in the determination of criteria 2 and 3 in step 202 in FIG. Next, in step 203, if the word "service" is included in the word index, the inquiry is successful. In the example of the word index in FIG. 3, since the word "service" is included in the word index,
The query succeeds and the ID of the document containing the word "service" is obtained. If there is no word including the word “service” in the search target document and the word is not included in the word index, a morphological analysis is performed in step 205 and the “service”
Is extracted as a search term. Then, if the query with the word index fails in step 206, it is understood that the search word is not included in the word index.

(2) When the search request sentence is composed of a plurality of search words If the search request sentence is "about NTT's ISDN service", the judgment of the criteria 2 and 3 in step 202 indicates that the search request sentence is Since it is determined that the sentence is composed of a plurality of search words, the search words are extracted by morphological analysis in step 205. Here, assuming that the independent word in the search request sentence is a search word, “NTT”, “ISD”
N "and" service "are extracted as search terms.

At step 206, a document containing each search term is obtained by querying each search term and the word index. When multiple search terms are extracted in this way, it is also possible to specify search conditions such as including all words, including some words, including only certain words, or not including It is. This condition can be specified by the user at the time of input.

The configuration of the document search device of the present invention is not limited to the configuration of the embodiment shown in FIG. 1, but each component is constructed by software (program) having the above-described procedure. By causing the computer system to execute the program, a general computer system can be used as the document search device of the present invention.

FIG. 4 is a block diagram showing an example of a hardware configuration of the computer system. The computer system includes a CPU 300 for executing processing, a memory 301 for storing programs and data, an external storage device 302 for storing programs and data used by the memory 301 or the CPU 300, a display 303 for displaying data, and inputting data or instructions. Keyboard 30
4. It comprises a communication processing device 305 for communicating with another computer system or the like via a network. The program is stored in the memory 301 or the external storage device 3
02 and executed by the CPU 300.

The recording medium of the present invention corresponds to the memory 301 or the external storage device 302. In addition, electronic memory,
Hard disk or floppy disk, CD-R
Portable recording media such as OM and magnetic tape can be used as the recording media of the present invention. By loading the program recorded on the document search program recording medium of the present invention into a general computer system as shown in FIG. 4, it becomes possible to execute the document search method of the present invention on the computer. .

It should be noted that the present invention is not limited to the above-described embodiment, but can be variously modified and applied within the scope of the claims.

[0044]

As described above, according to the present invention, the following effects can be obtained. According to the present invention, first, it is determined whether or not to perform a morphological analysis that requires a long calculation time. If the search request sentence is a single search word, the morphological analysis is not performed. Can be shortened.

That is, in the information search service on the Internet, for example, "N
Although a sentence used everyday called "TT's ISDN service" can be specified, a search request sentence actually input in the search service is very often composed of a single word. In the prior art, the morphological analysis was also performed on such a single word, and the search time was long. Does not perform morphological analysis, so that the search time is reduced.

Therefore, according to the present invention, it is possible to provide a comfortable document search environment with a short waiting time for the user.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a document search device according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating an operation of the document search device according to the embodiment of the present invention.

FIG. 3 is a diagram illustrating an example of a word index and an example of a query between a search request sentence or a search word and a word index in the embodiment of the present invention.

FIG. 4 is a block diagram illustrating a configuration of a computer system according to the embodiment of the present invention.

[Explanation of symbols]

 101 search request input unit 102 search word extraction execution determination unit 103 search word extraction unit 104 word index inquiry unit 105 word index 106 document output unit 300 CPU 301 memory 302 external storage device 303 display 304 keyboard 305 communication processing device

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Masakatsu Okubo 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Japan Telegraph and Telephone Corporation (72) Inventor Kazuo Tanaka 3--19, Nishishinjuku, Shinjuku-ku, Tokyo No. 2 Nippon Telegraph and Telephone Corporation F-term (reference) 5B075 ND03 NK02 NK54 PP25 QP10 UU06

Claims (9)

[Claims] 1. A document search apparatus for searching a document including a search word extracted as needed from a search request sentence from a plurality of documents using a word index, extracting the search word from the search request sentence. Search term extraction determining means for determining whether or not it is necessary to perform the search request. If the result of the determination indicates that the search term need not be extracted, the search request is not extracted from the search request sentence. Means for retrieving a document including a sentence from the plurality of documents. 2. The search term extraction determination means, comprising: first determination means for determining whether the search request sentence is a sentence composed of a plurality of search words based on a length of the search request sentence; If the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words, a second determination is made as to whether it is necessary to extract a search word by referring to the search request sentence and the word index. 2. The document search device according to claim 1, further comprising: a determination unit. 3. The search term extraction determination means, comprising: first determination means for determining whether or not the search request sentence is a sentence composed of a plurality of search words based on a character type constituting the search request sentence; If the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words, a determination is made as to whether or not it is necessary to extract a search word by referring to the search request sentence and the word index. 2. The document search apparatus according to claim 1, further comprising: a second determination unit. 4. A document search method for searching a document including a search word extracted from a search request sentence as needed from a plurality of documents using a word index, wherein the search word is extracted from the search request sentence. A search word extraction determining step of determining whether or not it is necessary to perform the search. If the result of the determination is that the search word need not be extracted, the search request is not extracted from the search request sentence. Retrieving a document containing a sentence from the plurality of documents. 5. The method according to claim 1, wherein the search term extraction determining step determines whether the search request sentence is a sentence composed of a plurality of search words based on a length of the search request sentence. If the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words, a second determination is made as to whether it is necessary to extract a search word by referring to the search request sentence and the word index. 5. The method according to claim 4, further comprising the steps of: 6. The search word extraction determining step includes: a first determination step of determining whether the search request sentence is a sentence composed of a plurality of search words based on a character type of the search request sentence; If the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words, a determination is made as to whether or not it is necessary to extract a search word by referring to the search request sentence and the word index. 5. The document search method according to claim 4, further comprising the following two determination steps. 7. A recording medium storing a document search program for causing a computer to execute a document search process for searching a document including a search word extracted from a search request sentence from a plurality of documents using a word index as needed. A search word extraction determination procedure for determining whether a search word needs to be extracted from the search request sentence; and as a result of the determination, if the search word does not need to be extracted, the search A step of searching a document including the search request sentence from the plurality of documents without extracting a search term from the request sentence; 8. The search term extraction determination step, comprising: a first determination step of determining whether the search request sentence is a sentence composed of a plurality of search terms based on a length of the search request sentence; If the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words, a second determination is made as to whether it is necessary to extract a search word by referring to the search request sentence and the word index. A recording medium on which the document search program according to claim 7 is recorded. 9. The search term extraction determining step includes: a first determination step of determining whether the search request sentence is a sentence composed of a plurality of search words based on a character type of the search request sentence; If the result of the determination is that the search request sentence is not a sentence composed of a plurality of search words, a determination is made as to whether or not it is necessary to extract a search word by referring to the search request sentence and the word index. 8. A recording medium storing the document search program according to claim 7, comprising: