JP2004151959A - Method and system of pseudo search - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a system of pseudo search, by which a high-precision search result can be obtained using a simple operation in a short time, irrespective of operator's skill level. <P>SOLUTION: A search condition document DB1 stores a search condition document as a search key in a text format. A document DB2 to be searched stores a multiplicity of documents to be searched in a text format. A keyword extracting part 3 extracts a plurality of keywords from the search condition document, stored in the search condition document DB1. A full-text search part 4 implements full-text search, using a keyword extracted by the keyword extracting part 3 as a search key, covering all the documents stored in the document DB2. A pseudo search part 5 implements pseudo search, using the search condition document as a search key covering the results of the full-text search by the full-text search part 4. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a similarity search method and system for documents, and more particularly to a similarity search method and system for searching a document similar to the text document from a large number of search target documents using a text document as a search key.
[0002]
[Prior art]
As a technique for searching a document containing desired information from an electronic document with high accuracy and efficiency, a "full text search" for extracting all documents including an input keyword and a text document as a search key are input. A "similarity search" for searching for a document similar to the contents of the text document is known.
[0003]
FIG. 5 is a functional block diagram of a conventional full-text search system. A search target document database (DB) 41 stores a large number of documents to be searched in a text format. The full-text search unit 43 performs a full-text search on all documents stored in the search target document DB 41 using a keyword manually input by the operator from the keyboard 42 as a search key. The full-text search result output unit 44 displays or prints out the full-text search result.
[0004]
FIG. 6 is a functional block diagram of a conventional similarity search system. Many documents to be searched are stored in a search target document DB 51 in a text format. The search condition document DB 52 stores a search condition document as a search key in a text format. The similarity search unit 53 performs a similarity search on all documents stored in the search target document DB 51 using a separately specified current search condition document as a search key. The similarity search result output unit 54 displays or prints out the similarity search result.
[0005]
As a typical method of the similarity search, a keyword is automatically extracted from a text, a document is weighted using a keyword appearance frequency or the like, and a search result is output in a vector space model ( A search model such as a Vector Space Model (VSM) or a probabilistic model (Probabilistic Model: PBM) has been proposed, and is disclosed in JP-A-10-269235 and JP-A-2001-331527.
[0006]
In the vector space model, each word in the search condition document and the search target document is represented by a vector having elements such as weights, and the most suitable document is searched from the viewpoint of “similarity of vector”. In the probability model, the search documents are ranked according to the probability regarding the relevance between the search condition document and the search target document.
[0007]
Japanese Patent Application Laid-Open No. 2002-222208 discloses a search system that combines "full-text search (keyword search)" and "similarity search". Here, a similarity search is first executed by inputting a keyword, and the similarity search is executed again using the search result (document) obtained as a search key. Further, a full-text search is executed this time using the characteristic word returned as a search result as a search key.
[0008]
[Problems to be solved by the invention]
The full-text search system for inputting a keyword has a relatively simple configuration and can operate at high speed even on a large scale, but requires a procedure for an operator to select and manually input a keyword. In addition, if the keyword is inappropriate, a large number of irrelevant documents will be searched, or a desired document will not be found at all. Therefore, the search accuracy greatly depends on the proficiency of the operator, and a search with high accuracy cannot always be performed. There was a problem.
[0009]
On the other hand, in the similar search system, the text of a document or a fragment thereof can be pasted as it is on the clipboard and searched, so that the usability is remarkably improved as compared with the full text search. However, when a high-precision search is required in the similarity search, the similarity calculation for all the words in the document is required, so that the calculation amount increases and the search time increases. There was a problem.
[0010]
Furthermore, even in a system combining a full-text search and a similar search, not only a manual input operation of a keyword is required at first but also a search operation three times (two similar searches and one full-text search) is required. There was a problem of becoming.
[0011]
An object of the present invention is to solve the above-mentioned problems of the related art and to provide a similar search method and system that can obtain a high-accuracy search result in a short time with a simple operation regardless of the skill level of an operator. is there.
[0012]
[Means for Solving the Problems]
In order to achieve the above object, the present invention provides a similar search system for searching a document similar to a specified search condition document from a plurality of search target documents, wherein a means for extracting a keyword from a search condition document; Means for performing a full-text search on the group of documents to be searched using the extracted keyword as a search key, means for performing a similarity search on the full-text search results using the search condition document as a search key, Means for outputting a similar search result.
[0013]
According to the above-described feature, a keyword for full-text search is automatically extracted only by designating a document (search condition document) as a search key. Therefore, an appropriate keyword can be selected regardless of the proficiency of a search operator. Can be extracted. Further, a full-text search is performed on all of the search target documents using the keyword as a search key, and a similarity search is performed on only the full-text search results using the search condition document as a search key. Improvement and reduction of search time are achieved at the same time.
[0014]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a main part of a similarity search system according to an embodiment of the present invention.
[0015]
A search condition document database (DB) 1 stores a search condition document as a search key in a text format. A large number of documents to be searched are stored in the search target document DB2 in text format. If the system is applied to a search system for patent publications, the search condition document DB1 stores articles and drafts written by the inventor, and the search target document DB2 stores patent publications and patents. A publicly known document such as a public publication is stored.
[0016]
The keyword extracting unit 3 extracts a keyword representing the content of the current search condition document from the current search condition document stored in the search condition document DB1. The full-text search unit 4 performs a full-text search on all the documents stored in the search target document DB 2 using the keyword extracted by the keyword extraction unit 3 as a search key. The similarity search unit 5 performs a similarity search on the full-text search result by the full-text search unit 4 using the current search condition document as a search key. The search result output unit 6 displays or prints out the similar search results.
[0017]
FIG. 2 is a functional block diagram of the keyword extracting unit 3. The morphological analyzing unit 31 performs a morphological analysis on the current search condition document to decompose the document into a plurality of words. The filtering unit 32 normalizes each word and performs filtering such as stop word removal. By this normalization and stop word processing, the inflected ending is returned to the original form, and the notation and abbreviation are unified. The statistical processing unit 33 weights each word based on a parameter such as an appearance frequency, and extracts a plurality of words with high weights as keywords.
[0018]
The statistical processing unit 33 obtains, for example, the relative frequency P (T = t | Ci) for each word t in the current search condition document Ci, and further calculates the relative frequency P for the word t in all search target documents. (T = t) is obtained. Then, all the words t satisfying the following equation (1) or the upper N words of the word t satisfying the following equation (1) are extracted as keywords. Note that S is a predetermined constant.
P (T = t | Ci) / P (T = t) ≧ S (1)
[0019]
That is, a word having a high appearance frequency [P (T = t | Ci)] in a document has a high probability of representing the contents of the document. However, a general word that often appears in any document cannot represent the contents of the document even if its appearance frequency is high. Thus, in the present embodiment, for each word t in the search condition document Ci, the appearance frequency [P (T = t | Ci)] in the document is obtained, and the appearance frequency [P (T (T = T)], and only words having a high [P (T = t | Ci)] and a low [P (T = t)] are extracted as keywords.
[0020]
Next, the operation of the present embodiment will be described in detail with reference to the flowchart of FIG. 3 and the sequence diagram of FIG.
[0021]
In step S1 of FIG. 3, one of a plurality of search condition documents Ci stored in the search condition document DB1 is designated. In step S2, the keyword extraction unit 3 performs morphological analysis (S21), filtering (S22), and statistical processing (S23) on the specified search condition document Ci, and the content of the search condition document Ci Are automatically extracted.
[0022]
In step S3, the full-text search unit 4 performs a full-text search on all documents stored in the search target document DB 2 using the keyword extracted by the keyword extraction unit 3 as a search key. The plurality of document IDs obtained by the full-text search are notified to the similarity search unit 5.
[0023]
As described above, in the present embodiment, only by specifying the search condition document, the keyword is automatically extracted based on the content, and a full-text search is performed using this keyword as a search key. Automatic search using a representative keyword becomes possible.
[0024]
In step S4, the similarity search unit 5 targets only the search target documents corresponding to the plurality of document IDs provided by the full text search unit 4 among a large number of documents stored in the search target document DB2. A similarity search is performed using the current search condition document Ci as a search key. In step S5, the search result output unit 6 outputs the similar search result.
[0025]
As described above, according to the present embodiment, since the similarity search is performed only on the documents limited in the range limited by the full-text search result, the time required for the similarity search can be reduced.
[0026]
Further, according to the present embodiment, a document having a content similar to the search condition document can be reliably obtained in a shorter time than in the past by simply performing the same procedure as the conventional similar search procedure of specifying the search condition document. Be able to search.
[0027]
【The invention's effect】
According to the present invention, the following effects are achieved.
(1) Just by performing the same procedure as a conventional similarity search procedure of designating a search condition document, an accurate similarity search can be performed in a shorter time than in the related art.
(2) Just by specifying a search condition document, a keyword is automatically extracted based on the content, and a full-text search is performed using this keyword as a search key. Search becomes possible.
(3) Since the similarity search is executed only for the search target documents limited in the range limited by the full-text search result, the time required for the similarity search is reduced and the search accuracy is simultaneously improved.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a main part of a similarity search system according to an embodiment of the present invention.
FIG. 2 is a functional block diagram of a keyword extracting unit 3 of FIG.
FIG. 3 is a flowchart showing an operation of the embodiment.
FIG. 4 is a sequence diagram showing an operation of the embodiment.
FIG. 5 is a functional block diagram of a conventional full-text search system.
FIG. 6 is a functional block diagram of a conventional similarity search system.
[Explanation of symbols]
1. Search condition document DB
2 ... Search target document DB
3. Keyword extraction unit 4 Full text search unit 5 Similarity search unit 6 Search result output unit

Claims (3)

In a similar search system for searching a document similar to a specified search condition document from a plurality of search target documents,
Means for extracting a keyword from the search condition document;
Means for performing a full-text search on the plurality of search target documents using the keyword as a search key;
Means for performing a similarity search on the full-text search result using the search condition document as a search key;
Means for outputting the similarity search result. The means for extracting the keyword includes:
Means for performing morphological analysis on search condition documents,
Means for statistically processing the result of the morphological analysis and weighting each word;
2. A system according to claim 1, further comprising means for extracting a word having a high weight as a keyword. In a similar search method for searching a document similar to a specified search condition document from a plurality of search target documents,
Extracting a keyword from the search condition document;
Performing a full-text search on the plurality of search target documents using the keyword as a search key;
Performing a similarity search on the full-text search result using the search condition document as a search key;
Outputting the similarity search result.

Images