JP3848014B2 - Document search method and document search apparatus - Google Patents

Description

【００６２】
そして、入力された文書（すなわち、ここでは、図３（ａ）の特許明細書）と検索対象の文書（例えば、図３（ｂ）の特許明細書）のそれぞれについて、特徴ベクトルを求めて、これらの間で内積を算出して、それを入力された文書と文書格納部２１１に格納されている各文書との間の類似度としてもよい。なお、内積の代わりにコサイン距離を求めてもよい。この場合、類似度の値が大きいほど類似度が高くなる。
【００６３】
ステップＳ４３で計算された入力された文書と文書格納部２１１に格納されている各文書との間の類似度は、その値が大きい順にソートされ（ステップＳ４４）、上位ｎ位（ｎは正の整数）の特許明細書のファイル名が出力部２０１に出力される（ステップＳ４５）。
【００６４】
上述したような処理によれば、文書検索の精度を向上させることができる。例えば、図３（ａ）の特許明細書と、図３（ｂ）の特許明細書とでは、どちらも都道府県名と市区郡名とを対応付けて辞書に格納しているにもかかわらず、どちらの特許請求の範囲にも「都道府県名」「市区郡名」という語は出現しない。そのため、拡張語抽出部２０７を持たない従来の文書検索装置では、特許請求の範囲の文章だけから索引語を抽出した場合、図３（ａ）の特許明細書と図３（ｂ）の特許明細書との間の高い類似度は得られない。これに対して本発明の文書検索装置では、図３（ａ）の特許明細書においても、図３（ｂ）の特許明細書においても、特許請求の範囲に出現しない「都道府県名」と「市区郡名」という語が拡張語として索引語に追加されるため両者の間で高い類似度が得られる。
【００６５】
なお、上記実施形態では、検索要求として入力した文書に類似する文書の検索要求の場合を例にとり説明したが、この場合に限らず、種々変形して応用可能である。例えば、文書検索部２０８での検索処理は、入力されたキーワードに合致する文書の検索要求の場合であっても、上記同様にして（すなわち、キーワードと索引語との類似度を求める）文書を検索することが可能である。
【００６６】
また、上記実施形態では、索引語として、主に名詞、動詞のみを抽出しているが、この場合に限るものではなく、種々変形して応用可能である。例えば、これらに加えて例えば形容詞、副詞等を選択してもよいし、動詞を選択しなくてもよい。
【００６７】
また、ここでは、検索対象の文書が特許明細書である場合を例にとり説明しているため、その内容の特徴を最も適切に記述している「特許請求の範囲」という項目の文章から基本語を抽出し、基本語に関連する拡張語（例えば、基本語をより具体化して表現している拡張語）を「発明の実施の形態」という項目の文章から抽出しているが、この場合に限るものではない。また、検索対象の文書が学術論文であれば、基本語を「アブストラクト」から抽出し、その基本語に関連する拡張語を本文から抽出するようにしてもよい。このように、検索対象の文書がどのような文書であるにしろ、基本語は、その文書の内容の特徴を最も適切に記述している構成要素から抽出し、拡張語は、それより詳細な記述がなされている構成要素から抽出することが望ましい。
【００６８】
図３２は、上記実施形態で説明した文書検索装置を適用した類似文書検索を行う他の文書検索装置の構成を概略的に示したものである。図３２に示した類似文書検索装置では、まず、入力した文書の類似文書を検索するに先だって、当該文書の大まかな分類を行う。例えば、文書の内容に応じて複数のクラス（例えば、電気、機械、化学等）が用意されているとする。各クラスは、例えば、そのクラスに属する文書にてよく使われる単語を羅列した辞書を有し、この辞書の単語と入力された文書内の単語とを照合して（類似度を算出して）、最も類似するクラスを特定する。このとき求めることができる当該文書中に出現する単語と、その出現頻度は、先に説明した図９に示したような索引語リストの作成の際に用いてもよい。
【００６９】
この大分け分類処理部においては、クラスの特定された文書は、例えば、その文書中の単語と出現頻度とに基づき、より詳細なサブクラスに分類され、さらに、サブクラスの特定された文書はより詳細なグループに分類され、さらに、グループの特定された文書はより詳細なサブグループに分類されてもよい。
【００７０】
次に、類似文書検索処理部において、上記実施形態にて説明した類似文書の検索を行い、検索された類似文書のリストを出力する。
【００７１】
なお、ここでの検索結果を大分け分類処理および類似文書検索処理にフィードバックすることにより、より精度の高い（ヒット率の高い）類似文書の検索が可能になる。すなわち、例えば、大分け分類処理において得られた入力された文書から抽出された単語を当該文書の属するサブグループ、グループ、サブクラス、クラスの辞書に追加する。また、検索された類似文書の索引語リストに当該入力文書にはあって類似文書にはない単語を追加する。
【００７２】
図３２に示した文書検索装置もコンピュータに実行させることのできるプログラムとして、磁気ディスク（フロッピーディスク、ハードディスクなど）、光ディスク（ＣＤ−ＲＯＭ、ＤＶＤなど）、半導体メモリなどの記録媒体に格納して頒布することもできる。
【００７３】
以上説明したように上記実施形態によれば、文書中の予め定められた構成要素の文章から索引語を抽出し、他の構成要素の文章中から適切な語を拡張語として抽出して索引語に追加することにより、大量の文書の中からユーザの検索要求に合致する文書を高精度に選択できるようになる。
【００７４】
【発明の効果】
以上説明したように、本発明によれば、文書の内容に即した精度の高い文書の検索を可能にする。
【図面の簡単な説明】
【図１】本発明の一実施形態に係る文書検索装置の機器構成例を示した図。
【図２】本発明の一実施形態に係る文書検索装置の機能ブロック図。
【図３】入力部より入力される特許明細書の例と、文書格納部に格納される特許明細書の具体例を示した図。
【図４】索引作成部における索引作成処理動作を説明するためのフローチャート。
【図５】基本語抽出処理動作を説明するためのフローチャート。
【図６】拡張語抽出処理動作を説明するためのフローチャート。
【図７】図３（ｂ）の特許明細書中の「特許請求の範囲」という項目に書かれた文章を示した図。
【図８】図７の文章に対し形態素解析を行った結果を示した図。
【図９】図８の形態素解析の結果から抽出した索引語リストの一例を示した図。
【図１０】図８の形態素解析の結果から抽出した名詞句リストの一例を示した図。
【図１１】図８の形態素解析の結果から抽出した述語−項リストの一例を示した図。
【図１２】図３（ｂ）の特許明細書中の「発明の実施の形態」という項目に書かれた文章を示した図。
【図１３】図１２の文章に対し形態素解析を施した結果から抽出した述語−項リストの一例を示した図。
【図１４】名詞句の言い換えを抽出するための文字列パターンの一例を示した図。
【図１５】図１２の文章に形態素解析を施した結果から抽出された名詞句の言い換えリストの一例を示した図。
【図１６】図１３の述語−項リストと図１５の名詞句の言い換えリストとを用いて選択した拡張語の一例を示した図。
【図１７】図１６の拡張語を単語に展開した場合を示した図。
【図１８】図９の索引語リストに拡張語を追加して得られた索引語リストの一例を示した図。
【図１９】拡張語選択処理動作を説明するためのフローチャート。
【図２０】頻度表の一例を示した図。
【図２１】索引語の文書頻度の算出結果を示した図。
【図２２】文書検索部における文書検索処理動作を説明するためのフローチャート。
【図２３】図３（ａ）に示す特許明細書中の「特許請求の範囲」という項目に書かれた文章を示した図。
【図２４】図２３の文章に対し形態素解析を施した結果から抽出した索引語リストの一例を示した図。
【図２５】図２３の文章に形態素解析を施した結果から抽出した名詞句リストの一例を示した図。
【図２６】図２３の文章に形態素解析を施した結果から抽出した述語−項リストの一例を示した図。
【図２７】図３（ａ）に示した特許明細書中の「発明の実施の形態」という項目の文章を示した図。
【図２８】図２７の文章に形態素解析を施した結果から抽出した述語−項リストの一例を示した図。
【図２９】図２８の述語−項リストを用いて選択された拡張語の一例を示した図。
【図３０】図２９の拡張語を単語に展開した場合を示した図。
【図３１】図２４の索引語リストに、図３０の拡張語を追加して得られた索引語リストの一例を示した図。
【図３２】他の文書検索装置の構成例を示した図。
【符号の説明】
２０１…出力部
２０２…制御部
２０３…入力部
２０４…索引作成部
２０５…語句抽出部
２０６…基本語抽出部
２０７…拡張語抽出部
２０８…文書検索部
２０９…索引格納部
２１０…文書構造認識部
２１１…文書格納部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a document search method for searching for a document according to a search request from a plurality of documents (keyword search, similar document search) and a document search apparatus using the same.
[0002]
[Prior art]
With the spread of personal computers in recent years, a large amount of digitized documents have been created, and with the spread of computer networks, it has become easier to access these large amounts of digitized documents. However, the more documents that can be accessed, the more difficult it becomes to find a document that the user needs, and there is a possibility that corner information will not be utilized. Therefore, a document search apparatus that selects a user's need from a large number of documents, in particular, a document search apparatus that uses not only bibliographic information such as titles and creators but also full-text search technology using the contents of the document. Demand is increasing.
[0003]
[Problems to be solved by the invention]
In a conventional document search apparatus, a word is extracted by performing a morphological analysis process on a document to be searched, and an index is created by weighting the extracted word / phrase by the appearance frequency or the number of appearance documents in the document. It is generally done. This method of extracting words and indexes from the whole document and creating an index is not important when it is a long document such as a patent specification or academic paper. There is a problem that words that appear are extracted.
[0004]
In order to avoid such a problem, in particular, there is a patent specification or an academic paper as an example of a structured document (structured) document (structured document). There are components for each item such as “claims”, “detailed description of the invention”, “embodiment of the invention”, and in the case of academic papers, there are components such as “abstract” and “text”. Among the constituent elements of the document, claims are created for patent specifications, abstracts are made for academic papers, etc., and words are extracted from only the main constituent elements that briefly express the gist of the document. Sometimes a method is taken. However, since such a part is often described with a word having a higher abstraction level, when the user's search request is described with a more specific word / phrase, there is a high risk of being leaked from the search result.
[0005]
On the other hand, when a user describes a search request with abstract words in consideration of such danger, there is a problem that many unnecessary documents are matched.
[0006]
The present invention has been made in view of such circumstances, and by extracting words that accurately represent the contents of a document and creating an index used for searching the document, the present invention is adapted to the contents of the document. It is an object of the present invention to provide a document search method and a document search apparatus using the document search method that can search documents with high accuracy.
[0007]
[Means for Solving the Problems]
(1) The document search method of the present invention is a document search method for searching for a document according to a search request input from a plurality of documents, wherein the document is a document structured by a plurality of components. , Extracting a first word / phrase from predetermined main components of the document, and a predetermined condition between the first word / phrase from components other than the main component of the document A second phrase that satisfies the above condition is extracted, and the document is searched based on the first and second phrases extracted from each of the plurality of documents and the search request.
[0008]
The document search method of the present invention is a document search method for searching a document similar to a document input from a plurality of documents, wherein the document is a document structured by a plurality of components, From each of the input document and the plurality of documents to be searched, a first word / phrase is extracted from predetermined main constituent elements of the document, and further, constituent elements other than the main constituent elements of the document A second word / phrase that satisfies a predetermined condition between the first word / phrase and the first word / phrase is extracted from each of the input document and the plurality of documents to be searched. The similarity between the first and second words is obtained, and a document similar to the input document is searched from the plurality of documents to be searched.
[0009]
According to the present invention, by extracting the first word (basic word) and the second word (extended word) that accurately represent the contents of the document and creating an index used for searching the document, Enables retrieval of documents with high accuracy in accordance with the contents of the documents.
[0010]
Preferably, a phrase associated with the first phrase is extracted as a second phrase with a predetermined language expression.
[0011]
Preferably, a phrase having the same predicate term as the predicate having the first word as a term is extracted as a second word.
[0012]
(2) The document search apparatus of the present invention is a document search apparatus for searching for a document according to a search request input from a plurality of documents, wherein the document is a document structured by a plurality of components. ,
First extraction means for extracting a first word from the main main constituent elements of the document;
Second extraction means for extracting a second phrase satisfying a predetermined condition with the first phrase from components other than the main component of the document;
Search means for searching for a document based on the first and second phrases extracted from each of the plurality of documents and the search request;
It is characterized by comprising.
[0013]
The document retrieval apparatus of the present invention is a document retrieval apparatus for retrieving a document similar to a document input from a plurality of documents, wherein the document is a document structured by a plurality of components,
A first extraction means for extracting a first word from a predetermined main component of the document from each of the input document and the plurality of documents to be searched; and the input document; Second extraction means for extracting, from each of the plurality of documents to be searched, a second word that satisfies a predetermined condition with the first word from among components other than the main component;
The similarity between the input document and the plurality of documents to be searched for the similarity between the first and second words extracted from each of the documents is obtained, and a document similar to the input document is obtained. A search means for searching among a plurality of documents to be searched;
It is characterized by comprising.
[0014]
According to the present invention, by extracting the first word (basic word) and the second word (extended word) that accurately represent the contents of the document and creating an index used for searching the document, This makes it possible to search for similar documents with high accuracy in accordance with the contents of the documents.
[0015]
Preferably, a phrase associated with the first phrase is extracted as a second phrase with a predetermined language expression.
[0016]
Preferably, a phrase having the same predicate term as the predicate having the first word as a term is extracted as a second word.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0018]
FIG. 1 shows an example of the device configuration of a document search apparatus according to this embodiment. As shown in FIG. 1, the document search apparatus of this embodiment executes a program for executing the document search process of the present invention, an external storage device 102 for storing various data, and a program stored in the external storage device 102. CPU 101, communication device 103 that reads predetermined data from another computer via a communication network such as a public network, a dedicated line, a keyboard 104 for inputting an instruction from a user such as a search request, a mouse 105, a search result, etc. Display devices 106 to be displayed are connected to each other via a bus.
[0019]
FIG. 2 is a functional block diagram of the document search apparatus according to the present embodiment. As shown in FIG. 2, the document search apparatus of this embodiment includes an input unit 201 for inputting an instruction from a user such as a search request, an output unit 203 for displaying a search result, and a document storage for storing a document group to be searched. Section 211, index creation section 204 that creates an index by extracting words from the document group, index storage section 209 that stores an index for searching the document group, and an index stored in the index storage section 209 with reference to the user Search unit 208 for selecting documents that match a search request from the user, a phrase extraction unit 205 for creating an index or extracting a phrase from a user search request, and a component of a structured document to be searched A document structure recognizing unit 210 for recognizing the document, and a control unit 202 for activating the index creating unit 204 and the document searching unit 208 according to an instruction from the user.
[0020]
The phrase extraction unit 205 includes a basic word extraction unit 206 that extracts a basic word from main constituent elements of a document to be searched, and an extended word extraction unit 207 that extracts a phrase to be added to the basic word from elements other than the main constituent element. Consists of.
[0021]
Each component (output unit 201, control unit 202, input unit 203, index creation unit 204, phrase extraction unit 205, document search unit 208, and document structure recognition unit 210) in FIG. 2 is the same as the external storage device 102 in FIG. The index storage unit 209 and the document storage unit 211 are recorded on the computer and are executed and controlled by the CPU 101. The index storage unit 209 and the document storage unit 211 are connected to the external storage device 102 or the external storage device of another computer connected via the communication device 103. It may be built on top. In this case, the input unit 203 receives an instruction from the user such as a search request input via the keyboard 104 and the mouse 105 in FIG. 1, and the output unit 201 displays the search result on the display device 106 in FIG. Is for.
[0022]
With the above configuration, the document search apparatus shown in FIG. 2 searches for a document similar to the content of the input document (similar document search).
[0023]
Here, as a search target document, for example, a description will be given of the operation of each unit when a patent specification as shown in FIG. 3A is input from the input unit 203 and a similar patent search is performed.
[0024]
Prior to the search, an index is created from each of a plurality of patent specifications already stored in the document storage unit 211. The index is created when the control unit 202 calls the index creation unit 204.
[0025]
The index creating unit 204 calls the word / phrase extracting unit 205 to extract words / phrases from the patent specification stored in the document storage unit 211 and creates an index from the extracted words / phrases. The phrase extraction unit 205 includes a basic word extraction unit 206 and an extended word extraction unit 207. The basic word extraction unit 206 calls the document structure recognition unit 210, extracts only the text of the component of the item “Claims” from the components of the patent specification, and searches for the index phrase from the entire extracted text To extract. The extended word extracting unit 207 calls the document structure recognizing unit 210, extracts only the text of the component of the item “invention embodiment” from the components of the patent specification, and extracts it by the basic word extracting unit 206 Extract phrases to expand the indexed phrases.
[0026]
When a patent specification is input from the input unit 203, the control unit 202 calls the document search unit 208. The document search unit 208 calls the phrase extraction unit 205 to extract a phrase from the inputted patent specification. Further, the document search unit 208 refers to the extracted word / phrase and the index stored in the index storage unit 209, so that the input between the patent specification and each document stored in the document storage unit 209 is performed. Calculate similarity. The control unit 202 presents a list of patent specifications having a high degree of similarity to the user from the output unit 201.
[0027]
Next, the processing of the index creation unit 204 will be described in detail by taking as an example the case of creating an index of the patent specification shown in FIG.
[0028]
FIG. 4 shows a processing flow of the index creation unit 204. The index creating unit 204 takes out patent specifications one by one from the document storage unit 211 and creates an index. In the document storage unit 211, one patent specification is stored as one file, and each patent specification is given a unique file name. For example, the patent specification shown in FIG. 3B is given a file name “JP-A-01-99999999.txt”.
[0029]
In the index storage unit 209, since each file is managed by a number, each file is registered with a number (step S2). Next, basic words are extracted (step S3).
[0030]
FIG. 5 shows the flow of processing in step 3 of FIG.
[0031]
In FIG. 5, first, the document structure recognition unit 210 is called to extract only the text of the component of the item “Claims” from the components of the patent specification (step S11). FIG. 7 shows an example of a sentence written in “Claims” taken from the patent specification of FIG. Morphological analysis is performed on the text written in the “claims” (step S12). The method of morphological analysis is widely known and will not be described in detail here.
[0032]
FIG. 8 shows a part of the result of performing the morphological analysis on the text of FIG. In FIG. 8, one morpheme information is output in one line, and is separated by a space from the beginning of the line, morpheme notation, reading, basic form notation, part of speech, part of speech number, fine part of speech, fine part of speech number, inflection type, inflection type Numbers, usage types, and usage type numbers are lined up. When there is no information, “*” is marked.
[0033]
From the result of morphological analysis in step S12, words having parts of speech such as nouns, verbs, adjectives, symbols, and unknown words are extracted as index words (step S13). FIG. 9 shows a part of the list of index words extracted from the result of the morphological analysis of FIG.
[0034]
On the other hand, a noun phrase list is also extracted from the result of the morphological analysis in step S12 (step S14). A noun phrase here refers to a noun, a symbol, an unknown word, an adjective stem, a conjunctive adjunct concatenation, or a continuation through the particle "no". FIG. 10 shows a part of the list of noun phrases extracted from the result of the morphological analysis of FIG. In FIG. 10, when the left side is a noun phrase notation and the noun phrase is composed of a plurality of morphemes, the boundary between the morphemes is indicated by “/”. The right side shows the part of speech of the morphemes that make up the noun phrase. As in the case of the notation, the boundary between morphemes is indicated by “/”.
[0035]
Further, a list of predicates and terms (predicate-term list) in a correspondence relationship is extracted from the result of the morphological analysis in step S12 (step S15). Here, predicates are limited to verbs, and terms are extracted in the form of noun phrases. FIG. 11 shows a part of the predicate-term list extracted from the result of the morphological analysis of FIG. In FIG. 11, for each extracted predicate, the term taken by each predicate is shown on the right side. When one predicate takes multiple terms, the terms are separated by “;”. Each term is described in the form of “notation (part of speech)”, and when one term is composed of a plurality of morphemes, the boundary between the morphemes is indicated by “/”.
[0036]
Returning to the description of FIG. When the basic word extraction is completed in step S3, the index word list extracted in step S13 is expanded based on the list extracted in steps S14 and S15 of FIG. 5 (step S4).
[0037]
FIG. 6 is a flowchart showing the flow of the process in step S4 of FIG.
[0038]
First, the document structure recognizing unit 210 is called to extract only the text written in the “Embodiment of the Invention” from the constituent elements of the patent specification (step S21). FIG. 12 shows a part of a sentence written in “Embodiment of the Invention” extracted from the patent specification of FIG. Morphological analysis is performed on the text written in the “embodiment of the invention” (step S22). The morpheme analysis process may be the same as step S12 in FIG.
[0039]
A predicate-term list is extracted from the result of the morphological analysis in the same manner as in step S15 in FIG. 5 (step S23). FIG. 13 shows a predicate-term list extracted from the result of performing the morphological analysis on the sentence of FIG. The format is the same as in FIG.
[0040]
Subsequently, the paraphrase of the noun phrase is extracted (step S24). The paraphrasing of the noun phrase is realized by preparing a character string pattern as shown in FIG. 14 and collating it with a sentence. For example, in the pattern of the character string of pattern number “1”, when a noun phrase appears before and after the character string “as”, the noun phrase that appears before is used as the extended noun phrase, Extract as an extended noun phrase. By using this pattern, for example, from the part of “prefecture name as a wide-area place name” in the sentence of FIG. 12, “wide-area place name” as an extended noun phrase and “prefecture name” as an extended noun phrase Is extracted. FIG. 15 shows a list showing the correspondence between the extended noun phrase extracted from the sentence of FIG. 12 and the extended noun phrase, that is, a paraphrase list of noun phrases.
[0041]
Next, an extended word is selected using the predicate-term list extracted in step S23 and the noun phrase paraphrase list extracted in step S24 (step S25).
[0042]
FIG. 19 is a flowchart showing the extended word selection process in step S25 of FIG. As an extended word selection process, there are a case where a predicate-term list is used and a case where a paraphrase list of noun phrases is used. Here, in FIG. 19, a case where a predicate-term list is used will be described as an example.
[0043]
One set of predicate and term in the predicate-term list extracted from the text of the “Embodiment of the Invention” is extracted one by one, and the text of “Claims” in step S15 of FIG. Are collated with the predicates in the predicate-term list extracted from (step S31). That is, one set of a predicate and a term is extracted from the predicate-term list extracted from the sentence “Embodiment of the Invention” (step S32). Then, it is checked whether or not the same predicate as the predicate is in the predicate in the predicate-term list extracted from the sentence “Claims”. If there is not the same, the process returns to step S31, and the next set of predicate and term is extracted from the predicate-term list extracted from the sentence of the “Embodiment of the Invention”.
[0044]
When there is the same thing (step S33), the term of the said predicate in the predicate-term list extracted from the sentence of "embodiment of invention" is investigated one by one (step S34, step S35). That is, the same noun phrase list extracted from the sentence “claims” in step S14 of FIG. 5 is the same as the term of the predicate in the predicate-term list extracted from the sentence “embodiment of the invention”. It is checked whether there is any (step S36). If there is the same item, the process returns to step S34 to check the next term. If there is no same item, the process proceeds to step S37, where the term is registered as an extended word, and the process returns to step S34 again to check the next term. The processes in steps S35 to S37 are repeated until there are no more predicate terms in the predicate-term list extracted from the sentence “Embodiment of the Invention”.
[0045]
As described above, an extended word may be selected using only the predicate-term list extracted from the sentence of the “Embodiment of the Invention” shown in FIG. An extended word may be selected using a paraphrase list of noun phrases. Furthermore, an extended word may be selected using both a predicate-term list and a paraphrase list of noun phrases.
[0046]
That is, after an extended word is selected by the procedure as shown in FIG. 9, next, the noun phrase as shown in FIG. 15 extracted from the sentence “Embodiment of the Invention” in step S24 of FIG. It is checked whether or not the same noun phrase as the extended original noun phrase in the paraphrase list exists in the noun phrase list extracted from the sentence “Claims” in step S14 of FIG. If there is the same thing, the extended noun phrase corresponding to the extended source noun phrase in the paraphrase list of the noun phrase is set as the extended word. At this time, the extended noun phrase already selected as the extended word is ignored.
[0047]
In addition, after an extended word is selected using a paraphrase list of noun phrases as shown in FIG. 15, an extended word may be selected using a predicate-term list as shown in FIG.
[0048]
FIG. 16 shows an extended word obtained by using both the predicate-term list of FIG. 13 and the paraphrase list of the noun phrase of FIG.
[0049]
Returning to the description of FIG. 6, in step S26, the expanded word in the form of a noun phrase selected in step S25 is expanded into a word. For example, FIG. 17 shows an expanded word of FIG. 16 expanded into words. Here, the expanded word frequency is calculated assuming that the expanded word appearance frequency is uniformly “1”.
[0050]
An index word list as shown in FIG. 9 extracted in step S13 in FIG. 5 and an expanded word (see FIG. 17) extracted in steps S25 to S26 in FIG. As shown in FIG. In FIG. 18, index word numbers “18” to “20” are newly added words, that is, expanded words.
[0051]
Here, it returns to description of FIG. 4 again. The next process is step S5. In step S5, the index word obtained up to step S4 and the information of its appearance frequency are written in the frequency table. The frequency table created here corresponds to an index stored in the index storage unit 209.
[0052]
An example of the frequency table is shown in FIG. The vertical axis takes the file number to identify the file in which each document is stored, and the horizontal axis takes the word number to identify each word extracted as a basic word and extended word. It shows how many times the word appears.
[0053]
The above processing (see FIG. 4) is executed for all patent specification files in the document storage unit 211. When the processing is completed for all the patent specification files in the document storage unit 211, the document frequency of each index word is counted in step S6. The document frequency of each index word is obtained by reading the frequency table vertically and counting the number of files having an appearance frequency of “1” or more. An example of calculating the document frequency of each index word is shown in FIG.
[0054]
Next, the processing of the document search unit 208 will be described in detail by taking as an example a case where a patent specification similar to the patent specification shown in FIG. FIG. 22 shows a processing flow of the document search unit 208.
[0055]
A basic word is extracted from the patent specification as shown in FIG. 3A input from the input unit 203 (step S41). The process in step S41 is the same as the process in step S3 of FIG. When the patent specification shown in FIG. 3A is used as input, information obtained in the basic word extraction process is shown in FIGS. FIG. 23 is a sentence “claims” extracted from the patent specification shown in FIG. FIG. 24 is an index word list extracted from the result of performing morphological analysis on the sentence of FIG. FIG. 25 is a noun phrase list extracted from the result of performing morphological analysis on the sentence of FIG. FIG. 26 is a predicate-term list extracted from the result of performing the morphological analysis on the sentence of FIG. Similarly, an extended word is extracted from the patent specification input from the input unit 203 in step S42. The process in step S42 is the same as step S4 in FIG.
[0056]
When the patent specification shown in FIG. 3A is used as input, data obtained in the process of the extended word extraction process are shown in FIGS. FIG. 27 is a text of an “Embodiment of the Invention” extracted from the patent specification shown in FIG. FIG. 28 is a list of predicates-terms extracted from the text of FIG. No paraphrase of noun phrase was extracted from the text of FIG. FIG. 29 is an extension word selected using the predicate-term list in FIG. FIG. 30 is an expanded word of FIG. 29 expanded into words. FIG. 31 is obtained by adding the extended word of FIG. 30 to the index word list of FIG.
[0057]
In step S43, using the phrases extracted in steps S41 and S42, referring to the index word list stored in the index storage unit 209, the similarity with each patent specification stored in the document storage unit 211 is determined. calculate. For example, the similarity between the patent specification of FIG. 3A and the patent specification of FIG. 3B is calculated by comparing the index word of FIG. 31 with the index word of FIG. Various methods for calculating the similarity between documents using frequency information of words appearing in the document are known, but any method may be used here.
[0058]
For example, for each document, the tf · idf value of each index word is obtained from the following equation.
[0059]
tf · log (N / df)
tf: appearance frequency of the index word in the document
N: Total number of documents
df: document frequency of the index word
Then, a feature vector having the index word number as a dimension and the tf · idf value of the index word corresponding to the index word number as an element of each dimension is obtained. A feature vector is obtained for each of the input document (that is, here, the patent specification in FIG. 3A) and the document to be searched (for example, the patent specification in FIG. 3B). Alternatively, instead of the tf · idf value, the appearance frequency of the index word in the document with respect to the index word number for each document may be used as an element of the feature vector.
[0060]
A part of the feature vector of the patent specification shown in FIG. 3B created using the appearance frequency of the index word based on the index word list shown in FIG.
[0061]
[Expression 1]

[0062]
Then, for each of the input document (that is, here, the patent specification of FIG. 3A) and the document to be searched (for example, the patent specification of FIG. 3B), a feature vector is obtained, An inner product may be calculated between them, and the similarity may be calculated between the input document and each document stored in the document storage unit 211. Note that the cosine distance may be obtained instead of the inner product. In this case, the greater the similarity value, the higher the similarity.
[0063]
The similarity between the input document calculated in step S43 and each document stored in the document storage unit 211 is sorted in descending order of the value (step S44). The file name of the (integer) patent specification is output to the output unit 201 (step S45).
[0064]
According to the processing described above, the accuracy of document search can be improved. For example, in both the patent specification of FIG. 3A and the patent specification of FIG. 3B, the prefecture name and the city name are associated with each other and stored in the dictionary. In both claims, the words “prefecture name” and “city name” do not appear. Therefore, in the conventional document search apparatus that does not have the extended word extraction unit 207, when the index word is extracted only from the sentences in the claims, the patent specification of FIG. 3 (a) and the patent specification of FIG. 3 (b) A high degree of similarity with a book cannot be obtained. On the other hand, in the document retrieval apparatus of the present invention, “prefecture name” and “name” that do not appear in the scope of claims in both the patent specification of FIG. 3A and the patent specification of FIG. Since the word “city name” is added to the index word as an extension word, a high degree of similarity can be obtained between the two.
[0065]
In the above embodiment, the case of a search request for a document similar to a document input as a search request has been described as an example. However, the present invention is not limited to this case, and various modifications can be applied. For example, the search processing in the document search unit 208 is performed in the same manner as described above (that is, the degree of similarity between a keyword and an index word) is obtained even in the case of a search request for a document that matches an input keyword. It is possible to search.
[0066]
In the above embodiment, only nouns and verbs are extracted as index words. However, the present invention is not limited to this case, and various modifications can be applied. For example, in addition to these, for example, adjectives, adverbs, etc. may be selected, or verbs may not be selected.
[0067]
In addition, here, since the case where the document to be searched is a patent specification is described as an example, the basic word is derived from the sentence of the item “Claims” that most appropriately describes the characteristics of the content. In this case, an extended word related to the basic word (for example, an extended word expressing the basic word more specifically) is extracted from the sentence of the item “Embodiment of the Invention”. It is not limited. If the document to be searched is an academic paper, the basic word may be extracted from the “abstract”, and the extended word related to the basic word may be extracted from the text. Thus, no matter what the document to be searched is, the basic word is extracted from the component that best describes the characteristics of the content of the document, and the extended word is more detailed than that. It is desirable to extract from the components that are described.
[0068]
FIG. 32 schematically shows the configuration of another document search apparatus that performs similar document search to which the document search apparatus described in the above embodiment is applied. In the similar document search apparatus shown in FIG. 32, first, prior to searching for similar documents of the input document, the documents are roughly classified. For example, it is assumed that a plurality of classes (for example, electricity, machine, chemistry, etc.) are prepared according to the contents of a document. Each class has, for example, a dictionary that lists frequently used words in documents belonging to the class, and collates the words in this dictionary with the words in the input document (calculates the similarity). Identify the most similar class. The words appearing in the document that can be obtained at this time and their appearance frequencies may be used when creating the index word list as shown in FIG. 9 described above.
[0069]
In this broad classification processing unit, a document whose class is specified is classified into a more detailed subclass based on, for example, words in the document and appearance frequency, and further, a document whose subclass is specified is more detailed. In addition, the identified documents of the group may be classified into more detailed subgroups.
[0070]
Next, the similar document search processing unit searches for similar documents described in the above embodiment, and outputs a list of searched similar documents.
[0071]
By feeding back the search results here to the broad classification processing and similar document search processing, it is possible to search for similar documents with higher accuracy (high hit rate). That is, for example, a word extracted from an input document obtained in the broad classification process is added to a dictionary of subgroups, groups, subclasses, and classes to which the document belongs. In addition, a word that is present in the input document but not in the similar document is added to the index word list of the retrieved similar document.
[0072]
The document retrieval apparatus shown in FIG. 32 is also stored as a program that can be executed by a computer in a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), semiconductor memory, etc. You can also
[0073]
As described above, according to the above-described embodiment, an index word is extracted from a sentence of a predetermined component in a document, and an appropriate word is extracted as an extension word from the sentence of another component element. As a result, it is possible to select a document that matches the user's search request from a large number of documents with high accuracy.
[0074]
【The invention's effect】
As described above, according to the present invention, it is possible to search for a document with high accuracy in accordance with the content of the document.
[Brief description of the drawings]
FIG. 1 is a diagram showing an example of the device configuration of a document search apparatus according to an embodiment of the present invention.
FIG. 2 is a functional block diagram of a document search apparatus according to an embodiment of the present invention.
FIG. 3 is a diagram showing an example of a patent specification input from an input unit and a specific example of a patent specification stored in a document storage unit.
FIG. 4 is a flowchart for explaining an index creation processing operation in an index creation unit.
FIG. 5 is a flowchart for explaining a basic word extraction processing operation;
FIG. 6 is a flowchart for explaining an extended word extraction processing operation;
7 is a diagram showing a sentence written in the item “Claims” in the patent specification of FIG. 3B;
FIG. 8 is a diagram showing a result of performing morphological analysis on the sentence of FIG. 7;
9 is a diagram showing an example of an index word list extracted from the result of the morphological analysis of FIG.
10 is a diagram showing an example of a noun phrase list extracted from the result of the morphological analysis in FIG. 8. FIG.
11 is a diagram showing an example of a predicate-term list extracted from the result of morphological analysis in FIG. 8. FIG.
FIG. 12 is a diagram showing a sentence written in the item “Embodiment of the Invention” in the patent specification of FIG. 3B;
13 is a diagram showing an example of a predicate-term list extracted from the result of performing morphological analysis on the sentence of FIG.
FIG. 14 is a diagram showing an example of a character string pattern for extracting paraphrases of noun phrases.
15 is a diagram showing an example of a paraphrase list of noun phrases extracted from the result of performing morphological analysis on the sentence of FIG.
16 is a diagram showing an example of an extended word selected using the predicate-term list of FIG. 13 and the noun phrase paraphrase list of FIG. 15;
FIG. 17 is a diagram showing a case where the extended word in FIG. 16 is expanded into words.
18 is a diagram showing an example of an index word list obtained by adding an extended word to the index word list of FIG.
FIG. 19 is a flowchart for explaining an extended word selection processing operation;
FIG. 20 is a diagram showing an example of a frequency table.
FIG. 21 is a diagram showing a calculation result of index word document frequency.
FIG. 22 is a flowchart for explaining a document search processing operation in a document search unit.
FIG. 23 is a diagram showing a sentence written in the item “Claims” in the patent specification shown in FIG.
24 is a diagram showing an example of an index word list extracted from a result of performing morphological analysis on the sentence of FIG. 23. FIG.
FIG. 25 is a diagram showing an example of a noun phrase list extracted from the result of performing morphological analysis on the sentence of FIG. 23;
26 is a diagram showing an example of a predicate-term list extracted from the result of performing morphological analysis on the sentence of FIG.
27 is a diagram showing a sentence of an item “Embodiment of the Invention” in the patent specification shown in FIG.
28 is a diagram showing an example of a predicate-term list extracted from the result of performing morphological analysis on the sentence in FIG. 27;
FIG. 29 is a diagram showing an example of an extension word selected using the predicate-term list in FIG. 28;
30 is a diagram showing a case where the extended word in FIG. 29 is expanded into words.
31 is a diagram showing an example of an index word list obtained by adding the extended word of FIG. 30 to the index word list of FIG.
FIG. 32 is a diagram showing a configuration example of another document search apparatus.
[Explanation of symbols]
201: Output unit
202 ... Control unit
203 ... Input unit
204 ... Index creation section
205 ... Phrase extraction unit
206: Basic word extraction unit
207 ... Extended word extraction unit
208: Document search section
209 ... Index storage unit
210: Document structure recognition unit
211 ... Document storage unit

Claims (12)

Document storage means for storing a plurality of documents structured by a plurality of components;
Index creation means for creating an index word list from each document stored in the document storage means;
Index storage means for storing an index word list of each document created by the index creation means;
Search means for searching for a document according to the input search request from among the documents stored in the storage means;
A document search method in a document search apparatus comprising:
A first extraction step in which the index creating means extracts a first word / phrase as an index word from predetermined main components of the document ;
The index creation means extracts a second word / phrase associated with the first word / phrase as an index word in a predetermined linguistic expression from constituent elements other than the main constituent element of the document . Extraction steps of
The index creating means creates an index word list including the first word and the second word and the appearance frequency of the first word and the second word in the document, and stores the index memory. Storing in the means;
The search means uses the index word list of each document stored in the index word storage means to determine the similarity between the keyword input as the search request and the index word in each index word list. A search step for searching a document that matches the input keyword by searching among a plurality of documents stored in the document storage means;
Search method including documents. Document storage means for storing a plurality of documents structured by a plurality of components;
Index creation means for creating an index word list from each document stored in the document storage means;
Index storage means for storing an index word list of each document created by the index creation means;
Search means for searching for a document according to the input search request from among the documents stored in the storage means;
A document search method in a document search apparatus comprising:
A first extraction step in which the index creating means extracts a first word / phrase as an index word from predetermined main components of the document ;
The index creation means extracts a second word / phrase associated with the first word / phrase as an index word in a predetermined linguistic expression from constituent elements other than the main constituent element of the document . Extraction steps of
The index creating means creates an index word list including the first word and the second word and the appearance frequency of the first word and the second word in the document, and stores the index memory. Storing in the means;
A third extraction step in which the search means extracts a third word as an index word from the main components of the document structured by a plurality of components input as the search request;
The search means extracts, as an index word, a fourth word / phrase associated with the third word / phrase in a predetermined language expression from components other than the main component of the input document. A fourth extraction step;
The search means creating an index word list including the third word and the fourth word and the appearance frequency of the third word and the fourth word in the input document; ,
The search means includes an index word in the index list created from the input document and its appearance frequency, and an index word in the index word list of each document stored in the index word storage means and its appearance frequency. A search step for searching a document similar to the input document from a plurality of documents stored in the document storage means by calculating a similarity between
Search method including documents. The second extraction step, according to claim 1 or 2 document search method wherein extracting the second word that is a section of the same predicate predicates to claim the first word. 3. The document search method according to claim 2, wherein the fourth extraction step extracts a fourth word / phrase having a predicate term that is the same as a predicate having the third word / phrase as a term.   Document storage means for storing a plurality of documents structured by a plurality of components;
  The first word / phrase is extracted as an index word from predetermined main constituent elements of each document stored in the document storage means, and predetermined from constituent elements other than the main constituent elements of the document. Extracting means for extracting, as an index word, a second phrase associated with the first phrase in a given language expression;
  For each document stored in the document storage means, the first word and the second word extracted from the document, and the appearance frequency of the first word and the second word in the document Creating means for creating an index word list including
  Index word storage means for storing the index word list created by the creation means for each document stored in the document storage means;
  By using the index word list of each document stored in the index word storage means to calculate the similarity between the keyword input as a search request and the index word in each index word list, Search means for searching a document matching the input keyword from a plurality of documents stored in the document storage means;
  A document retrieval apparatus comprising:   Document storage means for storing a plurality of documents structured by a plurality of components;
  The first word / phrase is extracted as an index word from predetermined main constituent elements of each document stored in the document storage means, and predetermined from constituent elements other than the main constituent elements of the document. First extraction means for extracting, as an index word, a second word / phrase associated with the first word / phrase in a given language expression;
  For each document stored in the document storage means, the first word and the second word extracted from the document, and the appearance frequency of the first word and the second word in the document First creation means for creating an index word list including:
  Index word storage means for storing the index word list created by the first creation means for each document stored in the document storage means;
A third word / phrase is extracted as an index word from the main constituent elements of the document structured by a plurality of constituent elements input as a search request, and the constituent elements other than the main constituent elements of the document are extracted. Second extraction means for extracting, as an index word, a fourth phrase associated with the third phrase in a predetermined language expression;
  Second creation means for creating an index word list including the third word and the fourth word, and the frequency of appearance of the third word and the fourth word in the input document;
Similarity between the index word in the index list created from the input document and its appearance frequency and the index word in the index word list of each document stored in the index word storage means and its appearance frequency Search means for searching a document similar to the input document from a plurality of documents stored in the document storage means by calculating a degree;
  A document retrieval apparatus comprising: The extraction means, the document search apparatus according to claim 5, wherein the extracting the second word that is a section of the same predicate predicates to claim the first word.   The first extraction means extracts a second phrase that is in the same predicate term as a predicate whose term is the first phrase;
  The second extraction means has a predicate term that is the same as the predicate having the third word as a term. 7. The document retrieval apparatus according to claim 6, wherein four words / phrases are extracted. Computer
  Document storage means for storing a plurality of documents structured by a plurality of components;
  The first word / phrase is extracted as an index word from predetermined main constituent elements of each document stored in the document storage means, and predetermined from constituent elements other than the main constituent elements of the document. Extracting means for extracting, as an index word, a second word / phrase associated with the first word / phrase in a given language expression;
  For each document stored in the document storage means, the first word and the second word extracted from the document, and the appearance frequency of the first word and the second word in the document Creating means for creating an index word list including
  Index word storage means for storing the index word list created by the creation means for each document stored in the document storage means,
  By using the index word list of each document stored in the index word storage means to calculate the similarity between the keyword input as a search request and the index word in each index word list, Retrieval means for retrieving a document that matches the input keyword from a plurality of documents stored in the document storage means;
  A computer-readable storage medium storing a program for functioning as a computer.   Computer
  Document storage means for storing a plurality of documents structured by a plurality of components;
  The first word / phrase is extracted as an index word from predetermined main constituent elements of each document stored in the document storage means, and predetermined from constituent elements other than the main constituent elements of the document. First extraction means for extracting, as an index word, a second word / phrase associated with the first word / phrase in a given language expression;
  For each document stored in the document storage means, the first word and the second word extracted from the document, and the appearance frequency of the first word and the second word in the document A first creation means for creating an index word list including
  Index word storage means for storing the index word list created by the first creation means for each document stored in the document storage means,
A third word is extracted as an index word from the main constituent elements of the document structured by a plurality of constituent elements inputted as a search request, and the constituent elements other than the main constituent elements of the document are extracted. Second extraction means for extracting, as an index word, a fourth phrase associated with the third phrase in a predetermined language expression;
  Second creation means for creating an index word list including the third word and the fourth word and the appearance frequency of the third word and the fourth word in the input document;
Similarity between the index word in the index list created from the input document and its appearance frequency and the index word in the index word list of each document stored in the index word storage means and its appearance frequency Search means for searching a document similar to the input document from a plurality of documents stored in the document storage means by calculating a degree;
  A computer-readable storage medium storing a program for functioning as a computer. Said extraction means, storage medium of claim 9, wherein the extracting the second word that is a section of the same predicate predicates to claim the first word.   The first extraction means extracts a second phrase that is in the same predicate term as a predicate whose term is the first phrase;
  11. The storage medium according to claim 10, wherein the second extraction unit extracts a fourth word / phrase having a predicate term that is the same as a predicate having the third word / phrase as a term.

Images