JP2005301856A - Method and program for document retrieval, and document retrieving device executing the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve precision of retrieval of a large quantity of text documents by input of a text and to save the trouble to specify a desired document from a retrieval result. <P>SOLUTION: A sum set of higher documents as results of retrieval based upon a keyword extracted from a plurality of partial character strings constituting an input text is generated, and a product set of retrieval results based upon a keyword extracted from the whole input text is generated. A document, specified by a user, which has retrieval order by a partial character string than retrieval order by the whole input text is specified, and the partial character string or keyword information used for the retrieval is displayed to the user. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a document search method, a document search program, and a document search apparatus that executes the document search method for searching a document including text data.

  As a technique for searching for a desired document from a large number of documents including text data, there is a method of inputting text (sentence or keyword set) as a search condition and searching for a document similar to the content of the input text. That is, a keyword vector composed of one or more weighted keywords extracted from the input text and one or more weighted keywords extracted in advance from each document constituting the search target document group. This is a technique for quantifying the similarity of contents between an input text and a search target document by calculating an inner product or cosine between keyword vectors and outputting a document having a high similarity as a search result.

  In the above method, when extracting keywords from the input text or the search target document, the word type (hiragana / katakana / kanji / alphabet / special symbol, etc.) and word information (headings, parts of speech) defined in the word dictionary are extracted. Etc.). Also, words that are clearly inappropriate as keywords are removed as unnecessary words.

  In addition, in the above method, a method called “TF / IDF method” is generally used when assigning weights to keywords. That is, a method of calculating a weight to be given to a keyword using two values of an appearance frequency (TF) of a keyword in a text and an inverse number (IDF) of the number of appearance documents of the keyword in the search target document group. is there. In document retrieval systems that are widely available on the market, these values are not used as they are, but are generally corrected by applying a logarithmic function (log) or the like. Further, since the appearance frequency (TF) increases as the text becomes longer, the value of the appearance frequency (TF) is often corrected by the text length or the like.

  Further, in the above method, as disclosed in Non-Patent Document 1, for example, a method is known that uses a structural characteristic of an input text or a search target document when extracting a keyword. For example, when the search target document is a patent specification, the range for extracting keywords is limited to “claims”, “summary”, and the like.

  In addition, when a weight is assigned to an extracted keyword, as disclosed in Non-Patent Document 2, a method is known in which the weight value is changed depending on where the keyword appears in the document. . For example, when the search target document is a patent specification, the noun appearing at the end of the claim (after the expression “characterized by”) by increasing the value of the weight of the keyword appearing in the “invention name” This is a technique of increasing the weight of keywords included in a phrase.

  Furthermore, as disclosed in, for example, Japanese Patent Application Laid-Open No. 11-085786, a technique for further expanding a search based on a search result is also known.

Japanese Patent Laid-Open No. 11-085786

Keio Mizuno: “On an automatic retrieval system for similar documents”, Journal of the Japan Patent Office Technical Conference, No.223, pp.9-15, 2002.5.15 Hisao Mase et al .: "Proposal of patent theme classification method and its evaluation experiment", Transactions of Information Processing Society of Japan, Vol.39, No.7, pp.2207-2216, July 1998

  In the above prior art, the search is performed using all the keywords included in the text input by the user. In this case, important keywords that are closely related to a desired document and noise keywords that are not related are often mixed. As a result, if the noise keyword has more influence on the similarity calculation than the important keyword, the desired document will not be output at the top of the search results even though the important keyword is included, resulting in a search omission. (Problem 1).

Further, in the above conventional technique, it takes a long time to find only the document desired by the user from the document search results (Problem 2).
The present invention solves the problem that a search omission is caused by the presence of a noise keyword even though an important keyword is included, and a document search that realizes a higher search accuracy than conventional methods with less omission of search. It is an object to provide a method, a document search program, and a document search apparatus that executes the method.

  In the present invention, in order to solve the above problem 1, a keyword is extracted from text input by a user, a weight corresponding to the importance of the extracted keyword is assigned, and each document in the search target document group is changed to a weighted keyword. In the document search method for outputting a corresponding document as a search result, the input text is divided into a plurality of partial character strings, and attention is paid to partial character string keywords extracted from each of the divided partial character strings. A search is executed using the search result searched and the search result searched focusing on the keyword of the whole input text.

  In addition, in order to solve the above-described problem 2, the present invention executes the search for only the documents selected and specified by the user from the document set output as the search results. Using the search result focused on and the search result focused on the partial character string keyword above, the partial search result with the higher ranking of the search result based on the search focused on the partial character string keyword is identified and used. Alternatively, partial character strings are specified, and these are displayed to the user in a different form. Further, the partial character string or keyword selected by the user using the search result focusing on the partial character string or keyword set selected by the user and the search result focusing on the keyword of the entire input text. A partial search result with a higher search result ranking by a search focusing on a set of is identified, a document with a higher search result ranking by a used partial string or keyword set is identified, and the identified Display the document to the user.

  According to the present invention, although an important keyword is included, a search omission is less likely to occur due to the presence of a noise keyword, and an omission of search is reduced. Further, a desired document can be easily found from the search result.

  Embodiments of the present invention will be described below in detail with reference to the drawings. Note that the present invention is not limited thereby.

  In the present embodiment, a large number of patent publication gazette text groups are searched and a document similar to the content of the claim text input by the user (document that invalidates the content of the invention described in the input claim) A patent search system for acquiring This system focuses on text data input by users and keywords appearing in patent publication gazette text groups to be searched, and documents similar to the contents of the input text using the “TF / IDF method” described above. Search for. In this embodiment, Japanese text is targeted, but text in English or the like is also applicable.

  FIG. 1 is a diagram showing an outline of the present embodiment. This system takes a text as input and outputs a document search result list similar to the text content. In the conventional method, the weighted keyword 102 is extracted from the entire input text 101, the search execution 103 is performed, and the search result 104 is output. In the present embodiment, the input text 101 is divided into partial character strings 101a, 101b, 101c, and 101d, the partial character string keywords 102a, 102b, 102c, and 102d are extracted from the partial character strings, and based on the partial character string keywords. Then, search execution 103 is performed, and partial search results 104a, 104b, 104c, and 104d are obtained. Then, the partial search result union set 105 is generated by merging these top N search results (here, N = 100). Finally, the search result (the entire search result 104) for the entire input text is filtered by the partial search result union set 105, and the documents included in the partial search result union set 105 are selected from the entire search result 104 in the order. The top M items (here, M = 200) are output as the final search results 106. In the present embodiment, the “final” output result is used. However, the search may be continued by inputting another search method using the document group appearing in the final search result 106 as a search target.

  The number written in the document ID used in FIG. 1 is equal to the rank in the overall search result 104. That is, the document “Doc100” is a document ranked 100th in the overall search result. Here, focusing on the partial search result 104a, the document Doc 164 has a search result for the entire input text in the 164th place, but it can be seen that the search result in the partial character string 101a is in the third place. As a result, the result of filtering with the partial search result union 105 is 120th, and a document that has a low rank in the search by the whole text but a high rank in the search by the partial character string is displayed in the rank in the search result display. Can be presented. As a result, the user's attention can be further attracted, so that omissions in search can be reduced.

  FIG. 2A is a block diagram in which the system described in the present embodiment is displayed in association with programs relating to user operations, various data, and data processing.

  A user inputs text data as search conditions, for example, claim text, via the input / output unit 1 of the system. The input text is stored and held in the input text 2. Here, the text may be input by the user from the keyboard, or may be input by dragging and dropping text data using a mouse or the like, voice, pen, OCR, or the like. Further, the text data may be automatically passed from another program.

  After inputting the claim text data as a search condition, when the user instructs execution of the search, the keyword extraction program 3 extracts the keyword, and gives a weight to each keyword. Hereinafter, the keyword extraction program 3 will be described.

  First, in the partial character string generation 31, the input text 2 is divided into syntactically / semantically / represented characters or symbols. In the present embodiment, the text is divided using a character string designated in advance as a boundary condition. These boundary conditions are stored as the partial character string extraction collation character string 172 in the parameter setting data 17 (see FIG. 7). In FIG. 7, symbol characters such as a punctuation mark, a punctuation mark, a comma, and a semicolon, a line feed code (¥ n), and a specific character string such as “characteristically” are defined as boundary conditions. Of course, in addition to this method, it is possible to forcibly divide by the number of characters or words defined in advance from the beginning of the text. You can specify string boundaries, or morphologically and parse the text to identify paragraphs (which can be easily identified from line breaks and indents), sentences (separated by breakpoints), clauses and phrases The text may be divided by these units. In some of the above methods, it is necessary to perform the morphological analysis 32 on the input text 2, and in this case, the processing order of the partial character string generation 31 and the morphological analysis 32 is reversed. Not a problem.

  Next, in the morphological analysis 32, the input text 2 and the grammar dictionary 4 in which attribute data relating to each word is registered and the grammar dictionary 5 in which the connection cost between parts of speech and grammar rules are defined are referred to. Each of the partial character strings is divided into words, and the heading, standard form, and part-of-speech data for each word are obtained from the word dictionary 4 and stored in the word table 8 in the order in which the words appear. Since the morphological analysis 32 is a well-known technique widely used in the field of document processing, details of its processing algorithm will not be described any further.

  Next, in the unnecessary word removal 33, it is determined whether each of the words divided by the morphological analysis 32 is an unnecessary word registered in the unnecessary word dictionary 6, and the word registered in the unnecessary word dictionary 6. Is excluded from keyword candidates as an unnecessary word. In the present embodiment, the unnecessary word dictionary 6 has a data structure in which one unnecessary word character string is stored in one record, and an unnecessary word (in the case of a patent publication, “ Invention, “claims”, “features”, etc.) and unnecessary words (“thing”, “thing”, “is”, etc.) that are generally used frequently and do not depend on the field or content of the search target document. The determination result as to whether or not each word obtained by the morphological analysis 32 is an unnecessary word is stored in the word table 8 as an unnecessary word flag (an unnecessary word if the value is 1). Although the unnecessary word dictionary 6 is different from the word dictionary 4 in this embodiment, information indicating whether or not a word is an unnecessary word may be embedded in the word dictionary 4 and defined. .

  Next, in the keyword specification 34, for the keyword candidates (words whose unnecessary word flag value is 0) stored in the word table 8, a word having a predefined part of speech is recognized as a keyword. In this embodiment, the part-of-speech information that defines a keyword is listed in the parameter setting data 17 (keyword part-of-speech list 171 in FIG. 7), but part-of-speech information that can be a keyword may be embedded in the processing program. The determination result as to whether or not each word obtained by the morphological analysis 32 is a keyword is stored in the word table 8 as a keyword flag (a keyword if the value is 1).

  Next, in the keyword weighting 35, for the extracted keyword, the appearance frequency (TF) in the input text and the reciprocal number (IDF) of the number of appearance documents in the search target document group stored in the document data 13 are used. The weight corresponding to the importance of the keyword is calculated and assigned to the keyword. In the present embodiment, the weight is calculated by equation (1).

Here, DF is the number of keyword appearing documents in the search target document group stored in the document data 13, and N is the total number of search target documents stored in the document data 13.

  In the keyword weighting 35, the appearance frequency is counted for each keyword stored in the word table 8, the number of appearance documents (DF) for each keyword stored in the document data index data 14 is acquired, and The weight of the keyword is calculated by a calculation formula. The standard form of each keyword, the appearance frequency, the number of appearing documents, and the weight value are stored in the keyword table 9.

  Once the keyword extraction program 3 determines the keyword and its weight, the search execution program 10 next searches the document data 13 using the weighted keyword stored in the keyword table 9. In this system, a weighted keyword is extracted in advance from each document stored in the document data 13 by the index generation program 15 and stored in the document index data 14. Also, the number of appearance documents of each keyword is calculated and stored in the document index data 14. An angle formed by a keyword vector composed of weighted keywords stored in the keyword table 9 (weight value is a vector component) and a keyword vector composed of weighted keywords stored in the document index data 14 And the similarity of the document (value range is between -1 and 1). Various methods have been proposed for the similarity calculation method, but the present invention does not particularly limit the similarity calculation method, and thus will not be described in further detail.

  The results searched by the search execution program 10 are paired with the document ID and similarity, sorted in descending order of similarity, and stored in the search result storage table 11.

  Next, the partial search result union set generation program 7 extracts the document of the number of cases defined in advance by the user in the parameter setting data 17 from the top of each of the partial character string search results stored in the search result, and duplicates them. The partial search result union 16 is generated by collecting and uniqueizing the document IDs.

  Next, in the search result generation display program 12, the product of the entire search result for the entire input text stored in the search result storage table 11 and the partial search result union set 16 generated by the partial search result union generation program 7. A set is generated, a final search result is generated in a form that maintains ranking in the overall search result, and a search document ID and a similarity score are displayed to the user via the input / output unit 1.

The system described in this embodiment described with reference to FIG. 2A is configured by an electronic computer. FIG. 2B is a block diagram showing the system described in this embodiment described in FIG. 2A as a configuration of an electronic computer, and the same components are denoted by the same reference numerals. Reference numeral 200 denotes a system bus. The system bus 200, a keyboard _{1 1} as an input means, a mouse _{1 2} are connected, the printing unit _{1 3} as an output means, the display means _{1 4} is connected. Further, a central processing unit (CPU) 201, a memory work area 203, and a memory storage area 204 are connected to the system bus 200. Here, an example in which a network 207 is connected to the system bus 200 and a client computer 205 is connected to the other end is shown. Various processes described with reference to FIG. 2A are executed by a client 205 connected via a network 207 with the system configuration illustrated in FIG.

  Various processes described with reference to FIG. 2A are executed by the CPU 201 by reading the necessary programs and data stored in the storage area 204 into the work area 203.

  FIG. 3 is a diagram illustrating an example of the configuration of the input text 2. In the present embodiment, description will be made assuming that the input text 2 in FIG. 3 has been input. The input text 2 is an arbitrary sentence. There may be a line break in the middle.

  FIG. 4 is a diagram illustrating a configuration example of the word table 8 corresponding to the input text 2. The word table 8 includes a partial character string ID 801 in which the word appears, a word heading 802, a standard form 803, a part of speech 804, an unnecessary word flag 805 for identifying whether or not the word is an unnecessary word, and whether it is a keyword. And is stored in the order of word appearance in the input text 2. Here, the partial character string ID 801 takes an integer value of 0 or more for each unit divided by the partial character string extraction collation character string 172 (see FIG. 7), and the value increases in the order of appearance in the input text 2. .

  FIG. 5 is a diagram illustrating an example of the configuration of the keyword table 9. The keyword table 9 is output from the keyword extraction program 3. The keyword table 9 includes a partial character string ID 901 in which the keyword appears, a standard form 902 serving as a keyword character string, a TF 903 equal to the appearance frequency of the keyword, and IDFs 904 and TF 903 calculated based on the reciprocal of the number of appearance documents of the keyword. And a weight 905 calculated from the IDF 904. Here, the value of the weight 905 is calculated by the above-described equation (1). Here, the partial character string ID 901 takes an integer value of 0 or more, and the value increases in the order of appearance in the input text 2. A word having a value of 0 represents a keyword extracted from the entire input text 2. The weight of the keyword extracted from the partial character string (partial character string ID is 1 or more) is calculated based on the TF in the partial character string in FIG. It may be calculated as follows. That is, the TF value in the entire input text of the keyword (the value of TF 903 of the keyword whose partial character string ID 901 is 0) may be used. Further, the weights may be calculated by fixing all the values of the TF 903 used when calculating the weights of the keywords extracted from the partial character strings (partial character string ID is 1 or more), or the values of the weights 905 May be fixed to the same value.

  FIG. 6 is a diagram illustrating an example of the configuration of the search result storage table 11. The search result storage table 11 includes a partial character string ID 1101 from which keywords used for the search are extracted, a search result rank 1102, a search document ID 1103, and a similarity 1104. The search order 1102 is assigned in order from the highest similarity 1104 value. Here, the partial character string ID 1101 matches the value of the partial character string in the word table 8 and the keyword table 9. In FIG. 6, the top 1000 documents with the highest similarity are stored as the overall search results, and the top 100 documents with the high similarity are stored as the partial character string search results, respectively. The setting of the number of cases is set to the maximum number 174 of the entire search results and the number of union generation application documents 173 of the parameter setting data 17 (FIG. 7), and can be freely set by the user.

FIG. 7 is a diagram illustrating an example of the configuration of the parameter setting data 17. This data stores the values of various parameters that are necessary for the system to search, and the values can be set by the user or the system administrator. The parameters stored in the parameter setting data 17 are as follows.
(1) Keyword part-of-speech list 171:
This indicates that the part of speech extracted as a keyword is only a common noun, sa-variant noun, verb, adjective, and suffix.
(2) Partial character string extraction collation character string 172:
Defines boundary conditions for dividing input text into syntactic, semantic, and expressed characters or symbolic substrings.
(3) Number of union generation application documents 173:
This represents the number of documents at the top of each partial character string search result used when generating a partial search result union from the partial character string search result.
(4) Overall search result maximum number of documents 174:
Represents the maximum number of search documents to be output as search results for the entire input text.
(5) Number of final search result output documents 175:
Indicates the number of documents to be output as the final search result.
(6) Number of consecutive keywords 176:
Represents how many consecutive keywords are taken out to generate a set of keywords.
(7) Keyword set 177:
Indicates how many keyword pairs are generated.

  8A and 8B are diagrams showing the first half and the second half of the details of the processing flow of the keyword extraction program 3, and move from A in FIG. 8A to A in FIG. 8B.

  In the partial character string generation 31, the input text stored in the input text 2 is read (step 3101). Next, the loop counter N is initialized to 0 (step 3102). Next, it is determined whether or not the value of N is smaller than the length of the input text (step 3103). If not, the process skips to step 3107. If the character string is small, it is verified whether the character string starting from the Nth byte from the beginning of the input text matches any one of the character strings defined in the partial character string extraction verification character string 172 of the parameter setting data 17 ( Step 3104), if there is a match, the part immediately after the matched partial character string extraction collating character string in the input text is recognized as the boundary of the partial character string (Step 3105). If not, skip to step 3106. Next, the value of the counter N is incremented by 1 (step 3106), and the process returns to step 3103. Next, the input text is divided at the portion recognized as the boundary, and the partial character string data is temporarily stored in the work area (step 3107).

  Next, in the morphological analysis 32, first, partial character string data temporarily stored in the work area is read (step 3201). Next, the word table 8 is initialized (step 3202). Next, it is determined whether or not there is a partial character string that has not been subjected to morphological analysis (step 3203). If there is a partial character string, morphological analysis is executed (step 3204), and the partial character string ID, the word heading, and the standard form of the word heading The part of speech information is stored in the word table 8 in the order of appearance of the words in the input text (step 3205). If there is no partial character string that has not been subjected to morphological analysis, the process proceeds to processing of unnecessary word removal 33.

  Next, in the unnecessary word removal 33, first, it is determined whether or not there is a word in the word table 8 that has not been subjected to unnecessary word removal processing (step 3301). Move on to processing. If there is an unprocessed word, it is checked whether or not the word is registered in the unnecessary word dictionary 6 (step 3302). If registered, the value of the unnecessary word flag 805 of the word in the word table 8 is set to 1. (Step 3303). If not registered, the process returns to step 3301.

  Next, in the keyword identification 34, first, it is determined whether or not there is a word in the word table 8 that has not been subjected to keyword identification processing (step 3401). If there is no unprocessed word, the loop is exited and step 3404 is exited. Skip to. If there is an unprocessed word, it is determined whether or not the part of speech of the word is in the keyword part of speech list 171 of the parameter setting data 17 (step 3402). If it is in the keyword part of speech list 171, the keyword flag 806 in the word table 8 is determined. Is set to 1 (step 3403), and the process returns to step 3401. If it is not in the keyword part-of-speech list 171, the process returns to step 3401. Next, it is determined whether or not there is an unprocessed word in the word table 8 for storing the keyword data in the keyword table 9 (step 3404). If there is no unprocessed word, the process skips to step 3409.

  If there is an unprocessed word, it is determined whether or not the value of the keyword flag 806 of the word in the word table 8 is 1 (step 3405). If not 1, the process returns to step 3404. If it is 1, it is further checked whether or not the standard form 803 of the word is already stored in the standard form 902 having the same partial character string ID in the keyword table 9 (step 3406). In this case, the value of the TF 903 of the word is increased by 1 (step 3407). If it is not stored yet, the standard form 803 of the word in the word table 8 is added to the standard form 902 of the keyword table 9, the value of TF903 is set to 1, and the IDF value of the word is obtained from the document index data 14 The ID is stored in the IDF 904 (step 3408). Next, the value of TF903 of the keyword in the entire input text is calculated by summing up the values of TF903 of the keyword for each keyword having a partial character string ID of 1 or more stored in the keyword table 9 (step 3409). Then, the value of the partial character string ID is set to 0, the standard form 902 of the keyword, the value of TF 903 calculated in step 3409, the IDF value of the keyword are stored in the keyword table 9, and the data storage in the keyword table 9 is finished. (Step 3410).

  Next, in keyword weighting 35, it is checked whether or not there is a keyword for which weighting processing has not been processed in the keyword table 9 (step 3411). If there is, a value obtained by applying natural logarithm to TF903 in the keyword table 9. Is multiplied by the value of IDF 904 and stored in the weight 905 (step 3412), and the process returns to step 3411. If there is no unprocessed keyword, the process ends.

  FIG. 9 is a diagram showing a processing flow for generating the partial search result union set 16 in the partial search result union generation program 7. First, the value of variable N is initialized to 1 (step 701). Next, it is determined whether or not the value of the variable N is equal to or less than the total number of partial character strings (step 702). If the value is equal to or less than the total number, the search for the partial character string ID 1101 in the search result storage table 11 is N. Search document IDs 1103 are extracted from the top of the result set by the number defined in the number of union generation application documents 173 in the parameter setting data 17 and stored in the working area (step 703). Next, the value of the variable N is incremented by 1 (step 704), and the process returns to step 702. If it is not less than the total number in step 702, the process exits the loop and proceeds to step 705.

  Next, a list is generated by merging all the search document IDs stored in the working area (step 705). Next, the list is sorted, and duplicate search document IDs are integrated and made unique (step 706). Then, all of the sorted and unique search document IDs are stored in the partial search result union set 16, and the process ends.

  FIG. 10 is an example of a screen example when the search result is displayed to the user.

This screen 100 is an example of a display screen of the display means _{1 4} of FIG. 2B, large, input text display area 120, the keyword list display area 140, consisting of the search result list display area 160.

  In the input text display area 120, when a user inputs a claim text as a search condition via the input / output unit 1, the input text data is once displayed. At this stage, the contents can be modified as necessary. Next, a search result can be obtained by pressing the search button 121, and this result is displayed in the search result list display area 160 in the form of a list. Further, by pressing the analysis button 122, the keyword group extracted by the keyword extraction program 3 can be displayed in the keyword list display area 140. In addition, by pressing the division button 123, the data in the input text display area 120 is switched to the display of the partial character string stored in the work area in step 3107. The display in FIG. 10 shows the result of pressing the division button 123.

  In the keyword list display area 140, the keyword group can be sorted by the selected item by selecting the item and pressing the sort button 142. Further, by pressing the edit button 141, it is possible to correct the display content corresponding to the result obtained by pressing the sort button 142, and by pressing the re-search button 143, a keyword corresponding to the correction content is displayed. You can search again. Here, the keyword list display area 140 is displayed together with the search result list 160. However, by pressing the analysis button 122 before the search, the keyword list 140 is displayed, and the search is performed after correcting the contents. The result can be displayed in the search result list 160.

  In the search result list display area 160, the results searched by the search execution program 10 are displayed. Display contents can be sorted (sort button 161), display scrolling (previous page button 162, next page button 163), and individual document contents (content display button 164) can be displayed.

  To end the search, the end button 180 may be pressed down.

  On this screen, when the user designates an arbitrary document displayed in the search result list display area 160 (check the check box at the left end of this area), the description content of the document is any partial character of the input text. Whether the relation is deep with the column is specified, and the partial character string having the deep relation is displayed in a different form (here, the symbol “◎” is attached to the left end) in the input text display area. Of course, the color, font, size, background color, etc. of the partial character string may be changed and displayed. Thus, the user can easily know the partial character string or keyword that determines the search level of the document designated in the search result list display area 160.

  The following describes two methods for specifying the depth of the relationship between a partial character string or keyword and a document.

  The first method is to determine how many ranks each document selected by the user is ranked in each partial search result, and regard the partial character string corresponding to the search result with the highest ranking as having a deep relationship. Is. Since the partial search result is stored in the search result storage table 11 together with the document ID, it is possible to easily realize the process of specifying the rank of each specific document. If a partial character string is determined, keywords included in the partial character string can be easily extracted from the keyword table 9.

  For example, when the ranking of the document Doc1 by the five partial character strings constituting the input text of FIG. 10 is 150th, 1st, 3rd, 5th, 2nd, respectively, the search order of the second partial character string Therefore, the symbol “◎” is added to the left end of the second partial character string in the input text display area 120. Since the keywords included in this partial character string are “keyword” and “extraction”, a symbol “◎” is added to the left end of these keyword display lines in the keyword list display area 140.

  The second method compares the search rank of the weighted whole keyword extracted from the input text with the partial search result rank of the partial character string for the document selected by the user, and the partial search result of the partial character string. Is higher, the partial character string corresponding to the partial search result is regarded as having a deep relationship. Here, not only “high ranking” but also a threshold “how high” may be provided. That is, for example, when the search ranking based on the entire keyword is 50th, the search ranking based on a partial character string is 20th, and the threshold of “how high” is “ranking difference is 20th or higher” Is higher by the 30th place from the 50th place to the 20th place, so the partial character string may be closely related. The setting of the threshold value may be a setting method such as an absolute value of the rank value (within 20th place) or a relative value (within 50%) instead of the rank difference.

  FIG. 11 is a diagram showing another example of a screen for outputting and displaying the search result to the user. The difference from FIG. 10 is the difference in content specified by the user. That is, in FIG. 10, the user designates an arbitrary document in the search result list display area 160, but in FIG. 11, designates an arbitrary partial character string displayed in the input text. At this time, in the keyword list display area 140, the symbol “◎” is added to the left end of the keyword included in the selected partial character string. In the search result list display area 160, a symbol “◎” is added to the left end of a document closely related to the partial character string. Of course, the color, font, size, background color, etc. of the document information that is closely related may be changed and displayed.

  Hereinafter, two methods for specifying a document closely related to the partial character string will be described.

  The first method is to display the search result by the designated partial character string as it is. For this purpose, the search result by the partial character string stored in the search result storage table 11 may be displayed as it is.

  The second method compares the search order of the weighted whole keyword extracted from the input text with the partial search result rank of the partial character string and compares the partial character string selected by the user with the partial character string. A document with a higher partial search result ranking is considered to be closely related to the partial character string. Here, not only “high ranking” but also a threshold “how high” may be provided. That is, for example, with respect to the partial character string, the search rank of the entire keyword of a document is 50th, the search rank of the partial character string is 20th, and the threshold of “how high” is “the rank difference is In the case of “20th or higher”, the ranking is higher by the 30th place from the 50th place to the 20th place, so the partial character string may be deeply related. The setting of the threshold value may be a setting method such as an absolute value of the rank value (within 20th place) or a relative value (within 50%) instead of the rank difference.

  When displaying the corresponding document in a different mode from the others, the corresponding document may be moved to the beginning of the search result list display area 160 and displayed together, or only the corresponding documents are displayed in the search result list. It may be displayed in the area 160.

  Next, a modification of the above embodiment will be described.

(1) Utilization of classification data In the above embodiment, the input text is divided into partial character strings, and the upper part of the search results performed for each partial character string is merged to generate a partial search result set, and the entire input text is searched. The search accuracy was improved by generating a product set with the results. Here, when searching for a partial character string as a unit, the number of keywords may decrease while the number of input keywords is reduced, and the number of keywords may decrease. Information is often missing. As a result, documents related to completely different fields appear at the top of the search results. Therefore, a field corresponding to the description is specified from the entire input text, and only documents related to the field are used as a partial character string search result, thereby excluding documents in completely different fields.

  The most commonly used method for identifying a field is to assign a classification code or controlled word (thesaurus) to each document. Since various patent codes are assigned to the patent publications dealt with in the present embodiment, it can be easily realized. In addition, as a method of specifying which field the content of the input text belongs to (which classification belongs to), a keyword is defined for each classification in advance, and the similarity with the keyword in the input text is calculated and related. A method of identifying a deep classification, or a search execution program 10 using an input text, and statistically analyzing which classification is assigned to a higher-order document of the search result, thereby providing a deeply related classification It is effective to specify this.

In FIG. 1, before performing the search execution 103, the classification code related to the input text 101 is recognized by the above method. For documents included in partial search results obtained by performing a search on each partial character string, it is determined whether the certified classification code is assigned. For documents that are not assigned, partial search results Exclude from Other processes are the same. In addition to the above search, the classification code may be input by the user by providing an input window in the input text display area 120, for example.
(2) Use of invention subject description part at the end of claim This modification is effective when the input text is the claim text. Instead of the search using the classification data described above, the last noun phrase in the input claim (the noun phrase described immediately after the proper expression such as “characterized by”, hereinafter “invention object description part” Information). This invention object description part is effective as a clue to limit the field as well as the classification code. Therefore, instead of simply dividing the input text into partial character strings and searching for each, the keyword appearing in the description part of the invention is added to the keyword extracted from each partial character string, and the search is performed. Prevent search results in different fields from being output. In FIG. 1, the character string of the partial character string 4 corresponds to the invention object description portion, and the keywords extracted from this partial character string are KW2, KW3, and KW8. Therefore, these keywords are added to the keywords 102a, 102b, and 102c extracted from each partial character string, and a search is executed.

In addition, you may apply both this modification and the method of using the classification | category mentioned above.
(3) Use of consecutive keywords in input text In this embodiment, the input text is divided into partial character strings, and keywords are extracted from each partial character string. In this modification, first, keywords are extracted from the input text. Is extracted to generate a list arranged in the order of appearance in the input text, a “keyword set” consisting of a plurality of consecutive keywords in the list is extracted, and this is used in place of the weighted partial character string keyword. The advantage of this modification is that it is not necessary to divide the input text into partial character strings.

FIG. 12 is a diagram showing an outline of the embodiment using the continuous keyword in the input text in a form corresponding to FIG. The difference between FIG. 1 and FIG. 12 is the method of extracting weighted partial character string keywords. That is, first, the input text 101 is analyzed to generate the appearance order keyword list 1000. This can be realized by sequentially extracting the standard forms 803 of the words whose keyword flag 806 is 1 in the word table 8 (see FIG. 4). Next, this list 1000 is scanned from the beginning, three consecutive keywords are extracted, and a set of keywords is generated. The setting of the value “three” is defined in the keyword sequence number 176 of the parameter setting data 17 and can be set by the user or the system administrator. Next, a weight is assigned to the keyword. At this time, the weight is calculated by setting the value of TF to 1. Here, instead of using the weight calculated with a TF value of 1, the weight may be calculated using the TF of the entire input text. That is, in FIG. 12, the weight value in the weighted partial character string keywords 102 a to 102 d matches the weight value in the weighted overall keyword 102.
(4) Use of arbitrary keywords in the input text Instead of using consecutive keywords in the input text, any N keywords are extracted from the keywords extracted from the entire input text, and a search is performed with this set of keywords. Is what you do. Important keywords that characterize the contents of the input text do not necessarily appear continuously, and may appear dispersed in the input text. This modification is effective in generating an important keyword set in such a case.

FIG. 13 is a diagram showing an outline of an embodiment using arbitrary N keywords from keywords extracted from the entire input text in a form corresponding to FIG. FIG. 12 differs from FIG. 13 in that FIG. 12 considers the appearance order of keywords, whereas FIG. 13 does not consider it. That is, the weighted overall keyword 102 is extracted from the input text 101, and any three types of keywords are extracted from the extracted keyword. The value “three types” is defined as “keyword set 177” of the parameter setting data 17 and can be freely set by the user or the system administrator. For weighting, the value of TF may be calculated as 1, or the weight value of the weighted overall keyword 102 may be inherited.
(5) Only one search is performed In the embodiment described above, the number of searches executed in the search execution program 10 increases as the number of partial character strings increases or the number of keyword sets increases. As a result, search performance is degraded. In this modification, the search performance is improved by limiting the number of searches to one.

  The search is performed only once using the weighted overall keyword 102 in FIG. As a result, a search result document is output, but each search result document has a similarity score. In the present modification, when the similarity is calculated for each document, the ratio of each keyword included in the weighted overall keyword 102 to the similarity calculation score (which may be a relative value or an absolute value) is extracted. For example, when the similarity is calculated by the inner product of the keyword vector extracted from the input text and the keyword vector extracted from the search target document, the similarity score is the sum of the products of the vector components of the keyword vector. Therefore, the value of the product of the vector components of each keyword represents the degree of contribution that the keyword occupies in the similarity score.

  In this way, in addition to the similarity score, when this “similarity by keyword” is obtained, instead of searching for a partial string keyword and obtaining a search result document, each keyword used as the partial string keyword is searched for. On the other hand, the keyword similarity is extracted, and the sum is regarded as the document similarity. Thereby, a search result document of each partial character string keyword can be obtained without performing many searches.

It is a figure which shows the outline | summary of embodiment of this invention. It is the block diagram which linked | related and displayed the program regarding a user's operation, various data, and data processing about the system described by embodiment of this invention. It is the block diagram which displayed the system described by this embodiment demonstrated in FIG. 2A as a structure of an electronic computer. It is a figure which shows an example of a structure of the input text 2 of embodiment of this invention. It is a figure which shows an example of a structure of the word table 8 corresponding to the input text 2 of embodiment of this invention. It is a figure which shows an example of a structure of the keyword table 9 of embodiment of this invention. It is a figure which shows an example of a structure of the search result storage table 11 of embodiment of this invention. It is a figure which shows an example of a structure of the parameter setting data 17 of embodiment of this invention. It is a figure which shows the first half of the processing flow of the keyword extraction program 3 of embodiment of this invention. It is a figure which shows the second half of the processing flow of the keyword extraction program 3 of embodiment of this invention. It is a figure which shows the processing flow of the partial search union production | generation program 7 of embodiment of this invention. It is a figure which shows an example of the search result output screen in embodiment of this invention. It is a figure which shows the other example of the search result output screen in embodiment of this invention. It is the figure which showed the outline | summary of embodiment of this invention by utilization of the continuous keyword in an input text in the form corresponding to FIG. It is the figure which showed the outline | summary of embodiment of this invention by the use of arbitrary N keywords from the keyword extracted from the whole input text in the form corresponding to FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Input / output part, 1 ₁ ... Keyboard, 1 ₂ ... Mouse, 1 ₃ ... Printing means, 1 ₄ ... Display means, 2 ... Input text, 3 ... Keyword extraction program, 4 ... Word dictionary, 5 ... Grammar dictionary, 6 ... unnecessary word dictionary, 7 ... partial search result union generation program, 8 ... word table, 9 ... keyword table, 10 ... search execution program, 11 ... search result data, 12 ... search result display program, 13 ... document data, 14 ... Document index data, 15 ... Index generation program, 16 ... Partial search result union, 17 ... Parameter setting data, 100 ... Display screen, 120 ... Input text display area, 121 ... Search button, 122 ... Analyze button, 123 ... Reset Button 140 ... Keyword list display area 141 ... Edit button 142 ... Sort button 143 ... Re Search button 160 ... Search result list display area 161 ... Sort button 162 ... Previous page button 163 ... Next page button 164 ... Content display button 180 ... End button 200 ... System bus 201 ... CPU 203 ... Memory work area 204. Memory storage area 205 Client 207 Network

Claims (12)

Document retrieval method using a computer comprising a storage device for storing text input by a user, a document to be searched, a weighted keyword extracted in advance from each document, and a central processing unit accessible to the storage device In
The central processing unit analyzing the input text in the storage device to extract keywords; and the central processing unit assigns a weight corresponding to the importance to each of the extracted keywords. The assigning step and the central processing unit comparing the weighted keyword with the pre-extracted weighted keyword in the storage device and calculating a similarity for each document in the storage device The central processing unit outputting a document having a high similarity as a search result, and the central processing unit dividing the inputted text in the storage device into a plurality of partial character strings; The central processing unit extracting a partial character string keyword from each of the divided partial character strings; and the central processing unit Assigning a weight corresponding to importance to each partial character string keyword, the central processing unit, a weighted partial character string keyword extracted from the partial character string, and the storage device in the storage device Comparing with pre-extracted weighted keywords and calculating a partial character string similarity for each document in the storage device; and the central processing unit performing a partial search for a document with a high partial character string similarity Extracting as a result; generating a union of documents included in the upper part of the partial search result extracted for each of the partial character strings; and the central processing unit Extracting a document included in the union from the search results for the entire text, and the central processing unit extracting the extracted document Document search method characterized by having a step of outputting a search result. Document retrieval method using a computer comprising a storage device for storing text input by a user, a document to be searched, a weighted keyword extracted in advance from each document, and a central processing unit accessible to the storage device In
The central processing unit analyzing the input text in the storage device to extract keywords; and the central processing unit assigns a weight corresponding to the importance to each of the extracted keywords. The assigning step and the central processing unit comparing the weighted keyword with the pre-extracted weighted keyword in the storage device and calculating a similarity for each document in the storage device The central processing unit outputting a document having a high degree of similarity as a search result; and the central processing unit includes a weighted keyword including two or more keywords from the extracted weighted keywords. A step of extracting a set; and the central processing unit includes the extracted weighted keyword set and the pre-extracted weight in the storage device. Comparing the keyword with the attached keyword, calculating a similarity for each document in the storage device, extracting the document with a high similarity as a partial search result, and executing the central processing A device generating a union of documents included in a higher rank of the partial search results extracted for each of the weighted keyword sets; and the central processing unit including the search results for the entire input text A document search method comprising: extracting a document included in the union from among the steps; and a step in which the central processing unit outputs the extracted document as a search result.   3. The document search method according to claim 2, wherein a set of weighted keywords including the two or more keywords is selected in the order of appearance of the keywords in the input text.   4. The document search method according to claim 1, wherein the step of dividing the input text into partial character strings is performed by the central processing unit, wherein the input text is a paragraph, sentence, clause, or phrase. A document search method characterized in that the division is performed for each unit, or the boundary of the partial character string is specified and divided depending on whether a specific description expression appears in the input text.   4. The document search method according to claim 1, wherein in the step of assigning a weight corresponding to the importance to each of the extracted partial character string keywords, the central processing unit includes: A document search method, comprising assigning a weight of the keyword to the entire input text as a weight of the partial character string keyword.   6. The document search method according to claim 1, wherein when the computer input device generates a union of the documents with high similarity, each of the documents with high similarity is obtained. Setting whether to generate a union by collecting, setting of how many consecutive keywords to extract when extracting a set of two or more consecutive keywords in the weighted appearance order keywords, and weighting A document search method characterized in that, when a set of keywords is extracted, one of settings for how many types of keywords is extracted is received from a user. In a program for causing a central processing unit that can access a storage device that stores text input by a user, a document to be searched, a weighted keyword extracted in advance from each document,
Analyzing the input text in the storage device to extract keywords; assigning weights corresponding to importance to each of the extracted keywords; the weighted keywords; Comparing the previously extracted weighted keywords in the storage device to calculate a similarity for each document, dividing the input text into a plurality of partial character strings, and the divided partial characters Extracting a partial character string keyword from each of the columns, assigning a weight corresponding to the importance to each of the extracted partial character string keywords, and a weighted portion extracted from the partial character string A partial sentence for each document in the storage device by comparing a character string keyword with the previously extracted weighted keyword in the storage device A step of calculating a column similarity, a step of extracting a document with a high degree of partial character string similarity as a partial search result, and a sum of documents included at the top of the partial search result extracted for each of the partial character strings Generating a set; extracting a document included in the union from the search results for the entire input text; and outputting the extracted document as a search result. A document search program to be executed by a processing device. In a program for causing a central processing unit accessible to a storage device that stores text input by a user, a document to be searched, a weighted keyword extracted in advance from each document,
Analyzing the input text in the storage device to extract keywords; assigning weights corresponding to importance to each of the extracted keywords; the weighted keywords; Comparing the previously extracted weighted keywords in the storage device to calculate the similarity for each document, and a set of weighted keywords comprising two or more keywords from the extracted weighted keywords A step of extracting, a step of calculating a similarity for each document by comparing the set of the extracted weighted keywords with the pre-extracted weighted keyword in the storage device, and a document having a high similarity A step of extracting as a partial search result, and a higher rank of the partial search result extracted for each of the set of weighted keywords Generating a union of documents, extracting a document included in the union from the search results for the entire input text, and outputting the extracted document as a search result A document search program for causing the central processing unit to execute steps.   9. The document search program according to claim 8, wherein a set of weighted keywords including the two or more keywords is selected in the order of appearance of the keywords in the input text.   In a document search apparatus comprising an input / output means connected to a system bus, a central processing unit, a memory work area, and a computer to which a memory storage area is connected, an input text for storing input text in the memory storage area, Acquired from a word dictionary in which attribute data related to each word is registered, a grammar dictionary in which connection costs between word parts of speech and grammar rules are defined, an unnecessary word dictionary that defines words to be excluded from keyword candidates, and input text A word table for storing stored words, a keyword table for storing words that do not correspond to words to be excluded from keyword candidates among the words stored in the word table, document data for storing a document to be searched, and the document data Document data index for search, search result data to store search results, given by user Parameter search data for storing various settings relating to search, a search program, and a document search apparatus comprising weighted keywords extracted in advance from each document, the search program being input by a user Analyzing text to extract keywords, assigning a weight corresponding to importance to each of the extracted keywords, the weighted keywords, and the pre-extracted weighted keywords in the memory And calculating a similarity for each document in the memory, dividing the input text into a plurality of partial character strings, and a partial character string from each of the divided partial character strings A step of extracting a keyword, and a degree of importance corresponding to each of the extracted partial character string keywords. A step of assigning a weight; and comparing the weighted partial character string keyword extracted from the partial character string with the pre-extracted weighted keyword in the memory to compare the partial character string similarity for each document in the memory A step of extracting a document with a high degree of partial character string similarity as a partial search result, and generating a union of documents included in the upper part of the partial search result extracted for each of the partial character strings A step of extracting a document included in the union from the search results for the entire input text, and outputting the extracted document as a search result to the central processing unit. A document search apparatus characterized by being executed.   In a document search apparatus comprising an input / output means connected to a system bus, a central processing unit, a memory work area, and a computer to which a memory storage area is connected, an input text for storing input text in the memory storage area, Acquired from a word dictionary in which attribute data related to each word is registered, a grammar dictionary in which connection costs between word parts of speech and grammar rules are defined, an unnecessary word dictionary that defines words to be excluded from keyword candidates, and input text A word table for storing stored words, a keyword table for storing words that do not correspond to words to be excluded from keyword candidates among the words stored in the word table, document data for storing a document to be searched, and the document data Document data index for search, search result data to store search results, given by user Parameter search data for storing various settings relating to search, a search program, and a document search apparatus comprising weighted keywords extracted in advance from each document, the search program being input by a user Analyzing text to extract keywords, assigning a weight corresponding to importance to each of the extracted keywords, the weighted keywords, and the pre-extracted weighted keywords in the memory And calculating a similarity for each document in the memory, extracting a set of weighted keywords composed of two or more keywords from the extracted weighted keywords, and the extraction A weighted keyword set and the pre-extracted weighted keyword in the memory; A step of calculating a similarity for each document in the memory in comparison, a step of extracting a document having a high similarity as a partial search result, and the partial search result extracted for each of the sets of weighted keywords Generating a union of documents included in a higher rank, extracting a document included in the union from the search results for the entire input text, and extracting the extracted documents as a search result The document search apparatus causing the central processing unit to execute the step of outputting as 12. The document search apparatus according to claim 11, wherein a set of weighted keywords including the two or more keywords is selected in the order in which the keywords appear in the input text.

Images