JP2006178599A - Document retrieval device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily and properly refine searches. <P>SOLUTION: A search condition A is inputted via a user interface part 10 and sent to a document retrieval part 13 (X01). Either a full-text search performing part 20 or an attribute search performing part 22 performs a search about the search condition A and outputs the search result to a search result acquisition part 14 (X02). The search result acquisition part 14 delivers the search result to an important word extracting part 14 and refers to a related document search index to request a group of characteristic important words to be obtained (X03). A display synthesizing part 16 receives the group of important words (characteristic words) and creates display data for display together with the search result, and the user interface part 10 displays the data (X04). A search refinement button B is operated to refine the search (X05). The display of the search result and the search refinement can be easily repeated by operation of the search button B. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a document search technique, and in particular, can narrow down search results appropriately and easily.

  When searching, the number of search results may be large. In this case, it is desirable to further perform a narrow search to reduce the number of accesses. Here, it is important to efficiently create an appropriate narrowing-down condition from the search result. In the prior art described below, the method for determining candidate search terms is the key point, but each has drawbacks.

  In Patent Document 1, information obtained by extracting and indexing a noun phrase from a search target document is prepared, and the index record for each noun phrase is used to narrow down a noun phrase including a search term input by a user. As a search term. This method has a drawback in that a word related to a word input by the user but having a different character face is not presented.

  In Patent Document 2, a search term candidate related to an input search term is obtained using a history of search conditions input from a user. However, this method has a drawback that candidates cannot be obtained unless search condition data accumulated as a history exists.

  In Patent Literature 3, each search result is subjected to morphological analysis at the time of search, a word in the result and an appearance frequency of the word in a document are obtained, and a search word having a threshold value or less is set as a search word candidate for narrowing down. The problem with this prior art is that each search result is morphologically analyzed at the time of search, and the processing time until the user obtains the search result is long. In other words, an appropriate search term cannot be selected. Specifically, the text length of each search result (problem in which a sentence having a short sentence tends to be selected), the search result group, or the document appearance frequency in the entire search target document is not taken into consideration.

Further, since a plurality of search terms for narrowing down can be designated, narrowing down from the result document group is extremely performed, and there is a high possibility that the narrowing down result becomes zero.
JP 2002-342373 A JP 2003-108594 A JP 2004-54619 A

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a document search technique that can narrow down search results appropriately and easily.

  In the principle configuration example of the present invention, index information for related document search (including words included in a predetermined word set and its importance level) is used for a document group as a result of attribute search or document content search. Thus, a characteristic word (related word and its importance. The importance is typically expressed by a score calculated from the index information) in the result document group is acquired, and the related word is obtained together with the search result document group. Present.

  At this time, the related words including the character string already used in the search condition and the related words included in all the search results are excluded from the related word group, and the higher related words are used for narrowing down in order of importance. It should be noted that no narrowing can be performed even if narrowing is performed using related terms included in all search result documents.

  Specifically, the following two types of index data are created when a document is stored in the document management system. The first is index data that can obtain the word appearing in the document and frequency information thereof from the document identifier, and is used for so-called related document search. The second is index data that can obtain an identifier group of a document including the word from the word (keyword or attribute), and is used for attribute search or content search (full-text search, keyword search).

  In this configuration example, index data for attribute search using attributes assigned to a document and full-text search using n-gram is used. After performing these attribute searches and full-text searches (content searches), the related document search index is used to acquire characteristic words in the result document group so that the search results can be narrowed down.

  As a method for calculating a word (related word) and its importance, a method described in Japanese Patent No. 3427774 or Japanese Patent Laid-Open No. 2002-32411 is used. In short, for example, the importance is calculated using a weight based on TF (Term Frequency) and IDF (Inverse Document Frequency). A weight vector having word weight as an element is used as index data for each document ("Information Retrieval and Language Processing" written by Takenobu Tokunaga, The University of Tokyo Press, 1999).

  In this configuration example, when a search result is displayed, a plurality of narrow search execution buttons and the like in which a narrow search condition is set are displayed. For example, a button is displayed for each narrowing word. The narrowing search condition is a narrowing search condition in which the related terms obtained as described above are connected to the original “search condition” by the AND search condition. By operating the search refinement button, it is possible to execute a refinement search / acquire search results in one step, and to repeat this.

  The search condition is not limited to “a set of document identifiers” that satisfies the condition. That is, it is sufficient if there is information that can limit the target to be narrowed down.

  Specifically, in this configuration example, the following procedure is executed.

[Step 1]: Enter search conditions.
[Step 2]: A search result matching the search condition is obtained.
[Step 3]: Find related terms in the search results. Here, excluding related terms including the search character string within the search condition and related terms appearing in all search result documents, the top X items (X is a positive integer) are obtained.
[Step 4]: A refined search condition is created by combining the related term obtained in Step 3 and the search condition in Step 1 or the set of document identifiers obtained in Step 2 by AND logic operation.
[Step 5]: A search result button with a search result and a narrow search condition expression set is displayed.

  Steps 2 to 5 described above can be repeated.

  Further, the present invention will be described. In addition, in order to make an understanding easy, the reference sign used in the Example mentioned later is attached | subjected, but this is not the intention which limits the content of rights.

  According to one aspect of the present invention, in order to achieve the above object, the document search device (100): search means (20, 22) for searching a document group; Important word / phrase information storage means (25) for storing; referring to the important word / phrase information storage means, obtaining important word / phrase information of the document included in the search result of the search means, and narrowing down based on the acquired important word / phrase information Refined word determining means (15) for determining the word of the search; refined search instruction means (10, 16) for instructing the search means to perform a refinement search of the search result using the refinement word. ).

  In this configuration, the important phrase information of the document included in the search result can be acquired from the search result with reference to the important phrase information storage unit, and the important phrase can be selected as a narrowing word.

  In this configuration, the search means includes, for example, a search means for performing full text search with reference to index data based on n-gram, a search means for performing full text search with reference to index data prepared for each word, a document Search means for performing an attribute search with reference to index data prepared on the basis of the attribute.

  The important phrase information is a set of weights of words included in a predetermined word set prepared for each document. In this case, the weight of each word for each document is preferably calculated based on the frequency of each word in the document and the number of documents in which the word appears.

  The important phrase information for each document stored in the important phrase information storage means is typically information on a related document search certificate index used for related document search means for searching for a document related to a predetermined word. .

  Further, it is preferable that the narrowing-down word determining means excludes the narrowing-down word candidates from the narrowing-down words if the search result document includes a predetermined ratio or more. The predetermined ratio may be selected according to the degree of narrowing down. The predetermined ratio can be acquired from the frequency information of each word in each document. By using tf for calculating tf * IDF (ratio of documents in which the word is tf> 0), the narrowing down ratio can be calculated.

  Typically, candidates included in all search result documents may be excluded from narrowing words. In this case, although it can also be determined from tf, IDF = log (N / df) (N is the total number of target documents, df is the frequency of documents including the word) is zero (tf * IDF is also zero), and is determined from this. You can also.

  The important phrase information for each document can be the importance (for example, a word vector) in the document of each word included in the predetermined word set. In this case, each element of the word vector is added to the search result document to obtain a score for each element, and can be used as an index for determining the importance in the entire search result. However, if the score is large, the probability of appearing in most of the documents in the search results is high, so the top X scores with a score smaller than a predetermined threshold may be selected.

  Further, for each of the narrowing words, the ratio of the document including the narrowing word in the search result document may be displayed. In this way, it is possible to know the narrowing result when each word is used, and it is possible to select a narrowing word using it as an index. As described above, the ratio of the document including the narrowing-down word can be calculated with reference to the value of tf, for example.

  According to another aspect of the present invention, the document retrieval apparatus (100) includes: retrieval means (20, 22) for retrieving a document group; important phrase information storage for storing important phrase information for each document. Means (25); means (15) for determining important phrase information of a document included in the search result of the search means by referring to the important phrase information storage means; and displaying the search result and the important phrase Search result / important phrase display means (10, 16); a refinement for instructing to perform a refinement search of the search result using the important phrase to be operated in response to a user operation on the displayed important phrase Search instruction means (10) is provided.

  The present invention can be realized not only as an apparatus or a system but also as a method. Of course, a part of the invention can be configured as software. Of course, software products used for causing a computer to execute such software are also included in the technical scope of the present invention.

  These and other aspects of the invention are set forth in the appended claims and will be described in detail below with reference to examples.

  According to the present invention, search results of attribute search and content search can be narrowed down easily and appropriately.

  Examples of the present invention will be described below.

  FIG. 1 shows an embodiment in which the present invention is applied to a document management system as a whole. In this example, the document management system 100 is realized by installing a computer program on the computer 101 using the recording medium 102. Each block shown in the figure represents a function realized by cooperating hardware resources and software resources of the computer 101. In this example, a mounting method using a single computer system is used, but a server computer and a client device may be connected via a network.

  In FIG. 1, a document management system 100 includes a user interface unit 10, a document management unit 11, a registered document group storage unit 12, a document search unit 13, a search result acquisition unit 14, a keyword extraction unit 15, a display synthesis unit 16, an index. The generation unit 17 and the like are included. In this example, the document search unit 13 includes a full-text search execution unit 20, a full-text search index storage unit 21, an attribute search execution unit 22, an attribute search index storage unit 23, a related document search execution unit 24, and a related document search index. The storage unit 25 and the like are included.

  The user interface unit 10 receives input from the user and displays output from the document management system 100. If limited to search, the user inputs search conditions using an input form as shown in FIG. Keyboard input may be used, and pull-down menus may be used for attribute search attributes and the like. Search conditions can be specified by keywords, attributes, natural sentences, and the like. The natural sentence is subjected to morphological analysis and extracted as a keyword, and can be used for searching in the form of the keyword or a word vector described later. The user interface unit 10 can be implemented on a web basis, for example, but is not limited thereto.

  The document management unit 11 manages the document group registered by the user by holding it in the registered document group storage unit 12. Each document is assigned a management attribute as shown in FIG.

  The document search unit 13 performs full-text search, attribute search, and related document search. These various searches may be performed in combination with logical expressions.

  The full text search execution unit 20 of the document search unit 13 performs a full text search using the full text search index held in the full text search index storage unit 21. For example, as shown in FIG. 5, the full-text search index is configured by recording a document identifier included in each n-gram and its appearance position (may be plural) as an index record. The index record is generated so as to cover a search target document, typically a document stored in the registered document group storage unit 12.

  From the n-grams constituting the keywords input as the search conditions and their positional relationships, the corresponding document identifiers are listed by referring to the index records as shown in FIG. Instead of the index record for each n-gram shown in FIG. 5, the index record for each morpheme (word, keyword) may be used after morphological analysis of the document. An example of this is shown in FIG.

  The attribute search execution unit 22 performs an attribute search using the attribute search index held in the attribute search index storage unit 23. For example, as shown in FIG. 6, the attribute search index is configured such that a document identifier corresponding to each attribute value (attribute value range) of each attribute is recorded as an index record. The index record is generated so as to cover a search target document, typically a document stored in the registered document group storage unit 12. The attribute is attribute information (FIG. 8) managed by the attribute management unit 11. The attribute search execution unit 22 lists the document identifiers of documents having attributes corresponding to the attribute values constituting the search condition with reference to the attribute search index to obtain a search result.

  The related document search execution unit 24 performs a related document search using the related document search index held in the related document search index storage unit 25. In this example, the related document search index holds an index record as shown in FIG. In each index, important phrase information, for example, a word vector is assigned to each document to be searched, typically for each document (document identifier) stored in the registered document group storage unit 12. The word vector is composed of weights w1, w2,. The weight is typically generated based on tf * IDF. Each word vector is normalized. tf is the frequency of occurrence of the word in the document, and IDF is log (N / df) as an example, but is not limited thereto. log (N / df) +1 or the like, and various logarithmic bases can be employed. N is the total number of target documents, and df is the number of documents in which the word appears.

  The related document search execution unit 24 performs an inner product or the like of the input one or more keywords (word vectors) and the word vectors of each document, and lists documents having a predetermined number or a value exceeding a predetermined threshold. Let it be a search result. A related document search using a keyword (word vector) obtained from a natural sentence or a word vector of a seed document can also be performed.

  The search result acquisition unit 14 acquires a document identifier set as a search result from the document search unit 13. A set of document identifiers as a search result is typically transferred to the display composition unit 16, and the display composition unit 16 generates display data for the search result list and supplies it to the user interface unit 10. The user interface unit 10 displays a search result list based on the display data.

  Further, when the search is other than the related document search, that is, when the search is a full-text search, an attribute search, or a combination thereof, the search result acquisition unit 14 also supplies the search result document identifier set to the keyword extraction unit 15. . The keyword extraction unit 15 acquires the keyword information (word vector) of the search result (document identifier set) with reference to the related document search index storage unit 25, and based on this, the keyword is expected to be important for the search result. The words to be extracted. For example, the word vectors of the search result document are accumulated for each element (word), and the word is selected according to the value of the element. That is, when the search result is composed of m documents, each element wi of the word vector is accumulated for the m documents, and a predetermined threshold or an upper predetermined number of elements i (words) is used as an important word. 16 is output.

  As shown in FIG. 3, the display synthesis unit 16 displays display data for displaying a refinement search button B for displaying the keyword A (also referred to as a feature word) determined by the keyword extraction unit together with the list A of search results. Generate. Based on this, the user interface unit 10 presents a display including a search result and a narrow-down button to the user. In FIG. 3, “feature word 1”, “feature word 2”, and the like are displayed, but specific words are displayed. When the user performs an operation by clicking any button B or the like, the user interface unit 10 supplies a narrowing search condition to the document search unit 13 based on this.

  If the initial search condition is the search condition A, the narrow-down search condition is “important word AND search condition A”. You may make it take a logical product with the document identifier set of a search result.

  Note that a plurality of important words may be selected and a narrowing search may be instructed by a logical expression such as a logical product or logical sum of them. For example, if you click on the search button for each refinement, the relevant key word is selected and transferred to the input form for conditional input, and this is combined with the operator input to start with a logical expression that includes key words (feature words). Search condition A (document identifier set) can be narrowed down.

  The index generation unit 17 generates / updates a full-text search index, an attribute search index, and a related document search index for a document group held in the registered document group storage unit 12. It is preferable to generate an index reflecting a new registered document or a changed document at an appropriate timing. When the weight calculated from tf and IDF, for example, is used as the related document search index, tf and IDF are held to reflect the new document information. Using this tf and IDF, auxiliary information such as a narrowing down ratio can be calculated.

  Next, the operation of the embodiment will be described with reference to the example shown in FIG. In this example, only a case where a full text search or an attribute search is performed under the search condition A will be described.

  First, the search condition A is input using the input form shown in FIG. 4 via the user interface unit 10 and sent to the document search unit 13 (X01). The full text search execution unit 20 or the attribute search execution unit 22 searches for the search condition A, and outputs the search result to the search result acquisition unit 14 (X02). The search result acquisition unit 14 delivers the search result to the keyword extraction unit 15 and requests to acquire a characteristic keyword (feature word) group with reference to the related document search index (X03). The important word group is, for example, from the top N words excluding words including the search character string in the search condition A (related words, words forming the elements of the word vector) and words included in all the search result documents. Become. Whether or not the word is included in all the search result documents can be determined by referring to tf of the search result document or by checking whether or not the IDF (tf * IDF) of the search result document becomes zero. The display synthesis unit 16 receives the important word (feature word) group, generates display data to be displayed together with the search result, and the user interface unit 10 displays the display data (X04). A refinement search is performed by operating the refinement search button B (X05). Processes X02 to X05 can be repeated. In addition, a button for canceling the narrowing down may be provided so as to return to the previous state. In this way, important words used for narrowing down may be selected again.

  Since the important word is a keyword, the narrow-down condition is typically a full-text search. However, if the important word can be grasped as an attribute, the narrow-down condition can be set as an attribute search.

  According to this embodiment, it is possible to acquire words representing the characteristics of the search result simply by referring to the related document search index, and it is possible to narrow down easily and accurately.

  That is, in this embodiment, when a document search is performed under a certain search condition, a plurality of search condition candidates effective for further narrowing down the search result can be presented together with the search result.

  In addition, using a system that holds an index, such as a related document search component, a word that is characteristically used in the search result document group is identified, and the original search condition is ANDed using this word. A refined search condition expression added as a search condition can be created, and search condition candidates effective for the refinement can be automatically generated.

  That is, in the system of the embodiment, words appearing in the document and frequency information thereof are obtained from the document, and characteristic word groups in the search result document group can be acquired in the order of score.

  Various types of index data for attribute search / document content search / related document search are available for documents held in the document management system. By using the index data, effective search word candidates for narrowing down by full-text search can be easily acquired.

  In addition, this invention is not limited to the above-mentioned Example, A various change is possible. For example, in the above example, the related document search index is used to select a word for narrowing down the search results. However, the frequency (importance) of each word is determined for each document regardless of the related document search index. Information may be retained, and a characteristic word may be selected in the search result document group using the information.

  Also, it is possible to extract important attribute values from the attribute information (FIG. 8) of the search result document group, and use this as an important word as a narrowing search condition by attribute search. In this case as well, the attribute values included in all search result documents are excluded from the important words. In this case, the search that is a premise (search before narrowing down) may be a related document search.

  Further, as shown in FIG. 10, the narrowing rate for each important word (feature word) may be calculated and displayed. The calculation of the narrowing-down rate may be performed by the keyword extraction unit 15 or may be performed by a function block (not shown) provided elsewhere. This narrowing rate is calculated with reference to tf of the search result document, for example. The value of tf may be stored as management data together with the registered document in the registered document group storage unit 12 together with the IDF value, or the related document search index as auxiliary information of the related document search index. You may memorize | store in the memory | storage part 25. FIG.

It is a block diagram which shows the structure of the Example of this invention as a whole. It is a figure explaining the operation example of the said Example. It is a figure explaining the example of a search result (narrowed search) display of the above-mentioned example. It is a figure explaining the example of the search condition input form of the above-mentioned Example. It is a figure explaining the example of the record of the index for full text search of the above-mentioned Example. It is a figure explaining the example of the record of the index for attribute search of the above-mentioned Example. It is a figure explaining the example of the record of the related document search index of the above-mentioned Example. It is a figure explaining the example of the document attribute record of the above-mentioned Example. It is a figure explaining the example of the record of the other full-text search index of the above-mentioned Example. It is a figure explaining the modification of the above-mentioned Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 User interface part 11 Attribute management part 11 Document management part 12 Registered document group memory | storage part 13 Document search part 14 Search result acquisition part 15 Key word extraction part 16 Display composition part 17 Index generation part 20 Full-text search execution part 21 Full-text search index Storage unit 22 Attribute search execution unit 23 Attribute search index storage unit 24 Related document search execution unit 25 Related document search index storage unit 100 Document management system 101 Computer 102 Recording medium

Claims (14)

A search means for searching a document group;
Important phrase information storage means for storing important phrase information for each document;
Narrow word determination means for obtaining important word information of a document included in a search result of the search means with reference to the important word information storage means, and determining a narrowing word based on the acquired important word information;
A computer program for searching a document, characterized in that it is used to realize a search refinement instruction means for instructing the search means to perform a refinement search of the search result using the refinement word. .   2. The computer program for document search according to claim 1, wherein the search means is search means for performing full text search with reference to index data based on n-gram.   2. The computer program for document search according to claim 1, wherein the search means is search means for performing full text search with reference to index data prepared for each word.   2. The computer program for searching a document according to claim 1, wherein the search means is a search means for performing an attribute search with reference to index data prepared based on the attribute of the document.   5. The computer program for searching a document according to claim 1, wherein the important phrase information is a set of weights of words included in a predetermined word set prepared for each document.   6. The computer program for searching a document according to claim 5, wherein the weight of each word for each document is calculated based on the frequency of each word in the document and the number of documents in which the word appears.   7. The related document search means for searching for a document related to a predetermined word with reference to the important phrase information for each document stored in the important phrase information storage means. Computer program for document retrieval.   8. The narrowed-down word determination means, when narrowed-down word candidates are included in a search result document in a predetermined ratio or more, are excluded from narrowed-down words. Computer program.   The computer program for searching a document according to any one of claims 1 to 8, wherein, for each of the narrowing-down words, a ratio of a document including the narrowing-down word in a search result document is displayed. A search means for searching a document group;
Important phrase information storage means for storing important phrase information for each document;
Narrowed word determination means for determining important phrase information of a document included in a search result of the search means with reference to the important phrase information storage means;
Search result / important phrase display means for displaying the search result and the important phrase;
A refinement search instruction means for instructing execution of a refinement search of the search result using the keyword to be operated in response to a user's operation on the displayed important phrase is used. A computer program for document retrieval. A search means for searching a document group;
Important phrase information storage means for storing important phrase information for each document;
Narrow word determination means for obtaining important word information of a document included in a search result of the search means with reference to the important word information storage means, and determining a narrowing word based on the acquired important word information;
A document search apparatus comprising: a search refinement instruction means for instructing the search means to perform a refinement search of the search result using the refinement word. A search means for searching a document group;
Important phrase information storage means for storing important phrase information for each document;
Means for determining important phrase information of a document included in a search result of the search means with reference to the important phrase information storage means;
Search result / important phrase display means for displaying the search result and the important phrase;
A document search apparatus comprising: a search refinement instruction means for instructing execution of a refinement search of the search result using the keyword to be operated in response to a user operation on the displayed keyword. . A search step for searching the document group by a search means;
The refined word determination means refers to the important phrase information storage means for storing the important phrase information for each document, acquires the important phrase information of the document included in the search result of the search means, and based on the acquired important phrase information A refinement word determination step for determining a refinement word;
A refinement search instruction means, further comprising a refinement search instruction step for instructing the search means to perform a refinement search of the search result using the refinement word. A search step for searching the document group by a search means;
A refinement word determination step for determining important phrase information of a document included in a search result of the search means by referring to an important phrase information storage means for storing important phrase information for each document by the refinement word determination means;
A display step of displaying the search result and the important phrase by the search result / important phrase display means;
A refinement search instruction means for instructing to perform a refinement search of the search result using the important word / phrase to be operated in response to a user's operation on the displayed important word / phrase. A document retrieval method as a feature.