JP4774081B2 - Document search system, document search method, and program - Google Patents

Description

  The present invention relates to a document search system, a document search method, and a program.

As represented by search engines that search for documents published on the Internet, search processing of documents according to keywords is widely performed, but because the amount of documents to be searched has increased, There is a need to efficiently narrow down search results. For example, Patent Literature 1 proposes a technique that preferentially displays search results that other users who have previously performed the same search condition as a search condition input by a certain user are useful.
JP 2003-108587 A

  However, if a keyword entered by a certain user hits a huge document, the huge document can be classified into various categories, and because other users use the same keyword, The category of the document that is determined to be useful does not necessarily match the category intended by the user who entered the keyword. In addition, when a user tries to input an additional keyword in order to narrow down an enormous number of search results, it is often impossible to come up with a new keyword.

  The present invention has been made in view of such a background, and an object of the present invention is to provide a document search system, a document search method, and a program capable of efficiently narrowing down a document search process.

  A main invention of the present invention for solving the above problems is a system for searching for a document, in which a keyword input unit that receives an input of a keyword, a document search unit that searches for a document corresponding to the keyword, a search result A group classification unit that classifies a document into a plurality of groups based on words included in the document, and a maximum group name that represents a maximum group that has the largest number of documents belonging to the group is assigned to the document that belongs to the maximum group. A maximum group name determination unit that is determined based on words included; a narrowing selection unit that receives input of selection information indicating whether the maximum group name matches a user's intention; and the selection information matches the intention. Display a list of documents belonging to the maximum group and indicate that the selection information does not match the intention. , And it and a search result display unit for displaying a list of those not belonging to the largest group among the documents included in the search results.

  According to the document search system of the present invention, the search result can be narrowed down to a document belonging to the maximum group or a document not belonging to the maximum group according to the user's selection. Therefore, the user can narrow down the search results easily by only answering questions from the system without adding a new keyword.

  In addition, when the selection information indicates that it matches the intention, a document belonging to the maximum group is used as the search result, and when the selection information indicates that it does not match the intention, a document included in the search result Of these, the group that does not belong to the maximum group is used as the search result, the group classification unit classifies the documents included in the search result, the maximum group name determination unit determines the maximum group name, and the selection screen An information transmission unit may transmit the selection screen information, the selection information reception unit may receive the selection information, and the search result transmission unit may transmit information for displaying the list.

  Further, the document search system of the present invention includes a feature word extraction unit that extracts a feature word for the document from the search result document by an LSI technique and determines the extracted feature word as a word included in the document. You may make it prepare.

  The group classification unit may classify the document into a plurality of groups by clustering words included in the document.

  The group classification unit may weight the words included in the document by TF-IDF and classify the weighted words by clustering.

  Further, the group classification unit may perform a plurality of clustering by a plurality of different methods. In this case, it is possible to narrow down from a plurality of viewpoints.

  The maximum group name determination unit may calculate the frequency of each word included in the document belonging to the maximum group, and determine the word with the highest frequency as the maximum group name.

  The document search system of the present invention includes a category database that stores words related to the group in association with a group name indicating a group, and the maximum group name determination unit is included in the document belonging to the maximum group. A plurality of words extracted, and for each of the group names stored in the category database, the number of words included in the word extracted from the document among the related words is counted, and the counted The group name having the largest number may be determined as the maximum group name.

  The document search system of the present invention includes a category database that stores words as nodes in a tree structure, and the maximum group name determination unit extracts a plurality of words included in a document belonging to the maximum group, and the category A node having the deepest hierarchy among the nodes in which all the extracted words are included as descendants may be acquired from a database, and the acquired node may be determined as the maximum group name.

  The maximum group name determination unit weights words extracted from the document using TF-IDF, and all of the words extracted from the document having a weight equal to or greater than a predetermined value among the nodes are included as descendants. May be determined as the maximum group name.

  The group classification unit may classify only a predetermined number of documents among the search result documents into a plurality of groups.

  In addition, the document search system of the present invention includes an index storage unit that stores the document and a feature word of the document in association with the keyword, and the document search unit includes the document corresponding to the keyword and The feature words may be acquired from the index storage unit, and the group classification unit may classify the documents into a plurality of groups based on the feature words.

  In addition, the document search system of the present invention includes a category database that stores one or more categories to which the document belongs for each document, and the group classification unit selects the search result document based on the corresponding category. May be classified into a plurality of groups.

  In addition, the document search system of the present invention includes a category database that stores a category to which the document belongs and a document included in the maximum group when the selection information indicates that the document matches the intention. A category display section that displays a list of the categories to which the document belongs, a category input section that receives input of the categories, and a narrowed display that displays a list of documents belonging to the input category among the documents included in the maximum group May be provided.

  Other problems and solutions to be disclosed by the present application will be made clear by the embodiments of the invention and the drawings.

  According to the present invention, it is possible to narrow down efficiently in a document search process.

  Hereinafter, a document search system according to an embodiment of the present invention will be described. The document search system according to the present embodiment searches for documents according to keywords, and narrows down a large number of search results while making an alternative inquiry to the user. In this embodiment, instead of searching for the document itself, index information including words and summaries of the document collected by the crawler is searched.

== System configuration ==
FIG. 1 is a diagram showing the overall configuration of the document search system of this embodiment. As shown in the figure, the document search system of this embodiment includes a user terminal 10 and a search server 20, and the user terminal 10 and the search server 20 are connected via a communication network 30. A plurality of user terminals 10 and search servers 20 may be included.

  The communication network 30 is, for example, the Internet or a LAN (Local Area Network), and is constructed by an optical fiber, Ethernet (registered trademark), a telephone line network, a wireless communication network, or the like. In this embodiment, it is assumed that the user terminal 10 and the search server 20 communicate with each other using HTTP (HyperText Transfer Protocol).

  The user terminal 10 is a computer used by a user. The user terminal 10 is, for example, a personal computer, a mobile phone, or a PDA (Personal Digital Assistance). In the present embodiment, a browser for browsing a web page is operating on the user terminal 10, and the user operates the browser operating on the user terminal 10 to transmit a keyword to the search server as an HTTP request. . In the following description, an HTTP request including a keyword for searching for a document is referred to as a query.

  The search server 20 is a computer that searches documents, and is, for example, a personal computer or a workstation. In the present embodiment, the search server 20 is assumed to be an Internet search engine. The search server 20 accesses another server (not shown) that provides a Web page (document), generates a document summary, generates a document index, and indicates a document corresponding to a keyword. (Uniform Resource Locator) and summary are presented to the user.

== Hardware ==
FIG. 2 is a diagram illustrating a hardware configuration of the search server 20. As shown in the figure, the search server 20 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 stores various data and programs, for example, a hard disk, a flash memory, a CD-ROM drive, and the like. The CPU 201 implements various functions by reading a program stored in the storage device 203 into the memory 202 and executing it. The communication interface 204 is an interface for connecting to the communication network 30. For example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a telephone line network, and a wireless communication network are used. Such as a wireless communication device. The input device 205 is a keyboard, mouse, touch panel, microphone, or the like that accepts data input. The output device 206 outputs data, for example, a display, a printer, a speaker, or the like.

  Note that the hardware configuration of the search server 20 is assumed to be that of a general personal computer or workstation. The hardware configuration of the user terminal 10 is the same as that of the search server 20.

== Software ==
FIG. 3 is a diagram illustrating a software configuration of the search server 20. As shown in the figure, the search server 20 includes a query reception unit 211, a search execution unit 212, a search result generation unit 213, a clustering analysis unit 214, a maximum cluster determination unit 215, a category determination unit 216, a proposal information generation unit 217, Each function part of the search result transmission part 218, the selection information reception part 219, and the crawler process part 220, and each memory | storage part of the index database 251 and the category database 252 are provided.

  The query receiving unit 211, the search execution unit 212, the search result generation unit 213, the clustering analysis unit 214, the maximum cluster determination unit 215, the category determination unit 216, the proposal information generation unit 217, the search result transmission unit 218, and the selection information reception unit. The 219 and the crawler processing unit 220 are realized by the CPU 201 included in the search server 20 reading out the program stored in the storage device 203 to the memory 202 and executing it. Further, the index database 251 and the category database 252 are realized as storage areas provided by the memory 202 and the storage device 203.

  The index database 251 stores information (hereinafter referred to as index information) including a URL of a document to be searched and a summary of the document. A configuration example of index information stored in the index database 251 is shown in FIG. As shown in the figure, the index information includes an ID for identifying the index information (hereinafter referred to as an index ID), a document URL, a document title, a document summary, and a word characterizing the document (hereinafter referred to as a feature word). Is included). In the present embodiment, it is assumed that the search server 20 performs document search processing by searching index information in which a keyword input by a user is included in a summary.

  The crawler processing unit 220 acquires documents published by various computers connected to the communication network 30 and extracts feature words from the acquired documents. In the extraction of feature words, for example, a predetermined number of words extracted from a document by general morphological analysis in descending order of appearance frequency can be used as feature words. The crawler processing unit 220 creates index information including a URL indicating the acquired document, a title included in the document, a summary of the document, and a feature word. The document title is a title set as a document attribute. For example, in the case of a document described in HTML, it can be the content of the TITLE tag, and in the case of text data, it can be the text on the first line. The document summary can be obtained by, for example, extracting text data before and after the feature word from the document. The crawler processing unit 220 registers in the index database 251. The crawler processing unit 220 is a so-called crawler, spider, robot, indexer, or the like. Note that a database creation process by a general crawler can be used for the process in which the crawler processing unit 220 creates index information. It is assumed that the crawler processing unit 220 continues to generate index information and register it in the index database 251 periodically.

  It should be noted that an LSI (Latent Semantic Indexing) technique can be used for extracting feature words of a document. In this case, the crawler processing unit 220 generates a matrix A having the words included in the document as rows, each document as a column, and the frequency of the words included in the document as elements, and the matrix A is subjected to special position decomposition. It is decomposed into three matrices U, S, V by (SVD; Singular Value Decomposition). Here, A = U × S × V. S is a diagonal matrix, and the crawler processing unit 220 extracts only a predetermined number of elements (feature components) of S from the larger one, compresses the dimensions to generate an example S ′, and A ′ = U × S The matrix A is calculated from “× V”. As a result, noise is eliminated in A ′, and the feature words are more emphasized. The crawler processing unit 220 can determine, as a feature word, a word in a row whose elements are greater than a predetermined threshold for the document in each column of A ′ (a feature word extraction unit).

  The category database 252 manages words belonging to the category for each category. FIG. 5 is a diagram illustrating a configuration example of the category database 252. As shown in the figure, the category database 252 of this embodiment manages each word in a tree structure, each word is a category name, and the words included in the category are descendants of the category. Is configured.

  In this embodiment, for simplicity of explanation, it is assumed that words are registered without overlapping in the tree structure, but a plurality of the same words may be included in the tree structure. For example, when the same word belongs to different categories, the word is included in the descendants of each category.

  Further, the structure of the category database 252 is not limited to the tree structure as shown in FIG. 5, and the category and the word only need to be associated with each other. For example, a category and a word belonging to the category may be associated with each other and managed in a table format. In the present embodiment, the category database 252 is preliminarily registered by the administrator so that the higher the concept, the higher the concept word.

The query receiving unit 211 receives a query transmitted from the user terminal 10.
The search execution unit 212 searches for a document corresponding to the keyword included in the query. Specifically, the search execution unit 212 acquires index information (hereinafter, referred to as a keyword search result) including a keyword included in the query as a feature word from the index database 251.

  The search result generation unit 213 generates screen data (hereinafter referred to as a search result screen) for displaying a summary list of index information included in the keyword search result. In this embodiment, the search result screen generated by the search result generation unit 213 is described in HTML (HyperText Markup Language), and the search result generation unit 213 attaches a link to the URL to the title of each index information. A list of data and summaries shall be described. It should be noted that a search result screen generation process generated by the search result generation unit 213 may employ a screen generation process for displaying a search result by a general search engine.

  The clustering analysis unit 214 performs a clustering analysis process on the index information included in the keyword search result, and classifies the index information into clusters. For example, the clustering analysis unit 214 calculates an index value (hereinafter referred to as a TFIDF value) obtained by TF-IDF (Term Frequency-Inverse Document Frequency) for each feature word. The clustering analysis unit 214 selects a predetermined number (for example, 1 to 5) of feature words in descending order of TFIDF values for each document, and generates a vector value from the TFIDF values of the selected feature words. The clustering analysis unit 214 calculates the inner product value of the vector values as the similarity for each of the combinations of all the two index information included in the keyword search result, and classifies the vector values having a short distance as a cluster. Note that the clustering analysis unit 214 can classify the index information into clusters by a general clustering analysis technique. The clustering analysis unit 214 stores the index ID of the index information included in the cluster in the memory 202 for each classified cluster. For example, the clustering analysis unit 214 can store a list of index IDs in the table 253 illustrated in FIG. 6 in association with cluster identification information (hereinafter referred to as cluster ID).

  The maximum cluster determination unit 215 determines the cluster with the largest number of classified index information (hereinafter referred to as the maximum cluster) among the clusters classified by the clustering analysis unit 214.

  The category determination unit 216 determines the name of the maximum cluster (hereinafter referred to as the maximum category name) based on the words included in the summary of the index information classified into the maximum cluster. In the present embodiment, the category determination unit 216 uses the tree structure stored in the category database 252 as the most out of the nodes in which all the words included in the summary of the index information classified into the largest cluster are included in the descendants. A deep category is determined as the maximum category name. Details of the maximum category name determination process by the category determination unit 216 will be described later.

  The proposal information generation unit 217 generates screen data (hereinafter referred to as a proposal screen) for inquiring whether or not the maximum category name matches the user's intention. In the present embodiment, the proposal information generation unit 217 describes a screen that displays a “Yes” button and a “No” button in HTML together with a message “What do you want to know is“ maximum category name ”?” The generated screen is generated as a proposal screen.

The search result transmission unit 218 transmits the search result screen and the proposal screen to the user terminal 10.
The selection information receiving unit 219 receives from the user terminal 10 information (hereinafter referred to as selection information) indicating that a proposal screen is displayed on the user terminal 10 and either “Yes” or “No” has been selected. To do.

== Processing ==
Hereinafter, the flow of processing in the document search system of this embodiment will be described with reference to FIGS. FIG. 7 is a diagram for explaining the overall flow of the document search process in the document search system, and FIG. 8 is a diagram showing an example of a screen displayed on the user terminal 10 in the document search process.

First, the user terminal 10 displays a screen 40 shown in FIG. The screen 40 includes a keyword input field 411.
When the search button 412 is pressed, the user terminal 10 transmits a query including the keyword input in the input field 411 to the search server (FIG. 7, S401).

  The query receiving unit 211 of the search server 20 receives a query from the user terminal 10, and the search execution unit 212 performs search result screen creation processing (S402). The flow of the search result screen creation process is shown in FIG. The search execution unit 212 extracts a keyword from the query (S501), reads index information including the extracted keyword as a feature word from the index database 251 and sets it as a keyword search result (S502). The search result generation unit 213 generates a search result screen by describing in HTML a list that lists titles with links to URLs of index information included in the keyword search results and summaries (S503). .

  After generating the search result screen, the search server 20 creates a proposal screen (S403). The flow of the proposal screen creation process is shown in FIG.

  The clustering analysis unit 214 performs clustering processing of index information included in the keyword search result (S521). In the clustering, as described above, TFIDF values are created for each feature word included in the index information, and vector values based on a predetermined number of TFIDF values are calculated in descending order, and are included in the keyword search result. It is possible to use a general cluster analysis process that calculates an inner product of vector values for each set of two pieces of index information and classifies those having a short distance as clusters. The clustering analysis unit 214 creates the table 253 shown in FIG. 6 in an empty state, and registers a list of index IDs of the index information belonging to each cluster classified by the clustering process in the table 253 of FIG. 6 (S522). . The maximum cluster determination unit 215 determines a cluster having the largest number of index information belonging to the clusters classified by the clustering analysis unit 214 as the maximum cluster (S523). The category determination unit 217 determines the maximum category name by the maximum category name determination process of FIG. 11 described later (S524), a message “What do you want to know is“ maximum category name ”?” And “Yes” Proposal information describing the button and the HTML tag for displaying the “No” button is generated (S525).

The maximum category name determination process by the category determination unit 216 is shown in FIG.
The category determining unit 216 first sets 0 to the variable n (S541). The category determination unit 216 acquires index information belonging to the maximum cluster. For example, the category determination unit 216 acquires an index ID corresponding to the cluster ID of the largest cluster from the table 256, and reads index information corresponding to the acquired index ID from the index database 251. The category determination unit 216 extracts feature words registered in the category database 252 among the feature words included in the read index information, and sets the extracted feature word list as a variable CL (S542). The category determination unit 216 sets a counter for each feature word to 0 (S543).

  The category determination unit 216 increments n (S544), and sets a feature word included in CL in the variable PL (S545). The category determination unit 216 acquires the n-th ancestor node of the feature word from the category database 252 (S546), sets the acquired node as CL (S547), and increments the counter for each node in the CL (S548). If the number of nodes included in the CL is not 1 (S549: NO), category determination unit 216 repeats the processing from step S544.

  When the number of nodes included in the CL is 1 (S549: YES), the category determination unit 216 includes the PL if the node included in the CL is the root of the tree structure (S550: YES). The node with the largest counter is the maximum category name (S551), and if the node included in CL is not the root (S550: NO), the node included in CL is the maximum category name. (S552).

  As described above, the category determining unit 216 determines, as the maximum category name, the node having the deepest hierarchy among the nodes including all the feature words included in the index information belonging to the maximum cluster as descendants. If there is no node other than the root, the highest category name is the node that is included in the summary of index information belonging to the largest cluster among the descendant nodes of the hierarchy immediately below the root Can be determined as By determining the maximum category name in this way, the maximum cluster can be expressed with the upper concept words including the words included in the maximum cluster. Therefore, the user can easily determine what kind of document is classified as the maximum cluster.

  As described above, when the maximum category name is determined and the proposal information screen is created, the search result device unit 218 of the search server 20 sends screen data including the proposal information screen and the search result screen to the user terminal 10. Transmit (S404). Based on this screen data, the user terminal 10 displays a screen such as the screen 42 in FIG. The screen 42 displays a message for inquiring whether or not the search target intended by the user includes the maximum category name (in the example of FIG. 8, “Do you want to know“ animal character ”?”). )) And a button 422 for selecting “Yes” and a button 423 for selecting “No” are displayed. A list of keyword search results (search result screen) is displayed in the display field 424 of the screen 42. The user presses the button 422 or the button 423 on the screen 42 to select whether or not the maximum category name matches the user's search intention (S405).

  When the user presses the button 422 or the button 423, the user terminal 10 transmits selection information indicating “Yes” or “No” to the search server 20 in accordance with the pressed button (S406).

Upon receiving the selection information transmitted from the user terminal 10, the selection information receiving unit 219 of the search server 20 performs a search result update process (S407). FIG. 12 is a diagram showing the flow of search result update processing.
When the selection information is “Yes” (S561: YES), the selection information receiving unit 219 reads the index ID corresponding to the cluster ID of the largest cluster from the table 253, and performs a keyword search for the index information corresponding to the read index ID. The result is taken (S562).
On the other hand, when the selection information is “No” (S561: NO), the selection information receiving unit 219 reads the index ID corresponding to the cluster ID other than the cluster ID of the maximum cluster from the table 253, and corresponds to the read index ID. The index information to be used is set as a keyword search result (S563).

The search result generation unit 213 generates a search result screen by describing in HTML a list that lists titles with links to URLs of index information included in keyword search results and summaries (S564). .
Next, the search server 20 performs a proposal information screen creation process shown in FIG. 10, and creates a proposal information screen (S565).
When the search result is updated, the search server 20 repeats the processing from step S404 in FIG. 7 to transmit to the screen data user terminal 10 including the proposal screen and the search result screen.

  After the search result update process is performed, a screen such as a screen 43 in FIG. 8 is displayed on the user terminal 10. The screen 43 has the same configuration as the screen 42. The maximum category name of the maximum cluster as a result of classifying the narrowed search results into a plurality of clusters by clustering is displayed in the display column 421, and the narrowed search results Is displayed in the display column 424.

  As described above, according to the document search system of this embodiment, the search server 20 can propose narrowing to the user. Therefore, the user simply selects “Yes” or “No” for the proposal from the search server 20 without actively performing a search such as adding a keyword or changing a search condition. Search results can be narrowed down easily. In general, even if the first keyword comes to mind, it is often difficult to add the second keyword. However, according to the document search system of this embodiment, the user needs to consider a new keyword. It is convenient because it disappears.

  Further, in the document search system of the present embodiment, the high-level concept of words included in the summary of index information classified into the maximum cluster can be presented to the user as the maximum category name. Therefore, the user can easily grasp what kind of document is classified into the largest cluster.

  Also, according to the document search system of the present embodiment, clustering is performed again for the index information classified into the maximum cluster or the index information classified into other clusters according to the selection from the user. For example, when a keyword is added or a new search condition is specified, another new document may be included in the result of the new search, as in the present embodiment. Further, by narrowing down the search results by clustering, the search results can be surely narrowed down.

  In the document search system according to the present embodiment, it is assumed that the feature word is included in the index information in advance. However, the present invention is not limited to this. You may make it perform.

  In the document search system of this embodiment, the index information is searched. However, the document may be directly searched. In this case, instead of the index database 251, a document database that manages documents can be adopted to search for words included in the documents.

  In addition, the index database 251 manages only URLs and feature words, searches for index information that matches the keyword included in the query, acquires the document indicated by the URL each time, and is included in the acquired document. A summary may be created by extracting words.

  In the present embodiment, clustering is performed for all index information that matches the keywords included in the query. For example, clustering is performed only for a predetermined number of index information such as 100 or 1000. You may do it.

  In this embodiment, clustering is performed using feature words. For example, each document is categorized in advance so that each document belongs to a plurality of categories, and clustering is performed using the categories. You may make it perform. In this case, the index information includes the category name in addition to the feature word, and the category database 252 manages the category name, and the category included in the index information that matches the keyword included in the query. To cluster. By performing clustering using a finite number of categories, the number of dimensions can be reduced. For example, if there are 1000 types of categories, 1000 dimensions may be used for clustering calculation.

  In addition, when documents are classified into categories in advance, the search results may be further narrowed down by category. In this case, for example, the search server 20 counts the documents included in the maximum cluster for each category, and transmits category data for selecting a predetermined number of categories to the user terminal 10 in descending order of the number of belonging documents. Screen transmission unit, category input unit for receiving a category selected from the user terminal 10, and screen data for displaying only a list of documents belonging to the selected category among documents included in the maximum cluster (hereinafter, narrowing down) The search result transmission unit 218 transmits the narrowing result screen to the user terminal 10. In this way, it is possible to easily narrow down the documents included in the maximum cluster according to the document category.

  In the present embodiment, the clustering is performed only once. However, the present invention is not limited to this. For example, the clustering is performed a plurality of times by different methods such as using a plurality of distance functions. Among all the clusters, a cluster having the largest number of documents belonging to the largest cluster is set as the maximum cluster. This makes it possible to narrow down search results from various viewpoints.

  In the present embodiment, the category determination unit 216 sets the maximum category name to the deepest hierarchy among nodes in which all words included in the summary of the index information classified into the maximum cluster are included in the descendants. However, the present invention is not limited to this. For example, the category determination unit 216 calculates the appearance frequency of words included in the summary of index information classified into the maximum cluster, and determines the word with the highest appearance frequency as the maximum category name. You may make it do. Further, the category determination unit 216 may calculate the TFIDF value of each word included in the summary of the index information classified into the maximum cluster, and determine the word having the highest TFIDF value as the maximum category name. The maximum category name including a predetermined number of words may be generated in descending order of the TFIDF value.

  In the present embodiment, the category database 252 manages words in a tree structure, but the words belonging to the category may be stored in association with the category name. In this case, for each index information, the clustering analysis unit 214 counts the number of words included in the summary for each category name registered in the category database 252, and indexes the category name having the highest count number. It is possible to classify the index information having the same category name as a cluster as a category name of information. Further, the category determining unit 216 can determine the category name as the maximum category name.

  Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention and is not intended to limit the present invention. The present invention can be changed and improved without departing from the gist thereof, and the present invention includes equivalents thereof.

  For example, the document search system of the present embodiment can be configured with only the search server 20. In this case, the search server 20 may receive a keyword input from the input device 205 such as a keyboard or a mouse, perform a search, and output the search result to the output device 206 such as a display. This configuration is useful, for example, when a large amount of documents are stored in a personal computer.

It is a figure which shows the whole structure of the document search system of this embodiment. 2 is a diagram illustrating a hardware configuration of a search server 20. FIG. 2 is a diagram illustrating a software configuration of a search server 20. FIG. It is a figure which shows the structural example of the index information memorize | stored in the index database. It is a figure which shows the structural example of the category database 252. It is a figure which shows the structural example of the table 253 which manages index ID for every cluster. It is a figure explaining the flow of the whole document search process in a document search system. It is a figure which shows the example of a screen displayed on the user terminal 10 in a document search process. It is a figure which shows the flow of a creation process of a search result screen. It is a figure which shows the flow of the creation process of a proposal screen. It is a figure which shows the flow of the determination process of a maximum category name. It is a figure which shows the flow of the update process of a search result.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 User terminal 20 Search server 30 Communication network 211 Query reception part 212 Search execution part 213 Search result generation part 214 Clustering analysis part 215 Maximum cluster determination part 216 Category determination part 217 Information generation part 217 Proposal information generation part 218 Search result transmission part 219 Selection information receiving unit 220 Crawler processing unit 251 Index database 252 Category database 253 Table

Claims (16)

A system for retrieving documents,
A keyword input unit that accepts keyword input;
A document search unit for searching for a document corresponding to the keyword;
A group classification unit for classifying the search result documents into a plurality of groups based on words included in the document;
A maximum group name determining unit that determines a maximum group name representing a maximum group having the largest number of documents belonging to the group based on words included in the documents belonging to the maximum group;
A refinement selection unit that receives input of selection information indicating whether or not the maximum group name matches a user's intention;
When the selection information indicates that it matches the intention, a list of documents belonging to the maximum group is displayed, and when the selection information indicates that the intention does not match the intention, the documents included in the search result A search result display section that displays a list of items that do not belong to the maximum group,
A document retrieval system comprising: The document search system according to claim 1,
When the selection information indicates that it matches the intention, a document belonging to the largest group is used as the search result. When the selection information indicates that it does not match the intention, the document included in the search result The group classification unit classifies documents included in the search result into groups, and the maximum group name determination unit determines the maximum group name, and the selection screen information is transmitted. The selection screen information is transmitted, the selection information reception unit receives the selection information, and the search result transmission unit transmits information for displaying the list,
Document search system characterized by The document search system according to claim 1,
A feature word extraction unit that extracts a feature word for the document from the search result document by an LSI technique and determines the extracted feature word as a word included in the document;
Document search system characterized by The document search system according to claim 1,
The group classification unit classifies the document into a plurality of groups by clustering words included in the document;
Document search system characterized by The document search system according to claim 4,
The group classification unit weights words included in the document by TF-IDF and classifies the weighted words by clustering;
Document search system characterized by The document search system according to claim 4,
The group classification unit performs a plurality of clustering by a plurality of different methods;
Document search system characterized by The document search system according to claim 1,
The maximum group name determination unit calculates the frequency of each word included in the document belonging to the maximum group, and determines the word with the highest frequency as the maximum group name;
Document search system characterized by The document search system according to claim 1,
A category database that stores words related to the group in association with the group name indicating the group,
The maximum group name determination unit extracts a plurality of words included in a document belonging to the maximum group, and extracts from the document among the related words for each of the group names stored in the category database. Counting the number of words included in the selected word, and determining the group name with the largest number as the maximum group name,
Document search system characterized by The document search system according to claim 1,
A category database that stores words as nodes in a tree structure
The maximum group name determination unit extracts a plurality of words included in a document belonging to the maximum group, and has the deepest hierarchy among the nodes in which all the extracted words are included as descendants from the category database. And determining the acquired node as the maximum group name;
Document search system characterized by The document search system according to claim 9,
The maximum group name determination unit weights words extracted from the document by TF-IDF, and among the nodes, all words extracted from the document having a weight equal to or greater than a predetermined value are included as descendants. Determining one as said maximum group name;
Document search system characterized by The document search system according to claim 1,
The group classification unit classifies only a predetermined number of documents among the search result documents into a plurality of groups;
Document search system characterized by The document search system according to claim 1,
An index storage unit that stores the document and feature words of the document in association with the keyword;
The document search unit acquires the document and the feature word corresponding to the keyword from the index storage unit,
The group classification unit classifies the documents into a plurality of groups based on the feature words;
Document search system characterized by The document search system according to claim 1,
A category database for storing one or more categories to which the document belongs for each document;
The group classification unit classifies the search result documents into a plurality of groups based on the corresponding category;
Document search system characterized by The document search system according to claim 1,
A category database storing a category to which the document belongs for each document;
A category display unit for displaying a list of the categories to which the document included in the maximum group belongs, when the selection information indicates that the intention is matched;
A category input unit that receives input of the category;
A refinement display unit for displaying a list of documents belonging to the input category among the documents included in the maximum group;
A document retrieval system comprising: A method for searching documents,
Computer
Receiving a keyword input;
Searching for a document corresponding to the keyword;
Classifying the search result documents into a plurality of groups based on words contained in the documents;
Determining a maximum group name representing a maximum group having the largest number of documents belonging to the group based on words included in documents belonging to the maximum group;
Receiving input of selection information indicating whether or not the maximum group name matches a user's intention;
When the selection information indicates that it matches the intention, a list of documents belonging to the maximum group is displayed, and when the selection information indicates that the intention does not match the intention, the documents included in the search result Displaying a list of items not belonging to the largest group;
The document retrieval method characterized by performing. A program for searching documents,
On the computer,
Receiving a keyword input;
Searching for a document corresponding to the keyword;
Classifying the search result documents into a plurality of groups based on words contained in the documents;
Determining a maximum group name representing a maximum group having the largest number of documents belonging to the group based on words included in documents belonging to the maximum group;
Receiving input of selection information indicating whether or not the maximum group name matches a user's intention;
When the selection information indicates that it matches the intention, a list of documents belonging to the maximum group is displayed, and when the selection information indicates that the intention does not match the intention, the documents included in the search result Displaying a list of items not belonging to the largest group;
A program for running

Images