JP6340351B2 - Information search device, dictionary creation device, method, and program - Google Patents

Description

  The present invention relates to an information search device, a dictionary creation device, a method, and a program that use speech or text as input.

  2. Description of the Related Art Conventionally, as an information search system, a method of searching a document that matches a query by a process such as keyword matching for a query input by a user is known. In the case of the keyword match search, the keyword of the query must match the keyword in the document completely, and there is a problem that the recall rate (Recall) of the search is lowered. Therefore, query expansion is known as a technique for matching a wider range of documents by automatically increasing the number of keywords included in the query (Patent Document 1).

  As a search method other than the keyword match type, concept search is known. Concept search is a technique in which a keyword is expressed by an n-dimensional vector of continuous values, and the center of gravity of the vector is regarded as a query vector. Similarly, the document vector is expressed by the center of gravity of the keyword vector in the document, and the similarity between the query vector and the document vector is calculated. Search is executed by outputting search results in descending order of similarity. Unlike keyword matching, conceptual search has the advantage that documents related to topics close to a query can be searched even if the keywords do not match completely.

JP 2010-123036 A JP 2010-182041 A

  However, in a conventional query expansion type search system, it is necessary to manually determine the number of keywords for expansion. Also, if there are too many keywords to expand, there is a higher risk that keywords that are less relevant to the keywords of the original query will match. On the contrary, when the number of keywords to be expanded is small, the possibility that the keywords do not match in the expanded query increases. Therefore, there is a problem that it is difficult to perform query expansion in which the keyword expansion number is reasonable.

  In addition, in the concept search type search using concept vectors, it is possible to search documents whose contents are conceptually close without expanding the query. In concept search, the weight of each word in the document is searched. There is a problem that even if important words are different, but other parts (function word part etc.) are matched, a high score may be shown. In addition, the concept vector search finds the center-of-gravity vector of all words in the document, so that there is a problem that the search accuracy decreases when the document is long.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide an information search device, a dictionary creation device, a method, and a program that can accurately search a document related to a query. And

  In order to achieve the above object, an information search device according to a first aspect of the present invention provides a weight of a search target document keyword included in a search target document created for each search target document included in a search target document set. , A search index that is a combination of the search target document keyword and a document ID representing the search target document, and the document content of the search target document and the search target document created for each of the search target documents A document database that is a combination of document IDs, a concept vector that is created based on a set of concept documents and that expresses each of the concept document keywords included in the concept document, and is represented by an n-dimensional vector; and the search index The search target document created for each of the search target documents based on the document database Based on the concept similarity dictionary in which the concept document keyword having the highest similarity is recorded together with the similarity to the search target document keyword included, the input query, the search index, and the concept similarity dictionary Then, for each of the search target documents included in the document database, the search using the similarity between the search target document keyword similar to the keyword included in the query and the weight of the search target document keyword And a score calculation unit for calculating a relevance score with the target document.

  According to a second aspect of the present invention, there is provided an information search method comprising: a weight of a search target document keyword included in the search target document created for each search target document included in the search target document set; the search target document keyword; A search index that is a combination of a document ID representing the search target document, and a document that is created for each of the search target documents and is a combination of the document content of the search target document and the document ID of the search target document The search target document based on a database, a concept vector represented by an n-dimensional vector representing each of the concept document keywords included in the concept document, created based on the concept document set, and the search index For each of the search target document keywords included in the search target document created for each of the The information search method in the information search apparatus includes a concept similarity dictionary that records the concept document keyword with a high degree of similarity together with a score calculation unit, and the score calculation unit includes an input query, Based on the search index and the concept similarity dictionary, for each of the search target documents included in the document database, similarity with the search target document keyword similar to the keyword included in the query, and A relevance score with the search target document is calculated using the weight of the search target document keyword.

  According to the first and second inventions, the score calculator calculates a query for each of the search target documents included in the document database based on the input query, the search index, and the concept similarity dictionary. The relevance score with the search target document is calculated using the similarity between the included keyword and the search target document keyword and the weight of the search target document keyword.

  As described above, based on the input query, the search index, and the concept similarity dictionary, for each search target document included in the document database, a search target document keyword similar to the keyword included in the query is determined. By calculating the relevance score with the search target document using the similarity and the weight of the search target document keyword, a document related to the query can be searched with high accuracy.

  According to a third aspect of the present invention, there is provided an information search apparatus including, for each search target document included in a search target document set, a weight of the search target document keyword included in the search target document, the search target document keyword, and the search target. A search index creation unit for creating a search index storing a combination of a document ID representing a document, a document content of the search target document, and a document ID of the search target document created for each of the search target documents A document database that is a combination of the above, a concept vector model creation unit that creates a concept vector represented by an n-dimensional vector that represents each of the concept document keywords included in the concept document based on the set of concept documents, and the concept Based on the vector, the search index, and the document database, the search target document A concept similarity dictionary creating unit that creates a concept similarity dictionary in which the concept document keyword having the highest similarity is recorded together with the similarity with respect to the search target document keyword included in the search target document. And the search target document keyword similar to the keyword included in the query for each of the search target documents included in the document database based on the query, the search index, and the concept similarity dictionary. A score calculation unit for calculating a relevance score with the search target document using the similarity and the weight of the search target document keyword.

  An information search method according to a fourth invention is a document database that is a combination of the document content of the search target document and the document ID of the search target document, created for each of the search target documents included in the search target document set. And an information search method in an information search apparatus including a search index creation unit, a concept vector model creation unit, a concept similarity dictionary creation unit, and a score calculation unit, wherein the search index creation unit For each search target document included in the search target document set, a search is a combination of the weight of the search target document keyword included in the search target document, the search target document keyword, and a document ID representing the search target document. An index is created, and the concept vector model creating unit creates a concept sentence included in the concept document based on the concept document set. A concept vector expressed by an n-dimensional vector that expresses each of the keywords is created, and the concept similarity dictionary creating unit generates, for each of the search target documents, based on the concept vector and the search index. For the search target document keyword included in the search target document, create a concept similarity dictionary in which the concept document keyword having the highest similarity is recorded together with the similarity, and the score calculation unit includes the input query, Based on the search index and the concept similarity dictionary, for each of the search target documents included in the document database, the similarity with the search target document keyword similar to the keyword included in the query, and The relevance score with the search target document is calculated using the weight of the search target document keyword.

  According to the third and fourth aspects of the invention, the search index creation unit determines, for each of the search target documents included in the search target document set, the weight of the search target document keyword included in the search target document, the search target document keyword, A search index that is a combination with a document ID representing a search target document is created, and a concept vector model creation unit represents each of the concept document keywords included in the concept document based on the concept document set. A concept vector expressed in vector is created, and based on the concept vector and the search index, the concept similarity dictionary creation unit creates the most similar to the search target document keyword included in the search target document for each of the search target documents. Create a concept similarity dictionary in which high-concept conceptual document keywords are recorded together with the similarity, and score calculator Thus, based on the input query, the search index, and the concept similarity dictionary, each search target document included in the document database is similar to the search target document keyword similar to the keyword included in the query. , And the weight of the search target document keyword, the relevance score with the search target document is calculated.

  Thus, for each search target document included in the search target document set, a search index is created, a concept vector is created based on the concept document set, and the search target document is set based on the concept vector and the search index. For each of the above, a concept similarity dictionary is created, and based on the input query, search index, and concept similarity dictionary, each search target document included in the document database is related to the search target document. By calculating the degree score, a document related to the query can be searched with high accuracy.

  The dictionary creation device according to a fifth aspect of the present invention provides a weight of a search target document keyword included in the search target document, the search target document keyword, and the search target for each search target document included in the search target document set. A search index creation unit for creating a search index storing a combination of a document ID representing a document, a document content of the search target document, and a document ID of the search target document created for each of the search target documents A document database that is a combination of the above, a concept vector model creation unit that creates a concept vector represented by an n-dimensional vector that represents each of the concept document keywords included in the concept document based on the set of concept documents, and the concept Based on the vector, the search index, and the document database, the search target document A concept similarity dictionary creating unit that creates a concept similarity dictionary in which the concept document keyword having the highest similarity is recorded together with the similarity to the search target document keyword included in the search target document. Consists of.

  A dictionary creation method according to a sixth aspect of the present invention is a document database that is created for each of the search target documents included in the search target document set and is a combination of the document content of the search target document and the document ID of the search target document. A dictionary creation method including a search index creation unit, a concept vector model creation unit, and a concept similarity dictionary creation unit, wherein the search index creation unit adds the search index to the search target document set. For each search target document included, create a search index that is a combination of the weight of the search target document keyword included in the search target document, the search target document keyword, and a document ID representing the search target document; The concept vector model creation unit is a concept for creating a concept vector in which a word is expressed by an n-dimensional vector. A concept vector represented by an n-dimensional vector that represents each of the concept document keywords included in the concept document based on the book set, and the concept similarity dictionary creating unit includes the concept vector, the search index, For each of the search target documents, a concept similarity dictionary is created in which the concept document keyword having the highest similarity is recorded together with the similarity with respect to the search target document keyword included in the search target document.

  According to the fifth and sixth inventions, for each of the search target documents included in the search target document set, the search index creation unit calculates the weight of the search target document keyword included in the search target document, the search target document keyword, Based on a concept document set for creating a search index that is a combination with a document ID representing a search target document and creating a concept vector in which words are expressed by n-dimensional vectors by a concept vector model creation unit A concept vector expressed by an n-dimensional vector that expresses each of the concept document keywords included in the document is created, and each of the search target documents is created based on the concept vector and the search index by the concept similarity dictionary creating unit. The concept document key having the highest similarity with respect to the search target document keyword included in the search target document The over de creates a record concepts similarity dictionary with similarity.

  Thus, for each search target document included in the search target document set, a search index is created, a concept vector is created based on the concept document set, and the search target document is set based on the concept vector and the search index. By creating a concept similarity dictionary for each of the above, it is possible to create a concept similarity dictionary for accurately searching for documents related to the query.

  In addition, the program of the present invention causes a computer to function as each unit constituting the information search device or the dictionary creation device, or causes the computer to execute each step of the information search method or the dictionary creation method. It is a program.

  As described above, according to the information search apparatus, method, and program of the present invention, each of the search target documents included in the document database based on the input query, the search index, and the concept similarity dictionary. On the other hand, a document related to the query is calculated by calculating a relevance score with the search target document by using the similarity between the search target document keyword similar to the keyword included in the query and the weight of the search target document keyword. Can be searched with high accuracy.

  Further, according to the information search device, the dictionary creation device, the method, and the program, a search index is created for each search target document included in the search target document set, and a concept vector is created based on the concept document set. By creating a concept similarity dictionary for each search target document based on the concept vector and the search index, it is possible to create a concept similarity dictionary for accurately searching for documents related to the query. .

It is a block diagram which shows the functional structure of the information search device which concerns on embodiment of this invention. It is a figure which shows an example of a search index. It is a figure which shows an example of a concept vector model. It is a figure which shows an example of a concept similarity dictionary. It is a figure which shows an example of the calculation content using the information search device which concerns on this embodiment. It is a figure which shows an example of the data used by an example of the calculation content using the information search device which concerns on this embodiment. It is a flowchart figure of the data creation process routine in the information search device which concerns on embodiment of this invention. It is a flowchart figure of the information search process routine in the information search device which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline of Embodiment of the Present Invention>
First, the outline | summary of embodiment of this invention is demonstrated.

  In the present embodiment, the keyword of the search target document included in the search index storing the combination of the weight of the search target document keyword included in the search target document and the combination of the search target document keyword and the document ID representing the search target document. The point is that a concept search is performed in units of keywords by calculating a similarity relationship with the keywords of the concept vector model in advance.

  For a keyword w in a document d in the search index of the search target document set D, the similarity in the concept space between the w and all keywords registered in the concept vector model is calculated and recorded. To do. This is applied to all the keywords in the document d and all the documents in the document set D.

  When the query Q is input to the search system in which the information search apparatus according to the present embodiment is implemented, the word similarity that is calculated and recorded in advance when calculating the relevance score between the query Q and the document d Is used to search the document d for a keyword having the highest similarity to the keyword q in the query Q.

  Here, when the keyword in the document d having the highest similarity with the keyword q is w, the score is calculated using the weight (TF, IDF, etc.) possessed by w and the similarity between the keywords q and w. This is calculated for all the keywords in the query Q, and finally, the sum of the scores of all the keywords in the query Q becomes the relevance score between the query Q and the document d. This is calculated for all the documents in the search target document set D, and finally sorted in the order of the relevance score to search for documents that match the query Q.

  Therefore, in the information search apparatus according to the present embodiment, a keyword match type search using query expansion and a search having the advantages of both concept vectors using concept vectors are possible. Since the query keyword and the keyword of the search target document are always matched on a one-to-one basis, it is not necessary to set the number of keywords to be expanded as in query expansion. Further, it is possible to eliminate the possibility that extended words match more than necessary. Since the weights of keywords such as TF / IDF can be used in the score calculation, all keywords are not handled in the same row as in the concept search, and a search considering important keywords can be performed.

  For example, when searching a set of Internet documents,

Query: “Credit card is not available for shopping”
Document A: “Online payment for our services”
Document B: “How to use e-mail for shopping services”

When there are documents A and B and a query, document B tends to show a high score for the query in the concept search. This is because the determination is based on the similarity of overall sentences such as “shopping” and “shopping service”, “use” and “use method”.

  However, a high score is assigned to the document A in the information search apparatus according to the present embodiment. This is because the weight of the keyword “online payment” can be taken into account in addition to the similarity between “credit card” and “online payment”.

  The information search apparatus according to the present embodiment is synonymous with replacing a keyword in a query with a keyword that appears in a search target document using a concept vector model. Therefore, it is effective for matching abbreviations and synonyms such as “Kureka” and “Credit Card”, “Net” and “Internet”. This can be said to be particularly useful for natural sentence search and voice search where the fluctuation of expression is large.

<Configuration of Information Retrieval Device According to Embodiment of the Present Invention>
Next, the configuration of the information search apparatus according to the embodiment of the present invention will be described. As shown in FIG. 1, an information search apparatus 100 according to an embodiment of the present invention stores a CPU, a RAM, a data creation processing routine to be described later, a program for executing an information search processing routine, and various data. It can be composed of a computer including a ROM. The information search apparatus 100 is functionally configured to include an input unit 10, a calculation unit 20, and a result output unit 90 as shown in FIG.

  The input unit 10 accepts a document set to be searched (hereinafter referred to as a search target document set) and a document set for creating the concept vector model 42 (hereinafter referred to as a concept document set), and outputs them to the similarity calculation unit 30. . Here, the collection method of the conceptual document set is not particularly specified, and when a query is performed on a Wikipedia (registered trademark) page set whose contents match the search target document set or a keyword extracted from the search target document set, A set of search result Web pages may be used. Each document included in the search target document set is set as a search target document, and each document included in the concept document set is set as a concept document.

  Further, the input unit 10 accepts a query (hereinafter referred to as an input query) input by the user and outputs it to the similarity matching search unit 50.

  The calculation unit 20 includes a similarity calculation unit 30, a storage unit 40, and a similarity match search unit 50.

  The similarity calculation unit 30 creates a search index 44 and a document database 46 based on the search target document set received from the input unit 10 and stores the search index 44 and the document database 46 in the storage unit 40. The similarity calculation unit 30 creates a concept vector model 42 based on the concept document set received from the input unit 10 and stores the concept vector model 42 in the storage unit 40. Further, the similarity calculation unit 30 creates a concept similarity dictionary 48 based on the search target document set and the concept document set received from the input unit 10 and stores them in the storage unit 40. In addition, the similarity calculation unit 30 includes a keyword extraction unit 32, a search index creation unit 34, a concept vector model creation unit 36, and a concept similarity dictionary creation unit 38. The similarity calculation unit is an example of a dictionary creation device according to the present invention.

  The keyword extraction unit 32 divides the search target document set received from the input unit 10, the search target document included in the concept document set, and the concept document into keyword units. Further, the keyword extraction unit 32 outputs each of the search target documents included in the search target document set divided into keyword units to the search index creation unit 34, and the concept document included in the concept document set divided into keyword units. Each is output to the concept vector model creation unit 36. A keyword unit obtained by dividing the search target document is set as a search target document keyword, and a keyword unit obtained by dividing the concept document is set as a concept document keyword.

  Here, if the keyword is English, it is divided into words, if it is Japanese, based on the result of morphological analysis, if nouns are consecutive, processing such as joining, processing such as extracting only nouns and verbs To create a keyword. At this time, the processing method and rules of the keyword creation processing can be freely set according to the search target. However, keywords are extracted according to the same processing procedure and rules (including a morphological analysis dictionary) for the search target document set and the concept document set.

  The search index creation unit 34 creates, for example, a search index 44 that is a search index as shown in FIG. 2 based on each of the search target documents input from the keyword extraction unit 32 and divided into keyword units. The document database 46 is created and stored in the storage unit 40. The search index 44 is generally the same as an inverted index used in an information search system.

  Here, the search index 44 shown in FIG. 2 includes a document ID that becomes the key of the search target document, a search target document keyword in the search target document, and a weight of the search target document keyword. As a weight calculation method, TF / IDF is used. The search index 44 is also used for score calculation in the similarity match search unit 50. As a weight calculation method, any weighting algorithm such as BM25 may be used.

  In the document database 46, the document ID is “Key” and the text of the search target document is recorded.

  Specifically, the search index creation unit 34 sets a document ID to be a key for each search target document divided into the obtained keyword units, and sets the search target included in the search target document. For each of the document keywords, the weight of the search target document keyword is calculated, and the search included in the search target document includes the document ID, the search target document keyword, and the weight of the search target document keyword as one index data. Index data for each target document keyword is added to the search index.

  The search index creation unit 34 also sets the document ID of the search target document set when creating the search index and the search target document of the search target document for each of the search target documents divided into the obtained keyword units. The text (document content) is combined and added to the document database 46.

  The concept vector model creation unit 36 creates a concept vector model 42 based on each of the concept documents divided from the keyword input from the keyword extraction unit 32 and stores the concept vector model 42 in the storage unit 40. Here, the concept vector model 42 is, for example, as shown in FIG. 3, a model composed of concept vectors in which words are expressed by vectors of n-dimensional continuous values for each of the concept document keywords. Here, in the present embodiment, an LSI using singular value decomposition is used as a method for creating the concept vector model 42. As a method for creating the concept vector model 42, any model such as a topic model or a model using a neural network may be adopted instead of an LSI using singular value decomposition.

  The concept similarity dictionary creation unit 38 calculates the word similarity between the concept document keyword of the concept vector model 42 and the search target document based on the search index 44 and the concept vector model 42 stored in the storage unit 40. The concept similarity dictionary 48 that summarizes the calculation results is created and stored in the storage unit 40.

  FIG. 4 shows an example of the concept similarity dictionary 48. The concept similarity dictionary 48 includes the concept document keyword of the concept vector model 42, the document ID, the search target document keyword of the search target document, and the similarity between the keywords. Here, with respect to the concept document keyword in the concept vector model 42, the search target document keyword having the highest similarity in each search target document of the search target document set is extracted and recorded.

  With this processing, when a concept document keyword in the concept vector model 42 is input as a query, it is possible to immediately refer to which search target document keyword in the search target document should be associated. In addition, since a search target document keyword having a higher similarity is a keyword having a higher degree of association, it is necessary to select an association having the highest degree of similarity. In the information search apparatus 100 according to the present embodiment, a search target document keyword that appears in the search target document other than the search target document keyword having the highest similarity is not used and is not recorded in this embodiment, so that the disk or memory Can be reduced.

  This processing is performed for each combination of all the concept document keywords of the concept vector model 42 and all the documents of the search target document set. When there are 100000 keywords and 500 search target documents in the concept vector model 42, a dictionary of combinations of 100,000 × 500 documents is created.

  As a method for calculating the similarity, a method that can be normalized while the range of similarity such as cosine distance is 0 to 1 is used. In order to reduce the amount of calculation and the memory capacity of the dictionary, if the keyword of the concept vector model 42 and the keyword of the search target document completely match, the similarity is not calculated and is also recorded in the concept similarity dictionary 48. Can be adopted (because it is obvious that the degree of similarity is maximum). Further, when the similarity is not more than the set threshold as a result of the similarity calculation, it is not possible to record it in the concept similarity dictionary 48 (it is assumed that the similarity is 0). The created similarity dictionary is used in the similarity match search unit. The calculation for creating the similarity dictionary can also be performed by distributed processing or the like.

  The storage unit 40 stores a concept vector model 42, a search index 44, a document database 46, and a concept similarity dictionary 48.

  Based on the input query input from the input unit 10, the search index 44 stored in the storage unit 40, the concept similarity dictionary 48, and the document database 46, the similarity match search unit 50 And the score with each of the search target documents is calculated, and a result based on each of the scores is output from the result output unit 90. The score is a score representing the degree of association between the input query and the target search target document.

  The similarity match search unit 50 includes a query keyword extraction unit 52 and a score calculation unit 60.

  The query keyword extraction unit 52 divides the input query input from the input unit 10 into keyword units according to the same processing (processing procedure and rule) as the keyword extraction unit 32 and transmits the keyword to the score calculation unit 60. Here, when the input query is a natural sentence or a voice input sentence, the processing in the query keyword extraction unit 52 is performed. However, when the input query is already in units of keywords, the query keyword extraction unit The process in 52 is not executed.

  The score calculation unit 60 uses the input query obtained from the query keyword extraction unit 52 and is divided into keyword units, the search index 44, the document database 46, and the concept similarity dictionary 48. A relevance score with each document is calculated, and a result based on each relevance score is output from the result output unit 90.

  The score calculation unit 60 includes a concept similarity reference unit 62 and a calculation unit 64.

  The concept similarity reference unit 62 selects one search target document included in the document database 46 and acquires the document ID of the search target document. In addition, the concept similarity reference unit 62 selects one keyword included in the input query based on the input query divided into keyword units acquired from the query keyword extraction unit 52. Further, the concept similarity reference unit 62 matches the selected keyword with the “keyword (concept)” field of the concept similarity dictionary 48 and the acquired document ID and the “document ID” of the concept similarity dictionary 48. The information in the “keyword (search document)” column that matches the “” column is acquired as a reference keyword. In addition, the process which acquires the said reference keyword is performed about all the keywords contained in an input query.

  Of the keywords included in the input query, the keyword matches the “keyword (concept)” field of the concept similarity dictionary 48, and the acquired document ID and the “document ID” of the concept similarity dictionary 48 When there is no information that matches the column “”, the keyword is excluded from the target of subsequent processing.

  In addition, the keyword determined to be excluded from the processing target is stored in a specific database, and after a series of processing of the information search apparatus 100 is completed, the search target related to the keyword is based on the keyword included in the database. The document set and the concept document set may be received by searching the Internet or the like based on the keyword, and the above-described similarity calculation unit 30 may perform the process.

  The calculation unit 64 calculates the relevance score between the search target document selected by the concept similarity reference unit 62 and the input query according to the following equation (1). In calculating the relevance score, the weight of the search target document keyword stored in the search index 44 and the similarity between the keywords in the concept similarity dictionary 48 are used. In the following formula (1), the selected search target document is d and the input query is Q. Further, in the present embodiment, the relevance score represents that the relevance is higher as the value is larger.

  Here, q is a keyword included in the input query Q, w is a reference keyword acquired in the concept similarity reference unit 62 (a search target document keyword in the search target document d that matches the keyword q with the maximum similarity), and weight (W) is the weight of the reference keyword w, and sim (q, w) is the similarity between the keyword q and the reference keyword w.

  The similarity is set to 0.0 as the minimum value and 1.0 as the maximum value (similar to the keyword exact match search).

  In addition, when the calculation unit 64 calculates the relevance score for all the search target documents included in the document database 46, the search target included in the document database 46 so that the relevance score is in descending order. Out of the sorted document texts, the top N items are output from the result output unit 90 as search results for the user query.

  In addition, when the relevance score is not calculated for all the search target documents included in the document database 46, the calculation unit 64 repeats the process of the concept similarity reference unit 62 and the process of the calculation unit 64. In this way, the relevance score can be calculated for all the search target documents in the search target document set.

  The similarity on the concept vector model is high between related keywords. In particular, the degree of similarity between keywords such as “Kureka” and “credit card”, “smartphone” and “smartphone” that have synonymous relationships such as shaking of expressions and abbreviations is extremely high (eg, a degree of similarity of 0.9 or more).

  In this case, the relevance score is such that the weight of the search target document keyword in the search target document is used as it is (for example, “Kureka” score = “credit card” score * 0.9). On the other hand, keywords with low relevance are low. In the method used in the present embodiment, the query keyword always matches one of the search target document keywords in the search target document.

  Therefore, when a low-relevance keyword is matched, the influence of the keyword can be reduced by multiplying the similarity (0.2 to the weight of the search target document keyword when calculating the relevance score). The one with a similarity of 0.3 is used).

  In FIG. 5, the example of the content of calculation of the relevance score performed in the information search device 100 which concerns on this embodiment is shown. In this example, the concept vector model 42 is created in advance. Further, the search target document set, the document database 46, the search index 44, and the concept similarity dictionary 48 used in the example of the calculation contents are assumed to use those shown in FIG.

<Operation of Information Retrieval Device According to Embodiment of the Present Invention>
Next, the operation of the information search apparatus 100 according to the embodiment of the present invention will be described. When the information search apparatus 100 receives the search target document set and the concept document set by the input unit 10, the information search apparatus 100 executes a data creation processing routine shown in FIG. When the information search apparatus 100 receives an input query by the input unit 10 after the data creation process routine, the information search apparatus 100 executes the information search process routine shown in FIG. The data creation processing routine is an example of a dictionary creation method according to the present invention.

  First, the data creation processing routine shown in FIG. 7 will be described.

  For each of the search target documents included in the search target document set received by the input unit 10 and each of the concept documents included in the conceptual document set in step S100 of the data creation processing routine shown in FIG. Alternatively, the concept document is divided into keyword units, and each of the search target document keyword and the concept document keyword is extracted.

  Next, in step S102, a search index 44 is created based on each of the search target documents divided in keyword units acquired in step S100, and stored in the storage unit 40.

  Next, in step S104, a document database 46 is created based on each of the search target documents divided in keyword units acquired in step S100 and the search index 44 acquired in step S102, and stored in the storage unit 40. To do.

  Next, in step S106, a concept vector model 42 is created based on each of the concept documents divided in keyword units acquired in step S100, and stored in the storage unit 40.

  Next, in step S108, based on the search index 44 acquired in step S102 and the concept vector model 42 acquired in step S106, a concept similarity dictionary 48 is created and stored in the storage unit 40, and data creation processing is performed. End the routine.

  Next, the information search processing routine shown in FIG. 8 will be described.

  First, in step S200 of the information search processing routine shown in FIG. 8, the search index 44, the document database 46, and the concept similarity dictionary 48 are read.

  Next, in step S202, as in step S100 described above, the input query received by the input unit 10 is divided into keyword units, and keywords are extracted.

  Next, in step S204, a search target document to be processed is determined from the search target documents included in the document database 46 acquired in step S200. In step S204, the document ID of the search target document to be processed is acquired from the document database.

  Next, in step S208, for each keyword acquired in step S202, reference is made based on the concept similarity dictionary 48 acquired in step S200 and the document ID of the search target document to be processed acquired in step S204. Get keywords.

  Next, in step S212, based on the search index 44 and the concept similarity dictionary 48 acquired in step S200, and each reference keyword of the keyword of the input query acquired in step S208, according to the above equation (1). The relevance score of the search target document to be processed is calculated.

  Next, in step S214, it is determined whether or not the processing from step S204 to step S212 has been completed for all search target documents included in the document database 46 acquired in step S200. If it is determined that the processing from step S204 to step S212 has been completed for all search target documents, the information search processing proceeds to step S216. On the other hand, if it is determined that the processing from step S204 to step S212 has not been completed for all search target documents, the information search process proceeds to step S204, and the search target document to be processed is changed. The processes from step S208 to step S214 are repeated.

  Next, in step S216, the document body of the search target document is arranged in descending order of the relevance score based on the relevance score of each search target document acquired in step S212 and the document database 46. Are output from the result output unit 90, and the information search processing routine is terminated.

  As described above, according to the information search device according to the embodiment of the present invention, each of the search target documents included in the document database based on the input query, the search index, and the concept similarity dictionary. On the other hand, a document related to the query is calculated by calculating a relevance score with the search target document by using the similarity between the search target document keyword similar to the keyword included in the query and the weight of the search target document keyword. Can be searched with high accuracy.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, in the present embodiment, the case where the similarity calculation unit and the similarity match search unit are configured to be included in the same information search device has been described. However, the similarity calculation unit, the similarity match search unit, May be configured as separate devices. In this case, the search index, the document database, and the concept similarity dictionary created by the apparatus including the similarity calculation unit are used in the apparatus including the similarity match search unit.

  In the present embodiment, the case where the process by the similarity matching search unit is performed after the process by the similarity calculation unit has been described, but the present invention is not limited to this. For example, the process of the similarity calculation unit may be processed offline in advance, and the process of the similarity match search unit may be executed online.

  Further, in the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium or provided via a network. It is also possible to do.

DESCRIPTION OF SYMBOLS 10 Input part 20 Calculation part 30 Similarity degree calculation part 32 Keyword extraction part 34 Search index creation part 36 Concept vector model creation part 38 Creation part 40 Storage part 42 Concept vector model 44 Search index 46 Document database 48 Concept similarity dictionary 50 Similarity degree Match search unit 52 Query keyword extraction unit 60 Score calculation unit 62 Concept similarity reference unit 64 Calculation unit 90 Result output unit 100 Information search device

Claims (7)

A combination of the weight of the search target document keyword included in the search target document, the search target document keyword, and the document ID representing the search target document, created for each of the search target documents included in the search target document set A search index that is
A document database created for each of the search target documents, which is a combination of the document content of the search target document and the document ID of the search target document;
The search based on a concept vector represented by an n-dimensional vector that expresses each of the concept document keywords included in the concept document created based on the concept document set, the search index, and the document database. A concept similarity dictionary that is created for each of the target documents and records the concept document keyword having the highest similarity with the similarity to the search target document keyword included in the search target document;
The search target document keyword that is similar to the keyword included in the query for each of the search target documents included in the document database based on the input query, the search index, and the concept similarity dictionary. A score calculation unit that calculates a relevance score with the search target document using the similarity to the search target document keyword and a weight of the search target document keyword;
Information retrieval device including For each search target document included in the search target document set, a combination of the weight of the search target document keyword included in the search target document, the search target document keyword, and a document ID representing the search target document is stored. A search index creation unit for creating a search index;
A document database created for each of the search target documents, which is a combination of the document content of the search target document and the document ID of the search target document;
A concept vector model creation unit for creating a concept vector represented by an n-dimensional vector that represents each of the concept document keywords included in the concept document based on the concept document set;
Based on the concept vector, the search index, and the document database, for each of the search target documents, the concept document keyword having the highest similarity is similar to the search target document keyword included in the search target document. A concept similarity dictionary creation unit that creates a concept similarity dictionary recorded with the degree,
The search target document keyword that is similar to the keyword included in the query for each of the search target documents included in the document database based on the input query, the search index, and the concept similarity dictionary. A score calculation unit that calculates a relevance score with the search target document using the similarity to the search target document keyword and a weight of the search target document keyword;
Information retrieval device including For each search target document included in the search target document set, a combination of the weight of the search target document keyword included in the search target document, the search target document keyword, and a document ID representing the search target document is stored. A search index creation unit for creating a search index;
A document database created for each of the search target documents, which is a combination of the document content of the search target document and the document ID of the search target document;
A concept vector model creation unit for creating a concept vector represented by an n-dimensional vector that represents each of the concept document keywords included in the concept document based on the concept document set;
Based on the concept vector, the search index, and the document database, for each of the search target documents, the concept document keyword having the highest similarity is similar to the search target document keyword included in the search target document. A concept similarity dictionary creation unit that creates a concept similarity dictionary recorded with the degree,
Dictionary creation device including A combination of the weight of the search target document keyword included in the search target document, the search target document keyword, and the document ID representing the search target document, created for each of the search target documents included in the search target document set A search index that is
A document database created for each of the search target documents, which is a combination of the document content of the search target document and the document ID of the search target document;
Each of the search target documents based on a concept vector expressed by an n-dimensional vector that expresses each of the concept document keywords included in the concept document, created based on the concept document set, and the search index A concept similarity dictionary in which the concept document keyword having the highest similarity is recorded together with the similarity to the created search target document keyword included in the search target document;
An information search method in an information search device, including a score calculation unit,
The score calculation unit is similar to the keyword included in the query for each of the search target documents included in the document database based on the input query, the search index, and the concept similarity dictionary. An information search method for calculating a relevance score with the search target document by using the similarity with the search target document keyword and the weight of the search target document keyword. A document database that is a combination of the document content of the search target document and the document ID of the search target document, each of the search target documents included in the search target document set, a search index creation unit, and a concept vector model An information search method in an information search device including a creation unit, a concept similarity dictionary creation unit, and a score calculation unit,
The search index creation unit represents the weight of the search target document keyword included in the search target document, the search target document keyword, and the search target document for each of the search target documents included in the search target document set. Create a search index that is a combination with the document ID,
The concept vector model creation unit creates a concept vector represented by an n-dimensional vector that represents each of the concept document keywords included in the concept document based on the concept document set,
The concept similarity dictionary creation unit has the highest similarity with respect to the search target document keyword included in the search target document for each of the search target documents based on the concept vector and the search index. Create a concept similarity dictionary that records concept document keywords along with similarities,
The score calculation unit is similar to the keyword included in the query for each of the search target documents included in the document database based on the input query, the search index, and the concept similarity dictionary. An information search method for calculating a relevance score with the search target document by using the similarity with the search target document keyword and the weight of the search target document keyword. A document database that is a combination of the document content of the search target document and the document ID of the search target document, each of the search target documents included in the search target document set, a search index creation unit, and a concept vector model A dictionary creation method in a dictionary creation device including a creation unit and a concept similarity dictionary creation unit,
The search index creation unit represents the weight of the search target document keyword included in the search target document, the search target document keyword, and the search target document for each of the search target documents included in the search target document set. Create a search index that is a combination with the document ID,
The concept vector model creation unit is an n-dimensional vector that represents each of the concept document keywords included in the concept document based on a concept document set for creating a concept vector in which a word is represented by an n-dimensional vector. Create a conceptual vector that represents
The concept similarity dictionary creation unit has the highest similarity with respect to the search target document keyword included in the search target document for each of the search target documents based on the concept vector and the search index. A dictionary creation method for creating a concept similarity dictionary in which conceptual document keywords are recorded together with similarities.   A computer is caused to function as each part of the information search device according to claim 1 or 2, or the dictionary creation device according to claim 3, or the information search method according to claim 4 or 5, or the computer according to claim 6. A program for executing each step of the dictionary search method.