JP3438947B2 - Information retrieval device - Google Patents

Description

BACKGROUND OF THE INVENTION [0001] Field of the Invention The present invention relates to a biopsy SakuSo location information in such document database, in particular, relates to an information search SakuSo location capable full-text search or full Property Search . 2. Description of the Related Art When searching for a property required by a user from a database holding a large number of information such as documents as properties, a keyword is assigned to each property in advance, and a keyword matching a search key is searched. Then, there are a keyword search method for searching for a desired property, and a direct search method for searching for an item including a search key by performing a full-text search of all information. In the keyword search method, a keyword is preliminarily assigned to each property stored in the database, so considerable work is required for the work. If a keyword is arbitrarily assigned, the number of keywords becomes enormous, so management using a thesaurus In addition, there is a problem that it is difficult to provide a proper keyword, and a desired property may not be reached. Also, direct search methods include:
When the number of properties to be searched is large or the data volume of each property is large, the search cannot be completed within a realistic time range, and so-called ambiguous search cannot be generally performed. There is a problem. [0003] The present applicant has disclosed Japanese Patent Application Laid-Open No. Hei 4-32661 in order to solve the problems of the conventional information retrieval method as described above.
No. 64 does not require the addition of a keyword,
We proposed a database search system that can perform high-speed search. In this database search system, the autocorrelation information is obtained in advance for each property to be searched, and at the time of search, the autocorrelation information of the search key is obtained and the degree of matching between the autocorrelation information of the search key and the autocorrelation information of the property is obtained. Is obtained for each property, and an attempt is made to output the property with the highest degree of matching. As the autocorrelation information, for example, a fixed-size binary matrix is used. This search system performs searches based on fixed-size autocorrelation information regardless of the data volume of each property, so that search time can be significantly reduced,
For properties that contain a character string that is slightly different from the search key, a fairly high degree of matching can be obtained, so fuzzy search can be performed and autocorrelation information can be calculated automatically. Compared with the search method, the amount of work required when constructing the database can be greatly reduced, and the property that includes the search key as it is is given the maximum matching degree, so that such a property will not be missed , Etc. Here, a specific example of the database search system described in Japanese Patent Application Laid-Open No. 4-326164 will be described. Here, it is assumed that the search target property is an English text, and each character of the text is represented by an ASCII (ASCII) code. The ASCII code is usually 8 bits, but only the lower 7 bits are considered since the most significant bit is not used as long as normal characters in English are used. As the autocorrelation information, since each character is represented by a 7-bit code by any integer from 0 to 127, a 128 × 128 binary matrix is used. Each element of this matrix is assumed to be initialized to “0”. [0005] First, for each character of the English text, a continuous character consisting of a predetermined number of characters starting from that character is extracted from the English text. If the English text is "This_is_a_pe
n. "(where" _ "represents a space), and if the predetermined number of characters is three," Thi "," his "," is_ "," s_
The consecutive characters of "a", "_ a _", "a_p", "_ pe", "pen", "en.", "n." are extracted.
The character code of the 2nd, 3rd and 3rd characters is c
_{If it is 1} , c ₂ , c ₃ , the values of the elements (c ₁ , c ₂ ) and (c ₁ , c ₃ ) of the binary matrix representing the autocorrelation information are set to “1”. By performing this operation for all extracted consecutive characters, the autocorrelation information of the target English text has been obtained. On the other hand, at the time of retrieval, first, the autocorrelation information of the retrieval key is obtained by the same procedure as described above. And
For each property, the binary matrix of the autocorrelation information of the property is compared with the binary matrix obtained from the search key, and each element which is "1" in the binary matrix from the search key is obtained from the property. Check whether it is "1" in the binary matrix. The ratio of the element of “1” in the binary matrix corresponding to the property among the elements of the matrix “1” in the binary matrix corresponding to the search key is defined as the matching degree. Then, properties are output in order from the one with the highest matching degree. In this specific example, autocorrelation information is calculated based on consecutive characters. Broadly speaking, the degree of matching is calculated according to how many consecutive characters included in the search key are included in the property. Then, rather than checking the coincidence of consecutive characters, the degree of matching is calculated after conversion to autocorrelation information, thereby enabling extremely high-speed retrieval. [0008] Japanese Patent Application Laid-Open No. 4-32616
By the database search system disclosed in No. 4,
As described above, a high-speed document search can be performed without using a keyword. However, the number of documents and the amount of information stored in the literature database are steadily increasing, and there is a demand for faster and more reliable literature search. An object of the present invention is to provide a information search SakuSo location that can be searched faster the intended searcher. An information search apparatus according to the present invention searches for a desired document from a set of search target documents based on an input search character string. An input unit for inputting the search character string; a continuous character extraction unit for extracting a continuous character having a predetermined character length from the search character string to form a search continuous character group; Importance storing means for storing the importance of each character, a combination of character types constituting the consecutive characters for each of the consecutive characters included in the retrieval consecutive character group , and determining the importance for the consecutive characters in accordance with the order. Then, for each of the search target documents, it is determined whether or not the consecutive characters are included in the search target document for each of the consecutive characters included in the search consecutive character group. The importance corresponding to the character A search means for adding to the importance of the document to be searched in the unit, and output means said reference the importance storage unit, to output the search results in accordance with the importance of each of said search target document. When performing a search of a document database, it is common to focus on the local structure of the document itself. It is highly unlikely that a search is to be performed based on the correlation between the content written in the place of interest and, for example, the content written five pages ahead. When focusing on the local structure, rather than using a search key with a length of several to several tens of words as it is, as described in `` Conventional Technology, '' the search key is decomposed into It is more efficient to search based on characters. By the way, considering a situation where a searcher is actually trying to perform a search, the searcher must select a search key with some meaning. In addition, when considering a document to be searched, not all the words included in the document have the same weight, but characteristic words useful for identifying the document are mixed with words that are not. In the conventional search method, especially in the case of continuous characters, the contribution of each continuous character to the search result is equal, and does not reflect the meaning entrusted by the searcher or the characteristic content of the document. , And often hit unintended documents. In the present invention, a search key (search character string) is decomposed into continuous characters, and it is determined whether or not the consecutive characters are characteristic for a search. Is made to greatly contribute to the search result, so that the search can be surely performed according to the intention of the searcher. Specifically, the percentage of each consecutive character that contributes to the search result of the consecutive characters is determined as importance, and if the consecutive characters are included in the search target document, the importance of the consecutive characters is determined by the search target. What is necessary is just to add to the importance (evaluation value) of a document. Examples of how to determine the importance for continuous characters are, for example,
For each consecutive character used in the search, calculate the frequency of occurrence of the consecutive characters in all the documents to be searched.The higher the frequency of the consecutive characters, the lower the importance. For example, there is a method of determining the importance of the consecutive characters according to the combination of the character types, and these methods may be used together. The above-described method of calculating the importance level is based on the fact that the more commonly appearing documents in each search target, the lower the degree of specifying a desired document. For example, various inflectional endings, "the" and auxiliary verbs in English sentences, and particle and auxiliary verbs in Japanese sentences correspond to common common words. On the other hand, the above method is based on the fact that kanji, hiragana, numbers, etc. are used in a mixture in Japanese sentences, for example, but the combination of kanji often has a characteristic meaning as a idiom. I have. Even in languages other than Japanese, for example, in Chinese, there is a distinction between kanji that can be a particle and kanji that cannot be a particle, and it is possible to determine the character type according to this distinction. Also, in English and the like, the character type can be determined according to whether it is a symbol such as uppercase and lowercase letters, numbers, Greek letters, hyphens, and the like. For English,
It is also possible to process such that the past ending "ed" and the adverb ending "ly" are treated as different character types. In the present invention, a specific method of calculating the importance of each consecutive character is, for example, the language or field of the document to be searched (for example, whether it is a technical document, a newspaper article,
Literary work), the habit of the searcher (such as what search keys are frequently used), the purpose of the search (whether or not to perform a fuzzy search), and also the search string itself. It can be varied and can be adaptively varied. By changing the method of calculating the importance as needed, it is possible to perform a more accurate search. Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an information retrieval apparatus according to an embodiment of the present invention. FIG. 2 is a flowchart showing a processing procedure when information retrieval is performed by the method of the present invention using this information retrieval apparatus. is there. The information retrieval apparatus 11 is for performing a search for document information stored in the database storage unit 10 as documents to be searched, and is a search character string for inputting a search character string designated by a searcher. An input unit 12, a continuous character extracting unit 13 for extracting continuous characters having a predetermined character length from the input search character string, and determining the importance of the continuous character for each continuous character extracted from the search character string. The search engine unit 14 for calculating and actually searching each search target document in the database storage unit 10 based on these consecutive characters, and calculating the importance for each consecutive character extracted from the search character string. Storage unit 15 for storing parameters used for
An importance storage unit 16 for storing the importance of each document to be searched, and an output unit for referring to the importance storage 16 and performing normalization according to the importance of each document to be searched and outputting a search result 17. A set of consecutive characters extracted from the retrieval character string is referred to as a retrieval consecutive character group. The search engine unit 14 calculates the importance of each of the consecutive characters in the retrieval consecutive character group. In this embodiment, the first type of importance determined from the appearance frequency of the consecutive characters. And the second type of importance determined from the character type of the consecutive characters is obtained for each consecutive character. The first type and the second type importance are each 0 or a positive real number, and are set so that the larger the value, the larger the contribution to the search result. Hereinafter, these degrees of importance will be described. The first type of importance is calculated by the search engine unit 14 by calculating the frequency of appearance of the consecutive characters through all the search target documents stored in the database storage unit 10, and the smaller the frequency is, the larger the value becomes. , The higher the frequency of appearance, the smaller the value. Here, the appearance frequency is obtained by dividing the number of documents including the consecutive characters by the total number of documents, and is represented by a real number from 0 to 1. FIG. 2 is a graph in which the frequency of appearance is plotted on the x-axis and the value of the first type importance is plotted on the y-axis, and shows an example of a function representing the relationship between the frequency of appearance and the first type importance. I have. As is clear from this graph, the first type of importance is represented by a function that decreases monotonically with the frequency of appearance. It should be noted that the consecutive characters appearing in all the search target documents are meaningless for use in the search. Therefore, the first type importance is set to 0 for such consecutive characters. desirable. If the character length of the consecutive characters is two, the second type of importance is determined by the search engine unit 14 in accordance with the combination of the first and second characters constituting the consecutive characters. It is determined. More specifically, a calculation table 21 representing a second type of importance value corresponding to the combination of character types is provided in the parameter storage unit 15, and by referring to this calculation table 21, Required for each character. FIG. 3 shows an example of the contents of the calculation table 21 when the search target character is a Japanese document. In the illustrated example, the character type is classified into hiragana, katakana, kanji, alphanumeric characters, and symbols, and when consecutive characters are composed only of kanji, the second type has the highest importance. . As can be seen from the combination of the kanji and the hiragana, when the character type of the first character and the character type of the second character are switched, the same second type importance value is not always obtained. When the first character is a kanji and the second character is a hiragana, the combination of the last character of the idiom (noun) + the particle is overwhelming, and the order of the combination of character types is reversed. Because it is considered to be less characteristic. The case where the character length of consecutive characters is three or more characters is basically the same as that described here. After obtaining the first and second types of importance, the search engine unit 14 executes the actual search processing. As a search algorithm,
For example, there is one described in the above-mentioned JP-A-4-326164. The search process determines, for each document to be searched, whether each of the consecutive characters in the group of consecutive characters for search is included in the document to be searched, and if the consecutive characters are included, This is performed by adding the first type and second type importance corresponding to the consecutive characters to the importance of the search target document in the importance storage unit 16. In practice, the search processing may be executed collectively after calculating the first and second types of importance for all the continuous characters in the search continuous character group, For each consecutive character, it takes out one consecutive character, finds the first and second importance for the consecutive characters, and then checks whether the consecutive characters are included in each search target document. May be executed. FIG. 4 shows an example of the configuration of the importance storage section 16. The importance storage unit 16 stores the database storage unit 10
The table is configured as a table including the document numbers assigned to the respective search target documents and the importance of the search target documents corresponding to the document numbers. The initial value of importance is 0. The degree of importance for each search target document corresponds to the degree of matching in the above-mentioned “conventional technology”. Next, an information retrieval procedure performed by using the information retrieval apparatus 10 based on the method of the present invention will be described with reference to FIG.
This will be described using. First, a search character string is input via the search character string input section 12 (step 101), and the continuous character extraction section 13 decomposes the search character string into continuous characters having a predetermined character length (step 102). . If the length of the consecutive characters is 2 characters and the search character string is, for example, "Business trip to Osaka.", "Osaka", "Osaka ni", "Nide", "Business trip", "Chang", "Yes" And "ru." Are extracted to form a search consecutive character group. Next, the following processing is performed for each consecutive character in the retrieval consecutive character group. That is, it is determined whether unprocessed continuous characters remain (step 103). If unprocessed continuous characters remain, the unprocessed continuous characters are searched for by the search engine unit 14 through all search target documents. An appearance frequency is calculated (step 104), and a first importance for the consecutive characters is calculated according to the calculated appearance frequency (step 1).
05), the second type importance for the consecutive characters is determined according to the combination of character types constituting the consecutive characters (step 105). Next, it is checked whether or not there is a document that is a search target document in the database storage unit 10 and that has not been searched for the consecutive characters (step 107). Select one document,
It is determined whether the consecutive characters are included in the document (step 108). If it is not included, the process directly returns to step 107. If it is included, the first and second types of importance for the consecutive characters are added to the importance of the document in the importance storage unit 16. (Step 1
09), returning to step 107. If it is determined in step 107 that there are no unsearched documents, the process proceeds to step 10 to search for the next consecutive character included in the retrieval consecutive character group.
Return to 3. When two consecutive characters are extracted from the above-described search character string "Travel to Osaka.", The consecutive characters "do" and "ru." Tend to appear commonly in each document. The first type becomes less important. In addition, when the calculation table as shown in FIG. 3 is used, the second type importance for the consecutive characters "Saka ni" becomes small. On the other hand, the consecutive characters "Osaka"
Is composed of two kanji characters, so the importance of the second type increases. Also, assuming that the frequency of occurrence of “Osaka” through each search target document is low and the frequency of occurrence of “business trip” is high to some extent, the importance of the first type is greater for “Osaka” than for “business trip”. Become. In the end, a search that has a large contribution degree of “Osaka” will be performed as a whole. If it is determined in step 103 that there are no unprocessed continuous characters, it means that the search process based on all continuous characters in the search continuous character group has been completed.
And normalizes the importance value of each document stored in the importance storage unit 16 (step 110). Here, the normalization is a process of multiplying each importance by the same coefficient so that the maximum importance becomes 1. The longer the search character string (search key), the larger the number of digraphs in the search digraphs, which tends to increase the raw importance of each document. ,
Such a difference in the search character string makes it possible to correct the variation in the value of the importance. Then, the normalized importance is arranged in descending order (step 111), and the search result is output by outputting the importance of each document as a list (step 112), and the process is terminated. As described above, according to the present invention, the degree of importance of each consecutive character is determined after extracting consecutive characters from the search character string, and the contribution ratio of each consecutive character to the search result is determined by this importance. , The characteristic continuous characters in the search are more reflected in the search result, and the search can be performed reliably according to the searcher's intention. As a method of determining the importance of the consecutive characters, for each consecutive character used in the search, the frequency of occurrence of the consecutive characters in all documents to be searched is determined, and the higher the appearance frequency, the lower the importance is. A more reliable search can be performed by adopting a method such as finding a character type constituting a consecutive character for each character and determining the importance of the consecutive character by a combination of the character types.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a configuration of an information search device according to one embodiment of the present invention. FIG. 2 is a graph showing an example of a function representing a relationship between an appearance frequency and a first type importance; FIG. 3 is a diagram illustrating an example of a configuration of a calculation table indicating a correspondence between a character type and a second type of importance; FIG. 4 is a diagram showing a configuration of an importance storage unit. FIG. 5 is a flowchart showing a processing procedure when information is searched by the method of the present invention using the information search apparatus of FIG. 1; [Description of Signs] 10 Database storage unit 11 Information search device 12 Search character string input unit 13 Continuous character extraction unit 14 Search engine unit 15 Parameter storage unit 16 Importance storage unit 17 Output unit 21 Calculation tables 101 to 112 Step

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G06F 17/30

Claims (1)

(57) [Claim 1] An information search device for searching for a desired document from a set of search target documents based on an input search character string, wherein the search character string is Input means for inputting, continuous character extracting means for extracting a continuous character having a predetermined character length from the search character string to form a continuous character group for search, and importance for storing the importance of each search target document A storage unit, for each of the consecutive characters included in the retrieval consecutive character group, a combination of character types constituting the consecutive characters , and determining the importance for the consecutive characters in accordance with the order thereof ; for each of the search target documents, For each consecutive character included in the search consecutive character group, it is checked whether the consecutive character is included in the search target document, and if the consecutive character is included, the importance corresponding to the consecutive character is set to the importance. Of the document to be searched in the degree storage means. An information search device comprising: a search unit for calculating a search result; and an output unit for outputting a search result according to the importance of each document to be searched by referring to the importance storage unit.