JP2715981B2 - Search result evaluation device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for evaluating a search result in a database, especially a document database search system, in which each record in the database is given an appropriate index word and its weight to meet a searcher's request. The present invention relates to a search result evaluation device that outputs search results in the order in which they are performed.

[0002]

2. Description of the Related Art Conventionally, when performing a search in a database, especially a document database search system, each record in the database has been searched using a search formula expressing a request of a searcher. Then, the search results output records that completely matched the search expression without any ordering. Therefore, the searcher had to go through all the records in the search result in order to find the record that satisfied the request most.

[0003] In order to solve this problem, there has been proposed a method of outputting search results in the order of the relevance of a searcher using the weight of a search word in one record.

For example, in the invention described in Japanese Patent Application Laid-Open No. 1-149127, "Information Retrieval Apparatus" (hereinafter referred to as Reference 1), one of the factors for determining the weight of a search term is how many times the search term appears in a record. The frequency of appearance of whether or not they do.

[0005] Japanese Patent Application Laid-Open No. 6-89211, "Database Apparatus and Data Management Method" (hereinafter referred to as Document 2)
And JP-A-4-262460, "Information Searching Device"
Also, in the invention described in (Reference 3), the appearance frequency in a record is used as the weight of the search term.

In the invention described in Japanese Patent Application Laid-Open No. Hei 4-281565, "Document Retrieval Apparatus" (hereinafter referred to as Reference 4), in a full-text search, the weight is changed depending on where a search term appears in a text. For example, the weight changes as 10 if it appears in the title and 2 if it appears in the preamble. Also, JP-A-3-29
Also in the invention described in Japanese Patent No. 4963, “Document Searching Apparatus” (hereinafter referred to as Document 5), weights are assigned according to parts where search words appear.

[0007]

However, in each of the systems described in the above-mentioned documents 1 to 5, the length of the record in the database is not considered. For example, even if a certain search word has the same frequency of occurrence once, it is considered that in the case of a record of about 20 characters and a record of 2,000 characters, the former search word is more important in the former case. Such differences are not reflected in the weighting.

In particular, the methods described in Documents 1 to 3 do not consider the number of records in which a word appears in a database. Words with a large number of records appearing, such as "information" in databases related to information processing technology, do not characterize a particular record and are of low importance, but according to these methods, "information" Also has a high weight.

In the methods described in Documents 4 and 5, in particular, the weight is changed depending on the item in which the search term appears. For example, a database in which the item in the database includes a title, an abstract, and a text is searched. In this case, a difference between a search word having 100 records in the title and a search word having 100 records in the abstract is not considered. In general, an abstract is considered to have more words as the text is longer than the title, and more records to appear in the database, and it is estimated that the importance of the words will be lower.

As described above, in the conventional method of calculating the relevance of a record based on the weight of a search word, there is a problem to be solved regarding the weight of the search word.

An object of the present invention is to solve the above-mentioned problems,
Determine the weight of the search term in consideration of the length of each item of each record in the database, the appearance frequency of the search term for each item of each record, and the number of records appearing for each item of the search term, and more appropriate for the searcher An object of the present invention is to provide a search result evaluation device for calculating the relevance of a record.

[0012]

Means for Solving the Problems The present invention, the first invention,
A database including a plurality of records and items, a search request for a searcher to input a search item and a search word, a search expression generation unit configured to generate a search expression from the search request, and searching the database using the search expression A search execution unit that performs a search, and outputs a search result in the order of the relevance of the searcher. In the search result evaluation device, a word appearing in each record in the database is extracted, and the extracted word is extracted. The record and the item are identified, and a record item-specific appearance frequency reference unit that further refers to an appearance frequency, and a record-specific item length reference unit that refers to an item length that is the length of each of the plurality of records, A word-based occurrence record number reference unit for referring to the number of records in which the extracted word appears in the database by item; the occurrence frequency, the item length, and the number of records An index word weight calculation unit that calculates a weight of the extracted word based on the extracted word, generates a weighted index word file specifying the record, item, and weight of the extracted word; and a search input by the searcher. A search word weight calculation unit that receives a word weight from the weighted index word file and calculates the weight of the search result obtained by the search execution unit;
A record relevance calculating unit that outputs a relevance order search result obtained by sorting the records that are the search results in relevance order with reference to the weight of the search result is provided.

In a second aspect based on the first aspect, the index term weight calculator holds a threshold value for selecting the extracted word, and calculates the weight when calculating the weight. An extracted word having a weight equal to or greater than the threshold is used as an index word,
It is characterized in that a weighted index word file that specifies the record, item and weight of the index word is generated.

In a third aspect based on the first and second aspects, the index word weight calculator multiplies the weight by the reciprocal of the number of records and divides the weight by the item length. Is calculated.

[0015] Further, a fourth aspect of the present invention provides the first, second, and third aspects.
The present invention is characterized in that the record item-specific appearance frequency reference unit uses natural language analysis when extracting words.

[0016]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of one embodiment of the present invention. FIG. 2 is a diagram showing an example of a result obtained by the appearance frequency reference unit for each record item in FIG. FIG. 3 is a diagram showing an example of a result obtained in the item length reference unit for each record in FIG. FIG. 4 is a diagram illustrating an example of a result obtained by the word-based occurrence record number reference unit in FIG. FIG. 5 is an explanatory diagram showing an example of the weighted index word file in FIG.

In FIG. 1, a database 101 is a database to be searched. The record item appearance frequency reference unit 102 extracts words appearing in each record in the database 101 for each item, and refers to the appearance frequency indicating how many times each word appears in the item. The database of the present embodiment is a database related to library and information science, and has three items of “title”, “abstract”, and “text”. Of course, the present invention is not limited to the present embodiment, but is applicable to various databases.

In extracting words in this embodiment, for example, items described in text such as titles, abstracts, and texts may be referred to, for example, in "Basic Technology of Natural Language Processing" (by Hirogo Nomura, Published by IEICE, 1988
The natural language analysis is performed by using the natural language analysis method described in Chapter 1 and Chapter 2 of (Year), and the word is segmented. Of the extracted words, words that exclude conjugative endings, auxiliary verbs, adjunct particles, final particles, accessory particles, case particles, and parallel particles are extracted, but this is only an example and other words are extracted. As a technique for performing the search, a table in which keywords are described in advance according to the type of the database to be searched is separately held, and a method of extracting words by referring to the table or a method of searching for a search word in the search request 107 Only various techniques such as a method of referring to the appearance frequency for each item in the database can be considered.

The item-by-record item length reference unit 103 refers to the item length of each record in the database 101.

The word-specific appearance record number reference unit 104 calculates, for each item, the number of records in which the words extracted by the record item-specific appearance frequency reference unit 102 appear in the database 101 in all records. refer.

The index word weight calculation unit 105 calculates the weight of the word extracted by the record item-specific appearance frequency reference unit 102, and determines a word having a weight equal to or greater than a certain threshold value as an index word.
The weight of each word is a value indicating the degree of importance that the word contributes to the contents of the record, and the greater the value, the higher the importance. In addition, the threshold value may be set to a value that is appropriate for eliminating words with low importance after the weight of each word is calculated, or may be set in advance as the threshold value. It doesn't matter.

The weight is determined by referring to the frequency of occurrence of the index term in the application frequency reference unit 102 for each record item, the number of records in the number-of-appearance records reference unit 104, and the item length in the item length reference unit 103 for each record. Weight = frequency of appearance × reciprocal number of record / item length (1)

The weighted index word file 106 is a file in which the index words and the weights calculated by the index word weight calculator 105 are described for each record item in the database 101. The weighted index word file will be described later in detail.

The search request 107 is a search item and a search word input by the searcher.

The search formula generation unit 108 generates a search formula from the search request 107. If the search request 107 particularly specifies a relationship between search words, the relationship is used. If not specified, all search terms are connected by OR. When the search formula generation unit 108 is equipped with a thesaurus, a synonym dictionary, or the like, the search formula input by the searcher is combined with a synonym, a different notation, an upper word, a lower word, etc. by OR, and the search formula is obtained. Generate

The search execution unit 109 includes a search expression generation unit 108
Executes the search formula generated by the server and retrieves a record as a search result from the database 101.

The search term weight calculation unit 110 refers to the weighted index word file 106 to obtain the weight of the search term of the search request 107 in each record of the search result obtained by the search execution unit 109.

The record matching degree calculation section 111 adds the weights calculated by the search word weight calculation section 110 to calculate a matching degree indicating how much each record matches the searcher's request. The records are sorted in the order of the relevance, and the relevance order search result 112 is output.

Here, an example will be described. As mentioned above, the search target is a library database related to library and information science.
It is assumed that there are 10,000 records and title, abstract, and text items.

First, in the index word weight calculation unit 105, the word appearing in each record of the database 101 is extracted by the record item appearance frequency referring unit 102 by item, and each of the records of the record of the word is extracted. Get the frequency of appearance in an item.

The record item-specific appearance frequency reference unit 102 in this embodiment analyzes the title, abstract, and body items of each record by natural language analysis, and uses inflected endings, auxiliary verbs, adjunct particles, final particles, adjunct particles, and cases. Extract words excluding particles and parallel particles. Further, the number of appearances of how many times each item appears is counted. FIG. 2 shows an appearance frequency reference unit 10 for each record item.
FIG. 2 is a diagram showing an example of a result referred to by No. 2; For example, the word "university library" appears once in the "title" item of the # 1 record. Similarly, in the “abstract” item of the # 1 record, it appears three times, and in the “body” item, it appears 149 times.

Next, the index word weight calculation unit 105 checks the length of each item of each record in the database 101 by the item length reference unit 103 for each record. FIG. 3 is a diagram showing an example of a result referred by the record-specific item length reference unit 103.
According to this figure, the length of the “title” item of the # 1 record is 76, the length of the “abstract” item is 264, and the length of the body is 24398.

The index word weight calculation unit 105 further obtains the number of records in which the words extracted by the record item appearance frequency reference unit 102 appear in the database 101 by the word appearance record number reference unit 104. FIG. 4 is a diagram illustrating an example of a result referred by the word appearance record number reference unit 104. According to this, all records 1 in the database 101
Of the 0000 records, 63 records have the word "university library" appearing in the "title" item, and 97 records in the "abstract" item.

Based on the above data, the index word weight calculation unit 105 calculates the weight of the word according to the above equation (1). For example, the weight of “university library” appearing in the “title” item of the # 1 record is (1/76) × (10000/63) = 2.089. The weight of “university library” in the “abstract” item of the # 1 record is (3/264) × (10000/97) = 1.172.

Finally, the index word weight calculation unit 105 describes, in the weighted index word file 106, words having a weight equal to or greater than a threshold value among words appearing in the database 101 together with their weights as index words. Here, an example of the weighted index word file 106 is shown in FIG. In this example, the threshold value is set to 0.05, and a word having a weight of 0.05 or less is not described. In addition, the threshold value may be freely set or may be set in advance.

Here, it is assumed that the searcher has requested a document relating to "network in a university library". First, the search request 107 includes "university library" and "network" as search terms, and "title" and "abstract" as search items. The searcher will be given an example of searching for a record in which "University Library" and "Network" are both listed in the title or abstract item.

The search formula generation unit 108 generates a search request 107
Convert to search expression. If the searcher specifies the relationship between the search terms, that is, if the search term is entered as “University Library AND Network”, the relationship is used as it is. If not specified, all search terms are
Connect with R. When a thesaurus or a synonym dictionary can be used, a synonym, a different notation, a synonym, or the like may be added to the search word by OR.

The search execution unit 109 executes the search formula generated by the search formula generation unit 108 and receives a search result from the database 101. In this example, it is assumed that among the 10000 records, the records satisfying the above conditional expression are the records # 1 to # 3.

The search term weight calculation unit 110 determines the weight of the search terms “university library” and “network” in the search items “title” and “abstract” of each record of the search result by using an index term described in FIG. The calculation is performed with reference to the file 106.

First, the search word weight calculation unit 110 searches the index word file 106 of FIG. 5 from the record number, search item, and search word of the search result, and refers to the weight. Here, it is assumed that the record of record number # 1 is included in the search result. The weight of “university library” in the “title” item of # 1 is 2.089, and the weight of the “abstract” item of # 1 is 1.172. Also, # 1
The weight of “network” in the “title” item is 2.651, and the weight in the “abstract” item is 3.1.
57.

The record matching degree calculation unit 111 calculates the matching degree of the record by adding the weights of the records of the search words “college library” and “network” output by the search word weight calculation unit 110. For example, in the record of # 1, when the weights of “university library” and “network” in the items of “title” and “abstract” are added, “university library”: 2.089 + 1.172 = 3.2
61, "network": 2.651 + 3.157 =
5.808, which together add 3.261+
5.808 = 9.069. This 9.069 is # 1
The relevance of the record.

Similarly, the weight of “college library” in the record of # 2 is 0.968 + 0.543 = 1.51
At 1, the weight of "network" is 0.487.
Therefore, the sum of these values is 1.511 + 0.487 =
1.998 is the fitness of # 2. Similarly, the weight of “university library” of # 3 is 0.469, the weight of “network” is 7.701, and the fitness of # 3 is 8.170.

The record matching degree calculation unit 111 sorts the results in order of record matching degree based on the above results.
In the case of this example, the data is output in the order of # 1, # 3, and # 2.

[0045]

As described above, according to the present invention, the length of the item of each record is taken into account in the calculation of the weight. The weight of a word that appears once in the cross record can be increased.

Also, since the number of occurrences of a word in a record in the database is taken into account, a word that appears only in a specific record is more frequently used than a word that appears more than once in each record. Can be increased.

Further, by setting a proper threshold value, unnecessary words can be eliminated and an index word effective for retrieval can be obtained.

As described above, according to the present invention, it is possible to calculate the weight of a search word in consideration of the items of each record and its length from the search word input by the searcher, and to arrange them in a more accurate order of conformity. The obtained search result can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of the configuration of the present invention.

FIG. 2 is a diagram illustrating an example of a result referred by an appearance frequency reference unit for each record item;

FIG. 3 is a diagram illustrating an example of a result referred by a record-specific item length reference unit;

FIG. 4 is a diagram illustrating an example of a result referred by a word appearance record number reference unit;

FIG. 5 is a diagram showing an example of a weighted index word file.

[Explanation of symbols]

 Reference Signs List 101 Database 102 Appearance frequency reference section by record item 103 Record item reference section 104 Record number of occurrence record reference section 105 Index term weight calculation section 106 Weighted index term file 107 Search request 108 Search expression generation section 109 Search execution section 110 Search Word Weight Calculator 111 Record Matching Calculator 112 Matching Order Search Results

Claims (4)

(57) [Claims] 1. A database comprising a plurality of records and items, a search request for a searcher to input a search item and a search word, a search expression generation unit for generating a search expression from the search request, and using the search expression A search execution unit that performs a search of the database, and in a search result evaluation device that outputs the results of the search in the order of the relevance of the searcher, extracting words that appear in each record in the database, A record item-specific appearance frequency reference unit that specifies the record and the item of the extracted word and further refers to an appearance frequency; and a record-specific item that refers to an item length that is the length of each of the plurality of records. A long reference part, a word-based occurrence record number reference part that refers to the number of records in which the extracted word appears in the database item by item, the occurrence frequency, the item length, and the An index word weight calculation unit that calculates a weight of the extracted word based on the number of records, and generates a weighted index word file that specifies records, items, and weights of the extracted word; Receiving the weight of the input search word from the weighted index word file, and calculating a weight of the search result obtained by the search execution unit, a search word weight calculation unit, and referring to the weight of the search result, A search result evaluation device, comprising: a record relevance calculating unit that outputs relevance order search results in which records as search results are sorted in relevance order. 2. The method according to claim 1, wherein the index word weight calculation unit holds a threshold value for selecting the extracted words, and calculates the extracted words having a weight equal to or greater than the threshold value when calculating the weights. 2. The search result evaluation apparatus according to claim 1, wherein a weighted index word file is generated by specifying a record, an item, and a weight of the index word as an index word. 3. The index word weight calculation unit calculates the weight by multiplying the appearance frequency by the reciprocal of the number of records and dividing the weight by the item length. Search result evaluation device. 4. The search result evaluation apparatus according to claim 1, wherein the appearance frequency reference unit for each record item uses a natural language analysis when extracting a word.