JP2004192374A - Document search system, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document search device searching an appropriate document desired by a user by selecting related terms from document groups based on various viewpoints. <P>SOLUTION: This document search device selects documents fitted to or unfitted to keywords input from document database 160 storing a plurality of documents, selects words appearing in the selected fitted documents and words having high relevance with the input keywords as the related terms of the keywords, and searches the document database 160 again. The selected fitted document groups are divided into a plurality of sets, the related words are found for every set, and the document is searched by the keywords formed by adding a sum of sets of the related words to the keywords. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a document search device, a program for executing a function of the document search device, and a computer-readable recording medium on which the program is recorded. More specifically, the present invention selects a document that matches a given keyword. A document search device capable of retrieving a document desired by the user by re-searching for a document that matches with a keyword to which a keyword extracted from the relevant document is added, and a program for executing the function of the document search device And a computer-readable recording medium recording the program.
[0002]
[Prior art]
To search for a document that the user needs from a document database that contains a large number of documents, search once using the keyword input by the user, and then input from the words that appear in the document that matches the keyword There is known a method of selecting a word related to a given keyword, adding the selected keyword to the first input keyword, and searching again to obtain a keyword closer to the one desired by the user.
[0003]
For example, as a method of selecting a related word of a keyword, for each word in a conforming document, the degree of relevance to the keyword is calculated using statistical information such as the appearance status in the conforming document, and the value of the keyword is large. A method of selecting the top several words has been proposed (see Non-Patent Document 1).
[0004]
Next, this related word extraction method will be described. Each word in the keyword input by the user is weighted according to the importance of the word. As a calculation formula for calculating the weight of the word, for example, Robertson's calculation formula (Formula 1) based on a probability model is known (see Non-Patent Document 2). In the technique of Non-Patent Document 2, the weight of each word in a keyword is assigned according to the appearance status Wp, Wq of each word in the entire search target document.
[0005]
W (weight) = Wp-Wq (Equation 1)
here
Wp = k4 + log (N / (Nn)),
Wq = log (n / (Nn)),
N: Total number of documents to be searched,
n: the number of documents in which the word appears,
k4: Adjustment parameter
[0006]
Next, the document relevance of each document is calculated based on the weight of each word in the keyword. The formula for calculating the document relevance is obtained, for example, by the following formula (Formula 2) in Non-Patent Document 2.
[0007]
F (fitness) = Σ (W × tf / (k1 + tf)) (Equation 2)
here
W: weight of word obtained by (Equation 1),
tf: number of words per document,
k1: adjustment parameter
[0008]
The document relevance of each document is determined, and the documents are ordered in descending order of relevance, and some of the top documents are regarded as conforming documents, and some of the bottom documents are regarded as non-conforming documents.
After selecting a conforming document, for all words except for unnecessary words (for example, article a) in the conforming document, the appearance status in the conforming document and the non-conforming document, that is, the feedback information is reflected, and each word is reflected. Recalculate the weights.
[0009]
The weight after the selection of the relevant document is determined by, for example, using the calculation formula (Expression 3) in Non-Patent Document 2 and the appearance status Wp, Wq (see the comment in (Expression 1) above) and the relevant document / The appearance statuses Wr and Ws in the non-conforming document are added by the ratios Cp and Cq.
[0010]
W ′ (weight) = (Cp · Wp + (1-Cp) · Wr) − (Cq · Wq + (1-Cq) · Ws) (Equation 3)
here
Wr = log ((r + 0.5) / (R-r + 0.5)),
Ws = log ((s + 0.5) / (S-s + 0.5)),
Cp = k5 / (k5 + √R),
Cq = k6 / (k6 + √S),
R: Number of conforming documents,
r: the number of documents in which the word appears in the set of conforming documents,
S: Number of non-conforming documents,
s: the number of documents in which the word appears in the non-conforming document set,
k5, k6: Adjustment parameters
[0011]
Further, the degree of relevance to the keyword is determined for each word obtained by removing unnecessary words in the matching document from the weight and the feedback information. As a method of calculating the degree of association, for example, there is a Boughanem equation (Equation 4) (see Non-Patent Document 3).
[0012]
Relevance = (r / R−α · s / S) × W ′ (Equation 4)
Where α: adjustment parameter
[0013]
In this way, for each word in the conforming document, the degree of relevance to the keyword is determined, selected as keyword-related words in descending order of relevance, and added to the input keyword to create a new keyword.
Using this new keyword, a matching document is selected again. At this time, the weight calculated by the above (Equation 3) is used for calculating the document relevance.
[0014]
On the other hand, the technique of Patent Document 1 prepares a set of synonyms of a concept classified by a different classification method and a broader term indicating a broader concept for an arbitrary search term, and generates a synonym including a search term given by a user. Present sets and broader terms.
As a result, desired information can be efficiently obtained by searching using the search words of the concepts classified by various classification methods.
[0015]
Further, the technique of Patent Literature 2 selects a main keyword that is a main keyword among search keywords given by a user, and classifies a search result obtained by the main keyword and its related words and associated words into a plurality of categories. . A classification corresponding to a search keyword is determined from among the classified taxonomic groups, and information belonging to the classification is efficiently searched by narrowing down the information using the previously given search keyword and its related words and associative words. can do.
[0016]
[Patent Document 1]
JP-A-10-21266
[Patent Document 2]
JP 2001-5830 A
[Non-patent document 1]
Robertson, SE "On term selection for query expansion," Journal of Documentation 46, Dec 1990, p359-364
[Non-patent document 2]
Robertson, SE and Walker, S. "On relevance weights with little relevance information," SIGIR97, ACM Press, pp. 16-24
[Non-Patent Document 3]
Walker, S.etal., "Okapi at TREC-6: Automated adhoc, VLC, routing, filtering and QSDR," The Sixth Text RetrievalConference (TREC-6), 1996, NIST
[0017]
[Problems to be solved by the invention]
However, in the above-described conventional technology, the entire set of conforming documents is regarded as one set, and related words are obtained from the entire set. Therefore, related words are selected from only one scale, and the following problem is caused. There was a point.
[0018]
(1) When a document regarded as a conforming document is not actually a conforming document, there is a high risk of selecting an unsuitable related word.
(2) Even if a document regarded as a conforming document is actually a conforming document, there is a possibility that many related words having a similar tendency may be selected, and it is desirable to obtain a document discussing the same theme from a different viewpoint. I can't respond to requests.
[0019]
According to the techniques of Patent Literature 1 and Patent Literature 2 described above, a related word for a search word is defined in advance, and a search expression expanded by adding a related word to a search word specified by a user can be created. However, this does not mean that a new keyword having an expanded viewpoint is effectively and automatically obtained.
[0020]
The present invention has been made in consideration of the above-described circumstances, and a document capable of searching for an accurate document desired by a user by selecting related words from various viewpoints from a document group. It is an object of the present invention to provide a search device, a program for executing a function of a document search device, and a computer-readable recording medium storing the program.
[0021]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, a first aspect of the present invention provides a document database that holds a plurality of documents, and a document ranking unit that selects a document that matches a keyword input from the document database and a document that does not match the keyword. A word ranking section that selects a word having a high degree of relevance between the keyword and the word appearing in the matching document selected by the document ranking section as a related word of the keyword; and a related word selected by the word ranking section. A keyword generation unit for adding to the keyword, wherein the document ranking unit searches again for a document that matches the new keyword generated by the keyword generation unit. Divide the conformable documents selected by the ranking section into multiple sets, and for each set From in search of the phrase, document retrieval apparatus which is characterized in that the related words to add the union of them related words to the original keyword.
[0022]
According to a second aspect of the present invention, in the document search device according to the first aspect, the division of the conformable document group is performed such that a specific item for the conformable document is divided into conformable documents having common or approximate values. Features.
[0023]
According to a third aspect of the present invention, in the document search device according to the second aspect, the specific item is a bibliographic item of the relevant document.
[0024]
According to a fourth aspect of the present invention, in the document search device according to the second aspect, when the document in the document database is a patent gazette, the specific item is an applicant.
According to a fifth aspect of the present invention, in the document search device according to the second aspect, when the document in the document database is a patent gazette, the specific item includes an application date, a publication date, a patent registration date, and the like. It is characterized by date information.
According to a sixth aspect of the present invention, in the document search device according to the second aspect, when the document in the document database is a patent gazette, the specific item includes an international patent classification, a facet classification, an F term, and the like. It is a patent classification.
A seventh aspect of the present invention is a program for causing a computer to execute the functions of the document search device according to any one of the first to sixth aspects.
An eighth aspect of the present invention is a computer-readable recording medium on which the program according to the seventh aspect is recorded.
[0025]
Therefore, even if a document regarded as a conforming document is not actually a conforming document, risk is diversified, and the risk of selecting only relevant words that are not suitable as relevant words is reduced.
In addition, by dividing a group of documents regarded as conforming documents into a plurality of document sets with different properties and selecting related words for each document set, it is possible to prevent only related words with similar tendency from being selected. be able to.
As a result, various words can be selected as the related words of the keyword, so that there is a high possibility that an accurate document desired by the user can be searched.
[0026]
When searching a set of documents such as patent gazettes, the related word is obtained by using the date information such as the applicant, filing date, publication date, registration date, or the union of related words found for each conforming document with the same patent classification. Therefore, related words can be selected from various viewpoints.
[0027]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a preferred embodiment of a document search device according to the present invention will be described with reference to the drawings.
[0028]
<First embodiment>
FIG. 1 is a block diagram showing a configuration of a document search device according to the present invention. In FIG. 1, the document search device includes a keyword input unit 110, a document ranking unit 120, a word ranking unit 130, a keyword generation unit 140, a document The output unit 150 includes a document database 160.
[0029]
The keyword input unit 110 allows a user to input, using a keyboard or the like, a character string obtained by combining keywords representing the characteristics of documents in the document database 160.
The input character string is subjected to morphological analysis using the word dictionary 170 to be decomposed into words as necessary. The word dictionary 170 includes at least the notation of each word, the part of speech, and the like.
Alternatively, instead of using the word dictionary 170, the input character string may be divided into n-grams and used as words.
[0030]
The document ranking unit 120 searches the document database 160 for the keyword passed from the keyword input unit 110, and selects a matching document and a mismatching document. The selected conforming document is passed to the word ranking unit 130, and becomes a source for extracting a word as a candidate of a related word.
[0031]
The document database 160 includes document information holding a document to be searched, and word statistical information of each word included in the document (see FIG. 2). For example, the document information holds the following information for each document.
[0032]
Document identifier (ID), document name, bibliographic information (creator, creation date, publishing office, etc.),
Pointer to document entity, etc.
[0033]
The word statistical information holds the following statistical information for each word.
[0034]
Word representation, frequency of occurrence of this word in the entire document database,
Word appearance information, etc.
[0035]
Here, the word appearance information holds the following information for each document in which the word appears.
[0036]
The document identifier of the document in which this word appears, the frequency of word occurrences in this document,
List of locations where this word appears in this document
[0037]
The word ranking unit 130 divides the entire group of matching documents into a plurality of matching document sets, extracts related word candidates for each of the divided matching document sets, and selects a related word from all of these related word candidates.
[0038]
Bibliographic items stored in the document database 160 are extracted from the document identifier of each conforming document, and documents having a common value or a close value with respect to the attribute (for example, creator or creation date) of interest are collected into a plurality of sets. To divide the relevant document.
By creating a set of conforming documents having the same attribute value, related words can be selected from various viewpoints.
When the document database 160 is composed of patent gazettes, “applicant”, “date information such as filing date, publication date, registration date” or “patent classification such as international patent classification, facet classification, F-term”, etc. By using the attribute as an attribute, the relevant document can be classified from a more appropriate viewpoint, and a more appropriate related word can be extracted.
[0039]
Next, a document stored in the document database 160 is extracted from the document identifier of the conforming document for each divided conforming document set, and words are extracted by morphological analysis or n-gram, and unnecessary words prepared in advance are extracted. If the extracted word is registered in the word table, the word is deleted, the remaining words are set as related word candidates, and the degree of relevance between the input keyword and the related word candidate is calculated by, for example, the following (Equation 5). I do.
[0040]
Relevance = Σ _i (rtf _i / K + rtf _i ) / R-β × Σ _j (stf _j / K + stf _j ) / S …… (Equation 5)
here,
R: Number of conforming documents,
S: Number of non-conforming documents,
rtf _i : Number of occurrences of a conforming document in document i;
stf _j : Number of occurrences of non-conforming document in document j,
K and β: tuning parameters.
The first term on the right side of (Equation 5) is the sum of
And the second term is the sum of each non-conforming document.
[0041]
Next, from among all the related word candidates extracted for each of the divided matched document sets, a predetermined number of related word candidates having a high degree of relevance (for example, about 10) are set as related words. elect. The related word selected in this way is passed to the keyword generation unit 140.
The keyword generation unit 140 passes to the document ranking unit 120 all of these related words, or a new keyword generated by adding selected ones from the related words presented to the user.
The document ranking unit 120 selects a document that matches again with the new keyword generated by the keyword generation unit 140, and passes the selected compatible document to the document output unit 150.
The document output unit 150 displays the list of compatible documents selected by the document ranking unit 120 on the display device, and calls the word ranking unit 130 when there is no desired document for the user. If there is a desired document, the document is output to a printer, a display device, a storage device, or the like, or transmitted to another computer device via a network.
[0042]
Next, the operation of the thus configured document search device of the present embodiment will be described with reference to the flowchart of FIG.
First, a keyword composed of, for example, English or Japanese words or a combination of words is input as a character string from an input device such as a keyboard, and is subjected to morphological analysis by the word dictionary 170 as necessary to be decomposed into words. (Step S100).
Alternatively, instead of using the word dictionary 170, the input character string may be divided into n-grams and used as words.
Thus, the keyword input unit 110 is configured.
[0043]
For each word in the input keyword, the word statistical information of the document database 160 is referred to, and for example, a weight corresponding to the importance of the word is calculated using (Equation 1) (step S110).
[0044]
Next, with respect to each document in the document database 160 to be searched, the word statistical information of the document database 160 and the weight of the keyword word calculated in step S110 are referred to, and the word in the keyword is added to the document. Is calculated using, for example, the above (Equation 2) to create a document list (step S120).
[0045]
Using the relevance as a key, the documents in the document list are ordered in descending order, a predetermined number of documents (for example, about 10) from the top are regarded as conforming documents, and a predetermined number of documents (for example, 500 Is regarded as a non-conforming document (step S130).
Alternatively, a list of ordered documents (a list of relevance, document names, bibliographic items, etc.) is presented to the user, and the user is instructed whether or not the document is conforming. A document designated as non-conforming may be a non-conforming document.
The document ranking unit 120 is configured by steps S110 to S130.
[0046]
The user is instructed whether or not there is a document desired by the user in the compatible document list selected in step S130 (step S140).
If there is no desired document, the process proceeds to step S150. If there is a desired document, the process proceeds to step S190.
[0047]
The desired document content or the like is presented to the user by transmitting it to an output device such as a display device, a printer or a storage device, or to another computer device connected via a network (step S190).
[0048]
The bibliographic items stored in the document database 160 are extracted from the document identifier of each conforming document obtained in step S130, and documents having a common value or a close value with respect to the noted attribute (for example, creator or creation date) are extracted. The relevant documents are divided by collecting and dividing them into a plurality of sets (step S150).
When the document database 160 is composed of patent gazettes, “applicant”, “date information such as filing date, publication date, registration date” or “patent classification such as international patent classification, facet classification, F-term”, etc. Is an attribute.
[0049]
Next, a document stored in the document database 160 is extracted from the document identifier of the conforming document for each divided conforming document set, and words are extracted by morphological analysis or n-gram, and unnecessary words prepared in advance are extracted. If the extracted word is registered in the word table, the word is deleted, the remaining words are set as related word candidates, and the degree of relevance between the input keyword and the related word candidate is calculated by, for example, the following (Equation 5). (Step S160).
[0050]
Next, from among all the related word candidates extracted for each of the divided matched document sets, a predetermined number of related word candidates having a high degree of relevance (for example, about 10) are set as related words. A selection is made (step S170).
The steps S150 to S170 constitute the word ranking unit 130.
[0051]
A new keyword is created by adding the related word extracted in step S170 or the related word selected by the user from the related words to the original keyword (step S180).
This constitutes the keyword generation unit 140.
[0052]
The new keyword is selected again in the same manner as in the processing from step S110 to step S130 (document ranking unit 120).
[0053]
When the present embodiment is configured as described above, instead of finding related words from the entire set of conforming documents, the group of conforming documents is divided into a plurality of sets, and related terms are found for each set. Since the union is used as a related word, even if the document regarded as a conforming document is not actually a conforming document, the risk is diversified and the risk of selecting only related words that are not suitable as related words is small. Become.
In addition, by dividing a group of documents regarded as conforming documents into a plurality of document sets with different properties and selecting related words for each document set, it is possible to prevent only related words with similar tendency from being selected. be able to.
As a result, various words can be selected as the related words of the keyword, so that there is a high possibility that an accurate document desired by the user can be searched.
[0054]
When searching a set of documents such as patent gazettes, the related word is obtained by using the date information such as the applicant, filing date, publication date, registration date, or the union of related words found for each conforming document with the same patent classification. Therefore, related words can be selected from various viewpoints.
[0055]
<Embodiment 2>
Furthermore, the present invention is not limited to only the above embodiment. For example, the document search device shown in FIG. 1 can be realized by a computer device 200 having a hardware configuration as shown in FIG.
That is, the computer device 200 includes a keyboard, a mouse, a touch panel, a scanner, and the like, and displays and outputs the input device 1 used for inputting information, various output information, information input from the input device 1, and the like. The display device 2, a CPU (Central Processing Unit; central processing unit) 3 for operating various programs, the programs themselves, and information temporarily created when the CPU 3 executes the programs. A memory 4 for holding, a storage device 5 for holding a document database 160, a word dictionary 170, a program, temporary information at the time of executing the program, and the like, which are handled by the document search device of the present invention, and a recording medium storing the program, data, and the like. And a medium drive used to read them and store them in the memory 4 or the storage device 5. A device 6, is composed of a network connecting device 7 for an interface for connecting to the network 9, which are connected by a bus 8.
The network 9 is a transmission path for coupling the computer device 200 to another computer device 200, and is generally realized by a cable, and uses TCP / IP as a communication protocol. However, the transmission path is not limited to cables, but may be wireless or wired as long as the communication protocol between them is the same. For example, LAN (Local Area Network), WAN (Wide Area Network), the Internet, etc. Can be used.
[0056]
In such a configuration of the computer apparatus 200, each function constituting the document search apparatus according to the above-described embodiment is programmed, written in advance on a recording medium such as a CD-ROM, and is stored in a CD-ROM drive mounted on the computer. By mounting the CD-ROM or the like on such a medium drive device 6, storing these programs in the memory 4 or the storage device 5 of the computer, and executing the programs, the object of the present invention can be achieved. Needless to say.
In this case, the program itself read from the recording medium implements the functions of the above-described embodiment, and the program and the recording medium on which the program is recorded also constitute the present invention.
[0057]
Note that a recording medium for storing the program is a semiconductor medium (for example, ROM, non-volatile memory, etc.), an optical medium (for example, DVD, MO, MD, CD, etc.), a magnetic medium (for example, magnetic tape, flexible disk, etc.). And so on.
[0058]
The functions of the above-described embodiments are realized not only by executing the program loaded into the memory 4 of the computer device 200 but also in cooperation with an operating system or other application programs based on the instructions of the program. The case where the functions of the above-described embodiments are realized by the processing is also included.
[0059]
When distributing to the market, the program is stored and distributed in a portable recording medium, or stored in a storage device of a server computer connected via a communication network such as the Internet, and another program is stored through the communication network. It can also be transferred to a computer. In this case, the storage device of the server computer is also included in the recording medium of the present invention. In the computer, the functions of the above-described embodiments are realized by installing a program on a portable recording medium or a transferred program in a storage device connected to the computer, and executing the installed program. Is done.
[0060]
<Operation in network environment>
FIG. 5 shows a configuration in which the present invention is connected to a wired or wireless communication network for operation.
For example, the server 300 holding the document search program and the terminal 310 used by a plurality of users are connected via the network 9.
In this case, the server 300 and the user terminal 310 are configured by the general-purpose computer device 200 shown in FIG.
The user logs in to the server 300 from the terminal 310, inputs a keyword for document search using the input device, and requests the document search program of the server 300 via the network 9 to execute the search.
The document search program of the server 300 returns, via the network 9, a search result that matches the specified keyword and the progress of the search to the requesting terminal 310. The user terminal 310 outputs the search result and the progress on the way to the output device. At the time of outputting the progress, an instruction to the server 300 is also performed depending on the progress.
By placing the document search program in the server 300 in this way, there is an advantage that the user can always use the latest document search program.
[0061]
【The invention's effect】
As described above, according to the present invention, even if a document regarded as a conforming document is not actually a conforming document, the risk is diversified, and there is a risk that only related words that are not suitable as related words are selected. Becomes smaller.
In addition, by dividing a group of documents regarded as conforming documents into a plurality of document sets with different properties and selecting related words for each document set, it is possible to prevent only related words with similar tendency from being selected. be able to.
As a result, various words can be selected as the related words of the keyword, so that there is a high possibility that an accurate document desired by the user can be searched.
[0062]
When searching a set of documents such as patent gazettes, the related word is obtained by using the date information such as the applicant, filing date, publication date, registration date, or the union of related words found for each conforming document with the same patent classification. Therefore, related words can be selected from various viewpoints.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a document search device according to the present invention.
FIG. 2 is a diagram illustrating a data structure of a document database.
FIG. 3 is a flowchart for explaining the flow of processing of the document search device according to the present invention.
FIG. 4 is a diagram illustrating a hardware configuration when the document search device according to the present invention is implemented by a computer.
FIG. 5 is a diagram for explaining a case where the document search device according to the present invention is operated in a network environment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Input device, 2 ... Display device, 3 ... CPU, 4 ... Memory, 5 ... Storage device, 6 ... Medium drive device, 7 ... Network connection device, 8 ... Bus, 9 ... Network, 110 ... Keyword input part, 120 ... document ranking section, 130 ... word ranking section, 140 ... keyword generation section, 150 ... document output section, 160 ... document database, 170 ... word dictionary, 200 ... computer apparatus, 300 ... server, 310 ... terminal.

Claims (8)

A document database that holds a plurality of documents, a document ranking unit that selects documents that match the keywords input from the document database and documents that do not match, and words that appear in the matching documents selected by the document ranking unit. A keyword ranking unit that selects a word having a high degree of relevance to the keyword as a related word of the keyword; and a keyword generation unit that adds a related word selected by the word ranking unit to the keyword. In the document search device for searching again in the document ranking unit for documents that match the new keyword generated in the above, the word ranking unit divides a group of compatible documents selected by the document ranking unit into a plurality of sets, After finding the related words for each set, calculate the union of those related words. Document retrieval system which is characterized in that the related words that you want to add to the keyword. 2. The document search apparatus according to claim 1, wherein the group of the conformable documents is divided into conforming documents in which specific items for the conforming documents have common or approximate values. 3. The document search device according to claim 2, wherein the specific item is a bibliographic item of the relevant document. 3. The document search device according to claim 2, wherein when the document in the document database is a patent publication, the specific item is an applicant. 3. The document search apparatus according to claim 2, wherein, when the document in the document database is a patent gazette, the specific item is date information such as an application date, a publication date, and a registration date. apparatus. 3. The document search apparatus according to claim 2, wherein when the document in the document database is a patent gazette, the specific item is a patent classification such as an international patent classification, a facet classification, or an F-term. Search device. A program for causing a computer to execute the functions of the document search device according to claim 1. A computer-readable recording medium on which the program according to claim 7 is recorded.

Images