JP4065346B2 - Method for expanding keyword using co-occurrence between words, and computer-readable recording medium recording program for causing computer to execute each step of the method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a keyword expansion method using co-occurrence between words that can avoid excessive search that occurs when a word having ambiguity is a keyword or a word having many synonyms is a keyword, and The present invention relates to a computer-readable recording medium on which a program for causing a computer to execute each step of the method is recorded.
[0002]
[Prior art]
With the development of computers, documents that have been recorded and stored on paper as a medium are generally handled as digitized information, and a large amount of digitized documents will be accumulated in the database. ing.
[0003]
By the way, as a large amount of electronic documents are accumulated, a search technique that can easily find a desired document from a large amount of documents becomes important. In particular, today, when everyone works using a computer, it is possible to construct a search system that allows a beginner of a computer to easily search for a desired document without using a special technique. It becomes an important point.
[0004]
It is also important to be able to perform highly accurate searches with simple operations. For example, as a method for improving the search accuracy, a keyword constituting the search condition is extracted from a search condition expressed by a search expression, a search sentence, or a search keyword group, and a synonym of the extracted keyword is used. A search method is known in which keywords are expanded and search processing is performed using the expanded keywords.
[0005]
[Problems to be solved by the invention]
However, according to the above conventional technique, the search accuracy is improved by expanding the keyword using the synonym. However, when the word having ambiguity is a keyword or when the word having a large number of synonyms is used. In the case of keywords, there are problems that search processing unintended by the user is performed and excessive search occurs due to excessive expansion of keywords.
[0006]
In other words, the search accuracy is improved when viewed from the system side that performs the search processing, but the search accuracy is not necessarily improved when viewed from the user side. It was. In particular, when a search process that is not intended by the user is performed, it is necessary to search for (search for) the target information from the search results, resulting in problems such as poor workability and poor usability, and loss of reliability in search accuracy. A malfunction occurs. In addition, when excessive search is performed, a problem that processing time becomes long occurs.
[0007]
The present invention has been made in view of the above, and even when a word having ambiguity is a keyword or when a word having many synonyms is a keyword, the search process intended by the user is efficiently performed. The purpose of this is to make it possible to avoid excessive searches.
[0008]
[Means for Solving the Problems]
To achieve the above object, expansion of keywords using the co-occurrence between words this is to enter the search expression, the search condition expressed by retrieval text or search keyword group, constitute the search condition A first step of extracting a keyword; a second step of obtaining a synonym of the keyword extracted in the first step; and a co-occurrence between words when a plurality of keywords extracted in the first step exist. A synonym obtained in the second step based on sex and a synonym obtained in the second step, and a synonym obtained in the second step or the low co-occurrence in the third step. And a fourth step of expanding the keyword extracted in the first step using the synonym after the synonym is excluded.
[0009]
Further, expansion of keywords using the co-occurrence between words this is the extracts keywords search expression, enter search criteria expressed in the search text or search keyword group, constitute the search condition 1 step, a second step for obtaining a synonym of the keyword extracted in the first step, and a plurality of keywords extracted in the first step, based on the co-occurrence between words After excluding a synonym having a low co-occurrence from the synonym obtained in the second step, a synonym obtained in the second step, or a synonym obtained in the second step or a synonym having a low co-occurrence in the third step A fourth step of selecting a synonym to be used for keyword expansion from the synonyms of No. 5 and a fifth step of expanding the keyword extracted in the first step by using the synonym selected in the fourth step And.
[0010]
Also, the keyword method extended using co-occurrence between words this is the keyword expansion method using co-occurrence between said word, said fourth step further arbitrary word as synonyms It can be added.
[0011]
Further, the computer-readable recording medium of this is obtained by recording a program for executing the steps of the keyword expansion method using co-occurrence between said word to the computer.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of a keyword expansion method using co-occurrence between words and a computer-readable recording medium storing a program for causing a computer to execute each step of the method will be described with reference to the accompanying drawings. However, this will be described in detail.
[0013]
FIG. 1 is a block configuration diagram showing a hardware configuration of a computer for realizing a keyword expansion method using co-occurrence between words according to the present embodiment. In FIG. 1, 100 is a CPU, 101 is a ROM, 102 is a RAM, 103 is a hard disk device storing search software 104 for performing search processing using a vector space method, and 105 is an input device such as a keyboard or a mouse. 106 denotes a display such as a CRT, 107 denotes a document DB storing a search target document, and 108 denotes a bus for connecting the above-described units.
[0014]
Although not shown, the bus 108 can further be connected to a floppy disk drive device, a CD-ROM drive device, a communication device for transmitting and receiving information via a network, and the like.
[0015]
Next, regarding specific processing by the keyword expansion method using co-occurrence between words according to the present embodiment, (1) search processing using vector space method (2) co-occurrence between words is used. The details will be described in the order of the keyword expansion processing.
[0016]
(1) Search Processing Using Vector Space Method FIG. 2 is a schematic block diagram showing processing of the search software 104 shown in FIG. With reference to FIG. 2, a search process using the vector space method by the search software 104 will be described to explain a technique that is a premise of a keyword expansion process using co-occurrence between words, which will be described later.
[0017]
The search software 104 performs a process of registering a search target document stored in the document DB 107 in the transposed file 205 and a search process using a vector space method in order to enable a search process using a vector space method.
[0018]
a. Processing for Registering Search Target Document in Transposed File Processing for registering a search target document in transposed file 205 is performed by natural language processing module 200 and database build component 204 shown in FIG.
[0019]
The natural language processing module 200 inputs a search target document from the document DB 107 and, for the input search target document, a format recognition process, a dictionary 201 storing part-of-speech information, and a grammar for analyzing the dependency of each word. Analysis processing such as morphological analysis and syntax analysis is performed using the dictionary 202. Then, based on the analysis result, the search target document is divided into sub-documents composed of a plurality of sentences, noun phrases are extracted from the sub-documents, and then a document set 203 composed of noun phrase lists for each sub-document is generated.
[0020]
The database build component 204 inputs the document set 203 of the search target document generated by the natural language processing module 200, and for each noun phrase of the input document set 203, the appearance frequency in the subdocument, the entire document DB 107 Statistical information such as the distribution in is obtained, and the sub-document is converted into a vector expression using the statistical information for each obtained noun phrase. This processing is performed for all the subdocuments in the document set 203, and a vector representation of the search target document is generated based on the vector representation of the converted subdocument and registered in the transposed file 205.
[0021]
In this way, each search target document in the document DB 107 is converted into a vector expression and registered in the transposed file 205. This makes it possible to perform a search process using the vector space method.
[0022]
b. Search Processing by Vector Space Method Search processing by the vector space method is performed by the natural language processing module 200, the query build component 207, and the search engine 209 shown in FIG.
[0023]
The natural language processing module 200 inputs a search condition 206 from the input device 105, and uses the dictionary 201 storing part-of-speech information and the grammar dictionary 202 for analyzing the dependency of each word for the input search condition 206. Analysis processing such as morphological analysis and syntax analysis is performed, and keywords constituting the search condition 206 are extracted. Then, synonyms and related words of the extracted keyword are extracted from the thesaurus dictionary 210, the keyword is expanded using the extracted synonym, and a document set 203 including the keyword is generated.
[0024]
The query build component 207 inputs the document set 203 of the search condition 206 from the natural language processing module 200, and for each keyword in the input document set 203, the appearance frequency in the document set 203, the transposed file 205. Statistical information such as distribution in the whole is obtained, and a query document 208 is generated by converting each keyword into a vector expression using the obtained statistical information.
[0025]
A search engine (for example, CLARTECH's CLARIT) 209 searches each query document 208 generated by the query build component 207 for the similarity to the vector expression of the search target document in the transposed file 205, and then performs a search. The target document is scored according to the similarity with the query document 208, and a search target document having a score exceeding the determination threshold is output as a search result.
[0026]
(2) Keyword Expansion Processing Using Word Co-occurrence Next, keyword expansion processing using word co-occurrence will be described. The keyword expansion process using the co-occurrence between words is performed in the natural language processing module 200 when the above-described search process using the vector space method is performed. That is, it is executed when the natural language processing module 200 extracts the keyword synonyms and related words extracted from the search condition 206 from the thesaurus dictionary 210 and expands the keywords using the extracted synonyms.
[0027]
FIG. 3 is a flowchart showing keyword expansion processing using co-occurrence between words. When the search software 104 is input with the search condition 206 from the input device 105 (S301), the natural language processing module 200 analyzes the input search condition 206 and performs a process of extracting keywords constituting the search condition 206 ( S302). Here, the search condition 206 may be expressed by any of a search expression, a search sentence, or a search keyword group.
[0028]
After extracting keywords constituting the search condition 206 in step S302, it is determined whether or not there are a plurality of extracted keywords (S303).
[0029]
If there are a plurality of extracted keywords, a process is performed for extracting corresponding synonyms and related words from the thesaurus dictionary 210 for each extracted keyword (S304).
[0030]
When synonyms and related words are extracted from the thesaurus dictionary 210, a process of excluding synonyms and related words having low co-occurrence from the extracted synonyms and related words is performed based on the co-occurrence between words (S305).
[0031]
Here, the process in step S305 will be described in detail. If the input search condition 206 is, for example, a search sentence “new model of passenger car”, “passenger car” and “model” are extracted as keywords of the search condition 206 in step S302. In step S303, synonyms and related words are extracted for each of the keywords “passenger car” and “model”.
[0032]
FIG. 4 is an explanatory diagram illustrating an example of a result of extracting synonyms and related words of the “model” that is a keyword. As is clear from FIG. 4, a number of synonyms and related terms are extracted for the keyword “model”. Therefore, in step S305, processing for excluding synonyms and related words having low co-occurrence with another keyword "passenger car" is performed.
[0033]
FIG. 5 is an explanatory diagram illustrating an example of a result of excluding those having low co-occurrence with “passenger car” from the synonyms and related words of “model” illustrated in FIG. 4. As is clear from FIGS. 4 and 5, synonyms and related terms such as “beauty” and “mode” having low co-occurrence with “passenger car” are excluded. If the search condition 206 is “swimsuit model”, the synonyms and related terms excluded in FIG. 5 are different.
[0034]
As described above, after synonyms and related words having low co-occurrence are excluded, the process proceeds to step S306 in FIG. 3, and keywords are used by using synonyms and related words after removing synonyms and related words having low co-occurrence. Process to extend.
[0035]
On the other hand, if a plurality of keywords extracted from the search condition 206 does not exist in step S303, the process proceeds to step S307, where a synonym or related word corresponding to the extracted keyword is extracted from the thesaurus dictionary 210 and then extracted. And processing for expanding keywords with related words (S306).
[0036]
Then, the keyword expanded in step S306 is output to the query build component 207 as the document set 203 as described with reference to FIG.
[0037]
In the above description, the keyword is expanded as it is with the synonym or related word extracted in step S307 or the synonym or related word after excluding the low-co-occurrence synonym or related word in step S305. It is also possible to select synonyms and related words used for keyword expansion before keyword expansion.
[0038]
FIG. 6 is an explanatory diagram illustrating an example of a selection screen for selecting synonyms and related words before keyword expansion. In FIG. 6, reference numeral 600 denotes a search condition input window, 601 denotes a search condition input field, and 602 denotes a synonym / related word selection window.
[0039]
When the search condition 206 is entered in the search condition input field 601, the processing up to step S305 in FIG. 3 is performed, and synonyms and related words extracted for keyword expansion are displayed in the synonym / related word selection window 602. . In FIG. 6, similar to FIG. 5, synonyms and related words after the removal of synonyms and related words with low co-occurrence for the keyword “model” are displayed in the synonym / related word selection window 602.
[0040]
The user uses the synonym / related word in the displayed synonym / related word selection window 602 to specify that a certain synonym or related word should be used for keyword expansion, or to use it for keyword expansion. Synonyms and related terms can be selected. In addition, when a desired synonym or related word does not exist in the synonym / related word selection window 602, an arbitrary word according to the user's preference can be added as a synonym or related word.
[0041]
As described above, according to the keyword expansion method using the co-occurrence between words according to the present embodiment, when there are a plurality of keywords extracted from the search condition 206, the thesaurus is based on the co-occurrence between words. Excludes synonyms and related words with low co-occurrence from synonyms and related words extracted from the dictionary 210, so that even if a word with multiple synonyms is a keyword or a word with many synonyms is a keyword, use The search process intended by the user can be performed efficiently, and excessive search can be avoided.
[0042]
In this embodiment, the computer shown in FIG. 1 can be used as a search server by connecting to a network. Therefore, in this case, the search condition 206 is input from a client connected to the network.
[0043]
In the present embodiment, as described with reference to FIG. 2, the description has been made on the assumption that the search processing by the vector space method is performed, but instead of the search processing by the vector space method, the search processing by Boolean search is performed. Obviously it can be done.
[0044]
Furthermore, the keyword expansion method using the co-occurrence between words described in this embodiment is executed by executing a prepared program (search software 104) on a computer such as a personal computer or a workstation shown in FIG. It is realized by doing. This program is recorded on a computer-readable recording medium such as a hard disk, floppy disk, CD-ROM, MO, or DVD, and is executed by being read from the recording medium by the computer. The program can be distributed via the recording medium or via a network.
[0045]
【The invention's effect】
As described above, according to the extended how keywords using co-occurrence between words of the present invention, by entering the search expression, the search condition expressed by retrieval text or search keyword group, the search condition Co-occurrence between words when there are a plurality of keywords extracted in the first step, a second step for obtaining synonyms of the keywords extracted in the first step, and a keyword extracted in the first step The third step of excluding synonyms with low co-occurrence from the synonyms determined in the second step based on gender, and the synonyms determined in the second step or synonyms with low co-occurrence in the third step are excluded And the fourth step of expanding the keyword extracted in the first step using the synonyms after being performed. Therefore, when a word having ambiguity is a keyword, or a word having a number of synonyms is a keyword. Even if there is, The search process of FIG efficiently can be performed, and it is possible to avoid excessive search.
[0046]
Further, according to the extended how keywords using co-occurrence between words of the present invention, queries, enter search criteria expressed in the search text or search keyword group, keywords that constitute the search condition Based on the co-occurrence between words when there are a plurality of first keywords extracted in the first step, a second step for obtaining synonyms of the keywords extracted in the first step, and keywords extracted in the first step The third step of excluding synonyms with low co-occurrence from the synonyms obtained in the second step, and the synonyms after excluding synonyms obtained in the second step or synonyms with low co-occurrence in the third step And a fourth step of selecting a synonym used for keyword expansion, and a fifth step of expanding the keyword extracted in the first step using the synonym selected in the fourth step. Therefore, the word with ambiguity is the keyword And if it is, even if the word has a number of synonyms is a keyword, it is possible to perform a search processing intended by the user effectively, and it is possible to avoid excessive search.
[0047]
Further, according to the extended how keywords using co-occurrence between words of the present invention, the keyword expansion method using co-occurrence between said word, the fourth step further any word Can be added as a synonym, so that the search process intended by the user can be performed more efficiently.
[0048]
Further, according to the computer-readable recording medium of the present invention, since the recorded program for executing the steps of the keyword expansion method using co-occurrence between said word in a computer, the program computer By executing the above, even if the word having ambiguity is a keyword or the word having a large number of synonyms is a keyword, the search process intended by the user can be performed efficiently and excessively. Search can be avoided.
[Brief description of the drawings]
FIG. 1 is a block configuration diagram showing a hardware configuration of a computer for realizing a keyword expansion method using co-occurrence between words according to the present embodiment.
FIG. 2 is a schematic block diagram showing processing of search software in the keyword expansion method using co-occurrence between words according to the present embodiment.
FIG. 3 is a flowchart showing keyword expansion processing using co-occurrence between words in the keyword expansion method using co-occurrence between words according to the present embodiment;
FIG. 4 is an explanatory diagram illustrating an example of a result of extracting synonyms and related words of a “model” that is a keyword in the keyword expansion method using co-occurrence between words according to the present embodiment;
5 is an explanatory diagram showing an example of a result obtained by excluding those having low co-occurrence with “passenger car” from the synonyms and related words of “model” shown in FIG. 4;
FIG. 6 is an explanatory diagram illustrating an example of a selection screen for selecting synonyms and related words before keyword expansion in the keyword expansion method using co-occurrence between words according to the present embodiment;
[Explanation of symbols]
100 CPU
101 ROM
102 RAM
103 Hard Disk Device 104 Search Software 105 Input Device 106 Display 107 Document DB
108 Bus 200 Natural Language Processing Module 201 Dictionary 202 Grammar Dictionary 203 Document Set 204 Database Build Component 205 Transposition File 206 Search Condition 207 Query Build Component 208 Query Document 209 Search Engine 210 Thesaurus Dictionary 600 Search Condition Input Window 601 Search condition input field 602 Synonym / related word selection window

Claims (4)

A keyword expansion method for expanding keywords using a computer equipped with a natural language processing means for performing analysis processing using a grammar dictionary,
A first step in which the natural language processing means acquires an input of a search condition expressed by a search expression, a search sentence, or a search keyword group;
A second step of extracting a keyword constituting the search condition input in the first step;
A third step in which the natural language processing means determines whether the keyword extracted in the second step is one or plural;
A fourth step in which the natural language processing means obtains a synonym of the keyword extracted in the second step;
The natural language processing means obtains in the fourth step based on the co-occurrence between words when there are a plurality of keywords extracted in the second step as a result of the determination in the third step. A fifth step of excluding synonyms having a low co-occurrence with a keyword different from the arbitrary keyword among synonyms of the arbitrary keyword;
The natural language processing means uses the synonym obtained in the fourth step when the keyword extracted in the second step is one as a result of the determination in the third step, and the third step If there are a plurality of keywords extracted in the second step as a result of the determination, the synonyms after excluding the low-co-occurrence synonyms in the fifth step are used in the second step. A sixth step of expanding the extracted keywords;
A keyword expansion method using co-occurrence between words, wherein the computer is executed. A keyword expansion method for expanding keywords using a computer equipped with a natural language processing means for performing analysis processing using a grammar dictionary,
A first step in which the natural language processing means acquires an input of a search condition expressed by a search expression, a search sentence, or a search keyword group;
A second step of extracting a keyword constituting the search condition input in the first step;
A third step in which the natural language processing means determines whether the keyword extracted in the second step is one or plural;
A fourth step in which the natural language processing means obtains a synonym of the keyword extracted in the second step;
The natural language processing means obtains in the fourth step based on the co-occurrence between words when there are a plurality of keywords extracted in the second step as a result of the determination in the third step. A fifth step of excluding synonyms having a low co-occurrence with a keyword different from the arbitrary keyword among synonyms of the arbitrary keyword;
A synonym selected by the user for use in keyword expansion from the synonym obtained by the natural language processing means after the synonym obtained in the fourth step or the synonym having the low co-occurrence in the fifth step is excluded. A sixth step of acquiring;
The natural language processing means uses the synonym obtained in the fourth step when the keyword extracted in the second step is one as a result of the determination in the third step, and the third step As a result of the determination by the above, when there are a plurality of keywords extracted in the second step, the synonym after the synonym having the low co-occurrence in the fifth step is excluded or selected by the sixth step A seventh step of expanding the keyword extracted in the second step by using a synonym;
A keyword expansion method using co-occurrence between words, wherein the computer is executed.   The keyword expansion method using co-occurrence between words according to claim 2, wherein in the sixth step, an arbitrary word can be added as a synonym.   A computer-readable recording, wherein a program for causing a computer to execute each step of the keyword expansion method using the co-occurrence between words according to any one of claims 1 to 3 is recorded. Medium.

Images