JPH11250097A - Key word extracting method, device therefor and storage medium stored with key word extracting program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To securely express an intention expressing a sentence and to generate a key word for new retrieval even in the case of parallel expression in the relation of modification by semantically sorting extracted key words and imparting degree of importance for every sorted key word. SOLUTION: A word importance decision part 101 inputs a syntactic tree after finishing morpheme analysis and semantic analysis and decides the importance of words included in the sentence by referring to a word importance defining rule storing part 102, which stores a rule defining the importance of a sentence pattern and a word. A word extracting part 103 extracts words by using an extracting word meaning defining table storing part 104 defining the meaning of a word to extract. A word sorting part 105 semantically sorts the words extracted by the part 103 by using a thesaurus storing part 106. A key word part 107 outputs key words of the pieces obtained by designating the words stored by the word sorting part in descending order of importance.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a keyword extracting method and apparatus, and a storage medium storing a keyword extracting program, and more particularly to a business for automatically extracting important keywords from sentences in a document retrieval system. And a storage medium storing a keyword extraction program.

[0002]

2. Description of the Related Art Conventionally, in a system for automatically extracting a keyword, a method based on frequency information by a statistical method is used.

[0003]

However, the method of extracting a keyword based on frequency information by the above-described conventional statistical method has the following problems. The first problem is that the guidance text displayed at stores and the like indicates that
Because the sentence is simply written with about 00 characters, words with the same meaning do not appear so many that statistical methods can be used. However, many words with similar meanings such as “rental video, dubbing, shooting, editing” appear. In the related art, these words can be handled only as independent keywords.

[0004] The second problem is that the same word has a different importance depending on the appearance position in the sentence. "We deal in TV, video, and various electrical products" and "Stores featured on TV" have different levels of "TV", but conventional technology treats them as the same. A third problem is that when a compound word is extracted as a keyword, it is not possible to extract a portion to be a keyword. For example, when the compound word “electric product sales” is extracted, the keyword should be “electric product”, but in the conventional technology, the keyword is “electric product sales” as a whole or “electric”. Since “product” and “sales” are individually divided into keywords, data unrelated to “electric products” may be searched.

[0005] The fourth problem is that in the case of a parallel expression having dependencies such as "various printing (new year's card, business card, flyer)", the conventional method uses "printing", "new year's card", "business card", "flyer". "New Year's card printing"
Is not a keyword. The present invention has been made in view of the above points, and it is possible to more accurately express the intention of expressing a sentence, to output appropriate words as keywords, and to execute parallel expressions having a dependency relationship. Also in this case, it is an object to provide a keyword extraction method and apparatus capable of generating a new keyword for search and a storage medium storing a keyword extraction program.

[0006]

FIG. 1 is a diagram for explaining the principle of the present invention. According to a first aspect of the present invention, in a keyword extraction method for extracting a keyword by inputting a sentence composed of a natural language, the importance of a word of the sentence is determined (step 1), and the keyword is determined based on the part of speech and the meaning of the word. Is extracted (Step 2), the extracted keywords are semantically classified, importance is given to each of the classified keywords (Step 3), and a predetermined number of keywords are assigned according to the importance. Output (Step 4).

According to the present invention (claim 2), when assigning the importance, the dependency pattern of the word included in the keyword is collated with the word importance definition rule storing the relation of the word, and the word importance is defined. Determine importance. The present invention (claim 3)
Defines keywords in the thesaurus when classifying keywords, converts keywords belonging to the same representative class and events, and regards keywords belonging to the same representative class as keywords of the same class.

According to the present invention (claim 4), when a keyword is classified, if the extracted keyword is a compound word,
If the result of the forward deletion or ending removal is in the thesaurus, the deleted result is regarded as a new keyword,
If the deleted part has the meaning of a verb, output keywords in pairs. According to the present invention (claim 5), when extracting a keyword, if a sentence includes a word having a parallel relationship with a dependency relationship, the portion of the parallel expression is decomposed into individual words, and the dependency is changed. The related words and individual words are combined and extracted as new keywords.

FIG. 2 is a diagram showing the principle of the present invention. The present invention (Claim 6) is a keyword extracting apparatus for extracting a keyword by inputting a sentence composed of a natural language,
Word importance determination means 10 for determining the importance of words in a sentence
1, a word extracting unit 103 for extracting a keyword based on the part of speech and meaning of the word, and a word classifying unit for semantically classifying the keyword extracted by the word extracting unit 103 and assigning importance to each classified keyword 105 and a keyword output unit 107 that outputs a predetermined number of keywords according to the degree of importance given by the word classification unit 105.

According to the present invention (claim 7), the word importance determining means 101 determines a word importance definition rule storing a relationship between words, a dependency pattern of a word included in the keyword, and a word importance definition rule. Means for collating and determining the importance of the word. According to the present invention (claim 8), in the word classifying means 105, a keyword belonging to the same representative class and a keyword belonging to the same representative class by referring to the thesaurus knowledge defining the representative class node in the thesaurus and the event belonging to the same representative class are referred to. Are included as keywords of the same classification.

According to the present invention (claim 9), the word classification means 10
In step 5, if the extracted keyword is a compound word and the result of forward deletion or ending removal is in thesaurus knowledge, the deleted result is regarded as a new keyword, and the deleted part is the meaning of the verb. Means for outputting keywords in pairs when

According to the present invention (claim 10), when a sentence includes a word having a parallel relationship with a dependency relationship, the word extracting / extracting means 103 decomposes the parallel expression portion into individual words. Means, and means for combining a dependency-related word and an individual word and extracting it as a new keyword.

[0013] The present invention (Claim 11) is a storage medium storing a keyword extraction program for inputting a sentence composed of a natural language and extracting a keyword, wherein the word importance is determined for judging the importance of the word of the sentence. A degree determination process, a word extraction process of extracting keywords based on the part of speech and meaning of the words, and semantically classifying the extracted keywords.
It has a word classification process of assigning importance to each classified keyword, and a keyword output process of outputting a predetermined number of keywords according to the importance.

According to the present invention (claim 12), in a word importance determination process, a dependency pattern of a word included in a keyword is collated with a word importance definition rule storing a relation between words, and the importance of the word is determined. The process of determining According to the present invention (claim 13), in the word classification process, the keywords belonging to the same representative classification and the events are converted with reference to thesaurus knowledge defining the representative classification nodes in the thesaurus, and the keywords belonging to the same representative classification are classified into the same classification. Includes processes that are considered keywords.

According to the present invention (claim 14), in the word classification process, when the extracted keyword is a compound word, the result of forward deletion or ending deletion is in the thesaurus knowledge defining a representative classification node in the thesaurus. In such a case, a process of considering the deleted result as a new keyword and outputting the keyword as a pair when the deleted portion has the meaning of a verb is included.

According to the present invention (claim 15), in the word extraction and extraction process, when a sentence includes a word having a dependency relation with a parallel expression, a part of the parallel expression is decomposed into individual words. And a process of combining the words of the dependency relationship with individual words and extracting them as new keywords.

As described above, in the present invention, words belonging to the same representative classification node are treated as the same,
Even for short sentences, it is possible to determine the importance of the keyword based on the frequency. In addition, since the importance of the word can be defined by the dependency relationship, it is possible to determine the difference in the importance of the word from the sentence pattern.

Further, by treating words belonging to the same representative classification node as the same, it is possible to determine the importance of a keyword based on the frequency of a short sentence. In addition, since forward deletion and ending deletion are performed from the keyword of the compound word, a portion suitable as the keyword can be extracted. In addition, since the product keyword and the verb keyword are extracted as a pair, it is possible to improve search accuracy.

Further, in the case of a dependent parallel expression,
Since a keyword is newly generated with a product and a verb that are arranged in parallel, it is possible to search even if a condition that is not parallel is input.

[0020]

FIG. 3 shows the configuration of a keyword extracting apparatus according to the present invention. The keyword extraction device shown in FIG.
Word importance determination unit 101, word importance definition rule storage unit 102, word extraction unit 103, extracted word meaning definition table storage unit 103, word classification unit 105, thesaurus storage unit 1
06 and a keyword output unit 107.

The word importance determining unit 101 performs morphological analysis /
Using the syntax tree after the semantic analysis as input, the importance of the words included in the sentence is determined with reference to the word importance definition rule storage unit 102. Word importance definition rule storage unit 102
Stores rules defining sentence patterns and importance of words.

The word extraction unit 103 extracts words using an extracted word meaning definition table storage unit 104 which defines the meaning of the word to be extracted. The extracted word meaning definition table storage unit 104 stores information in which the meaning is defined for each word. The word classifying unit 105 includes the thesaurus storage unit 10
6, the words extracted by the word extraction unit 103 are semantically classified.

The thesaurus storage unit 106 stores representative classification nodes of words in a tree structure. The keyword output unit 107 outputs a number of keywords in which the words classified by the word classification unit 105 are specified in descending order of importance.
Next, the operation of the above configuration will be described.

FIG. 4 is a flowchart showing the operation of the keyword extracting process according to the present invention. Step 100) The input guide text is cut out for each sentence and the text is formatted. Step 200) A morphological analysis is performed on the sentence that has undergone sentence shaping. Step 300) A semantic analysis is performed for each word divided by the morphological analysis, and the meaning of the word and the dependency relationship between the words are obtained.

Step 400) Next, the word importance determination unit 101 refers to the word importance definition rule storage unit 102 to determine the importance of the word. (Step 500) Next, the word extraction unit 103 refers to the extracted word meaning definition table storage unit 104 and determines a word to be extracted based on the meaning of the word. For example, in a shop guide, words are usually nouns and verbs representing goods and services, so words that have meanings such as things and things are extracted,
Pronouns, conjunctions and adverbs are not extracted.

Step 600) The word classifying unit 105
By performing forward deletion and ending deletion, the thesaurus storage unit 106
For a word in. Step 700) The word classifying unit 105 refers to the thesaurus storage unit 106 and combines words having the same meaningful word into one. For example, in the case of the sentence "English, French, German, etc., studying abroad", "English", "French" and "German" have the same representative classification node of foreign language, so the frequency is "3", “Study abroad” is another representative classification, so the frequency can be “1”.

Step 800) Keyword output unit 10
In step 7, the specified number of keywords are output in descending order of importance.

[0028]

Embodiments of the present invention will be described below with reference to the drawings. First, the process of determining the importance of a word will be described. In the configuration shown in FIG. 3 described above, the word importance definition rule storage unit 102 is provided inside the keyword extraction device. However, in the following description, when determining the word importance (step 400), the importance of the word is determined. An explanation will be given using an example in which an external file is used so that the content of the importance definition rule storage unit 102 can be freely changed when the degree is determined based on the dependency relationship.

FIG. 5 is a diagram showing the meaning of the symbols used in the word importance definition rule according to one embodiment of the present invention.
Shows an example of a word importance definition rule according to an embodiment of the present invention. FIG. 7 is a diagram illustrating an operation of determining the importance of a word according to an embodiment of the present invention. Step 401) The word importance determination unit 101 repeats the following processing while shifting the collation position of the word importance definition rule from the beginning to the end.

Step 402) Rules are extracted one by one from the word importance definition rules. Step 403) With the dependency relation output by the semantic analysis processing as input, it is checked whether there is a rule that matches the word importance definition rule while shifting the matching position of the rule from the beginning to the end of the sentence. Step 404) If there is a matching rule, the evaluation value of the word is stored. Note that the evaluation value is defined as that corresponding to the word frequency. The evaluation value “1” has the meaning of appearing once in the sentence. A word for which there is no matching rule by checking until the end of the sentence is stored as “1”.

For example, in the case of the sentence "It is famous for a well-established Japanese restaurant", it matches the rule 502 shown in FIG.
“Japanese food” is “2”, “long established” is “1”, and “famous” is “1”. In the case of the sentence “Introduced on TV”, the rule 502 shown in FIG. 6 is matched.
“TV” becomes “0” (that is, the same as not appearing in a sentence), and “introduce” becomes “1”.

Step 405) The next rule is retrieved from the word importance definition rule storage unit 102. Next, the word classification processing in the word classification unit 105 will be described. The word classification unit 105 defines a representative classification node on the thesaurus, and expresses a word other than the upper / lower relation such as a relation between a tool and a purpose of use (for example, a car and a driving), such as a word having the same representative classification node as an upper word. When converted by the event conversion table, words belonging to the same representative classification node are regarded as belonging to the same classification.

FIG. 8 shows a flowchart of a thesaurus retrieval process according to an embodiment of the present invention, FIG. 9 shows an example of a thesaurus according to an embodiment of the present invention, and FIG. 4 shows an example of an event conversion table. Step 601) The word classification unit 105 obtains a representative classification node of all words by searching the thesaurus in the thesaurus storage unit 106.

Step 602) Next, the frequency is counted for each representative classification. For example, "XX driving school.
Wheel, regular car, training camp license available. In the case of the sentence "
According to the thesaurus, "normal cars" ranked higher than "automobiles"
Therefore, the frequency of the representative classification “car” is “2”. Step 603) Next, it is checked whether the representative classification can be converted into an event.

Step 604) If there is a word to be converted into the same representative classification, it is merged and the frequency is counted.
For example, “license” is a qualification necessary for driving a car, and therefore, the representative classification node that has been converted using the event conversion table shown in FIG. 10 is the same. After all, "normal car"
Since "car" and "license" are the same representative classification, the frequency is "3".

Next, in the thesaurus search by the word classifying unit 105 (step 600), when the extracted word is a compound word, forward deletion or ending deletion is performed to determine a word to be a keyword. If the determined word has the meaning of a verb, the word can also be output as a pair. FIG.
FIG. 1 is a flowchart showing an operation of a process when a word extracted in a thesaurus search process according to an embodiment of the present invention is a compound word.

Step 701) The word classifying section 105
A thesaurus is searched for the extracted word, and if the word exists in the thesaurus, the extracted word is used as a keyword as it is. Step 702) When the compound word is not in the thesaurus, if the end of the compound word is an ending that indicates an auxiliary meaning such as “supply” or “information”, or if it is an inflected noun such as “sales” or “rental”, delete the compound word Search (move to step 701).

Step 703) When there is no longer any ending or noun that can be deleted, the ending is restored. Step 704) Delete one word at a time from the front, shift to step 701, and thereafter repeat the processing from step 701 to step 703. Step 705) If there is no word or only the ending or the noun, it is not in the thesaurus and the original compound word is used as a keyword.

Step 706) If the thesaurus is found on the thesaurus as a result of searching the thesaurus, it is determined whether the deleted portion has the meaning of a verb. Step 707) If the verb has no meaning, the part in the thesaurus is used as a keyword. Step 708) If the verb has a meaning, a pair of the part in the thesaurus and the deleted part is used as a keyword.

For example, in the case of the word "repair of various home electric appliances", first, "repair" at the end is deleted, but "the various home electric appliances" is not in the thesaurus. Next, "repair" at the end is returned, "various" ahead is deleted, and "home appliance repair" is searched, but this compound word is not in the thesaurus either. Next, when "repair" at the end is also deleted and searched, the keyword is "home electric appliances" because "home electric appliances" exists in the thesaurus. Since the deleted "repair" is a paranoun, it is remembered that "repair" is also a keyword in combination with "home electric appliances".

Next, in the word extraction process (step 500) in the word extraction unit 103, a description will be given of a process in a case where there is a dependency relation in the parallel expression. If there is a dependency relationship in the parallel expression, words in parallel are decomposed individually, and words in the dependency relationship are added to generate a new keyword. FIG. 12 is a flowchart showing an operation of the word extraction processing according to the embodiment of the present invention when there is a dependency relation in the parallel expression.

Step 501) First, the word extracting unit 10
3 determines whether the extracted word has a parallel expression.
In parallel expression, parentheses "various printing (postcard, flyer)"
There are punctuation marks such as "postcards, flyers, and various types of printing." Step 502) Next, words in parallel are divided into individual words. For example, the judgment is made based on the fact that the word between “various” and parentheses is a paravariable noun, and that the end of the parallel expression is “various” + variable noun.

Step 503) Next, the decomposed word and the sa-variable noun are combined and output. In the case of the example, "postcard printing" and "flyer printing" are used. By performing the above-described processing, it is possible to generate a new keyword using a product and a verb in parallel, and to extract a keyword even when a non-parallel condition is input.

Although the above embodiment has been described based on the components shown in FIG. 3, the present invention is not limited to this example. The components shown in FIG. Device, floppy disk, CD connected to computer used as
-The present invention can be easily realized by storing it in a portable storage medium such as a ROM and installing it when implementing the present invention.

It should be noted that the present invention is not limited to the above-described embodiment, but can be variously modified and applied within the scope of the claims.

[0046]

As described above, according to the present invention, the importance of a keyword is determined by the position of a word in a sentence and the frequency of words that are semantically close, so that the intention expressed by the sentence is represented more accurately. It is possible. As a result, search accuracy can be improved. Also, since compound words are subjected to forward deletion and ending deletion, and words suitable as keywords are output, search accuracy is improved as compared to searching for compound words by individual keywords.

Also, when the deleted ending has an important meaning such as a verb, it can be used as a keyword. Also, in the case of a parallel expression having a dependency relationship, a keyword is newly generated for a search, so that improvement in search accuracy can be expected.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a configuration diagram of a keyword extraction device of the present invention.

FIG. 4 is a flowchart illustrating an operation of a keyword extraction process according to the present invention.

FIG. 5 is a diagram illustrating meanings of symbols used in a word importance definition rule according to an embodiment of the present invention.

FIG. 6 is an example of a word importance definition rule according to an embodiment of the present invention.

FIG. 7 is a flowchart illustrating an operation of determining importance of a word according to an embodiment of the present invention.

FIG. 8 is a flowchart of a thesaurus retrieval process according to one embodiment of the present invention.

FIG. 9 is an example of a thesaurus according to an embodiment of the present invention.

FIG. 10 is an example of an event conversion table according to an embodiment of the present invention.

FIG. 11 is a flowchart illustrating an operation of a process when a word extracted in the thesaurus search process according to an embodiment of the present invention is a compound word.

FIG. 12 is a flowchart illustrating an operation of a process in a case where a parallel expression has a dependency relationship in a word extraction process according to an embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 101 Word importance determination part, word importance determination means 102 Word importance definition rule storage part 103 Word extraction part, word extraction means 104 Extracted word meaning definition table storage part 105 Word classification part, word classification means 106 Thesaurus storage part 107 Keywords Output unit, keyword output means

Claims (15)

[Claims] 1. A keyword extracting method for extracting a keyword by inputting a sentence composed of a natural language, determining the importance of the word of the sentence, extracting the keyword based on the part of speech and meaning of the word, and extracting the keyword. A keyword extraction method characterized by semantically classifying the classified keywords, assigning importance to each classified keyword, and outputting a predetermined number of keywords in accordance with the importance. 2. When assigning the importance, a dependency pattern of a word included in the keyword is collated with a word importance definition rule storing a relationship between the words, and the importance of the word is determined. The keyword extraction method according to claim 1. 3. When classifying the keywords, a representative classification node is defined in a thesaurus, and keywords belonging to the same representative class and events are converted. Keywords belonging to the same representative class are regarded as keywords of the same class. The keyword extraction method according to claim 1, wherein the keyword is extracted. 4. When categorizing the keywords, if the extracted keyword is a compound word, and if the result of forward deletion or ending removal is in a thesaurus, the deleted result is regarded as a new keyword. 2. The keyword extracting method according to claim 1, wherein the keywords are output in pairs when the deleted part has the meaning of a verb. 5. When extracting the keyword, if the sentence includes a word having a parallel relationship with a dependency relationship, a part of the parallel expression is decomposed into individual words. Combine the word with individual words,
2. The keyword extracting method according to claim 1, wherein the keyword is extracted as a new keyword. 6. A keyword extracting device for extracting a keyword by inputting a sentence composed of a natural language, comprising: a word importance judging unit for judging the importance of a word of the sentence; Word extracting means for extracting a keyword by means of: a word classifying means for semantically classifying the extracted keywords and assigning an importance to each of the classified keywords; a number designated in advance according to the importance And a keyword output means for outputting the keyword. 7. The word importance determining means compares a word importance definition rule storing a relationship between words, a dependency pattern of a word included in the keyword and the word importance definition rule, and 7. The keyword extracting device according to claim 6, further comprising means for determining the importance of the keyword. 8. The word classifying means converts the keywords belonging to the same representative class and the events by referring to the thesaurus knowledge defining the representative class nodes in the thesaurus and the keywords, and classifies the keywords belonging to the same representative class to the same class. 7. The keyword extracting apparatus according to claim 6, further comprising a unit that regards the keyword as a keyword. 9. The word classification means, when the extracted keyword is a compound word, when the result of forward deletion or ending deletion is in the thesaurus knowledge, the deleted result is regarded as a new keyword. 9. The keyword extracting apparatus according to claim 6, further comprising means for outputting a keyword as a pair when the deleted part has the meaning of a verb. 10. The word extracting / extracting means, when the sentence includes a word having a dependency relationship with a parallel expression, means for decomposing a part of the parallel expression into individual words, Combine related words with individual words,
Means for extracting as a new keyword.
The described keyword extraction device. 11. A storage medium storing a keyword extraction program for inputting a sentence composed of a natural language and extracting a keyword, comprising: a word importance determination process for determining importance of a word of the text; A word extraction process of extracting a keyword according to the part of speech and meaning of a word, a word classification process of semantically classifying the extracted keywords, and assigning an importance to each of the classified keywords, A keyword output process for outputting a predetermined number of keywords. 12. The word importance determination process, wherein a dependency pattern of a word included in the keyword and a word importance definition rule storing a relationship between words are compared,
12. A storage medium storing the keyword extraction program according to claim 11, including a process of determining the importance of the word. 13. The word classification process, wherein keywords belonging to the same representative class and events are converted with reference to a thesaurus knowledge defining a representative classification node in the thesaurus, and keywords belonging to the same representative class are determined to be keywords of the same class. 12. A storage medium storing the keyword extraction program according to claim 11, which includes a process of determining the keyword. 14. The word classification process, wherein when the extracted keyword is a compound word, if the result of forward deletion or ending deletion is in the thesaurus knowledge defining a representative classification node in the thesaurus, the word is deleted. 12. The storage medium storing the keyword extraction program according to claim 11, further comprising a process of considering the result as a new keyword and outputting the keyword as a pair when the deleted portion has the meaning of a verb. 15. The word extraction and extraction process, wherein, when the sentence includes a word having a dependency relationship with a parallel expression, a process of decomposing the parallel expression portion into individual words, Combine related words with individual words,
12. A storage medium storing a keyword extraction program according to claim 11, further comprising: a process of extracting a keyword as a new keyword.