JPH0728823A - Coocurrence word extracting method and device therefor - Google Patents

Abstract

PURPOSE:To automatically extract a coocurrence relation which is effective in the selection of a homonym and has high reliability. CONSTITUTION:A homonym supply part 1 successively supplies each word of homonyms. A text extraction part 2 extracts plural texts including the words supplied from the homonym supply part 1. A word extraction part 3 extracts a specified word in the extracted text. A number of text counting part 4 determines the number of text where each word appears for each word in the texts. A number of word counting part 5 determines the number of word every number of text. A coocurrence word selection part 6 extracts a coocurrence word group based on the number of text counted by the number of text counting part 4 and the number of word counted by the number of word counting part 5. A coocurreence word deletion part 7 eliminates words included in at least two coocurrence word groups of the coocurrence word group corresponding to each word of homonyms from each coocurrence word group.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a co-occurrence word extraction method and apparatus for extracting a co-occurrence relation between words.

[0002]

2. Description of the Related Art In recent years, attempts have been made to perform natural language analysis processing using the co-occurrence relationship between words. For example, in kana-kanji conversion, a co-occurrence word pair composed of two co-occurring words is created in advance, and if there is a co-occurrence word pair among conversion candidates, then the candidate is prioritized and a homophone is selected. Method has been developed. Since such a co-occurrence word was initially created manually, the work was enormous. In order to improve this, a method of extracting the co-occurrence relation based on the number of appearances of two words appearing simultaneously in the text has been developed.

FIG. 5 is a block diagram of a conventional co-occurrence word extraction device. In the figure, 504 is a text database, which holds a plurality of texts. A word extraction unit 501 extracts words from the text in the text database 504. 502 is an appearance frequency counting unit, which is a word extracting unit 5.
A set of two words is created from the words extracted by 01, and the frequency of appearance is counted. A co-occurrence word selection unit 503 selects a co-occurrence word based on the appearance frequency.

The operation of the conventional co-occurrence word extraction device configured as described above will be described below. (1) The word extraction unit 501 uses the text database 504.
Independent words are extracted from each text.

(2) The appearance frequency counting unit 502 creates a combination of two words from the words in each text and obtains the appearance frequency thereof. Furthermore, the appearance frequencies obtained from all the texts are added for each combination.

(3) The co-occurrence word selection unit 503 selects a combination of two words based on the appearance frequency (end).

The specific operation of the conventional co-occurrence word extraction device that operates as described above will be described below.

The text database 504 stores a plurality of texts delimited by a certain unit.
Here, it is assumed that the text database 504 has a plurality of texts divided for each sentence.

(1) The word extraction unit 501 morphologically analyzes each text in the text database 504 and extracts independent words in each text. For example, from the text "stress on information devices is called techno-stress, and techno-stress is increasing in recent years."
“Equipment”, “Stress”, “Techno”, “Call”, “Recent”, and “Increase” are extracted.

(2) The appearance frequency counting unit 502 creates a combination of two words in each text and obtains the appearance frequency thereof. For example, in the case of the above-mentioned text "for information device ...",
The result of FIG. 6 is obtained.

(3) The appearance frequency counting unit 502 further adds the appearance frequencies obtained from all the texts for each combination.

(4) The co-occurrence word selection unit 503 selects a combination of two words based on the frequency. For example, if a combination with a frequency of 3 or more is selected, a combination of “information-stress” and “device-stress” will be extracted from the above-mentioned text “for information device”.

[0013]

In the above-described specific example of the conventional co-occurrence word extraction method and its apparatus, sentences including "information" and "stress" or "device" and "stress" at the same time are stored in the text database. Even if there is only one in
A combination of "stress" and "equipment-stress" is extracted. As described above, in the conventional co-occurrence word extraction method and apparatus, when there is a word having a high appearance frequency in a certain text, a combination of two words having the word on one side is easily extracted as a co-occurrence word. There was a property of becoming. This property indicates that the conventional co-occurrence word extraction method and the co-occurrence word extracted by the apparatus are extremely sensitive to the contents of the prepared text. Therefore, there is a problem that a combination of unsuitable words is easily extracted as a co-occurrence word for selecting a homophone.

The present invention is capable of automatically extracting a highly reliable co-occurrence relation in view of the above problems of the conventional co-occurrence word extraction method and apparatus.
It is an object of the present invention to provide a co-occurrence word extraction method and apparatus which are effective for selecting homophones.

[0015]

[Means for Solving the Problems]

(1) In order to achieve the above-mentioned object, the present invention provides a homophone supply unit that sequentially supplies each of the homophones, and a text extraction unit that extracts a plurality of texts including the words supplied by the homophone supply unit. , A word extraction unit for extracting a specific word in the extracted text, a text number counting unit for determining the number of texts in which each word appears in each word in the text, and a text counted by the text number counting unit A co-occurrence word extraction device comprising a co-occurrence word selection unit that extracts a co-occurrence word group based on the number of co-occurrence words.

(2) A co-occurrence word deletion unit for removing a word included in at least two co-occurrence word groups from the co-occurrence word groups corresponding to each homophonic word is added. Configure a word extraction device.

(3) Co-occurrence word extraction in which a co-occurrence word selection unit extracts a co-occurrence word group based on the number of words by adding a word number counter for obtaining the number of words for each number of texts counted by the text number counter Configure the device.

(4) The word extracting unit constitutes a co-occurrence word extracting device for extracting an independent word in the text. (5) A co-occurrence word extraction device that adds a co-occurrence word candidate storage unit that stores candidates for a co-occurrence word in advance and causes the word extraction unit to extract the words in the co-occurrence word candidate storage unit from the words in the text. Constitute.

[0019]

According to the co-occurrence word extracting method and the apparatus thereof of the present invention, when the homophone supplying section supplies each word of the homophone to the text extracting section, the text extracting section is supplied by the homophone supplying section. Extracting a plurality of texts including words, the word extraction unit extracts a specific word in the extracted text, the text number counting unit counts the number of texts in which each word appears for each word in the text, The word number counting unit counts the number of words for each number of texts counted by the text number counting unit, and the co-occurrence word selection unit based on the number of texts counted by the text number counting unit and the number of words counted by the word number counting unit. The co-occurrence word group is extracted, and the co-occurrence word deletion unit removes the words included in at least two co-occurrence word groups from the co-occurrence word groups among the co-occurrence word groups corresponding to the respective words of the homophone.

[0020]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the co-occurrence word extraction device of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of a co-occurrence word extraction device according to an embodiment of the present invention. In the figure, 8 is a text database, which holds a plurality of texts. A co-occurrence word candidate storage unit 9 stores co-occurrence word candidates. 1 is a homophone supply unit,
Each of the homophones is sequentially supplied. Reference numeral 2 is a text extraction unit, which extracts a text including the word supplied from the homophone supply unit 1 from the text database 8. Reference numeral 3 denotes a word extraction unit, which extracts a specific word contained in the text extracted by the text extraction unit 2. Reference numeral 4 denotes a text number counting unit which, for each word extracted by the word extracting unit 3, calculates the number of texts in which the word was included. Reference numeral 5 is a word number counting unit which obtains the number of words for each text number counted by the text number counting unit 4. Reference numeral 6 denotes a co-occurrence word selection unit for selecting a valid co-occurrence word. 7
Is a co-occurrence word deletion unit, which compares the co-occurrence word groups corresponding to each of the homophones and excludes words included in at least two co-occurrence word groups.

The operation of the co-occurrence word extracting device of the present embodiment configured as described above will be described with reference to the flowchart of FIG.

(201) The homophone word supply unit 1 supplies one of the homophone words to the text extraction unit 2. (202) The text extracting unit 2 extracts from the text database 8 the text including the word given from the homophone supplying unit 1.

(203) The word extraction unit 3 extracts the words stored in the co-occurrence word candidate storage unit 9 from the words in each text extracted by the text extraction unit 2.

(204) For each word extracted by the word extraction unit 3, the text number counting unit 4 obtains the number of texts in which each word appears.

From the combination of the words and the number of texts obtained in (205) and (204), a set of words having the same number of texts can be assumed. The word number counting unit 5 obtains the number of words having the same number of texts for each number of texts.

(206) The co-occurrence word selection unit 6 includes the word counting unit 5
A reference value for selecting co-occurrence words is obtained from the number of texts and the number of words obtained by, and the co-occurrence word is extracted based on the reference value. Generally, the smaller the number of texts, the larger the number of words. Therefore, the number of texts immediately before the increase in the number of words becomes extremely large is set as a reference value, and words having the number of texts equal to or larger than the reference value are extracted.

(207) If all homophones have been processed, the process proceeds to step (208). If there is an unprocessed homonym, proceed to (201).

(208) The co-occurrence word deleting unit 7 compares at least two co-occurrence word groups in association with each homophone and extracts at least two.
The words included in one co-occurrence word group are removed from each co-occurrence word group (end).

The specific operation of the co-occurrence word extraction device of this embodiment, which operates as described above, will be described below.

The homonym word supply section 1 is assumed to hold, for example, the homonym words "apparatus" and "crisis" whose reading character string is "ki". Further, the text database 8 stores a plurality of texts divided into a certain unit. In the present embodiment, the text database 8 is assumed to have a plurality of texts divided by sentence.

The co-occurrence word candidate storage unit 9 also stores co-occurrence word candidates. For example, it is conceivable to store words in the co-occurrence word candidate storage unit 9 based on grammatical information such as a part of speech or a semantic code, or obtained by individually selecting a specific word. In this embodiment, the co-occurrence word candidate storage unit 9 stores a part of the independent words excluding the words such as "recently" which are generally used.

(1) The homophone supplying unit 1 takes out one of the homophones "device" and gives the extracted word "device" to the text extracting unit 2.

(2) The text extraction unit 2 extracts a plurality of texts including the word "device" given from the homophone supply unit 1 from the text database 8, and, for example, the text "stress on information device is referred to as techno-stress. In recent years, techno-stress has been increasing. "

(3) The word extraction unit 3 performs morphological analysis on each text extracted by the text extraction unit 2 and extracts only the words stored in the co-occurrence word candidate storage unit 9. For example, in the case of the text “for information devices ...”, the independent words are “information”, “device”, “stress”, “techno”, “call”, “recent”.
Although “increased”, “recent” is not stored in the co-occurrence word candidate storage unit 9, and thus is removed. As a result, the word extraction unit 3 extracts the independent words “information”, “device”, “stress”, “techno”, “call”, and “increase”.

(4) The text number counting section 4 includes a word extracting section 3
For each word extracted by, the number of texts in which each word appears is counted and, for example, the result as shown in FIG. 3A is obtained.

(5) The word counting section 5 obtains the number of words having the text number for each text number, and obtains the result shown in FIG. 4, for example.

(6) In FIG. 4, the increase in the number of words having a text number smaller than 3 is much larger than the increase in the number of words having a text number of 3 or more. Therefore, the co-occurrence word selection unit 6
From the word group corresponding to “device”, only words having a text number of 3 or more are extracted from the word group.

Here, the word "stress" appears three times in the above text, but if the word "stress" does not appear in other texts including "apparatus", "stress".
The number of texts corresponding to is 1 and does not remain as a valid co-occurrence word.

(7) The homophone supply unit 1 takes out the next homophone "crisis" and extracts the corresponding word group by the same procedure as in the case of "apparatus". For example, the result of FIG. 3 (b) is obtained. .

(8) Since all the homophones have been processed, the process proceeds to the next step. (9) The co-occurrence word deletion unit 7 compares the co-occurrence word groups corresponding to “equipment” and “crisis” and removes the words included in at least two co-occurrence word groups from each co-occurrence word group. For example, FIG.
3 (b), the word "structure" is removed from both.

As a result, in the case of the example of FIG. 3, "apparatus", "electronic", "sales" ... are co-occurrence words of "equipment", and "military""era" as "crisis" co-occurrence words. "Theories" ... can be automatically extracted.

In this embodiment, after the text extracting unit 2 extracts the necessary text from the text database 8 storing a plurality of texts, the word extracting unit 2 extracts the independent word contained therein. Alternatively, a method may be used in which an independent word is extracted in advance corresponding to each text in the text database 8, an independent word database that stores this is provided, and a necessary independent word group is extracted from the independent word database. . Further, when the word extracting unit 3 extracts a word from the text, the morphological analysis process is performed, but a simplified method such as matching in terms of a prepared dictionary may be used. Also, the co-occurrence word selection unit 6
Instead of using the number of texts in which the number of words increases as a reference value, a predetermined number of texts may be set in advance and, for example, a word that appears in at least two texts may be selected. Alternatively, the method of executing the process of the co-occurrence word deleting unit 7 prior to the process of the co-occurrence word selecting unit 6 may be used.

[0044]

In the co-occurrence word extraction method and apparatus according to the present invention, the co-occurrence words are selected based on the number of texts in which a set of words that appear in the text at the same time appears. The utterance is less likely to be affected by the contents of the prepared text.

Further, among the co-occurrence word groups extracted in association with each homophone word, words included in at least two co-occurrence word groups are removed from each co-occurrence word group. In this case, it is possible to prevent the same word in the conversion candidates from having a co-occurrence relationship with a plurality of words in the same phoneme.

Also, the number of words increases as the number of texts decreases, but co-occurrence words are selected based on the number of texts that significantly increases as the number of words increases. It is possible to extract co-occurrence words with good cost performance by comparing the sexes.

As described above, in the co-occurrence word extraction method and apparatus of the present invention, a co-occurrence word that is effective in selecting a homophone and has high reliability can be automatically extracted. Is big.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a co-occurrence word extraction device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing the operation in one embodiment of the present invention.

FIG. 3 (a) is a schematic diagram showing an example of a word corresponding to “device” in one embodiment of the present invention and the number of texts thereof. (B) is a word corresponding to “crisis” in one embodiment of the present invention. And schematic diagram showing an example of the number of texts

FIG. 4 is a schematic diagram showing an example of the number of texts and the number of words in one embodiment of the present invention.

FIG. 5 is a block diagram of a conventional co-occurrence word extraction device.

FIG. 6 is a schematic diagram showing an example of a combination of two words and an appearance frequency in a conventional co-occurrence word extraction device.

[Explanation of symbols]

 1 homophone supply unit 2 text extraction unit 3 word extraction unit 4 text number counting unit 5 word number counting unit 6 co-occurrence word selection unit 7 co-occurrence word deletion unit 9 co-occurrence word candidate storage unit

Claims (10)

[Claims] 1. A homophone supplying step for sequentially supplying each of the homophones, a text extracting step for extracting a plurality of texts including the words supplied by the homophone supplying step, and a specific word in the extracted text. The word extraction step for extracting the co-occurrence word group based on the number of texts counted in the text number counting step and the number of texts for each word in the text A method of extracting a co-occurrence word, which comprises performing a co-occurrence word selecting step. 2. A homophonic word supply section for sequentially supplying each word of the homonym word, a text extraction section for extracting a plurality of texts including the word supplied by the homonym word supply section, and a specific word in the extracted text. A coexistence word group is extracted based on the number of texts counted by the text extraction unit, a text number counting unit that obtains the number of texts in which each word appears for each word in the text, and the text number counting unit A co-occurrence word extraction device, comprising: 3. A co-occurrence word group corresponding to each of the homophones,
The co-occurrence word extracting method according to claim 1, further comprising a co-occurrence word deleting step of removing a word included in at least two co-occurrence word groups from each co-occurrence word group. 4. A co-occurrence word group corresponding to each of the homophones,
The co-occurrence word extracting device according to claim 2, further comprising a co-occurrence word deleting unit that removes words included in at least two co-occurrence word groups from each co-occurrence word group. 5. A word number counting step of obtaining the word number for each text number counted in the text number counting step,
The co-occurrence word extraction method according to claim 1, wherein the co-occurrence word selection step extracts the co-occurrence word group based on the number of words. 6. The co-occurrence word selecting unit extracts a co-occurrence word group based on the number of words, the word number counting unit obtaining the number of words for each text number counted by the text number counting unit. Origin extraction device. 7. The co-occurrence word extracting method according to claim 1, wherein the word extracting step extracts independent words in the text. 8. The co-occurrence word extraction device according to claim 1, wherein the word extraction unit extracts an independent word in the text. 9. The co-occurrence word extracting method according to claim 1, wherein the word extracting step extracts a word from a co-occurring word candidate prepared in advance among the words in the text. 10. A co-occurrence word candidate storage unit for storing candidates of a co-occurrence word in advance, wherein the word extraction unit extracts a word in the co-occurrence word candidate storage unit from the words in the text. Co-occurrence word extraction device.