JP2016122398A - Subject word extraction device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract a word becoming a subject from a document even without a dictionary in which all relations between words are described.SOLUTION: A subject word extraction device 1 comprises: an extraction unit 12 which extracts a word from document data; a related word acquisition unit 13 which acquires, from related word dictionary data in which a plurality of related words are described in association, a word related to a word extracted by the extraction unit 12 as a related word; an indirect related word acquisition unit 14 which acquires a word related to the related word from the related word dictionary data as an indirect related word; an importance calculation unit 15 which calculates the importance quantitatively showing the importance of a word, for each word extracted by the extraction unit 12, on the basis of the number of times of appearance that a related word of the word appears in the document data, and the number of times of appearance that an indirect related word of the word appears in the document data; and a selection unit 16 which selects a word representing a subject on the basis of the importance calculated by the importance calculation unit 15 from among words extracted by the extraction unit 12.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a subject word extraction device and a program.

  One technique for extracting important words that are subject words from words included in a document is tf-idf (see, for example, Non-Patent Document 1). tf-idf calculates the importance of a word using the frequency of the word applied for in the document and the frequency of the document in which the word appears. However, since the context is not considered in tf-idf, the subject word may not be extracted correctly. Therefore, there is a technique for calculating the importance of a word in a document in consideration of not only tf-idf but also a value that considers the relationship between words in the document, and extracting a subject word (for example, Patent Documents). 1).

JP2013-242791A

“Tf-idf”, [online], Wikipedia, [October 22, 2014 search], Internet <URL: http://en.wikipedia.org/wiki/Tf-idf>

  In the technique of Patent Document 1, the importance does not increase unless there is any direct relationship between words appearing in the document. For example, suppose that there is a document that says "Ginger is on the stiff shoulders ... You may enter a hot spring. ... Yoga may be good." In this case, if the relationship paths between words such as “Stiff shoulder” and “Ginger”, “Stiff shoulder” and “Hot spring”, “Stiff shoulder” and “Yoga” are obtained from the dictionary, the importance of “Stiff shoulder” increases. However, existing dictionaries indicate words having higher and lower relationships or similar words, and it is difficult to actually have a dictionary describing direct relationships between all words. Therefore, it is not easy to realize the technique of Patent Document 1.

  The present invention has been made in view of such circumstances, and provides a subject word extraction apparatus and program for extracting a subject word from a document without a dictionary that describes all relationships between words.

According to one aspect of the present invention, an extraction unit that extracts a word from document data and another word related to the word extracted by the extraction unit from related word dictionary data that is described in association with a plurality of related words A related word acquisition unit that acquires as a related word, an indirect related word acquisition unit that acquires other words related to the related word from the related word dictionary data as an indirect related word, and each of the words extracted by the extraction unit The degree of importance that quantitatively represents the importance of the word based on the number of occurrences of the related word of the word in the document data and the number of occurrence of the indirect related word of the word in the document data. An importance calculation unit to calculate, and a selection unit that selects a word representing a subject based on the importance calculated by the importance calculation unit from the words extracted by the extraction unit. Toss Which is the subject word extraction apparatus.
According to the present invention, the subject word extraction apparatus is configured to extract a related word that is another word related to a word extracted from document data and an indirect related word that is another word related to each related word. Get from data. The subject word extraction apparatus calculates the importance of a word extracted from document data based on the number of related words that appear in the document data and the number of indirect related words that appear in the document data. . The subject word extraction device selects a word representing the subject based on the importance from the words extracted from the document data.
As a result, even if there is no dictionary that describes all the relationships between words, the subject word extraction device can extract a word suitable as a subject from words included in a document using, for example, an existing related word dictionary. it can.

One aspect of the present invention is the subject word extraction device described above, wherein the extraction unit extracts a word having a predetermined part of speech included in the document data.
According to this invention, the subject word extraction device selects a word representing a subject from words of a predetermined part of speech included in document data.
As a result, the subject word extraction device can extract, from the document data, a word with a part of speech that is easy to understand as a subject, such as a noun, as the subject word.

One aspect of the present invention is the subject word extraction device described above, wherein the importance calculation unit indicates the importance of the word extracted by the extraction unit, and the related word of the word appears in the document data. It is calculated by adding the weighted result to each of the number and the number of the indirectly related words of the word appearing in the document data.
According to this invention, the subject word extraction apparatus determines the importance of a word extracted from document data by the number of occurrences of related words of the word in the document data and the number of occurrences of indirect related words of the word in the document data. Are calculated by adding the weighted results to each of.
As a result, the subject word extraction device places a greater weight on the number of occurrences of directly related words out of the number of occurrences of related words and the number of occurrences of indirect related words in the document data. It is possible to appropriately calculate the importance, such as making the word more frequently appear in the data so that the importance is likely to be higher.

One aspect of the present invention is the subject word extraction device described above, wherein the indirect related word acquisition unit performs processing for acquiring other words related to the indirect related words from the related word dictionary data as indirect related words. It repeats predetermined times, It is characterized by the above-mentioned.
According to the present invention, the subject word extraction device can calculate the importance of a word by further using a word related to the indirect relation word of the word extracted from the document data.
As a result, the subject word extraction apparatus further extracts the word suitable as the subject from the words included in the document by further utilizing the number of occurrences of the related words that are interleaved with a plurality of words in the document data. be able to.

  According to another aspect of the present invention, there is provided a computer related to an extraction unit that extracts a word from document data and a related word dictionary data that is described by associating a plurality of related words with the word extracted by the extraction unit. A related word acquisition means for acquiring the word of the related word as a related word, an indirect related word acquisition means for acquiring another word related to the related word as an indirect related word from the related word dictionary data, and the extraction means extracted For each word, quantitatively represent the importance of the word based on the number of occurrences of the related word of the word in the document data and the number of occurrence of the indirect related word of the word in the document data. Importance level calculating means for calculating importance level, and selecting a word representing the subject based on the importance level calculated by the importance level calculating means from the words extracted by the extracting means Is a program for functioning as a subject word extraction apparatus comprising a-option means.

  According to the present invention, a word as a subject can be extracted from a document without a dictionary that describes all the relationships between words.

It is a figure for demonstrating the operation | movement outline | summary of the theme word extraction apparatus by one Embodiment of this invention. It is a block diagram which shows the structure of the theme word extraction apparatus by the embodiment. It is a flowchart which shows the subject word extraction process of the subject word extraction apparatus by the embodiment. It is a flowchart which shows the example of the word list by the embodiment. It is a flowchart which shows the word list production | generation process of the theme word extraction apparatus by the embodiment. It is a flowchart which shows the 1st importance element calculation process of the theme word extraction apparatus by the embodiment. It is a flowchart which shows the 2nd importance element calculation process of the theme word extraction apparatus by the embodiment. It is a figure which shows the example of the document data by the same embodiment. It is a figure which shows the word which the subject word extraction device by the embodiment extracted from the document data of FIG. It is a figure which shows the example of the related word dictionary data by the embodiment. It is the figure which added the related word and the indirect related word to the word list by the embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram showing an outline of operation of a subject word extraction apparatus according to an embodiment of the present invention. The subject word extraction apparatus according to the present embodiment extracts words that are the subject from sentences included in document data by using related word dictionary data.

  The related word dictionary data is data in which a plurality of related words are described in association with each other. As such related term dictionary data, existing synonym dictionary and concept dictionary data can be used. For example, the Kadokawa new dictionary, classification vocabulary table, etc. can be used for the thesaurus. In addition, for example, the data of the concept dictionary includes “National Institute for Communications Research (NICT),“ ALAGIN Language Resource / Speech Resource Site ”, Advanced Language Information Fusion Forum, [online], Internet <URL: https: //alaginrc.nict .go.jp / opensource.html> "can be used.

  In the related word dictionary data, words like "Stiff shoulder"-"Heat", "Stiff shoulder"-"Gymnastics", "Heat"-"Ginger", "Heat"-"Hot spring", "Gymnastics"-"Yoga" The relationship is registered. “Word T1” − “word T2” indicates that there is a direct relationship between the word T1 and the word T2. The word T2 is a related word of the word T1, and the word T1 is a related word of the word T2.

  If the document data contains the text “Ginger is on the shoulders… You may enter the hot springs… Yoga may be good.” From this document data, “Stiff shoulders”, “Ginger” , “Hot spring”, “yoga” and other words are extracted. In the related word dictionary data, the direct relationship between these extracted words is not registered. However, even when a direct relationship (path) between words is not registered in the related word dictionary data, a path between words may be obtained with another word interposed therebetween. For example, “shoulder stiffness” – “heat” – “ginger”, “shoulder stiffness” – “heat” – “hot spring”, “shoulder stiffness” – “gymnastics” – “yoga” By this, you can get from “Stiff shoulder” to “Ginger”, “Onsen”, “Yoga” (make a pass). In this way, when the word T1 and the word T3 have a common relation word, and the word T3 can be reached from the word T1 with the common relation word in between, the word T3 is indirectly related to the word T1. A word.

  The subject word extraction apparatus according to the present embodiment arrives at a word extracted from a sentence included in document data by sandwiching a common related word from the number of related words that appear in the document data. The degree of importance is given according to the number of indirect relation words that can be generated in the document data. The subject word extraction apparatus of this embodiment selects a subject word from the extracted words based on importance.

  FIG. 2 is a block diagram showing the configuration of the subject word extraction device 1 according to an embodiment of the present invention, and shows only functional blocks related to the present embodiment. The subject word extraction device 1 is realized by a computer device, and, as shown in the figure, an input unit 11, an extraction unit 12, a related word acquisition unit 13, an indirect related word acquisition unit 14, and an importance level calculation unit 15. And a selection unit 16.

  The subject word extraction device 1 is connected to the related word dictionary storage device 3. The related word dictionary storage device 3 stores related word dictionary data. For example, the related word dictionary data includes a plurality of item data described in the format of “word T1 [relation name] word T2”. “Word T1 [Relation Name] Word T2” indicates that the word T1 and the word T2 have a relationship represented by a relationship name. For example, the item data indicates “fruit [upper / lower] apple”, “typhoon [cause / result] heavy rain”, and the like. The related word dictionary data may include a plurality of item data indicating the relationship between the same word and other different words (related words).

  The input unit 11 receives input of document data. The document data includes text data indicating one or more sentences. As the document data, for example, a web page, an electronic book, a program summary sentence of an electronic program guide (EPG), and the like can be used. The extraction unit 12 extracts words from text data included in the document data input by the input unit 11. The related word acquisition unit 13 is a relation of words extracted by the extraction unit 12 from other related words extracted by the extraction unit 12 from the related word dictionary data stored in the related word dictionary storage device 3. Get as a word. The indirect related word acquisition unit 14 acquires, from the related word dictionary data stored in the related word dictionary storage device 3, other words related to the related word of the word extracted by the extraction unit 12 as indirect related words. The importance level calculation unit 15 calculates an importance level that quantitatively represents the importance of the word for each word extracted by the extraction unit 12. The importance level of a word is calculated based on the number of related words that appear in the document data and the number of indirect related words of the word that appear in the document data. The selection unit 16 selects a subject word that is a subject word based on the importance calculated by the importance calculation unit 15 from the words extracted by the extraction unit 12, and outputs the selected subject word.

FIG. 3 is a flowchart showing the subject word extraction process of the subject word extraction device 1.
First, the input unit 11 of the subject word extraction device 1 receives input of document data (step S110). The extraction unit 12 selects one piece of document data from the document data input in step S110 (step S115). The extraction unit 12 extracts a word with a predetermined part of speech from the text data included in the document data selected in step S115. In this embodiment, a predetermined part of speech is a noun. The extraction unit 12 generates a word list that is data in which a list of extracted words is set (step S120). The i-th word set in the word list is described as word Ai (i is an integer of 1 or more). In the following, the words A1, A2,... Are collectively referred to as the word A.

The related word acquisition unit 13 acquires each related word of the word Ai set in the word list from the related word dictionary data stored in the related word dictionary storage device 3. It acquired the words of Ai _{n i} number _{(n i} is an integer of 1 or more) of relational terms, respectively, the relationship between language Bi1, Bi2, ..., and Bin _i. Relational terms Bij (j is 1 or _{n i} an integer) is the j-th relation of the word Ai. Word Ai of relational terms Bi1, Bi2, ..., are collectively referred to Bin _i described as a relational terms Bi. The related word acquisition unit 13 calculates the number of occurrences of the related word Bi of the word Ai in the document data as the first importance factor V1-i and sets it in the word list (step S125). In the following, the related words B1, B2,... Are collectively referred to as the related word B, and the first importance elements V1-1, V1-2,... Are collectively referred to as the first importance element V1.

Indirect relationship word acquisition unit 14 acquires the n _i-number of relational terms Bi respective relations of the word Ai from the relationship dictionary data related word dictionary storing unit 3 stores, and indirect relations of the word Ai . K _ij indirect related words (k _ij is an integer of 1 or more) obtained as related words of the related word Bij are set as indirect related words Cij1, Cij2,..., Cijk _ij . In the following, the indirect relationship between language Cij1 of the word Ai, Cij2, ..., described as indirect relationship word Cij are collectively Cijk _ij, the indirect relationship between language Ci1 of the word Ai, Ci2, ..., are collectively referred to as Cin _i, indirect relationship It is written as the word Ci. The indirect related word acquisition unit 14 excludes from the indirect related words Ci, related words that match the word Ai among the related words of the related word Bi. The indirect related word acquisition unit 14 calculates the number of occurrences of the indirect related word Ci of the word Ai in the document data as the second importance element V2-i, and sets it in the word list (step S130). In the following, the indirect related words C1, C2,... Of the words A1, A2,... Are collectively referred to as indirect related words C, and the second importance elements V2-1, V2 of the words A1, A2,. -2, ... are collectively referred to as a second importance factor V2.

  The importance calculation unit 15 uses the first importance element V1-i and the second importance element V2-i of the word Ai set in the word list, and calculates the importance of the word Ai according to the following equation (1). V-i is calculated.

  Importance V−i = (1−u) × [First Importance Element V1-i] + u × [Second Importance Element V2-i] (0 ≦ u ≦ 0.5) (1)

  The importance calculator 15 sets the importance Vi calculated for each word Ai in the word list (step S135). Hereinafter, the importance V-i is collectively referred to as importance V.

  The selection unit 16 selects, from the word list, a word A whose importance level V satisfies a predetermined condition as a subject word (step S140). The predetermined condition may be, for example, a condition that the importance level is the highest, a condition that the importance degree is high to a predetermined number, or a condition that the importance degree is equal to or greater than a predetermined ratio of the highest importance degree. The selection unit 16 outputs the subject word selected in step S140 (step S145).

The extraction unit 12 determines whether there is unselected document data in the document data input in step S110 (step S150). If the extraction unit 12 determines that there is unselected document data (step S150: YES), the extraction unit 12 repeats the processing from step S115.
In addition, when the word of a predetermined part of speech cannot be extracted from the text data included in the document data in step S120, the extraction unit 12 performs the process of step S150 without performing the processes from step S125 to step S145.
When the extraction unit 12 determines that all the document data input in step S110 has been selected (step S150: NO), the process ends.

  FIG. 4 is a diagram illustrating an example of a word list. As shown in the figure, the word list includes a word Ai (i = 1, 2,...) That is a noun extracted from document data, a first importance element V1-i, and a second importance element V2 of the word Ai. This is data in which -i and importance V-i are associated with each other.

FIG. 5 is a flowchart showing word list generation processing of the subject word extraction device 1. The word list generation process shown in the figure is a detailed process of step S120 in FIG.
The extraction unit 12 selects one sentence from the text data included in the selected document data (step S210). The extraction unit 12 performs morphological analysis on the selected sentence (step S215). Existing techniques can be used for morphological analysis. For example, “Mecab”, [online], Internet <URL: https://code.google.com/p/mecab/> ”can be used as an existing morphological analysis technique.

  Based on the result of the morphological analysis, the extraction unit 12 extracts a noun word from the selected sentence (step S220). The extraction unit 12 adds a word that is not registered in the word list as the word A among the extracted words (step S225). The extracting unit 12 may further set the appearance frequency indicating the total number of times that the word A appears in the document data in the word list. When the appearance frequency is set, the total number of times that the related word B of the word A appears in the document data is used as the first importance element V1, or the indirect related word C of the word A is used as the second importance element V2. Is the total number of times that appears in the document data. In this case, when the word extracted in step S220 is not registered in the word list, the extraction unit 12 adds the extracted word and the appearance frequency indicating the number of extracted words from the selected sentence to the word list. Set. When the extracted word is already registered in the word list, the extracting unit 12 sets the appearance frequency of the word set in the word list to a value obtained by adding the number of extracted words from the selected sentence. Update.

When the extraction unit 12 determines that the sentence selected in step S210 is not the last sentence of the text data included in the document data (step S230: NO), the extraction unit 12 repeats the process from step S210.
If the extraction unit 12 determines that the sentence in step S210 is the last sentence of the text data included in the document data (step S230: YES), the extraction unit 12 ends the process.

FIG. 6 is a flowchart showing the first importance factor calculation process of the subject word extraction device 1. The first importance factor calculation process shown in the figure is a detailed process of step S125 of FIG.
The related word acquisition unit 13 sets an initial value 1 to the variable i (step S310). The related word acquisition unit 13 selects the i-th word Ai set in the word list (step S315). The relationship word acquisition unit 13, relational terms Bi1 word Ai, Bi2, ..., the Bin _i is related word dictionary storing unit 3 acquires from the relation dictionary data stored (step S320). The relationship word acquisition unit 13, the acquired relational terms Bi1, Bi2, ..., to count the number of relational terms Bi appear as word A in the word list of the Bin _i, the first importance elements V1-i (step S325).
Incidentally, the relationship word acquisition unit 13, relational terms Bi1, Bi2, ..., each Bin _i is the sum of the total number of occurrences in the document data may be first importance elements V1-i. When totaling the total number of times to obtain the first importance factor V1-i, the extraction unit 12 records the appearance frequency of the word A in the word list in step S225 of FIG. Then, the relationship between language acquisition unit 13, relational terms Bi1, Bi2, ..., Bin of the _i, about the relationship between language Bi that appear as a word A in the word list to get its frequency of occurrence from the word list, the word A in the word list As for the related word Bi that does not appear as, the appearance frequency is set to “0”. The relationship word acquisition unit 13, relational terms Bi1, Bi2, ..., and the total frequency of occurrence of Bin _i, respectively, and the first importance elements V1-i of the word Ai.
When the related word Bi cannot be acquired in step S320, the related word acquiring unit 13 sets the first importance factor element V1-i to “0”.
The related word acquisition unit 13 associates the first importance element V1-i obtained in step S325 with the word Ai and sets it in the word list (step S330).

When the related word acquisition unit 13 determines that all the words A set in the word list have not been selected (step S335: NO), 1 is added to the variable i and the processing from step S315 is repeated (step S315). S340).
If the related word acquisition unit 13 determines that all the words A set in the word list have been selected (step S335: YES), the related word acquisition unit 13 ends the process.

FIG. 7 is a flowchart showing the second importance level element calculation process of the subject word extraction device 1. The second importance factor calculation process shown in the figure is a detailed process of step S130 of FIG.
The indirect related word acquisition unit 14 sets an initial value 1 to the variable i (step S410). The indirectly related word acquisition unit 14 selects the i-th word Ai set in the word list (step S415). Indirect relationship word acquisition unit 14, each of relational terms Bi1, Bi2 word Ai, ..., the Bin _i is related word dictionary storing unit 3 acquires from the relation dictionary data stored (step S420). Overhead relationship word acquisition unit 14, relational terms Bi1 relationship word acquisition unit 13 has acquired in the step S320 of FIG. 6, Bi2, ..., it may obtain the Bin _i.

The indirect related word acquisition unit 14 sets an initial value 1 to the variable j (step S425). The indirect related word acquisition unit 14 acquires the related words of the related word Bij from the related word dictionary data stored in the related word dictionary storage device 3, and sets the indirect related words Cij1, Cij2, ..., Cijk _ij of the word Ai. (Step S430).

The indirect related word acquisition unit 14 calculates the number of indirect related words Cij that appear as the word A in the word list among the indirect related words Cij1, Cij2,..., Cijk _ij , and the second importance degree subelement V2- related to the related word Bij. i−j (step S435).
Note that the indirect related word acquisition unit 14 may add up the total number of times each of the indirect related words Cij1, Cij2,..., Cijk _ij appears in the document data to obtain the second importance subelement V2-ij. When totaling the total number of times to obtain the second importance subelement V2-ij, the extraction unit 12 records the appearance frequency of the word A in the word list in step S225 of FIG. Then, the indirect related word acquisition unit 14 acquires the appearance frequency of the indirect related words Cij that appear as the word A in the word list from the word list among the indirect related words Cij1, Cij2, ..., Cijk _ij. For the indirectly related word Cij that does not appear as the word A, the appearance frequency is set to “0”. The indirect related word acquisition unit 14 sums up the appearance frequencies of the indirect related words Cij1, Cij2,..., Cijk _{ij to} obtain the second importance subelement V2- _{ij of} the word Ai.
When the indirect related word Cij cannot be acquired in step S430, the indirect related word acquiring unit 14 sets the second importance subelement V2-ij of the word Ai to “0”.

Indirect relationship word acquisition unit 14, the variable j is less than _{n i,} relational terms Bi1 acquired in step S420, Bi2, ..., if it is determined that no already selected all the Bin _i (step S440: NO) , 1 is added to the variable j, and the processing from step S430 is repeated (step S445).
The indirect relationship word acquisition unit 14, the variable j reaches _{n i,} relational terms Bi1 acquired in step S420, Bi2, ..., when it is determined that selects all Bin _i (step S440: NO), step The process of S450 is executed. In other words, the indirect relation word acquisition unit 14 sums the second importance subelements V2-i-1~V2-i- n i obtained in step S435 for each relational terms Bi1～Bin _i word Ai, word A second importance factor V2-i of Ai is calculated (step S450). The indirect related word acquisition unit 14 sets the second importance element V2-i obtained in step S450 in the word list in association with the word Ai (step S455).

If it is determined that the word A set in the word list has not been selected (step S460: NO), the indirect related word acquisition unit 14 adds 1 to the variable i and repeats the processing from step S415 ( Step S465).
If the related word Bi cannot be acquired in step S420, the indirect related word acquisition unit 14 sets the second importance element V2-i to “0” and performs the processing from step S455.
If the indirect related word acquisition unit 14 determines that all the words A set in the word list have been selected (step S460: YES), the process ends.

  After the process of FIG. 7, the process from step S135 of FIG. 3 is executed. That is, the importance calculation unit 15 calculates the importance V of each word A set in the word list using the first importance element V1 and the second importance element V2, and sets the importance V in the word list ( Step S135). The selection unit 16 selects a subject word based on the importance V from the words A set in the word list (step S140) and outputs the subject word (step S145).

Next, the subject word extraction process of the subject word extraction device 1 will be described using a specific example.
FIG. 8 is a diagram illustrating an example of document data. The extraction unit 12 of the subject word extraction device 1 selects the document data D shown in FIG. 8 from the document data received by the input unit 11 in step S110. The extraction unit 12 extracts a noun word from a sentence included in the document data D.

  FIG. 9 is a diagram illustrating words extracted from the document data D by the extraction unit 12. In the figure, the frequency at which each extracted word appears in the document data D is also shown. The extraction unit 12 generates a word list in which the noun words “stiff shoulder”, “elimination”, “method”, “body”,... Shown in FIG.

  FIG. 10 is a diagram illustrating an example of related word dictionary data stored in the related word dictionary storage device 3. The related word acquisition unit 13 acquires the related word B of each word A with reference to the related word dictionary data shown in FIG. 10, and the number of related words B of the word A appearing in the word list (or related words of the word A) (Total number of times B appears in the document data) is set in the word list as the first importance factor V1. Further, the indirect related word acquisition unit 14 refers to the related word dictionary data to acquire the indirect related word C of each word A, and the number of indirect related words C appearing in the word list (or the indirect related word C of the word A). Is set in the word list as the second importance factor V2.

FIG. 11 is a diagram in which the related word B and the indirect related word C of the word A are added to the word list.
As shown in the figure, for the word A “stiff shoulder”, the related words B “gymnastics”, “heat”,... Are obtained from the related word dictionary data shown in FIG. (O) described after the related word B indicates that it appears in the word list, and (X) indicates that it does not appear in the word list. Of the related words B “Gymnastic”, “Heat”,... Of the word A “Stiff shoulder”, only “Gymnastic” appears in the word list, so the first importance factor V1 is “1”.

For the related word B “gymnastics” of the word A “stiff shoulder”, the related words “stiff shoulder”,... Are obtained from the related word dictionary data. For the related word B “warmth” of the word A, the related words “stiff shoulder”, “hot spring”, “ginger”,... Are obtained from the related word dictionary data. Among the related words obtained for the related word B, the related word that matches the word A “stiff shoulder” is excluded from the indirect related word C. Therefore, “hot spring”, “ginger”,... Are obtained as the indirect relation word C of the word A “stiff shoulder”. (◯) described after the indirect relation word C indicates that it appears in the word list, and (×) indicates that it does not appear in the word list. Among the indirect relation words C “hot spring”, “ginger”,... Of the word A “stiff shoulder”, two words “hot spring” and “ginger” appear in the word list, so the second importance factor V2 is “2”.
The importance V of the word A “stiff shoulder” is calculated as 1.5 (= 0.5 × 1 + 0.5 × 2) based on the formula (1).

  Similarly, for the word A “hot spring”, the related words B “relax”, “heat”,... Are obtained, and only “relax” appears in the word list, so the first importance factor V1 is “1”. It becomes. For the word A “hot spring”, the indirect relation word C “ginger”,... Is obtained, and only “ginger” appears in the word list, so the second importance factor V2 is “1”. The importance level V of the word A “hot spring” is calculated as 1.0 (= 0.5 × 1 + 0.5 × 1) based on the formula (1).

  Moreover, since only “hot spring” appears in the word list among the related words B of the word A “relax”, the first importance factor V1 is “1”. Further, since none of the indirectly related words C of the word A “relax” appears in the word list, the second importance factor V2 is “0”. The importance level V of the word A “relax” is calculated as 0.5 (= 0.5 × 1 + 0.5 × 0) based on the formula (1).

  In addition, since only “cooking” appears in the word list among the related words B of the word A “ginger”, the first importance factor V1 is “1”. Further, since only “hot spring” of the indirect relation word C of the word A “ginger” appears in the word list, the second importance factor V2 is “1”. The importance V of the word A “ginger” is calculated as 1.0 (= 0.5 × 1 + 0.5 × 1) based on the equation (1).

When the total number of times that the related word B and the indirect related word C appear in the document data is the first importance element V1 and the second importance element V2, respectively, the appearance frequency of each word A shown in FIG. Set to.
In the case of the word A “Stiff shoulder”, the appearance frequency of “Gymnastic” among the related words B is “1” and the appearance frequency of the other related words B is “0”. “1”. In addition, among the indirect related words C of the word A “stiff shoulder”, the appearance frequency of “hot spring” is “2”, the appearance frequency of “ginger” is “2”, and the appearance frequency of other indirect related words C is “0”. Therefore, the second importance factor V2 is “4”. Therefore, the importance V of the word A “stiff shoulder” is calculated as 2.5 (= 0.5 × 1 + 0.5 × 4) based on the equation (1).
Similarly, in the case of the word A “hot spring”, the appearance frequency of “relaxed” among the related words B is “1” and the appearance frequency of the other related words B is “0”. Becomes “1”. Moreover, since the appearance frequency of “ginger” is “2” and the appearance frequency of other indirect relation words C is “0” among the indirect relation words C of the word A “hot spring”, the second importance factor V2 Becomes “2”. Therefore, the importance level V of the word A “hot spring” is calculated as 1.5 (= 0.5 × 1 + 0.5 × 2) based on the formula (1).

As a result of calculating the importance of each word A as described above, the importance V of the word A “stiff shoulder” is the highest. The selection unit 16 selects the word A “stiff shoulder” having the highest importance V as the subject word.
Note that the value of the coefficient u in the equation (1) is the value of the coefficient u when the subject word is extracted while changing the coefficient u for the document data for learning, and the probability of matching with the manually extracted subject word is high. Determine based on.

Although the subject word of the document data D is “stiff shoulder”, when tf-idf is calculated under the condition that the document data group consists only of the document data D, the word “valid” having the highest appearance frequency as shown in FIG. Is selected as the subject word.
In addition, there is a conventional technique that refers to related word dictionary data and assigns a weight when a direct relationship is obtained between words in a document, but related word dictionary data that includes all directly related word pairs is actually It ’s hard to exist. If the relation word dictionary data contains only the relationship between two words, such as “ginger [food] cooking”, “hot spring [effective] relaxation”, and “shoulder stiffness [measure] gymnastics”, the weight of “stiff shoulder” is It is possible that it will not be so large.
The subject word extraction device 1 of the present embodiment adds the importance of words extracted from document data to the number of directly related words appearing in the document data, and is indirectly related with other words in between. Importance is calculated using the number of words that appear in the document data. By using the importance calculated in this way, the subject word extraction apparatus 1 can extract a word suitable as a subject word from words appearing in the document data.

  Note that the selection unit 16 may extract the subject from the word A by combining frequency information such as tf-idf. For example, the selection unit 16 extracts, from the words A having a tf-idf value higher than a predetermined value, a word A whose importance V satisfies a predetermined condition as a subject word. The word A having a tf-idf value higher than a predetermined value can be a word A having a tf-idf value equal to or higher than a predetermined value, a word A having a high tf-idf value to a predetermined number of words A, or the like. Alternatively, the selection unit 16 may extract a predetermined number of words A as subject words from words A having the highest tf-idf value among the words A whose importance level V satisfies the predetermined condition.

In addition, when the related word acquisition unit 13 sets the first importance element V1 of the word A as the number of occurrences of the related word B in the word list, the related word B belonging to the predetermined field has an appearance count larger than 1. You may multiply by a value coefficient.
For example, the relationship of the word Ai Bi1, Bi2, ..., is the Bin _i, in the word list, the emergence of relational terms Bi1, Bi2, Bi3, the relationship between language Bi4~Bin _i and does not appear. When the related words Bi1 and Bi2 are words belonging to a predetermined field, the related words Bi1 and Bi2 are counted as 1 × a (a> 1) times. Accordingly, the first importance factor V1-i of the word Ai is 2 × a + 1.
Similarly, when the indirectly related word acquisition unit 14 sets the second importance element V2 of the word A as the number of occurrences of the indirectly related word C in the word list, the indirect related word C belonging to the predetermined field is set to the number of appearances. You may multiply by a coefficient larger than one.

In addition, when the related word acquisition unit 13 sets the first importance element V1 of the word A as the sum of the appearance frequencies of the related words B in the document data, the related word B belonging to the predetermined field has an appearance frequency of 1 You may multiply by the coefficient of a bigger value. For example, the relationship of the word Ai Bi1, Bi2, ..., is the Bin _i, the document data, relational terms Bi1 twice, appeared relational terms Bi2 three times, the relationship between language Bi3~Bin _i do not appear . When the related word Bi1 is a word belonging to a predetermined field, the appearance frequency “2” of the related word Bi1 is multiplied by a coefficient a (a> 1). Therefore, the first importance level element V1-i of the word Ai is the appearance frequency “2” of the related word Bi1 × the coefficient a + the appearance frequency “3” of the related word Bi2.
Similarly, when the related word acquisition unit 13 sets the second importance element V2 of the word A as the sum of the appearance frequencies of the indirect related words C in the document data, the related word C belonging to a predetermined field appears. The frequency may be multiplied by a coefficient greater than 1.

  The indirect related word acquisition unit 14 reads a related word related to the indirect related word C of the word A from the related word dictionary data and sets the indirect related word of the word A m times (m is an integer of 1 or more). ) You may repeat. A related word related to the indirect related word C is an indirect related word E1, a related word related to the indirect related word E1 is an indirect related word E2,. The importance calculation unit 15 includes the number V1 of the related word B of the word A that appears in the document data, the number V2 of the indirect related word C of the word A that appears in the document data, and the indirect related word E1 of the word A that appears in the document data. The number V3 and the number V4 of the indirectly related word E2 of the word A appearing in the document data are multiplied by a predetermined weighting coefficient and then added to calculate the importance of the word A. Note that the weighting coefficient of V1> the weighting coefficient of V2> the weighting coefficient of V3> the weighting coefficient of V4, and so on.

  According to the above-described embodiment, the subject word extraction device 1 can extract words taking into account topicality from document data by using the relationship between words appearing in the document data. For the relationship between the words, not only the directly related words but also indirectly related words are used. Accordingly, even if words appearing in the document data are not registered as words directly related to the related word dictionary data, a word having a high degree of importance is given to a word having a lot of relations with other words in the document data, and the subject word The word suitable for can be extracted. Thus, the subject word extracted from the document data by the subject word extracting apparatus 1 can be used for extracting information from the document, summarizing the document, or recommending content related to the document data.

  The subject word extraction device 1 described above has a computer system therein. The process of the operation of the subject word extraction device 1 is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by the computer system reading and executing this program. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a portable dictionary such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a conceptual dictionary storage unit such as a hard disk built in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

DESCRIPTION OF SYMBOLS 1 Subject word extraction apparatus 11 Input part 12 Extraction part 13 Relation word acquisition part 14 Indirect relation word acquisition part 15 Importance calculation part 16 Selection part 3 Relation word dictionary memory | storage device

Claims (5)

An extraction unit for extracting words from document data;
A related word acquisition unit that acquires, as related words, other words related to the word extracted by the extraction unit from related word dictionary data in which a plurality of related words are described in association with each other;
An indirect related word acquisition unit that acquires other words related to the related word as indirect related words from the related word dictionary data;
For each of the words extracted by the extraction unit, based on the number of the related words of the word that appear in the document data and the number of the indirect related words of the word that appear in the document data. An importance calculation unit for calculating importance that quantitatively represents importance,
A selection unit that selects a word representing a subject based on the importance calculated by the importance calculation unit from the words extracted by the extraction unit;
A subject word extraction apparatus comprising: The extraction unit extracts a word of a predetermined part of speech included in the document data;
The subject word extraction device according to claim 1. The importance calculator calculates the importance of the word extracted by the extraction unit, the number of the related words of the word appearing in the document data, and the indirect related word of the word appearing in the document data. Calculate by adding the weighted result to each number.
The subject word extraction device according to claim 1, wherein the subject word extraction device is a feature word extraction device. The indirect related word acquisition unit repeats a process of acquiring other words related to the indirect related word as indirect related words from the related word dictionary data a predetermined number of times.
The subject word extraction device according to any one of claims 1 to 3, wherein: Computer
Extraction means for extracting words from document data;
A related word acquisition unit that acquires, as related words, other words related to the word extracted by the extraction unit, from related word dictionary data in which a plurality of related words are described in association with each other;
Indirect related word acquisition means for acquiring other words related to the related word as indirect related words from the related word dictionary data;
For each of the words extracted by the extraction means, based on the number of occurrences of the related word of the word in the document data and the number of occurrences of the indirect related word of the word in the document data. Importance calculation means for calculating importance that quantitatively represents importance,
A selecting unit that selects a word representing a subject based on the importance calculated by the importance calculating unit from the words extracted by the extracting unit;
A program for causing a function to function as a subject word extraction device.

Images