JP6433045B2 - Keyword extraction apparatus and program - Google Patents

Description

  The present invention relates to a keyword extraction device and a program thereof using a natural language processing technique using a computer or the like.

  Use keywords such as subtitles of broadcast programs for meaningful keywords to search for broadcast programs, search for documents related to broadcast programs, and recommend broadcast programs. The extraction process is often performed. In order to extract keywords from text, a basic technique that is often performed is a process in which sentences included in the text are morphologically analyzed to classify parts of speech of morphemes and to extract nouns or noun phrases. Then, after extracting the noun or noun phrase, matching with the corpus data or applying a rule defined implicitly or explicitly with respect to the target document, the part of the desired noun etc. Is extracted as a keyword.

  For example, Patent Document 1 discloses a technique for navigating playback content so that playback information can be easily understood by using keyword information extracted from subtitle information during playback of a video playback device. In the keyword extraction process here, morphological analysis is performed on the caption sentence to extract a noun part. After this, the process of extracting the noun part surrounded by symbols such as parentheses and quotations as an important word, and the process of extracting whether the meaning of the corresponding noun is registered in the genre database created in advance Including.

  Further, for example, Patent Document 2 discloses a system that provides detailed information related to program contents from keywords related to program contents via the Internet, and here, keyword extraction processing is used. . In this case as well, the keyword extraction process is realized by the matching process between the noun part extracted by the morpheme analysis process and the very dictionary after performing the morpheme analysis as a basic process.

JP 2012-034235 A JP 2013-168091 A

  As described above, the keyword extraction process according to the conventional technique is based on the morphological analysis process. Therefore, the problem of the morphological analysis process itself remains as a problem of the keyword extraction process. The problem of morphological analysis processing is that, for example, there is a problem in dealing with compound words, foreign words, specific expressions, colloquial sentences, and the like.

  The present invention has been made to solve the above-described problems, and provides a keyword extraction device and a program for extracting a keyword from text including a sentence without assuming morphological analysis processing. To do.

  [1] In order to solve the above problems, a keyword extraction device according to an aspect of the present invention includes a corpus data storage unit that stores a set of keywords, a text acquisition unit that acquires text data, and the text acquisition unit. Is applied to only text data within a predetermined range that is currently acquired by a local corpus data storage unit that stores a set of keywords, and a keyword based on the first text data acquired by the text acquisition unit A local corpus data generation unit that registers the extracted keyword in the local corpus data storage unit, the keyword read from each of the corpus data storage unit and the local corpus data storage unit, and the text acquisition Matching the second text data acquired by the department Therefore, it is characterized in that it comprises a and a keyword extraction section that extracts the keyword included in the second text data.

  [2] Further, according to one aspect of the present invention, in the keyword extraction device, the local corpus data storage unit stores a set of keywords that differ for each of the predetermined ranges, and the keyword extraction unit includes: When the second text data is changed beyond the range to which the second text data belongs, the second text data is switched to the keyword set corresponding to the changed predetermined range, and the keyword is read from the local corpus data storage unit. It is used for matching with data.

  [3] Further, according to one aspect of the present invention, in the keyword extracting device, the text acquisition unit acquires text data based on a television broadcast signal, and the first text data is a television program. The second text data is subtitle text data in the program, and the predetermined range is a range of the subtitle text in the program.

[4] Further, according to one aspect of the present invention, in the keyword extraction device, each of the corpus data storage unit and the local corpus data storage unit stores the keyword in association with a word length of each keyword. The keyword extracting unit gives priority to the longest word length and reads the keyword read from each of the corpus data storage unit and the local corpus data storage unit, and the second text acquired by the text acquisition unit. It is characterized by matching data.
With this configuration, the keyword extraction unit performs processing for preferentially matching long keywords. Since long keywords have good accuracy in the result of the matching process, the accuracy of keyword extraction as a whole device increases.

[5] In addition, according to one aspect of the present invention, in the keyword extraction device, the keyword is a noun or a noun phrase, and the keyword extraction unit first gives priority to a long word length, A keyword included in the second text data is extracted by matching the keyword up to the lower limit word length with the second text data, and then the extracted keyword is extracted from the second text data. Alternatively, the morphological analysis process is performed under the premise that the phrase is a noun phrase, and the noun or noun phrase obtained as a result of the morphological analysis process is further obtained from each of the corpus data storage unit and the local corpus data storage unit. It is included in the result of the morphological analysis process by matching with the read keyword. Extracting a keyword, and wherein the.
With this configuration, keywords having a predetermined length or more are matched before morphological analysis processing. This further improves the accuracy of long keyword extraction. This leads to avoiding the problem that it is difficult to correctly output a noun part having a long word length as a characteristic of the morphological analysis process itself.

[6] Further, according to one aspect of the present invention, in the keyword extraction device, the keyword extraction unit passes the extracted keyword information to the local corpus data generation unit, and the local corpus data generation unit The keyword extracted from the second text data by the extraction unit is further registered in the local corpus data storage unit.
With this configuration, local corpus data can be generated using not only the first text data but also the second text data, that is, using the keywords included in the second text data.

[7] Further, according to one aspect of the present invention, in the keyword extraction device, the corpus data storage unit and the local corpus data storage unit may each have keywords in a relationship that can be paraphrased among the plurality of keywords to be stored. The keyword extraction unit refers to the association information held by the corpus data storage unit and the local corpus data storage unit, thereby extracting the plurality of keywords extracted from the second text data. Is in a paraphrasable relationship, it is output after performing a name identification process that integrates a plurality of the keywords in the paraphrased relationship.
With this configuration, it is possible to output a name identification result corresponding to paraphrasing or fluctuation of expression in text data.

[8] Further, according to one aspect of the present invention, in the keyword extraction device, the keyword extraction unit reads the keyword read from each of the corpus data storage unit and the local corpus data storage unit, and the text acquisition unit. With respect to the keyword detected by matching with the second text data acquired by the above, it is confirmed whether or not it is a detection error by applying a rule according to the type of characters constituting the keyword. Only when it is confirmed that there is no keyword, the keyword is extracted.
With this configuration, the accuracy of keyword detection can be increased. That is, erroneous detection of keywords can be prevented. In addition, knowledge about the keyword can be applied for each type of keyword (for example, classification by character type such as kanji, hiragana, katakana, alphanumeric characters, special characters, etc.).

  [9] Further, according to one aspect of the present invention, the computer includes a corpus data storage unit that stores a set of keywords, a text acquisition unit that acquires text data, and a predetermined range currently acquired by the text acquisition unit. In order to apply only to the text data, a local corpus data storage unit that stores a set of keywords, a keyword is extracted based on the first text data acquired by the text acquisition unit, the extracted keyword, The local corpus data generation unit registered in the local corpus data storage unit, the keyword read from each of the corpus data storage unit and the local corpus data storage unit, and the second text data acquired by the text acquisition unit By matching, the second text data Keyword extracting unit for extracting the keyword included is a program for functioning as a.

According to the present invention, highly accurate keyword extraction can be performed in a form in which local corpus data is locally used only within a predetermined range.
Further, according to the present invention, it is possible to accurately detect a long keyword that is difficult to be correctly processed in the morphological analysis process.
In addition, according to the present invention, keywords can be accurately extracted from text data such as subtitles of television, and such keywords can be used in a service linked with television broadcasting.

It is a block diagram which shows the function structure of the keyword extraction apparatus by embodiment of this invention. It is the schematic which shows the basic composition of the corpus database (large-scale corpus database and local corpus database) by the embodiment. It is a flowchart which shows the procedure of the process which the keyword extracting device by the embodiment extracts a keyword from a subtitle text. It is the schematic which shows the procedure which builds the large-scale corpus database by the same embodiment. It is the flowchart which showed the construction procedure of the local corpus database by the local corpus database production | generation part 13 in the embodiment. It is a part (1/2) of the flowchart which shows the procedure of the process of the stepwise keyword matching by the same embodiment. It is a part (2/2) of the flowchart which shows the procedure of the process of the stepwise keyword matching by the same embodiment.

Next, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a functional configuration of the keyword extracting apparatus according to the present embodiment. In this figure, reference numeral 1 denotes a keyword extracting device. As shown in the figure, the keyword extraction device 1 includes a corpus database construction processing unit 11, a large-scale corpus database storage unit 12 (corpus data storage unit), a local corpus database generation unit 13 (local corpus data generation unit), A corpus database storage unit 14 (local corpus data storage unit), a caption / program description sentence extraction unit 16 (text acquisition unit), a keyword extraction unit 17, and a category extraction unit 18 are configured.

Note that the keyword extracting device 1 is provided, for example, inside a television receiver or connected to the television receiver by a cable or the like. Thereby, the keyword extracting device 1 extracts keywords using the digital broadcast signal received by the television receiver.
The function of the keyword extraction device 1 is roughly divided into an offline processing unit that performs offline and an online processing unit that performs real-time processing. A typical offline process is a process for constructing a large-scale corpus database by the corpus database construction processing unit 11. The center of online processing is processing (keyword extraction unit 17) that extracts keywords from subtitle text in real time based on the digital broadcast signal being received.

The corpus database construction processing unit 11 constructs a large-scale corpus database, for example, based on dictionary resource data that can be acquired from the outside, and writes the large-scale corpus database storage unit 12.
The large-scale corpus database storage unit 12 is a storage unit that stores a set of keywords for use in matching processing with text.

The local corpus database generation unit 13 generates a local corpus in the program and writes it in the local corpus database storage unit 14. The local corpus database generation unit 13 generates this local corpus database based on two types of data. The first is information extracted by analyzing the program description obtained from the caption / program description extractor 16. The second is keyword information extracted by the keyword extraction unit 17 from the subtitles of the program. The text of the program description is data representing the characteristics of the program.
In another aspect, the local corpus database generation unit 13 extracts keywords based on the first text data (program description) acquired by the caption / program description extraction unit 16, and uses the extracted keywords as a local corpus database. Register in the storage unit 14.

The local corpus database storage unit 14 stores local corpus data in the target program. For example, paraphrases or abbreviations of nouns (typically names of performers of programs) are included in the local corpus database. The paraphrasing of the name is, for example, a nickname or a nickname obtained by adding the suffix “chan” to only one of the surname / first name. The local corpus data stored in the local corpus database storage unit 14 can be used for name identification purposes such as shaking of expressions that are likely to appear locally in a program.
Note that the local corpus database exists in a form distinguished for each program. As will be described later, when the user switches the program while viewing the television program, the local corpus database to be used is also switched.
The local corpus database storage unit 14 stores a set of keywords in order to target only text data within a predetermined range currently being processed (acquired by the caption / program explanation extracting unit 16). The predetermined range is, for example, the range of one program in television broadcasting.

  The caption / program description extracting unit 16 obtains a digital television broadcast signal from the outside, and extracts text data from the signal. The text data extracted here is an explanatory text of a broadcast program or caption text included in a predetermined format in a digital broadcast signal. In addition, since the structure of the signal of digital television broadcasting is known, description here is abbreviate | omitted. The subtitle / program description sentence extraction unit 16 passes the extracted program description text (first text data) to the local corpus database generation unit 13. Also, the caption / program explanation extracting unit 16 passes the extracted caption text (second text data) to the keyword extracting unit 17.

The keyword extraction unit 17 performs matching processing on the large-scale corpus data read from the large-scale corpus database storage unit 12 and the local corpus data read from the local corpus database storage unit 14, so that the subtitle / program description sentence extraction unit 16 performs the matching process. Extract keywords included in the received subtitle text.
Further, the keyword extraction unit 17 gives priority to the keywords read from the large-scale corpus data read from the large-scale corpus database storage unit 12 and the local corpus data read from the local corpus database storage unit 14 in preference to those having a long word length. The subtitle text received from the subtitle / program explanation extractor 16 is matched.
In addition, the keyword extraction unit 17 refers to the association information (paraphrasable relationship) between keywords held in the large-scale corpus database and the local corpus database so that a plurality of keywords extracted from the caption text data can be rephrased. In some cases, a name identification process that integrates a plurality of keywords that can be paraphrased is performed and then output.

  The category extraction unit 18 adds the category information to the keyword extracted by the keyword extraction unit 17 and outputs it. The category extraction unit 18 acquires category information corresponding to the keyword from the corpus database.

FIG. 2 is a schematic diagram showing the basic structure of the corpus database. The data structure shown in the figure is common to both the large-scale corpus database (large-scale corpus database storage unit 12) and the local corpus database (local corpus database storage unit 14). As shown in the figure, the corpus database is expressed as tabular data in which word lengths and word information are associated with each other. The word length is the length (number of characters) of a word. In the present embodiment, the database is constructed in a state where the word lengths are arranged in descending (long) order. As a specific implementation method for arranging the entries in the order of word length, for example, data may be physically arranged in the order of the word length on the data recording medium. Data may be arranged in the order of logically long word length using index (index) data or the like.
That is, the corpus database stores each keyword in association with the word length of the keyword.

  In the figure, data is arranged in the order of word length, and a total of M words w (1), w (2),..., W (M) are registered. In the illustrated example, the word length is K (K is a positive integer) up to the word w (i + 1). From the word w (i + 2), the word length is (K-1). As will be described later, K is a preset value, and usage is slightly different between a corpus element having a word length of K or more and a corpus element having a word length of less than K.

For each word (keyword), the corpus database has, in addition to the word itself, information on the category to which the word belongs and information on a set of words for paraphrasing for use in name identification processing.
In the corpus database, keywords are stored together for each type. Here, the types of keywords are types such as kanji, hiragana, katakana, alphanumeric characters, and other special characters. This type of usage will be described later.

In addition, if the source of an entry in the local corpus database is an entry in a large corpus database, link information (information indicating the location) from the entry on the local corpus database to the entry on the large corpus database is also provided. It is held as an accessory.
In addition, the corpus database also holds link information to the keyword before paraphrase for the keyword based on paraphrase.

FIG. 3 is a flowchart showing a procedure of processing in which the keyword extracting device 1 extracts keywords from subtitle text. The keyword extraction processing procedure will be described below with reference to this flowchart.
Note that the large-scale corpus database and the local corpus database have already been constructed before entering the processing of this flowchart. Each construction method and the like will be described later with reference to another drawing. In particular, a large corpus database is first constructed as an off-line process.

First, in step S <b> 1, the caption / program explanation extracting unit 16 extracts caption text data from the acquired digital broadcast signal, and passes the extracted data to the keyword extracting unit 17. The keyword extraction unit 17 temporarily stores the subtitle text in the buffer storage unit.
Next, in step S2, the keyword extracting unit 17 converts the subtitle text passed from the subtitle / program description extracting unit 16 into a large-scale corpus database (large-scale corpus database storage unit 12) and a local corpus database (local corpus database storage). Stepwise keyword extraction is performed by matching with the unit 14). Details of this keyword extraction processing will be described later with reference to another drawing.

  In step S3, the keyword extraction unit 17 removes the keyword matched in step S2 from the buffer storage unit. The keyword extraction unit 17 passes the keyword to the local corpus database generation unit 13. Then, when the keyword is not registered, the local corpus database generation unit 13 additionally registers in the local corpus database (local corpus database storage unit 14).

In addition, supplementally, the meaning of “stepwise” described in step S2 is as follows. That is, a keyword is matched in step S2, and in step S3, the keyword is removed from the caption text data on the buffer storage means. Thereafter, when returning to step S2 again by a processing loop (control to return from step S5 to S2 described later), keyword matching is performed on the subtitle text data after removal. In this way, while matching the keywords one by one, the keyword is registered in the local corpus database, deleted from the caption text data on the buffer storage means, and matching for the caption text data after the deletion is performed as follows. Do it in stages.
In other words, the keyword extraction device 1 continues to detect keywords for subtitle sentences excluding the keywords detected from the subtitles. Then, the same processing is repeated until there is no subtitle element to be detected, or there are no elements in the large corpus database and the local corpus database.

  After step S3, in step S4, the category extraction unit 18 extracts information on the category to which the keyword extracted by the keyword extraction unit 17 belongs. Then, the category extraction unit 18 outputs a pair of the keyword and the category information to the outside.

  Next, in step S5, the keyword extraction device 1 determines whether or not the final stage of keyword detection has been reached in the current caption text. Here, the final stage is a stage where there are no more elements to be subjected to keyword matching. Specifically, the final stage is a state in which there are no more elements in the subtitle text in the buffer storage means, or there are no elements in the large corpus database and the local corpus database. If the final stage has already been reached (step S5: YES), the process proceeds to the next step S6, and if not yet the final stage (step S5: NO), the process returns to step S2 for further matching.

  Next, in step S6, the keyword extracting device 1 determines whether or not the processed subtitle text is the last subtitle text. In other words, the keyword extraction device 1 determines whether or not it remains. If it is the last caption text (step S6: YES), the keyword extracting device 1 ends the process of the entire flowchart. If subtitle text to be processed still remains (step S6: NO), the process returns to step S1 to process the next subtitle taste.

  FIG. 4 is a schematic diagram showing a procedure for constructing a large-scale corpus database. This process is performed offline in advance by the corpus database construction processing unit 11. The large-scale corpus database has a structure that can be easily used in the keyword extraction processing according to the present embodiment. The corpus database construction processing unit 11 shapes the dictionary resource data acquired from the outside into such a structure. The information that is the basis of the large-scale corpus database is an open resource dictionary database such as Wikipedia. Of course, instead of using such an existing database, a large-scale corpus database may be constructed by collecting corpus elements. Note that the number of words held in the large-scale corpus database assumed in the present embodiment is about one million to several hundred thousand words, but actually, the number of words is larger or smaller than that range. A corpus database may be constructed.

As shown in the figure, the corpus database construction processing unit 11 acquires dictionary resource data, and first performs a process of classifying each keyword included in the dictionary resource by type (process 81). Here, the type is a classification according to the type of characters constituting the keyword, such as kanji, hiragana, katakana, alphanumeric characters, special characters. This can be determined by the character code. Then, the corpus database construction processing unit 11 performs a sorting process based on the keyword length (number of characters) for each classified type (process of reference numeral 82). This is because the words in the corpus database are used in the order of word length as described in FIG. The sorting process is performed so that descending results are obtained using the word length as a key. Then, for each keyword after sorting, a process of adding word attachment information is performed. The process of adding the attached information includes a process of generating and adding word length index information (reference 83), a process of adding alias (alias, paraphrase) information of the word (reference 84), and word category information. Is a process (reference numeral 85).
The alias information in the above process can be used for the name identification process.

  When selecting the Wikipedia as a dictionary resource, the entire dictionary data can be downloaded and used. The category information and alias information for each keyword can be obtained directly from the downloaded file, or can be obtained by analyzing the dictionary text (explanatory text).

  FIG. 5 is a flowchart showing a procedure for constructing a local corpus database by the local corpus database generation unit 13. Hereinafter, the processing will be described along this flowchart.

First, in step S <b> 31, the local corpus database generation unit 13 acquires a program description from the caption / program description extraction unit 16. This program description is typically extracted from the digital broadcast signal by the caption / program description extraction unit 16 at the start of the program. In this step, the local corpus database generation unit 13 selects a local corpus database corresponding to the current program.
Next, in step S32, the local corpus database generation unit 13 extracts main keywords from the received program description text.

  Next, in step S <b> 33, the local corpus database generation unit 13 reads alias information that can be a paraphrase of the extracted keyword from the large-scale corpus database (large-scale corpus database storage unit 12). Then, the local corpus database generation unit 13 additionally registers a set of paraphrase keywords related to the keyword in the local corpus database (local corpus database storage unit 14).

Next, in step S34, the local corpus database generation unit 13 determines whether or not the program has been switched. This determination can be made based on, for example, a signal obtained from the television receiver side. The television receiver switches the receiving channel by a remote control operation by the user.
If the result of determination is that the program has been switched (step S34: YES), the process proceeds to step S35. If the program has not been switched (step S34: NO), the process jumps to step S37.

  When the process proceeds to step S35, the local corpus database generation unit 13 switches the local corpus database in the same step. That is, the local corpus database used before the program switching is different from the local corpus database used after the program switching. As already described, a local corpus database exists for each program.

Next, in step S36, the local corpus database generation unit 13 determines whether or not the program has ended. This determination can also be made based on a signal obtained from the television receiver side. Based on the control information included in the television broadcast signal, the television receiver can detect the end of the program.
If the result of determination is that the program has ended (step S36: YES), the process proceeds to step S38. If the program has not ended (step S36: NO), the process proceeds to step S37.

  When the process proceeds to step S37, in this step, the keyword extraction unit 17 performs a stepwise keyword extraction process from the caption text. The procedure of keyword extraction processing by the keyword extraction unit 17 is as described in another figure.

  When the process proceeds to step S38, in this step, the local corpus database generation unit 13 resets the local corpus database. When the program ends, the local corpus database corresponding to the program is selected to be targeted as the next program starts (as described in step S31).

  As described in the above processing procedure, when neither the program switching nor the program end occurs, the keyword extracting device 1 performs keyword extraction from the caption text (step S37). Then, regarding the extracted keyword, if the paraphrase keyword is not yet registered in the local corpus database, the paraphrase keyword is registered in the local corpus database (step S33). Note that the keywords registered in the local corpus database are to be matched when the keyword extraction unit 17 performs a process of extracting keywords from the subtitles. That is, the keyword extraction unit 17 treats both the large-scale corpus database and the local corpus database equally and uses them for keyword matching.

  6 and 7 are flowcharts showing the procedure of the stepwise keyword matching process. Note that a series of processing represented by steps S51 to S68 is shown separately in these two figures. Hereinafter, it demonstrates along this flowchart.

  First, in step S51 (FIG. 6), the keyword extraction unit 17 performs matching between subtitle text and corpus data (both large-scale corpus database and local corpus database; the same applies to the following). At this time, the keyword extraction unit 17 performs matching in order from an element having a long word length in the corpus database. That is, at first, an element having the longest word length in the corpus database is a matching target.

  Next, in step S52, the keyword extraction unit 17 determines whether or not matching between the caption text and the corpus database is detected in the matching process in step S51. If detected, that is, if an element of the current corpus database exists in the caption text (step S52: YES), the process proceeds to the next step S53. If not detected, that is, if the current element does not exist in the caption text (step S52: NO), the process jumps to step S57.

  Next, in step S53, the keyword extraction unit 17 performs confirmation according to the type of word. This is to correct when the keyword detection in step S52 is incorrect. The confirmation here is performed for each keyword type. As described with reference to FIG. 4, since registration is performed for each type of keyword in the corpus database, the keyword extraction unit 17 uses the type to perform confirmation according to the type of matched keyword. The keyword extraction unit 17 holds rules for matters to be confirmed for each type of keyword, and applies those rules. That is, the keyword extraction unit 17 confirms whether or not it is a detection error by applying a rule according to the type of characters constituting the keyword. Only when it is confirmed that there is no detection error, the keyword extraction unit 17 Let it be an extraction result.

An example of the rules is as follows (a) and (b).
(A) For Kanji keywords: If there is katakana or alphanumeric characters before or after the keyword detected in the text, the detection is correct.
(B) In the case of a kanji keyword: If four or more kanji characters are consecutive, the detection is correct.
(C) In the case of a keyword of katakana: If there is katakana before or after the keyword detected in the text, the detection is incorrect.
The rules are not limited to these.

  Next, in step S54, the keyword extraction unit 17 determines the confirmation result in step S53 above. If the keyword detected in the determination in step S52 is a false detection (step S54: YES), the process jumps to step S57. If the keyword detected in the determination in step S52 is correctly detected (step S54: NO), the process proceeds to the next step S55.

  When the process proceeds to step S55, in the same step, the keyword extraction unit 17 removes the keyword that has been successfully detected from the subtitle text temporary buffer storage means.

  In step S56, the keyword extraction unit 17 checks the word length of the next corpus element. Specifically, it is determined whether or not the word length of the next element is smaller than K. Here, K is a preset value. This word length check is performed so that only keywords stored in the corpus database having a length (number of characters) of K or more are targeted for matching. If “word length <K” is true (step S56: YES), the process proceeds to step S58. If “word length <K” is false (FALSE) (step S56: NO), that is, if the word length of the next corpus element is sufficiently long, the process proceeds to step S57.

  When the process proceeds to step S57, the keyword extraction unit 17 selects the next element from the corpus database. Here, the keyword extraction unit 17 gives the highest priority to the word length, and selects the element with the longest word length from the corpus elements that are not yet used for matching. That is, in the selection of the next element, a long word length is given priority over the type of keyword (kanji, hiragana, katakana, alphanumeric characters, special symbols).

  Next, in step S58, the keyword extraction unit 17 performs restriction under the restriction that the part of the keyword already detected in the subtitle text (the part where the removed keyword originally existed) is identified as the noun part. The attached morpheme analysis process is executed. As a result of the morpheme analysis process, the caption text is decomposed into morphemes, and information on the part of speech of each morpheme is obtained. Note that the morphological analysis process itself can be performed using existing technology.

  Next, moving to FIG. 7, in step S59, the keyword extraction unit 17 extracts one noun part from the result of the morphological analysis process in step S58. In the following process, keyword matching is performed between the extracted noun part and the corpus database.

  Next, in step S60, the keyword extracting unit 17 performs matching between the noun part extracted in step S59 and elements of the corpus database. In this step, a corpus element having a length of less than K that has not been used for matching so far is targeted.

  Next, in step S61, the keyword extraction unit 17 determines whether or not matching between the noun part and the corpus element is detected in the matching process in step S60. If detected, that is, if the current corpus database element corresponds to the current noun part (step S61: YES), the process proceeds to the next step S62. If not detected, that is, if the current corpus element is different from the current noun part (step S61: NO), the process jumps to step S67.

  When it progresses to step S62, in the same step, the keyword extraction part 17 performs confirmation according to the kind of word. This is to confirm the keyword detected in step S61. The confirmation method uses a rule for each type of keyword, and is the same as the confirmation described in step S53.

  Next, in step S63, the keyword extraction unit 17 determines the result of the confirmation in step S62 above. If the keyword detected in the determination in step S61 is a false detection (step S54: YES), the process jumps to step S67. If the keyword detected in the determination in step S61 is correctly detected (step S54: NO), the process proceeds to the next step S64.

  When the process proceeds to step S64, the keyword extraction unit 17 determines whether or not the noun part still remains in the morphological-analyzed caption data in the same step. When the noun part exists (step S64: YES), the process proceeds to step S65. If the noun part no longer exists (step S64: NO), the process proceeds to step S68.

  When the process proceeds to step S65, the keyword extraction unit 17 selects one of the noun parts left unprocessed in the subtitles subjected to morphological analysis as the next noun. And in order to take out the selected noun, it returns to step S59.

  In step S66, the keyword extracting unit 17 determines whether all elements in the corpus database have been checked. If all elements are checked (step S66: YES), the process proceeds to step S68. If there is an unchecked element (step S66: NO), the process proceeds to step S67.

In step S67, the keyword extraction unit 17 selects the next element in the corpus database as a matching target. Then, the process returns to step S60.
When the process proceeds to step S68, the keyword extraction unit 17 returns to step S51 in FIG. 6 for processing the next caption text. The next subtitle text data is also passed from the subtitle / program explanation extraction unit 16.

  Although the processing has been described with reference to the flowchart, the same determination as in step S56 may be performed when the process proceeds from step S52 to S57 or when the process proceeds from step S54 to S57. That is, when the word length of the next corpus element is smaller than K, matching using the next corpus element may not be performed, and the process of the restricted morphological analysis in step S58 may be performed.

As described above, the keyword extraction unit 17 first performs matching processing with subtitle text sequentially for corpus elements having a word length equal to or longer than a predetermined length (K) in the longest word length from the corpus database. To go. When matching with a corpus element having a length equal to or longer than K is completed, morpheme analysis processing is performed, and matching of the noun part obtained as a result with a corpus element having a length less than K is sequentially performed. Thereby, it is possible to perform extraction as a keyword while avoiding the morphological analysis process for a noun part longer than a predetermined length. That is, the long noun problem that is a weak point of the morphological analysis process can be solved. For noun parts shorter than a predetermined length, keyword extraction with higher accuracy than simple character string matching can be performed by performing morphological analysis processing.
That is, the keyword extraction unit 17 first extracts the keywords included in the subtitle text data by prioritizing the longest word length and matching the keywords up to a predetermined lower limit word length (K) with the subtitle text data. Next, morphological analysis processing is performed on the premise that the extracted keywords for subtitle text data are nouns or noun phrases, and the nouns or noun phrases obtained as a result of this morphological analysis processing are further scaled up. The keyword included in the result of the morphological analysis process is extracted by matching with the keyword read from each of the corpus database and the local corpus database.

  In other words, as a process of the keyword extraction device, first, a keyword having a word length of K or more is extracted without performing the morpheme analysis process, and then the morpheme analysis process is performed. A keyword matching process is performed to extract keywords whose word length is smaller than K. This keyword extraction process is repeated until all the noun parts of the subtitles disappear or the keyword matching process with all the constituent elements of the corpus database is completed. When the processing with the target subtitle is completed, the same processing is performed for the next subtitle read from the digital broadcast.

  For the keywords extracted in this process, not only the keywords are extracted, but also the category information attached to the keywords is extracted. It can be carried out. Further, there is an advantage that an effective way of showing the processing result can be easily realized, for example, the accompanying category information is presented when the user touches the keyword displayed on the screen as a user interface with a finger or the like.

  Note that the function of the keyword extraction device in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible disk, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, a “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included, and a program that holds a program for a certain period of time. The program may be a program for realizing a part of the above-described functions, or may be a program that can realize the above-described functions in combination with a program already recorded in a computer system.

The embodiments described above are briefly summarized below.
The keyword extraction method according to the above embodiment solves the problem of conventional keyword extraction processing by extracting keywords from text sentences such as subtitles through exhaustive matching processing with a dictionary corpus database with large-scale category information. It consists of the following items.
(1) Collation processing with each element of a large-scale dictionary corpus database. Collation accuracy is improved by performing collation processing in word length order and by word type.
(2) Generation and use of a local dictionary corpus for name identification processing. The data structure of the dictionary corpus database is devised in order to cope with word paraphrasing, and name identification information is added to generate a name collation dictionary corpus as needed during keyword extraction, and matching processing is also performed with the dictionary corpus. The name identification process is realized.
(3) Keyword information with category information can be output.

The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.
For example, a local corpus database is constructed for each program, but a local corpus database that has become unnecessary after the program is finished can be created at an appropriate timing (at the end of the program, after a predetermined time elapses from the end of the program, etc.). It may be deleted. Thereby, the memory | storage means in the keyword extraction apparatus 1 can be used efficiently.

  In the embodiment described above, TV program description text data and subtitle text data are targeted for processing. However, the present invention is not limited to this, and general text data including sentences may be targeted for processing.

  Moreover, although the above-described embodiment includes all the features of the configuration described in the means for solving the problem, the embodiment may be implemented as a configuration including only a part of them. Moreover, you may make it implement combining the some structure of them.

  The present invention can be used, for example, as a part of equipment constituting a broadcast receiving facility. For example, it can be used to automatically extract a keyword that well represents a program being viewed on a television receiver in a general home. In addition, the category to which the keyword belongs can be automatically extracted. By accumulating such keyword information and category information, it is possible to extract a keyword set or concept set that well expresses the preference of the user who uses the television receiver. Furthermore, the present invention can be used for the purpose of improving the accuracy of recommendation based on these keyword sets and concept sets.

1 Keyword Extraction Device 11 Corpus Database Construction Processing Unit 12 Large-scale Corpus Database Storage Unit (Corpus Data Storage Unit)
13 Local corpus database generator (local corpus data generator)
14 Local corpus database storage (local corpus data storage)
16 Subtitle / program description extractor (text acquisition unit)
17 Keyword extractor 18 Category extractor

Claims (8)

A corpus data storage for storing a set of keywords;
A text acquisition unit for acquiring text data;
A local corpus data storage unit for storing a set of keywords in order to apply only to text data within a predetermined range currently acquired by the text acquisition unit;
A local corpus data generation unit that extracts keywords based on the first text data acquired by the text acquisition unit, and registers the extracted keywords in the local corpus data storage unit;
By matching the keyword read from each of the corpus data storage unit and the local corpus data storage unit with the second text data acquired by the text acquisition unit, the keyword included in the second text data A keyword extractor for extracting keywords;
Equipped with,
Each of the corpus data storage unit and the local corpus data storage unit stores the keyword in association with the word length of each keyword,
The keyword extraction unit gives priority to the longest word length and reads the keyword read from each of the corpus data storage unit and the local corpus data storage unit, the second text data acquired by the text acquisition unit, Match
Keyword extracting device you wherein a. The keyword is a noun or noun phrase,
The keyword extraction unit first gives priority to the longest word length, and matches the keyword up to a predetermined lower limit word length with the second text data to determine the keyword included in the second text data. Next, morpheme analysis processing is performed on the premise that the extracted keyword for the second text data is a noun or a noun phrase, and a noun obtained as a result of the morpheme analysis processing For a noun phrase, further extract a keyword included in the result of the morphological analysis process by matching with the keyword read from each of the corpus data storage unit and the local corpus data storage unit,
The keyword extracting device according to claim 1 , wherein: The local corpus data storage unit stores a set of keywords different for each predetermined range,
When the predetermined range to which the second text data acquired by the text acquisition unit belongs is switched , the keyword extraction unit switches to the set of keywords corresponding to the changed predetermined range, and The keyword is read from a local corpus data storage unit and used for matching with the second text data.
The keyword extraction device according to claim 1 or 2 , wherein The text acquisition unit acquires text data based on the broadcast signal of the television,
The first text data is program description data relating to a TV program,
The second text data is subtitle text data in the program,
The predetermined range is a range of the subtitle text in the program.
The keyword extracting device according to claim 3 , wherein The keyword extraction unit passes the extracted keyword information to the local corpus data generation unit,
The local corpus data generation unit further registers the keyword extracted from the second text data by the keyword extraction unit in the local corpus data storage unit;
The keyword extracting device according to any one of claims 1 to 4, wherein The corpus data storage unit and the local corpus data storage unit each hold association information between keywords in a paraphrasable relationship between the plurality of keywords to be stored,
When the keyword extracting unit refers to the association information held by the corpus data storage unit and the local corpus data storage unit, the plurality of keywords extracted from the second text data are in a paraphrasable relationship. Is output after performing a name identification process that integrates a plurality of the keywords in a relationship that can be paraphrased,
The keyword extracting device according to any one of claims 1 to 5, wherein The keyword extraction unit is related to the keyword detected by matching the keyword read from each of the corpus data storage unit and the local corpus data storage unit with the second text data acquired by the text acquisition unit. Confirming whether or not it is a detection error by applying a rule according to the type of characters constituting the keyword, and extracting the keyword only when it is confirmed that it is not a detection error,
The keyword extracting device according to any one of claims 1 to 6 , wherein the keyword extracting device includes: Computer
The keyword extracting device according to any one of claims 1 to 7,
Program to function as.

Images