JP6279354B2 - Topic identification device and topic identification method - Google Patents

Description

  The present invention relates to a topic identification device and a topic identification method.

  The following topic word extraction devices are known. In this topic word extraction device, important words are extracted from written information obtained from social media servers by excluding pronouns, words used for greetings, words related to time, and calculated for each extracted important word A topic word is extracted based on importance (for example, patent document 1).

JP 2013-69246 A

  However, sentences such as written information include a plurality of words, and these words often constitute one topic in association with each other. For this reason, there is a possibility that a topic word for specifying a topic of a sentence cannot be extracted with high accuracy only by extracting an important word in the sentence.

A topic identification device according to the present invention includes a posting information acquisition unit that acquires posting information in which text information posted by a user is accumulated, and a word included in the posting information by analyzing the posting information acquired by the posting information acquisition unit. The word extraction means for extracting the appearance frequency, the appearance frequency calculation means for calculating the appearance frequency in the posted information for each word extracted by the word extraction means, and the appearance frequency for each word calculated by the appearance frequency calculation means Based on the above, an upper frequent word specifying means for specifying a predetermined number of words having a higher appearance frequency as an upper frequent word and an upper frequent word specified by the upper frequent word specifying means A connection strength calculating means for calculating a connection strength between two words indicating the strength of a connection between other words included in the word; By means of an upper frequent word pair extracting means for extracting, as upper frequent word pairs, a combination of upper frequent words having a connection strength calculated by the path strength calculating means that is equal to or higher than a predetermined threshold set in advance, and by an upper frequent word pair extracting means. Among the extracted top frequent word pairs, two words constituting the top frequent word pair having the highest connection strength are used as core words (hereinafter referred to as “core words”) constituting the topic included in the posted information. For each of the core word identification means to be identified and the two core words identified by the core word identification means, an association that relates the upper frequent words whose connection strength calculated by the connection strength calculation means is a predetermined threshold or more to the core word A related word specifying means for specifying as a word, two core words specified by the core word specifying means, A related word identified by communicating word specifying means, characterized in that it comprises a topic structure word group specifying means for specifying a set of topics constituent words that constitute the subject.
The topic identification method according to the present invention includes a posting information acquisition procedure for acquiring posting information that accumulates text information posted by a user, and analyzing the posting information acquired in the posting information acquisition procedure to determine a word included in the posting information. Based on the word extraction procedure to be extracted, the appearance frequency calculation procedure for calculating the appearance frequency in the post information for each word extracted in the word extraction procedure, and the appearance frequency for each word calculated in the appearance frequency calculation procedure The upper frequent word specifying procedure for specifying a predetermined number of words having a higher appearance frequency as a higher frequent word and the higher frequent word specified in the higher frequent word specifying procedure are included in the higher frequent word. The connection strength calculation procedure for calculating the connection strength between two words indicating the strength of the connection between other words and the connection strength calculation procedure. Out of the top frequent word pair extraction procedure that extracts a combination of top frequent words that are greater than or equal to a predetermined threshold that has a predetermined strength as a top frequent word pair, and the top frequent word pair extracted in the top frequent word pair extraction procedure, A core word specifying procedure for specifying two words constituting the top frequent word pair having the highest connection strength as core words (hereinafter referred to as “core words”) constituting a topic included in the posted information; For each of the two core words specified in the specifying procedure, a related word specifying procedure for specifying, as a related word related to the core word, an upper frequent word whose connection strength calculated in the connection strength calculating procedure is equal to or higher than a predetermined threshold, and the core word A set of topic structures that constitute a topic, with the two core words specified in the specifying procedure and the related words specified in the related word specifying procedure. A method for executing the topic structure word group specific procedures on a computer for identifying a group of words.

  According to the present invention, two words constituting a top frequent word pair having the largest connection strength between two words are identified as core words, and the top frequent occurrences having a connection strength of a predetermined threshold or more for each of the core words. Since a word is specified as a related word, and the specified two core words and related words are specified as a set of topic constituent words constituting a topic, a plurality of words included in a sentence posted by the user A word for specifying a topic with high accuracy can be extracted in consideration of the connection strength of words.

1 is a block diagram showing a configuration of an embodiment of a topic identification device 100. FIG. It is a figure which shows the example of calculation of the connection intensity | strength between two words. It is a figure which shows the specific example of topic text extraction result information. 3 is a flowchart showing a flow of processing executed by the topic identification device 100. FIG.

  FIG. 1 is a block diagram showing a configuration of an embodiment of the topic identification device 100 according to the present embodiment. As the topic specifying device 100, for example, an information processing device such as a server device or a personal computer is used. FIG. 1 shows a configuration of an embodiment in which a server device is used as the topic specifying device 100. The topic identification device 100 includes an operation member 101, a connection IF (interface) 102, a control device 103, and a recording device 104.

  The operation member 101 includes various devices operated by an operator of the topic identification device 100, such as a keyboard and a mouse.

  The connection IF 102 is an interface for connecting the topic identification device 100 to a communication line such as a LAN or the Internet. For example, a wired LAN module for connecting to the LAN by wire, or a wireless LAN for connecting to the LAN wirelessly. Modules are used.

  The control device 103 includes a CPU, a memory, and other peripheral circuits, and controls the entire topic identification device 100. The memory constituting the control device 103 is a volatile memory such as SDRAM, for example. This memory is used as a work memory for the CPU to expand the program when the program is executed and a buffer memory for temporarily recording data.

  The recording device 104 is a recording device for recording various data stored in the topic identification device 100, data of a program to be executed by the control device 103, and the like, for example, HDD (Hard Disk Drive) or SSD (Solid State). Drive) or the like is used. The program data recorded in the recording device 104 is provided by being recorded on a recording medium such as a CD-ROM or DVD-ROM, or provided via a network, and records the program data acquired by the operator. By installing in the device 104, the control device 103 can execute the program.

  The topic identification device 100 according to the present embodiment uses a service that allows a user to input and publish text on the WEB, analyzes the text posted by the user and published, and currently Execute processing to identify whether the content is posted as the main topic. As a service that allows a user to input and publish text on WEB, a service such as Twitter (registered trademark) that allows a user to post a short text and publish it on WEB is assumed.

  In this embodiment, in order to perform an analysis on the text posted by the user, the control device 103 is posted by the user from an external server operated by a service provider that provides the service as described above. Get post information that has accumulated text information. For example, the topic identification device 100 is equipped with an API (Application Programming Interface) for acquiring post information from an external server operated by a service provider, and the control device 103 is configured at predetermined time intervals. The API is activated, the posting information is acquired from the external server via the connection IF 102, and is recorded in the recording device 104.

  The control device 103 executes filtering processing for removing postal information that is not necessary for specifying a topic constituent word group to be specified by processing to be described later from the post information, with respect to post information recorded in the recording device 104. To do. As a result, sentence information that is highly likely to be unrelated to topic identification is removed from the posted information. In the present embodiment, the following determination processing is performed as filtering processing, and the posted content is news, advertising, retweeted, or mechanically posted in large quantities ( bot), those that are not suitable for the topic identification purpose, those whose contents overlap with other sentences, and those whose hiragana rate is a predetermined value or less are removed as unnecessary sentence information.

  In determining whether the posted content is news, a news pattern list that lists sentence patterns when the posted content is news is prepared in advance, and the control device 103 refers to the news pattern list. Then, pattern matching using regular expressions is performed on each piece of text information included in the posted information, and if there is a character string that matches the text information, the text information is determined to be news and removed from the posted information. In addition, a news account list in which accounts of posters who post news are listed is prepared in advance, and the control device 103 refers to the news account list and posts accounts included in the news account list in the posting information. If there is any written text information, it is determined that the text information is news and is removed from the posted information.

  The process for determining whether or not the posted content is an advertisement is prepared in advance with an advertisement pattern list that lists sentence patterns when the posted content is an advertisement. The control device 103 refers to the advertisement pattern list. Then, pattern matching using regular expressions is performed on each piece of text information included in the post information, and if there is a character string that matches the text information, the text information is determined to be an advertisement and removed from the post information. In addition, an advertisement account list in which the accounts of posters who post advertisements are listed is prepared in advance, and the control device 103 refers to the advertisement account list and posts accounts included in the advertisement count list in the posting information. If there is any written text information, it is determined that the text information is an advertisement and is removed from the posted information.

  Whether the posted content is retweeted or not is determined. If the text starts with “RT” for each text information, it is determined that the text information is retweeted, and the posted information Remove from.

  Whether or not the posted content is mechanically posted in large quantities (bots) is determined by preparing a bot pattern list that lists sentence patterns when the posted content is bot in advance. The device 103 refers to the bot pattern list, performs pattern matching with regular expressions for each piece of text information included in the posted information, and if there is a character string that matches the text information, the text information is “bot”. Judgment and removal from posting information. Also, a bot account list that lists the accounts of the posters who post bots is prepared in advance, and the control device 103 refers to the bot account list and posts the account included in the bot account list in the posting information. If there is any written text information, it is determined that the text information is “bot” and is removed from the posted information. In addition, a bot profile sentence list in which sentence patterns that are likely to be used as profile sentences of posters who post bots are listed in advance is prepared, and the control device 103 refers to the bot profile sentence list, Pattern matching by a regular expression is performed on the profile sentence of the poster, and if there is a character string that matches the profile sentence, it is determined that the text information posted by the poster is bot and is removed from the posted information. In addition, when the screen name set as an alias for the account name of the poster includes the character string “bot”, the control device 103 determines that the text information posted by the poster is “bot”. And remove it from the posted information.

  In the determination process of whether or not the posted content is not suitable for the topic identification purpose in the present embodiment, the control device 103 removes the URL, screen name, and mail address from the text for each text information, If the remaining text does not include a preset keyword, it is determined that the sentence information is not suitable for the topic identification purpose, and is removed from the posted information.

  In the process of determining whether or not the posted content overlaps with other text, the control device 103 converts the text content into a hash value for each text information one by one. It is regarded as duplicate text information, and one text is left and the other text information is removed from the posted information.

  In the process of determining whether or not the posted content has a hiragana rate equal to or lower than a predetermined value, the control device 103 calculates the ratio of the number of hiragana characters included in the text to the total number of characters for each text information. When the calculated ratio is equal to or less than a predetermined ratio, for example, 10% or less, the sentence information is determined to have a hiragana ratio equal to or less than a predetermined value and removed from the posted information.

  The control device 103 performs the following process on the posted information after the filtering process as described above.

  The control device 103 performs a morphological analysis process on the sentence information included in the posted information, extracts only words of a specific part of speech, and sets a removal target that is set in advance from the extracted words of a specific part of speech. Remove a character or string. In the present embodiment, the control device 103 uses a known morphological analysis engine to extract nouns, adjectives, and adjective verbs from the posted information, and then, from the extracted words, non-independent words, one number character, One alphabetic character is identified as a removal target word and removed. Note that whether the word extracted by the morphological analysis process is an independent word or a non-independent word is determined in the morpheme analysis process, and each extracted word is an independent word or a non-independent word. It is assumed that the control device 103 identifies and removes non-independent words based on the information.

  A specific part-of-speech extraction process and a removal target word removal process according to the present embodiment will be described using sentence information “goal seems close” as an example. For example, the sentence information “Goal seems close” is divided into five words “Goal”, “Ha”, “Close”, “Yo”, “Da” by morphological analysis processing, `` Goal '' is a noun, `` ha '' is a particle, a coordinator, `` close '' is an adjective, `` yo '' is a noun, `` da '' is an auxiliary verb, and the result is a noun, adjective, adjective verb `` goal '', “Near” and “Yo” are extracted. Each extracted word is attached with information indicating “goal” and “close” as independent words and “yo” as non-independent words, and the control device 103 removes “yo” as non-independent words. To do. As a result, from the sentence information “Goal seems to be close”, the two words “Goal” and “Close” are finally processed as words to be processed in the process for specifying the topic constituent words described below. Extracted.

  The control device 103 calculates the appearance frequency of each word extracted as the processing target in the above processing from the posted information, and identifies the 10 words with the highest appearance frequency, that is, the 10 words with the highest application frequency as the upper frequent words. To do.

The control device 103 calculates, for each of the identified upper frequent words, the connection strength between the two words indicating the strength of the connection with the other words included in the upper frequent words. Specifically, the control device 103 calculates an index value R (A, B) indicating the strength of connection between the two words A and B by the following equation (1). As a result, for each of the 10 words that are the top frequent words, a total of 45 connection index values R (A, B) are calculated for all combinations between two words. In the following equation (1), U is a variable indicating the number of text information included in the posted information, df (A) is a variable indicating the number of text information including the word A, and df (B) is Df (A∩B) is a variable representing the number of sentence information including both the word A and the word B. The variable represents the number of sentence information including the word B.

  The control device 103 calculates the connection strength between two words by converting the 45 index values R (A, B) calculated by the equation (1) into deviation values.

  The control device 103 extracts a combination of upper frequent words whose connection strength between two words is equal to or higher than a preset threshold value as an upper frequent word pair. Then, of the extracted top frequent word pairs, two words constituting the top frequent word pair having the highest connection strength are replaced with the core word constituting the main topic included in the posted information (hereinafter referred to as “core word”). Specified). Further, for each of the two specified core words, the upper frequent word having a connection strength equal to or higher than a predetermined threshold is specified as a related word related to the core word. The control device 103 identifies the two core words identified in this way and the related words related to them as a set of topic constituent words constituting the topic.

  After identifying a set of topic constituent words, the control device 103 removes the upper frequent words that have already been identified as core words and the words that have already been identified as related words from among the remaining frequent words pairs. The two words constituting the highest frequent word pair having the highest intensity are specified as new core words, and the related words for the new core word from the higher frequent words excluding the higher frequent words already identified as core words By specifying, the topic constituent word groups in the second and subsequent sets are specified. The control device 103 repeats the process for specifying the topic constituent word group until the upper frequent word pairs cannot be specified.

  A calculation example of the connection strength between two words and a specific example of a topic constituent word group based on the calculation will be described with reference to FIG. In the example shown in FIG. 2, “Japan”, “Political Party”, “National”, “Secret Protection Act”, “Deputy Member”, “Prime Minister”, “Government”, “Politics”, “Opposition”, “Bill” A word is identified as an upper frequent word, and the deviation value of the index value calculated by equation (1) is calculated as the connection strength between the two words. Here, the topic constituent word group specifying method will be described by taking, as an example, a case in which a deviation value 49.0 is set as a threshold for extracting a top frequent word pair and a threshold for specifying a related word. explain.

  In the example shown in FIG. 2, the combination of the top frequent words having the largest connection strength between the two words having a connection strength of 49.0 or more as the threshold is “Political Party” and “Government” indicated by reference numeral 2a. It becomes a combination. The control device 103 extracts the combination of the “political party” and the “government” as the first upper frequent word pair, and identifies “the political party” and “the government” as core words.

  The control device 103 specifies related words having a connection strength of 49.0 or more, which is a threshold value, for each of “political party” and “government” specified as core words. In the example of FIG. 2, “primary” whose connection strength is 55,1 is identified as a related word for “political party”, and “national” whose connection strength is 55,2 for “government” Is identified as a related word. Based on this result, the control device 103 identifies four words of “political party”, “government”, “prime”, and “national” as the first set of topic constituent word groups.

  Next, among the top frequent words excluding “Political Party”, “Government”, “Prime Minister”, and “National” that have already been identified as core words and related words, the connection strength between the two words is a threshold value of 49.0 or more The combination of the top frequent words having the highest connection strength is a combination of “secret protection method” and “opposite” indicated by reference numeral 2b. The control device 103 extracts the combination of the “secret protection method” and “opposite” as the second upper frequent word pair, and identifies “secret protection method” and “opposite” as the core words.

  For each of the “secret protection law” and “opposite” specified as the core word, the control device 103 excludes “political party”, “government”, “secret protection law”, and “opposite” already specified as the core word. A related word having a connection strength of 49.0 or more, which is a threshold value, is identified from words other than the frequent words. In the example of FIG. 2, for “Secret Protection Law”, “Prime Minister” with a connection strength of 51, 3 and “Bill” with 49.0 are specified as related words. “National” with a connection strength of 49.8 and “Bill” with 49.3 are identified as related words. Based on this result, the control device 103 identifies five words “secret protection law”, “opposite”, “prime”, “bill”, and “national” as the second set of topic constituent word groups.

  In the example illustrated in FIG. 2, since it is not possible to extract a higher-rank frequent word pair at this stage, the control apparatus 103 ends the topic constituent word group specification. As a result, according to the first group of topic constituent words, one of the four words “political party”, “government”, “prime”, and “national” is mainly the topic in the text information included in the posted information. It is specified to be a word group of eyes. In addition, according to the second set of topic constituent words, five words of “Secret Protection Law”, “Opposite”, “Prime Minister”, “Bill” and “National” are mainly included in the text information included in the posted information. The second word group is identified.

  The control device 103 selects a topic corresponding to the topic constituent word group based on the core word and the related word included in the topic constituent word group specified by the above-described processing from the posted information recorded in the recording device 104. Is extracted as topic sentence information. Hereinafter, processing for extracting topic sentence information executed by the control device 103 will be described.

  The control device 103 determines the ratio of the total number of core words and related words excluding duplicates included in the sentence information to the total number of core words and related words included in the topic constituent word group. Calculate as word coverage. Specifically, the text information included in the posted information that "the secret protection law was enacted. This bill is against many citizens" and "the opposite to the secret protection law. However, it is strange that it will be passed. You should respect the opinions of the people more. "

  Looking at the first group of topical word groups, the text information that “the secret protection law was enacted. Many of the citizens are against this bill” includes “party”, “government”, “prime”, Of the four words “national”, one “national” is included. In this case, 1/4 = 0.25, and the coverage rate is calculated as 25%. In addition, in the text information that says, “We are against the confidentiality protection law. It is strange that many people judge it. It is strange that we should respect the opinions of the people.” Two are included. In this case, since “National” is duplicated, the number of duplicates is eliminated to be one, 1/4 = 0.25, and the coverage rate is calculated as 25%.

  Next, looking at the second set of topical word groups, the text information that “the secret protection law was enacted. Many of the citizens are against this bill” includes “secret protection law”, “opposition” Of the five words "Prime Minister", "Bill", and "National", each of the four words "Secret Protection Law", "Opposite", "Bill", and "National" is included. In this case, 4/5 = 0.8, and the coverage rate is calculated as 80%. In addition, the text information that says, “I am against the Secret Protection Law. It is strange that many people have judged it. It should be passed. ”And“ opposite ”are included one by one, and two“ nationals ”are included. In this case, since “National” is duplicated, the number of duplicates is eliminated to be one, 3/5 = 0.6, and the coverage rate is calculated as 60%.

  The control device 103 also scores each piece of sentence information as described below for sentence information including at least one of the core words and related words included in the topic constituent word group. In the present embodiment, the control device 103 calculates three scores of a word frequency score, a noun content rate score, and a hiragana content rate score for each piece of sentence information, and scores the sentence information based on these three scores. I do.

  First, a method for calculating a word frequency score will be described. The control device 103 uses the sentence information including at least one word of the core word and the related word included in the topic constituent word group in the posting information as the target sentence information, from the target sentence information, as in the above-described process, , Extract adjectives, adjective verbs, and calculate the appearance frequency of each word included in all target sentence information for the word after removing non-independent words, one number character, and one alphabetic character from the extracted words A weight value is set for each word so that the word having a higher application frequency has a higher weight. The control device 103 calculates a word frequency score by adding a weight value set to a word included in each target text information for each target text information.

Next, the noun content rate score will be described. Based on the number of words included in each target sentence information and the number of nouns included in each target sentence information, the control device 103 uses the following formula (2) for each target sentence information. Calculate the noun content rate.
Noun content rate = number of nouns / total number of words (2)
As shown in the following equation (3), the control device 103 calculates the reciprocal number of the business card content rate calculated by the equation (2) as a noun content rate score.
Noun content score = 1 / Noun content rate (3)

Next, the hiragana content rate score will be described. For each target sentence information, the control device 103 determines the hiragana content of each target sentence information according to the following equation (4) based on the number of characters included in each target sentence information and the number of hiragana included in each target sentence information. Calculate the rate score.
Hiragana content score = Hiragana character count / total character count (4)

Based on the word frequency score, the noun content rate score, and the hiragana content rate score calculated in the above-described processing, the control device 103 calculates the score Score (di) of each sentence information by the following equation (5). In the following equation (5), Score _freq (di) is a variable indicating a word frequency score, Score _noun (di) is a variable indicating a noun content rate score, and Score _hira (di) is a hiragana content rate score. Is a variable indicating Α, β, and γ are mixing ratios. Here, α = β = γ = 1/3, and the word frequency score, noun content rate score, and hiragana content rate score are multiplied by the same ratio. .
Score (di) = Score _freq (di) ^α · Score _noun (di) ^β · Score _hira (di) ^γ (5)

  Based on the above-described coverage rate and score, the control device 103 extracts, for each topic constituent word group, sentence information that matches the topic represented by the topic constituent word group from the posted information. Note that the number of sentence information extracted here is set in advance. For example, three pieces of sentence information are extracted for each topic constituent word group.

  In order to extract sentence information matching the topic represented by each topic constituent word group from the posted information, the control device 103 first extracts sentence information whose coverage rate is higher than a preset threshold value so as to satisfy the extraction number. To do. At this stage, if text information corresponding to the set number of extractions can be extracted, the extraction is completed.

  When the extracted text information does not satisfy the set number of extractions, the control device 103 further extracts text information whose score calculated by the equation (5) is higher than a preset threshold value so as to satisfy the number of extractions. To do.

  When the sentence information is extracted based on the coverage rate and the score for each topic constituent word group, the control device 103 extracts the topic sentence information that lists the extracted topic sentence information for each topic constituent word group. Generate result information. The topic sentence extraction result information is, for example, as shown in FIG. 3, the topic sentence information extracted as the sentence information along the topic represented by these words after enumerating each word included in the topic constituent word group Is generated as a text file. In FIG. 3, as the first topic (topic 1), the topic constituent words of the political party / the government / the prime minister / the public are listed, and below the three topic sentence information matching topic 1 are displayed. . Also, as the second topic (topic 2), the secret protection law / opposition / prime / bill / national topic composition word group is listed, and below it, three topic sentence information matching topic 2 is displayed. An example is shown. The control device 103 indicates that there is no main topic in the posted information when the topic constituent word group cannot be extracted or when no sentence information based on the coverage rate and the score can be extracted. A text message may be displayed to generate topic sentence extraction result information.

  If the topic sentence extraction result information generated by the above-described processing is provided to a company or person who wants to know what topic is currently posted, the topic identification device 100 according to the present embodiment. It is possible to effectively utilize the topic sentence extraction result information generated in the above. Further, if the control device 103 creates topic sentence extraction result information at predetermined time intervals and transmits it to a pre-registered mail address, it is always related to what topic is posted. The latest information can be distributed to those who wish.

  FIG. 4 is a flowchart showing a flow of processing executed by the topic identification device 100 according to the present embodiment. The process shown in FIG. 4 is executed by the control device 103 as a program that starts when the topic sentence extraction result information is created. In the process illustrated in FIG. 4, the post information filtering process described above has already been completed, and post information after filtering is recorded in the recording device 104.

  In step S10, the control device 103 performs a morphological analysis process on sentence information included in the posted information, extracts nouns, adjectives, and adjective verbs from the posted information, and further removes the above-described removal target words and performs processing. Extract the target word. Then, it progresses to step S20.

  In step S <b> 20, the control device 103 calculates the appearance frequency of each word extracted from the posted information, and identifies the ten cases with the highest appearance frequency as upper frequent words. Then, it progresses to step S30.

  In step S30, the control device 103 calculates the connection strength between two words indicating the strength of connection between each of the upper frequent words specified in step S20 and other words included in the upper frequent words. Thereafter, the process proceeds to step S40.

  In step S40, the control apparatus 103 extracts a combination of upper frequent words having a connection strength between two words equal to or higher than a preset threshold value as an upper frequent word pair. Thereafter, the process proceeds to step S50.

  In step S50, the control device 103 identifies, as core words, two words constituting the upper frequent word pair having the highest connection strength among the upper frequent word pairs extracted in step S40, and the identified two core words For each of the above, the top frequent words whose connection strength is greater than or equal to a predetermined threshold are specified as related words related to the core word, and the two specified core words and related words related thereto are included in the topic. It is specified as a group of topic constituent words. Thereafter, the process proceeds to step S60.

  In step S60, as described above, the control device 103 determines whether or not a higher frequent word pair can be identified by excluding higher frequent words that have already been identified as core words. If a positive determination is made in step S60, the process returns to step S40. On the other hand, if a negative determination is made in step S60, the process proceeds to step S70.

  In step S70, as described above, for each topic constituent word group, the control device 103 eliminates duplication included in the sentence information with respect to the total number of core words and related words included in the topic constituent word group. The ratio of the total number of core words and related words is calculated as the coverage rate of the topic constituent words. Thereafter, the process proceeds to step S80.

  In step S80, as described above, for each topic constituent word group, the control device 103 targets word information including at least one of the core words and related words included in the topic constituent word group as a word frequency. A score, a noun content rate score, and a hiragana content rate score are calculated, and a score Score (di) of each sentence information is calculated by the equation (5). Thereafter, the process proceeds to step S90.

  In step S90, as described above, the control device 103 posts sentence information that matches the topic represented by the topic constituent word group for each topic constituent word group based on the calculated coverage rate and score. Extract from Then, it progresses to step S100.

  In step S <b> 100, the control device 103 generates topic sentence extraction result information in which the extracted topic sentence information is displayed as a list for each topic constituent word group, and records it in the recording device 104. Thereafter, the process ends.

According to the present embodiment described above, the following operational effects can be obtained.
(1) The control device 103 acquires post information that accumulates text information posted by the user, analyzes the acquired post information, extracts words included in the post information, and for each extracted word, The appearance frequency in the posted information is calculated, and based on the calculated appearance frequency for each word, a predetermined number of words having a higher appearance frequency are specified as upper frequent words. Then, the control device 103 calculates the connection strength between the two words indicating the strength of the connection between each of the identified upper frequent words and the other words included in the upper frequent words, and the calculated connection strength is A combination of top frequent words exceeding a predetermined threshold set in advance is extracted as a top frequent word pair, and two words constituting the top frequent word pair having the highest connection strength are extracted from the extracted top frequent word pairs. For each of the two identified core words, the top frequent words whose connection strength is a predetermined threshold or more are identified as related words related to the core word, and the identified two core words and related words are It was specified as a set of topic constituent words that make up a topic. Thereby, it is possible to extract a word for specifying a main topic posted by the user with high accuracy in consideration of the connection strength of a plurality of words included in the text information posted by the user.

(2) The control device 103 determines the two words constituting the upper frequent word pair having the highest connection strength from the remaining upper frequent word pairs, except for the words already identified as the core words and related words, as the core words. This is repeated until there are no higher frequent word pairs that can identify the core word. Thereby, when the post information includes text information related to a plurality of topics, the core word can be specified for each of the plurality of topics.

(3) The control device 103 identifies related words from the top frequent words excluding the core words already identified as core words. As a result, it is possible to prevent the upper frequent word already specified as the core word from being redundantly specified as the related word of another topic.

(4) The control device 103 selects a noun, an adjective, and an adjective verb included in the posted information and extracts words from the posted information. As a result, it is possible to specify a topic group word group by limiting to words with parts of speech that are likely to constitute a topic.

(5) The control device 103 extracts nouns, adjectives, and adjective verbs from the posted information, and then identifies and removes non-independent words, one numeric character, and one alphabetic character from the extracted words as removal target words. I did it. As a result, it is possible to narrow down the words to be processed into words that are highly likely to constitute a topic, and to improve the extraction accuracy of the words that constitute the topic. Further, by narrowing down the words to be processed, the processing load on the control device 103 can be reduced and the processing speed can be improved.

(6) Based on the core word and the related word included in the identified topic constituent word group, the control device 103 converts the sentence information that matches the topic corresponding to the topic constituent word group from the posted information. As extracted. This makes it possible to specify text information that matches the main topic posted by many users from text information posted by the user.

(7) The control apparatus 103 is a ratio of the total number of core words and related words excluding duplicates included in the sentence information to the total number of core words and related words included in the topic constituent word group. Is calculated as the coverage rate of the topic constituent words, and a predetermined number of text information with a high coverage rate is extracted as the topic text information. As a result, text information with high coverage of core words and related words in the text is more text information that reflects the topic represented by the topic word group, Text information that matches the main topic posted by users can be extracted with high accuracy.

(8) The control device 103 sets sentence information including at least one of the core words and related words included in the topic constituent word group as target sentence information, and based on the appearance frequency of each word included in the target sentence information. The word frequency score is calculated by setting a weight value for each word and adding the weight value set to the word included in the target sentence information, and the target sentence for the number of words included in the target sentence information Based on the number of nouns included in the information, the noun content score in the target sentence information is calculated, and the hiragana content in the target sentence information is calculated based on the number of hiragana included in the target sentence information relative to the number of characters included in the target sentence information. Rate score, calculate score based on word frequency score, noun content rate score, hiragana content rate score, and a predetermined number of sentence information with high score It was to be extracted as a die sentence information. This improves the extraction accuracy of sentence information that matches the main topic, taking into account the weighting based on the appearance frequency of words in the sentence, the content ratio of nouns in the sentence, and the content ratio of hiragana in the sentence. it can.

(9) The control device 103 calculates a word frequency score for a word after removing a non-independent word, one number character, and one alphabetic character from the noun, adjective, and adjective verb included in the target sentence information. did. As a result, the word frequency score can be calculated only for words with parts of speech that are likely to constitute a topic. Further, by narrowing down the words to be calculated for the word frequency score, the processing load on the control device 103 can be reduced and the processing speed can be improved.

(10) The control device 103 generates topic sentence extraction result information in which topic sentence information is displayed as a list for each topic constituent word group, and records it in the recording device 104. Thus, for each topic constituent word group, it is possible to display a list of what text information is posted as being in line with the topic specified based on those word groups.

(11) The control device 103 executes a filtering process for removing sentence information that is not necessary for specifying the topic constituent word group from the posted information, and specifies the topic constituent word group for the post information after filtering. I made it. In this way, the accuracy of specifying the topic constituent word group can be improved by removing in advance the sentence information that is not necessary for specifying the topic constituent word group from the posted information.

-Modification-
Note that the topic identification device 100 according to the above-described embodiment can be modified as follows.
(1) In the above-described embodiment, the control device 103 performs filtering processing on the posted information acquired from the external server, and the posted content is news, advertising, retweeted, machine Explains an example of removing unnecessary text information that has been submitted in large numbers (bots), not suitable for topic specific purposes, content that overlaps with other text, or that has a hiragana rate below a predetermined value did. However, the removal target by these filtering processes is an example, and when other sentence information exists as sentence information that is not necessary for specifying the topic constituent word group, these may be also targeted for the filtering process. .

(2) In embodiment mentioned above, the control apparatus 103 performs the filtering process for removing the text information which is not necessary for pinpoint word group specification from posting information, and is a topic for post information after filtering. An example of specifying a constituent word group has been described. However, the topic constituent word group may be specified using the posting information acquired from the external server as it is without performing the filtering process.

(3) In the above-described embodiment, the control apparatus 103 calculates the appearance frequency of each word extracted from the posted information, and has described an example in which the top 10 appearance words are identified as the top frequent words. . However, the specific number of upper frequent words is not limited to ten, and the number of cases may be arbitrarily changed by the operator of the topic identification device 100.

(4) In the embodiment described above, the control device 103 calculates the connection strength between two words by converting the 45 index values R (A, B) calculated by the equation (1) into deviation values. The example to do was demonstrated. However, the index value R (A, B) calculated by the formula (1) may be used as it is as the connection strength between the two words, or the index value R (A, B) calculated by the formula (1) is used. An evaluation value other than the deviation value may be calculated, and the calculation result may be used as the connection strength between two words.

(5) In the above-described embodiment, the example in which the threshold value of the deviation value for extracting the upper frequent word pair and the threshold value of the deviation value for extracting the related word is 49.0 has been described. However, this threshold value is not limited to 49.0. If a threshold value for improving the extraction accuracy of upper frequent word pairs and related words is found based on experimental results and operation results, the numerical value is used. May be set as a threshold value. Further, the threshold value of the deviation value for extracting the top frequent word pairs and the threshold value of the deviation value for extracting the related word may be a common value or may be set individually.

(6) In the above-described embodiment, the control device 103, for each topic constituent word group, based on the coverage rate and the score, the sentence information that matches the topic represented by the topic constituent word group from the posted information. An example of extraction has been described. However, the control device 103 extracts, for each topic constituent word group, sentence information that matches the topic represented by the topic constituent word group from the posted information based on only one of the coverage rate and the score. May be.

(7) In embodiment mentioned above, the control apparatus 103 performs the morphological analysis process on the text information contained in contribution information, extracts a noun, an adjective, and an adjective verb from contribution information, Then, By identifying and removing non-independent words, one numeric character, and one alphabetic character as removal target words, the word to be processed is extracted in the process for specifying the topic constituent word. Further, when calculating the word frequency score, the control device 103 uses the sentence information including at least one of the core word and the related word included in the topic constituent word group in the posted information as the target sentence information. Extract nouns, adjectives, and adjective verbs from information, and remove words from non-independent words, numbers, and alphabets from the extracted words. The appearance frequency was calculated. However, the control device 103 only extracts nouns, adjectives, and adjective verbs in these processes, and does not perform the process of removing non-independent words, single numeric characters, and single alphabetic characters from the extracted words. Also good.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired. Moreover, it is good also as a structure which combined the above-mentioned embodiment and a some modification.

100 Topic Identification Device 101 Operation Member 102 Connection IF
103 control device 104 recording device

Claims (24)

Post information acquisition means for acquiring post information accumulating text information posted by the user;
Analyzing the posted information acquired by the posted information acquiring means, and extracting a word included in the posted information;
Appearance frequency calculating means for calculating the appearance frequency in the posted information for each word extracted by the word extracting means;
Based on the appearance frequency for each word calculated by the appearance frequency calculation means, a high-frequency word specifying means for specifying a predetermined number of words having a high appearance frequency as a high-frequency word;
For each of the top frequent words specified by the top frequent word specifying means, a connection strength calculation that calculates the strength of the connection between two words indicating the strength of the connection with the other words included in the top frequent words. Means,
Upper frequent word pair extraction means for extracting a combination of upper frequent words having a connection strength calculated by the connection strength calculation means equal to or higher than a predetermined threshold value as a higher frequent word pair;
Of the top frequent word pairs extracted by the top frequent word pair extraction means, two words constituting the top frequent word pair having the largest connection strength, a core constituting a topic included in the post information, and Core word specifying means for specifying as a word (hereinafter referred to as “core word”),
For each of the two core words specified by the core word specifying means, the higher-order frequent word whose connection strength calculated by the connection strength calculation means is equal to or higher than a predetermined threshold is used as a related word related to the core word. Related word identification means to identify;
Topic constituent word group for specifying the two core words specified by the core word specifying means and the related word specified by the related word specifying means as a set of topic constituent word groups constituting the topic A topic specifying device comprising: a specifying unit. In the topic identification device according to claim 1,
The core word specifying means excludes the upper frequent word pair that already includes the core word or the word identified as the related word, and among the remaining upper frequent word pairs, the upper frequent word having the highest connection strength 2. A topic identification device characterized in that two words constituting a pair are identified as the core word, and this is repeated until there is no higher frequent word pair that can identify the core word. In the topic identification device according to claim 2,
The related word specifying means specifies the related word from the upper frequent words excluding the word already specified as the core word. In the topic specific device according to any one of claims 1 to 3,
The topic identifying device, wherein the word extracting unit selects a noun, an adjective, and an adjective verb included in the posted information and extracts a word from the posted information. In the topic identification device according to claim 4,
The topic identifying device, wherein the word extracting means removes a non-independent word, a single character, and a single alphabet from the word extracted from the posted information. In the topic identification device according to any one of claims 1 to 5,
Matches the topic corresponding to the topic constituent word group based on the core word and the related word included in the topic constituent word group specified by the topic constituent word group specifying unit from the posted information. A topic identification device further comprising topic sentence information extraction means for extracting sentence information to be extracted as topic sentence information. In the topic identification device according to claim 6,
The topic sentence information extraction unit is configured to detect the overlap included in the sentence information with respect to the total number of the core words and the number of related words included in the topic component word group specified by the topic component word group specifying unit. A ratio of the total number of the excluded core words and related words is calculated as a coverage rate of topic constituent words, and a predetermined number of the sentence information having a high coverage rate is extracted as the topic sentence information. Topic identification device. In the topic identification device according to claim 6 or 7,
The topic sentence information extracting means uses sentence information including at least one of the core word and the related word included in the topic constituent word group specified by the topic constituent word group specifying means as target sentence information. Based on the appearance frequency of each word included in the target sentence information, a word frequency score is set by setting a weight value for each word and adding the weight value set for the word included in the target sentence information. The noun content rate score in the target sentence information is calculated based on the number of nouns included in the target sentence information with respect to the number of words included in the target sentence information, and the number of characters included in the target sentence information Hiragana content rate score in the target sentence information is calculated based on the number of hiragana included in the target sentence information for the word sentence score A sentence information extraction score is calculated based on the noun content ratio score and the hiragana content ratio score, and a predetermined number of the sentence information having a high sentence information extraction score is extracted as the topic sentence information. Topic identification device. In the topic identification device according to claim 8,
The topic identifying device, wherein the topic sentence information extracting means calculates the word frequency score for nouns, adjectives and adjective verbs included in the target sentence information. In the topic identification device according to claim 9,
The topic sentence information extracting unit calculates the word frequency score after removing non-independent words, one number character, and one alphabetic character from nouns, adjectives and adjective verbs included in the target sentence information. Topic identification device. In the topic specific device according to any one of claims 6 to 10,
Topic sentence extraction result information for generating topic sentence extraction result information displaying a list of the topic sentence information extracted by the topic sentence information extracting means for each topic constituent word group specified by the topic constituent word group specifying means Generating means;
A topic identifying device further comprising topic sentence extraction result information recording means for recording the topic sentence extraction result information generated by the topic sentence extraction result information generating means in a recording device. In the topic specific device according to any one of claims 1 to 11,
A topic identification device further comprising filtering means for performing a filtering process for removing the sentence information that is not necessary for identifying the topic constituent word group from the posted information. Post information acquisition procedure for acquiring post information that accumulates text information posted by the user,
Analyzing the posting information acquired in the posting information acquisition procedure, and extracting a word included in the posting information;
For each word extracted in the word extraction procedure, an appearance frequency calculation procedure for calculating an appearance frequency in the post information;
Based on the appearance frequency for each word calculated in the appearance frequency calculation procedure, an upper frequent word identification procedure that identifies a predetermined number of words that are higher in appearance frequency as upper frequent words,
For each of the top frequent words identified in the top frequent word identification procedure, a connection strength calculation procedure for calculating a connection strength between two words indicating the strength of the connection with other words included in the top frequent words When,
An upper frequent word pair extraction procedure for extracting a combination of upper frequent words having a connection strength calculated in the connection strength calculation procedure equal to or higher than a predetermined threshold value as a higher frequent word pair;
Of the top frequent word pairs extracted in the top frequent word pair extraction procedure, the two words constituting the top frequent word pair having the largest connection strength serve as the core constituting the topic included in the posted information. A core word identification procedure for identifying as a word (hereinafter referred to as a “core word”);
For each of the two core words specified by the core word specifying procedure, the upper frequent words having the connection strength calculated by the connection strength calculation procedure equal to or higher than a predetermined threshold are specified as related words related to the core word. Related word identification procedure,
Topic constituent word group specifying procedure for specifying the two core words specified by the core word specifying procedure and the related word specified by the related word specifying procedure as a set of topic constituent word groups constituting the topic Topic identification method to make computer execute. In the topic identification method according to claim 13,
In the core word specifying procedure, the top frequent words having the highest connection strength among the remaining top frequent word pairs, excluding the top frequent word pairs that already include the core word or the words identified as the related words. 2. A topic identification method characterized in that two words constituting a pair are identified as the core word, and this is repeated until there is no higher frequent word pair that can identify the core word. In the topic identification method of Claim 14,
In the related word specifying procedure, the related word is specified from the upper frequent words excluding the word already specified as the core word. In the topic specific method as described in any one of Claims 13-15,
In the word extracting procedure, a noun, an adjective, and an adjective verb included in the posted information are selected and a word is extracted from the posted information. The topic identification method according to claim 16,
The word identifying procedure includes removing a non-independent word, a single numeral, and a single alphabet from a word extracted from the posted information. In the topic identification method as described in any one of Claims 13-17,
Matches the topic corresponding to the topic constituent word group based on the core word and the related word included in the topic constituent word group specified by the topic constituent word group specifying procedure from the posted information. A topic identification method further comprising a topic sentence information extraction procedure for extracting sentence information as topic sentence information. The topic identification method according to claim 18,
The topic sentence information extraction procedure excludes duplication included in the sentence information with respect to the total number of the core words and related words included in the topic constituent word group specified in the topic constituent word group specifying procedure. Calculating a ratio of the total number of the core words and the number of related words as a coverage rate of topic constituent words, and extracting a predetermined number of the sentence information having a high coverage rate as the topic sentence information, Topic identification method. In the topic identification method according to claim 18 or 19,
The topic sentence information extraction procedure uses sentence information including at least one of the core word and the related word included in the topic constituent word group specified in the topic constituent word group specifying procedure as target sentence information, Based on the appearance frequency of each word included in the target sentence information, a weight value is set for each word, and a word frequency score is obtained by adding the weight value set to the word included in the target sentence information. Calculating, based on the number of nouns included in the target sentence information relative to the number of words included in the target sentence information, calculating a noun content rate score in the target sentence information, and for the number of characters included in the target sentence information Based on the number of hiragana included in the target sentence information, the hiragana content rate score in the target sentence information is calculated, the word frequency score, the name Topic specification characterized in that a sentence information extraction score is calculated based on a content ratio score and the hiragana content ratio score, and a predetermined number of the sentence information having a high sentence information extraction score is extracted as the topic sentence information Method. In the topic identification method according to claim 20,
The topic sentence information extraction procedure calculates the word frequency score for nouns, adjectives and adjective verbs included in the target sentence information. In the topic identification method according to claim 21,
The topic sentence information extraction procedure calculates the word frequency score after removing non-independent words, one number character, and one alphabetic character from nouns, adjectives, and adjective verbs included in the target sentence information. Topic identification method. In the topic specific method as described in any one of Claims 18-22,
A topic sentence extraction result information generation procedure for generating topic sentence extraction result information that lists the topic sentence information extracted in the topic sentence information extraction procedure for each topic constituent word group specified in the topic composition word group specification procedure When,
A topic identification method further comprising a topic sentence extraction result information recording procedure for recording the topic sentence extraction result information generated in the topic sentence extraction result information generation procedure in a recording device. In the topic identification method as described in any one of Claims 13-23,
A topic identification method further comprising a filtering procedure for performing filtering processing for removing the sentence information that is not necessary for identifying the topic constituent word group from the posted information.

Images