JP2016045938A - Comment grouping program, server and method extracting area specific comments from many comments - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program and the like that can extract area specific comments from large amounts of comments in which comments with positional information and comments without positional information are mixed.SOLUTION: A comment grouping program comprises: topic grouping means which groups a lot of comments into a plurality of topic groups based on the distribution of appearance frequency of the word and applies topic number to respective comments; positional information added comment grouping means which groups a plurality of positional information added comments into the prescribed area ranges respectively where the positional information belongs; area specific topic number extraction means which extracts a most common topic number among the positional information added comments as an area specific topic number for each of the prescribed area ranges by use of the prescribed standard based on the maldistribution degree; and area specific comment grouping means which groups the positional information added comments and the comments without positional information on the basis of the area specific topic number of the respective area ranges into the area specific comments.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for classifying comments posted from an unspecified number of third parties.

  A blog (Web log) server and a mini blog (for example, twitter (registered trademark)) server are connected to the Internet. Such a blog server receives comments (posts) from an unspecified number of third parties and publishes them to other third parties. These comments are written for the purpose of publishing various topics that occur in current affairs. For example, it often describes regional topics closely related to the posting location, such as traffic jams, suspension or delays in trains. These posted sentences often transmit events in various places such as comic market and fireworks display, and the situation of storm and flood damage in almost real time. These posted texts may be supplemented with information provided by local governments, government offices and mass media, and are attracting attention as detailed on-site situation grasp media.

  Conventionally, there is a technique for performing topic classification only on comments (sentences) to which position information is given, and extracting local topics and terms (for example, see Non-Patent Document 1). According to this technology, topics unique to the region are extracted from a large number of comments with position information, and various social phenomena can be discovered.

J. Eisenstein, et.al. A Latent Variable Model for Geographic Lexical Variation, EMNLP2010, [online], [searched August 16, 2014], Internet <URL: http://www.cs.cmu.edu/ ~ nasmith / papers / eisenstein + oconnor + smith + xing.emnlp10.pdf> DM Blei, AY Ng, and MI Jordan. Journal of Machine Learning Research, 3: 993-1022, 2003, [online], [Search August 16, 2014], Internet <URL: http: / /machinelearning.wustl.edu/mlpapers/paper_files/BleiNJ03.pdf>

  However, the number of comments with position information is less than 1% of the total number of comments. If a topic specific to a region is extracted from comments sent from only a small number of contributors, the information itself may be unevenly distributed. In fact, according to the technique described in Non-Patent Document 1, even if the slang words specific to the region and the names of the local baseball teams can be extracted, topics such as traffic jams that are currently occurring in the region can be extracted. Is difficult.

  On the other hand, the inventors of the present application considered that the comments posted from the same area range would be similar regardless of whether or not the location information is included. In other words, isn't it possible to distinguish regional ranges according to topical similarity in comments that occur in current affairs? I thought.

  In addition, the inventors of the present application considered that it is important to extract only posted texts of sudden specific topics that are unevenly distributed in the region. In other words, it was thought that the importance of the postings on the topic in that area range is low. For example, in the case of Narita City, there is a problem that a posted text regarding a disaster that occurred suddenly is buried, for example, because there are always a large number of posted texts about overseas travel.

  Therefore, an object of the present invention is to provide a comment classification program, server, and method capable of extracting region-specific comments from a large number of comments in which comments with position information and comments without position information are mixed. Another object of the present invention is to extract only unexpected posted sentences that have occurred in the area.

According to the present invention, a comment classification program that causes a computer to function to extract region-specific comments from a large number of comments in which comments with position information (sentences) and comments without position information (sentences) are mixed,
A topic classification means for classifying a large number of comments into a plurality of topic groups based on the distribution of word frequencies, and assigning a topic number to each comment;
Comment classification means with position information for classifying a plurality of comments with position information for each predetermined area range to which the position information belongs;
An area-specific topic number extracting means for extracting the most topic number as an area-specific topic number among a plurality of comments with position information using a predetermined criterion based on the uneven distribution degree for each predetermined area range;
The computer is caused to function as an area-specific comment classification unit that classifies comments with position information and comments without position information as area-specific comments based on the area-specific topic number of each area range.

According to another embodiment of the comment classification program of the present invention,
The topic classification means causes the computer to function so as to classify each comment into any one topic group using an LDA (Latent Dirichlet Allocation) algorithm that calculates the probability (topic ratio) belonging to each topic group. Is also preferable.

According to another embodiment of the comment classification program of the present invention,
It is also preferable to make the computer function so that the predetermined criterion based on the uneven distribution degree in the region-specific topic number extracting means is the Akaike information amount criterion.

According to another embodiment of the comment classification program of the present invention,
Comment storage means by period for storing region-specific comments for each predetermined period;
A frequent word extracting means for extracting frequent words that appear more than a predetermined number (or a predetermined ratio) using a predetermined criterion based on the uneven distribution degree in a region-specific comment based on each region-specific topic number of each region range;
A constant topic number extracting means for comparing a frequent word in a current period with a frequent word in a predetermined past period and determining, as a constant topic number, a region-specific topic number from which the same frequent word is extracted by a predetermined ratio or more. ,
It is also preferable to further cause the computer to function as a permanent comment deleting means for deleting a comment based on a permanent topic number in the region-specific comments.

According to another embodiment of the comment classification program of the present invention,
It is also preferable to make the computer function so that the predetermined criterion based on the uneven distribution degree in the frequent word extraction means is the Akaike information amount criterion.

According to another embodiment of the comment classification program of the present invention,
The topic classification means preferably analyzes the predicate term structure of the comment with position information and the comment without position information, and causes the computer to function so as to remove the comment including the hearing estimation expression.

According to another embodiment of the comment classification program of the present invention,
If the region-specific topic numbers in both adjacent region ranges are the same, the computer may further function as a region range clustering means for clustering both region ranges and integrating region-specific comments as one region range. preferable.

According to another embodiment of the comment classification program of the present invention,
It is also preferable to make the computer function so that the comment is posted on a mini Web log server by an unspecified number of third parties.

According to the present invention, a comment classification server that collects a large number of comments in which comments (sentences) with position information and comments (sentences) without position information are mixed, and extracts region-specific comments,
A topic classification means for classifying a large number of comments into a plurality of topic groups based on the distribution of word frequencies, and assigning a topic number to each comment;
Comment classification means with position information for classifying a plurality of comments with position information for each predetermined area range to which the position information belongs;
An area-specific topic number extracting means for extracting the most topic number as an area-specific topic number among a plurality of comments with position information using a predetermined criterion based on the uneven distribution degree for each predetermined area range;
A region-specific comment classification unit that classifies comments with position information and comments without position information as region-specific comments based on the region-specific topic number of each region range.

According to another embodiment of the comment classification server of the present invention,
Comment storage means by period for storing region-specific comments for each predetermined period;
A frequent word extracting means for extracting frequent words that appear more than a predetermined number (or a predetermined ratio) using a predetermined criterion based on the uneven distribution degree in a region-specific comment based on each region-specific topic number of each region range;
A constant topic number extracting means for comparing a frequent word in a current period with a frequent word in a predetermined past period and determining, as a constant topic number, a region-specific topic number from which the same frequent word is extracted by a predetermined ratio or more. ,
It is also preferable to further include a permanent comment deleting means for deleting a comment based on the permanent topic number in the region-specific comments.

According to the present invention, there is a comment classification method for an apparatus that collects a large number of comments in which comments with a position information (sentence) and comments without a position information (sentence) are mixed, and extracts region-specific comments,
A first step of classifying a number of comments into a plurality of topic groups based on a distribution of word appearance frequencies, and assigning a topic number to each comment;
A second step of classifying a plurality of comments with position information for each predetermined area range to which the position information belongs;
A third step of extracting the largest topic number as a region-specific topic number among a plurality of comments with position information using a predetermined criterion based on the degree of uneven distribution for each predetermined region range;
A comment classification method for an apparatus, comprising: a fourth step of classifying comments with position information and comments without position information as area-specific comments based on the area-specific topic number of each area range.

According to another embodiment of the comment classification method of the present invention,
A fifth step of storing region-specific comments for each predetermined period;
A sixth step of extracting frequent words appearing more than a predetermined number (or a predetermined ratio) using a predetermined criterion based on the uneven distribution in the region-specific comments based on each region-specific topic number of each region range;
A seventh step of comparing the frequent words of the current period with the frequent words of a predetermined past period, and determining, as a permanent topic number, a region-specific topic number from which the same frequent words are extracted by a predetermined ratio or more;
It is also preferable to further include an eighth step of deleting a comment based on the permanent topic number among the region-specific comments.

  According to the comment classification program, server, and method of the present invention, it is possible to extract region-specific comments from a large number of comments in which comments with position information and comments without position information are mixed. In addition, according to the present invention, it is possible to extract only a sudden posted sentence that occurs in the area range.

It is a system configuration diagram in the present invention. It is a functional block diagram of the comment classification | category server in this invention. It is explanatory drawing showing the example of the comment accumulate | stored in the comment storage part. It is explanatory drawing showing the classification | category of the comment in a topic classification part. It is explanatory drawing represented with the comment with a positional information, and an area specific topic number. It is explanatory drawing which specifies the comment classified by the area specific topic number. It is a sequence diagram in the present invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a system configuration diagram according to the present invention.

According to FIG. 1, a comment classification server 1 according to the present invention is connected to the Internet. The comment classification server 1 collects a large number of comments from the blog server 2 that publishes comments posted by an unspecified number of third parties. The comment classification server 1 can classify as a region-specific comment for each region range from a large number of comments in which comments (text) with position information and comments without position information (text) are mixed.
The blog server 2 is a mini blog server such as a twitter (registered trademark) server, for example.
The terminal 3 is owned by an unspecified number of third parties, and can freely post comments to the miniblog server 2.
The terminal 4 is operated by a viewer, can access the comment classification server 1 via the access network and the Internet, and can receive region-specific comments in a desired region range.

  FIG. 2 is a functional configuration diagram of the comment classification server in the present invention.

As shown in FIG. 2, the comment classification server 1 includes a communication interface unit 10, a comment collection unit 101, and a comment storage unit 102.
The comment collection unit 101 receives comments posted from an unspecified number of third parties from the blog server 2 via the network, and outputs the comments to the comment storage unit 102. Note that the comment collection unit 101 may collect only comments including a plurality of keywords specified in advance corresponding to a desired application.

  FIG. 3 is an explanatory diagram illustrating an example of comments accumulated in the comment accumulation unit.

  The comment accumulation unit 102 accumulates a large amount of comments collected by the comment collection unit 101. The “comment” is, for example, a Japanese “tweet” (maximum number of characters: 140 characters) transmitted by twitter (registered trademark). The comment includes at least a user id (from_user_id), a tweet ID (id_str), a transmission time (created_at), and a tweet (texts). Here, the comment may be “with position information” or “without position information”. The position information is generally latitude / longitude information acquired by GPS (Global Positioning System) in the comment poster terminal. Note that the position information is not limited to this, and may be position information based on positioning by a plurality of base stations, or position information associated with an access point.

  The comment storage unit 102 preferably divides a large number of comments into comment groups for each predetermined time range according to the transmission time. For example, it may be every hour. As a result, it is possible to know changes in the region-specific comments every hour.

  According to FIG. 2, the comment classification server 1 further includes a morphological analysis unit 110, a topic classification unit 111, a comment classification unit with position information 112, a region-specific topic number extraction unit 113, and a region-specific comment classification unit 114. And an area range clustering unit 115. These functional components are realized by executing a program that causes a computer mounted on the server to function. Further, the processing flow of these functional components can be understood as a comment classification method of the apparatus.

[Morphological analyzer 110]
The morpheme analysis unit 110 classifies a large number of comments in the comment storage unit 102 into morphemes by morpheme analysis. “Morphological analysis” refers to a technique in which sentences are divided into meaningful words and the part of speech and contents are discriminated using a dictionary. The “morpheme” means the smallest unit having meaning among the elements of the sentence. The classified morphemes are output to the topic classification unit 111.

[Topic classification unit 111]
The topic classification unit 111 classifies a large number of comments into a plurality of topic groups based on the distribution of word appearance frequencies, and assigns a topic number to each comment.

<LDA analysis of topic classification unit 111>
The topic classification unit 111 classifies each comment into any one topic group using an LDA (Latent Dirichlet Allocation) algorithm that calculates a probability (topic ratio) belonging to each topic group. Topic classification is a probabilistic model of a document generation process expressed in bag-of-words in order to analyze comments (see, for example, Non-Patent Document 2).

  LDA introduces a probabilistic framework based on fluctuations in a feature vector of a word to a technique (LSI (latent Semantic Indexin)) for dimensional compression of a word document matrix. The compressed set of dimensions is called a topic.

The topic classification unit 111 executes processing in the following steps.
(S1) From a large number of comments, the appearance frequency (number of appearances) for each word is input to the LDA process. And the appearance frequency of each word is counted for every comment.
(S2) Next, the word distribution for each topic and the topic ratio for each comment are acquired. This topic ratio classifies the topic group to which the comment belongs. And the appearance frequency of each word contained in all the comments is counted for every topic group.
(S3) Finally, the words belonging to each topic group are counted for each comment. Then, the comments are classified into topic groups with high count values.

  FIG. 4 is an explanatory diagram showing the classification of comments in the topic classification unit.

According to FIG. 4, the topic classification unit 111 classifies the large number of comments in FIG. 3 into three topic groups as a result. Each comment is given a classified topic number.
The comments with position information are output to the comment classification section with position information 112 and the region-specific comment classification section 114.
In addition, the comment without position information is output to the region-specific comment classification unit 114.

<Predicate term structure analysis of topic classifier 111>
It is also preferable that the topic classifying unit 111 further analyzes the predicate term structure of the comment with position information and the comment without position information, and removes the comment including the hearing estimation expression.

  “Predicate term structure” refers to a noun phrase or the like that becomes a “term” to a predicate in a sentence. By using the predicate term structure, it is possible to grasp the skeleton of the meaning of the sentence. The predicate term structure is composed of “object” for “predicate” and its case. As the predicate term structure analysis, for example, a predicate term structure analyzer such as Syncha which is free software can be used.

Instead of expressing a document as a bug of words (BOW), topic classification is performed using a predicate term structure. The following predicate term structure is acquired from each example sentence.
For example, a comment is analyzed as follows.
Comment 1 “Helicopters are gathering”
-> Predicate “Gather (situation)”, Ga ’s “Helicopter”
Comment 2 “Helicopters are gathering”
-> Predicate “Gather (situation)”, Ga ’s “Helicopter”
Comment 3 “Helicopters are gathering”
-> Predicate “Gather (hearing)”, Ga ’s “Helicopter”
Comment 4 “Helicopters seem to be gathering”
-> Predicate “Gather (estimate)”, Ga ’s “Helicopter”

  According to the above-described example, even if the same predicate is “gather”, it is possible to distinguish between “gather (situation)” of comments 1 and 2 and “gather (hearing / estimation)” of comments 3 and 4. . If BOW, which is an existing topic classification, is used, the feature quantities of these comments are the same and cannot be analyzed separately. That is, according to the present invention, the comments 1 and 2 and the comments 3 and 4 can be distinguished and classified into topics, and different topic numbers are assigned.

  According to the predicate term structure analysis, the comments 1 and 2 can be analyzed as situation expressions posted from the local range of the site. In addition, the comments 3 and 4 can be analyzed to be hearing estimation expressions posted from positions other than the area range of the site. In this case, the comments 3 and 4 including the hearing estimation expression are removed. As a result, only comments (regardless of position information or without position information) posted from the local area of the site can be associated on the map.

[Comment classification unit 112 with position information]
The comment classification unit 112 with position information classifies a plurality of comments with position information for each predetermined area range to which the position information belongs.

  FIG. 5 is an explanatory diagram showing comments with position information and region-specific topic numbers.

  According to FIG. 5, the position information of the comments with position information is plotted on the map. The map is divided in advance for each predetermined area range. As a result, the comments with position information included in the predetermined area range are classified.

[Regional topic number extraction unit 113]
The region-specific topic number extraction unit 113 extracts a topic number having the largest number as a region-specific topic number among a plurality of comments with position information, using a predetermined criterion based on the uneven distribution degree for each predetermined region range. Here, the predetermined criterion based on the uneven distribution degree may be, for example, Akaike's Information Criterion (AIC).

  Here, the topic numbers of the comments with position information assigned to the area ranges C1, C2,... Are t1, t2,. In the following, a calculation method of an index Info (C, t) indicating whether the topic number t is useful for discrimination of the area range C will be described.

(S1) The following four types of frequencies are obtained from the set of comments U included in the area range.
n11 = number of topic numbers j in region range i
n12 = Number other than topic number j in region range i
n21 = number of topic numbers j outside the regional range i
n22 = Number other than topic number j outside region range i

(S2) Next, with respect to n11, n12, n21, and n22, the value MLL_IM (C, t) for the independent model and the value MLL_DM (C, t) for the dependent model are calculated using the Akaike information criterion. This calculates an unreasonable ratio for each set of area range and topic number.
MLL_IM (C, t) = (n11 + n12) log (n11 + n12)
+ (N11 + n21) log (n11 + n21)
+ (N21 + n22) log (n21 + n22)
+ (N12 + n22) log (n12 + n22) -2N log N
MLL_DM (C, t) = n11 log n11 + n12 log n12 + n21 log n21 + n22 log n22−N log N
However, N = n11 + n12 + n21 + n22

(S3) The following Info (C, t) is calculated from the aforementioned MLL_IM (C, t) and MLL_DM (C, t).
AIC_IM (C, t) = -2 × MLL_IM (C, t) + 2 × 2
AIC_DM (C, t) = -2 × MLL_DM (C, t) + 2 × 3
Info (C, t) = AIC_IM (C, t) − AIC_DM (C, t)

  Info (C, t) calculated above represents an unreasonable proportion in which the topic number t appears with a bias toward the regional range C. In Info (C, t), according to the Akaike information criterion, the value of Info (C, t) increases as the topic number t is useful for discrimination of the region range C. According to the present invention, m topic numbers t can be extracted in descending order of Info (C, t) for each area range Ci.

[Region specific comment classification unit 114]
The region-specific comment classification unit 114 classifies the comment with position information and the comment without position information as region-specific comments based on the region-specific topic number of each region range.

  FIG. 6 is an explanatory diagram that clearly shows comments classified by region-specific topic numbers.

  According to FIG. 6, for example, it is assumed that the number of topic numbers 1 is the largest among comments with position information located in the area range a. In this case, it can be estimated that the comment of the topic number 1 is posted from within the area range a. Thereby, the comment with the position information and the comment without the position information of the topic number 1 can be classified as the region-specific comments in the region range a.

[Region Range Clustering Unit 115]
If the region-specific topic numbers in both adjacent region ranges are the same, the region range clustering unit 115 clusters both region ranges and integrates the region-specific comments as one region range.

  The area range may be a mesh section based on latitude and longitude, or may be a municipal area based on a place name. These regional ranges are geographically hierarchized and can be integrated in a cluster from a lower regional range to a higher regional range (lattice structure). When the Akaike information criterion is used, the information amounts n11, n12, n21, and n22 of the lower regional range have already been calculated. Therefore, even when the regional ranges are simply integrated, it is possible to calculate the topic numbers of the higher regional ranges simply by integrating these information amounts.

  According to FIG. 2, the comment classification server 1 further includes a period comment storage unit 116, a frequent word extraction unit 117, a constant topic number extraction unit 118, and a constant comment deletion unit 119. These functional components are also realized by executing a program that causes a computer mounted on the server to function. Further, the processing flow of these functional components can be understood as a comment classification method of the apparatus.

[Period comment storage unit 116]
The period-specific comment storage unit 116 stores region-specific comments for each predetermined period. An area-specific comment based on each area topic number in each area range is stored as a predetermined period, for example, in units of one day.

[Frequent word extraction unit 117]
The frequent word extraction unit 117 extracts a frequent word that appears more than a predetermined number (or a predetermined ratio) using a predetermined criterion based on the degree of uneven distribution in a region-specific comment based on each region-specific topic number in each region range. To do. The predetermined criterion based on the degree of uneven distribution in the frequent word extraction unit 117 may be an Akaike information amount criterion. In the following, a calculation method of an index Info (C, t, w) indicating whether or not the word w is useful for discrimination of the area range C and the topic number t will be described.

(S1) For a word w that appears ubiquitously in the posted text in the area range Ci and topic number j,
The following four types of frequencies are obtained.
m11 = In the posting of topic number j in the regional range Ci,
Number of posts containing the word w
m12 = In a post other than topic number j in regional area Ci,
Number of posts containing the word w
m21 = In the post with topic number j in the regional range Ci,
Number of posts without word w
m22 = In a post other than topic number j in regional range Ci,
Number of posted sentences not including the word w This frequency is calculated, for example, every day according to a predetermined period.

(S2) For m11, m12, m21, and m22, the value MLL_IM (C, t, w) for the independent model and the value MLL_DM (C, t, w) for the dependent model are calculated using the Akaike information criterion. . This calculates an unreasonable ratio for each set of area range and topic number.
MLL_IM (C, t, w) = (m11 + m12) log (m11 + m12)
+ (M11 + m21) log (m11 + m21)
+ (M21 + m22) log (m21 + m22)
+ (M12 + m22) log (m12 + m22) -2M log M
MLL_DM (C, t, w) = m11 log m11 + m12 log m12 + m21 log m21 + m22 log m22−M log M
However, M = m11 + m12 + m21 + m22

(S3) The following Info (C, t, w) is calculated from the aforementioned MLL_IM (C, t, w) and MLL_DM (C, t, w).
AIC_IM (C, t, w) =-2 × MLL_IM (C, t, w) + 2 × 2
AIC_DM (C, t, w) =-2 × MLL_DM (C, t, w) + 2 × 3
Info (C, t, w) = AIC_IM (C, t, w) −AIC_DM (C, t, w)

  The Info (C, t, w) calculated above represents an unreasonable proportion in which the word w appears unevenly in the posted text posted in the area range C with the content of the topic number t. In Info (C, t, w), according to the Akaike information criterion, the value of Info (C, t, w) increases as the word w is more useful in determining the region range C and topic number t. According to the present invention, for each area range Ci and topic number t, l words w can be extracted in descending order of Info (C, t, w).

[Constant topic number extraction unit 118]
The constant topic number extraction unit 118 uses the frequent word extraction unit 117 to compare the frequent words in the current period with the frequent words in a predetermined past period, and extract the same frequent words for a predetermined ratio or more. The topic number is determined as a permanent topic number. As a user, as the posted sentence including the word w based on the region range C is an unexpected posted sentence, it is more necessary to browse and can be regarded as news. On the other hand, it is possible to grasp that a permanent post is less necessary to browse.

Whether or not the word w satisfying w∈W (C, t, today) has appeared in the past n days ago is expressed as follows.
Exist (day number indicating n days ago, today's day number, C, t, w)
= 0 (if Info (C, t, w) n days ago is less than the predetermined threshold θ), or
= 1 (If Info (C, t, w) n days ago is greater than or equal to the predetermined threshold θ)
As
E (n) = Σw {Exist (Day number n days ago (= n), Today's day number (= 0), C, t, w)
× Info (C, t, w)} ÷ Σw {Info (C, t, w)}
0.0 ≦ E (n) ≦ 1.0

E (n) approaches 1.0 when the ratio of the word w appearing in the posted text of today's unevenly distributed topic is high in the posted text of the same area range and the same topic (topic number) appearing n days ago.
Eval = Σn E (n) / {number of n types}
Eval determines whether the topic t is constant in the area range C. Specifically, if the value of Eval is equal to or greater than a predetermined threshold, it is determined that the topic t in the area range C is constant.

[Permanent comment deletion unit 119]
The permanent comment deletion unit 119 deletes a comment based on the permanent topic number in the region-specific comments.

For example, in Narita, the number of posts based on topic number 17 on April 8, 2014 is as follows.
Number of posts with topic number = 17 posted from Narita City: 26 Number of posts with topic number = 17 posted from Narita City: 68 Number of posts with topic number = 17 posted from outside Narita City: 756 Number of posts other than topic number = 17 posted from other than Narita City: 28,677 cases. For example, the word “airport” is calculated as follows.
m11 = In the post with topic number 17 in Narita,
Number of posts containing the word “airport”
m12 = In posts other than topic number 17 in Narita City,
Number of posts containing the word “airport”
m21 = In the post with topic number 17 in Narita City,
Number of posts without the word “airport”
m22 = In posts other than topic number 17 in Narita City,
Number of posts that do not include the word “airport” And, in the post based on topic number 17 in Narita City on April 8, 2014, the frequent words = “Airport” “Narita” “Airplane” “International” are extracted Suppose that It is assumed that these frequent words w are extracted as frequent words even in a predetermined past period.
In this case, the Eval value is relatively large for these words w appearing in the posted text of topic number 17 in Narita. As a result, the posted message with topic number 17 in Narita is determined as a permanent topic number, and these posted messages are deleted.

  FIG. 7 is a sequence diagram in the present invention.

According to FIG. 7, processing of the comment classification server that collects a large number of comments in which comments (sentences) with position information and comments (sentences) without position information are mixed and extracts region-specific comments (device comment classification method) Represents.
(S111) First, the comment classification server 1 classifies a large number of comments into a plurality of topic groups based on the distribution of word appearance frequencies, and assigns topic numbers to the comments (topics in FIG. 2 described above). Classification unit 111).
(S112) Next, the comment classification server 1 classifies a plurality of comments with position information for each predetermined area range to which the position information belongs (see the comment classification section with position information 112 in FIG. 2 described above).
(S113) Next, the comment classification server 1 extracts, as a region-specific topic number, a topic number that is the largest among a plurality of comments with position information using a predetermined criterion based on the uneven distribution degree for each predetermined region range. (See the above-mentioned region-specific topic number extraction unit 113 in FIG. 2).
(S114) Then, the comment classification server 1 classifies the comment with position information and the comment without position information as the area-specific comment based on the area-specific topic number of each area range (area-specific comment classification in FIG. 2 described above). Part 114). Here, a plurality of regional ranges may be integrated and clustered.
Then, the terminal 4 operated by the viewer accesses the comment classification server 1 via the access network and the Internet. The terminal 4 can receive an area-specific comment of the area range from the comment classification server 1 by transmitting the area range as a query to the comment classification server 1.

(S116) The region-specific comments are stored for each predetermined period (see the period-specific comment storage unit 116 in FIG. 2 described above).
(S117) Of the region-specific comments based on each region-specific topic number in each region range, a frequent word that appears more than a predetermined number (or a predetermined ratio) is extracted using a predetermined criterion based on the uneven distribution degree (described above) (See the frequent word extraction unit 117 in FIG. 2).
(S118) The frequent words in the current period are compared with the frequent words in a predetermined past period, and the region-specific topic number from which the same frequent words are extracted by a predetermined ratio or more is determined as the permanent topic number (see the above-described figure). 2 constant topic number extraction unit 118).
(S119) The comment based on the permanent topic number is deleted from the region-specific comments (see the above-mentioned permanent comment deletion unit 119 in FIG. 2).

  As described above in detail, according to the comment classification program, server, and method of the present invention, it is possible to extract region-specific comments from a large number of comments in which comments with position information and comments without position information are mixed. . In particular, according to the present invention, even if a comment does not necessarily include a word specifying a position such as a place name or a landmark, a topic that is unevenly distributed in a region can be detected based on the topic classification.

  In addition, according to the present invention, it is possible to extract only a sudden posted sentence that occurs in the area range. In other words, it is possible to delete a posting message that is a constant topic in the area range. As a result, for example, a posted text related to a disaster or the like that occurred suddenly is easily transmitted to the user.

  In addition, when the fire broke out in a certain area, the inventors of the present application collected a large number of twitter comments posted during that time zone. At this time, out of approximately 26,000 comments including the word “fire”, there were only 54 comments with location information. The present invention was applied, and LDA topic classification was executed for these approximately 26,000 comments, and the same comments as the 54 topic numbers of the comments with position information were associated with the position information. As a result, even comments without position information could be associated with actual fire site positions.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Comment classification | category server 10 Communication interface part 101 Comment collection part 102 Comment accumulation | storage part 110 Morphological analysis part 111 Topic classification part 112 Comment classification part with a positional information 113 Area specific topic number extraction part 114 Area specific comment classification part 115 Area range clustering part 116 Comment storage unit by period 117 Frequent word extraction unit 118 Permanent topic number extraction unit 119 Permanent comment deletion unit 2 Miniblog server 3 Contributor terminal 4 Reader terminal

Claims (12)

A comment classification program that causes a computer to function to extract region-specific comments from a large number of comments in which comments with position information (sentences) and comments without position information (sentences) are mixed,
A topic classification means for classifying a large number of comments into a plurality of topic groups based on the distribution of word frequencies, and assigning a topic number to each comment;
Comment classification means with position information for classifying a plurality of comments with position information for each predetermined area range to which the position information belongs;
An area-specific topic number extracting means for extracting the most topic number as an area-specific topic number among a plurality of comments with position information using a predetermined criterion based on the uneven distribution degree for each predetermined area range;
A comment classification program that causes a computer to function as an area-specific comment classification unit that classifies comments with position information and comments without position information as area-specific comments based on the area-specific topic number of each area range. The topic classification means causes the computer to function so as to classify each comment into any one topic group using an LDA (Latent Dirichlet Allocation) algorithm that calculates a probability (topic ratio) belonging to each topic group. The comment classification program according to claim 1, wherein: The comment classification program according to claim 1 or 2, wherein the computer functions so that the predetermined criterion based on the degree of uneven distribution in the region-specific topic number extraction unit is an Akaike information amount criterion. Comment storage means by period for storing the region-specific comments for each predetermined period;
A frequent word extracting means for extracting frequent words that appear more than a predetermined number (or a predetermined ratio) using a predetermined criterion based on the uneven distribution degree in a region-specific comment based on each region-specific topic number of each region range;
A constant topic number extracting means for comparing a frequent word in a current period with a frequent word in a predetermined past period and determining, as a constant topic number, a region-specific topic number from which the same frequent word is extracted by a predetermined ratio or more. ,
The comment according to any one of claims 1 to 3, wherein the computer is further caused to function as a permanent comment deletion unit that deletes a comment based on the permanent topic number among the region-specific comments. Classification program. 5. The comment classification program according to claim 4, wherein the computer is caused to function such that the predetermined criterion based on the uneven distribution degree in the frequent word extracting means is the Akaike information amount criterion. 6. The topic classifying unit analyzes a predicate term structure of a comment with position information and a comment without position information, and causes the computer to function so as to remove a comment including a hearing estimation expression. The comment classification program according to item 1. If the region-specific topic numbers in the two adjacent region ranges are the same, cluster the both region ranges to further function the computer as a region range clustering unit that integrates region-specific comments as one region range. The comment classification program according to any one of claims 1 to 6, wherein: The computer according to any one of claims 1 to 7, wherein the computer functions so that the comment is posted to a mini blog (mini Web log) server by an unspecified number of third parties. The comment classification program described. A comment classification server that collects a large number of comments in which comments (sentences) with location information and comments (sentences) without location information are mixed, and extracts region-specific comments,
A topic classification means for classifying a large number of comments into a plurality of topic groups based on the distribution of word frequencies, and assigning a topic number to each comment;
Comment classification means with position information for classifying a plurality of comments with position information for each predetermined area range to which the position information belongs;
An area-specific topic number extracting means for extracting the most topic number as an area-specific topic number among a plurality of comments with position information using a predetermined criterion based on the uneven distribution degree for each predetermined area range;
A comment classification server, comprising: a region-specific comment classification unit that classifies comments with position information and comments without position information as region-specific comments based on the region-specific topic number of each region range. Comment storage means by period for storing the region-specific comments for each predetermined period;
A frequent word extracting means for extracting frequent words that appear more than a predetermined number (or a predetermined ratio) using a predetermined criterion based on the uneven distribution degree in a region-specific comment based on each region-specific topic number of each region range;
A constant topic number extracting means for comparing a frequent word in a current period with a frequent word in a predetermined past period and determining, as a constant topic number, a region-specific topic number from which the same frequent word is extracted by a predetermined ratio or more. ,
The comment classification server according to claim 9, further comprising: a permanent comment deleting unit that deletes a comment based on the permanent topic number in the region-specific comment. A comment classification method for a device that collects a large number of comments in which comments (sentences) with location information and comments (sentences) without location information are mixed, and extracts region-specific comments,
A first step of classifying a number of comments into a plurality of topic groups based on a distribution of word appearance frequencies, and assigning a topic number to each comment;
A second step of classifying a plurality of comments with position information for each predetermined area range to which the position information belongs;
A third step of extracting the largest topic number as a region-specific topic number among a plurality of comments with position information using a predetermined criterion based on the degree of uneven distribution for each predetermined region range;
A comment classification method for an apparatus, comprising: a fourth step of classifying comments with position information and comments without position information as area-specific comments based on the area-specific topic number of each area range. A fifth step of storing the region-specific comments for each predetermined period;
A sixth step of extracting frequent words appearing more than a predetermined number (or a predetermined ratio) using a predetermined criterion based on the uneven distribution in the region-specific comments based on each region-specific topic number of each region range;
A seventh step of comparing the frequent words of the current period with the frequent words of a predetermined past period, and determining, as a permanent topic number, a region-specific topic number from which the same frequent words are extracted by a predetermined ratio or more;
The comment classification method according to claim 11, further comprising an eighth step of deleting a comment based on the permanent topic number among the region-specific comments.

Images