CN107273496B - A detection method for regional emergencies in Weibo network - Google Patents

Abstract

The invention discloses a detection method of a regional emergency of a microblog network, which comprises the steps of (1) acquiring a regional microblog from a microblog network to obtain a microblog set P L MB, preprocessing the microblog to obtain a microblog set L MB, (2) extracting emergency words from the microblog set L MB to obtain an emergency word set EW, and (3) clustering the emergency words in the EW to obtain an emergency word cluster EWC { EWC ═ E } E₁，ewc₂，…，ewc_qSuppose there are q word clusters. The method provided by the invention calculates the burst value of the word by using 4 types of indexes of word frequency, word-associated users, word distribution regions and word social behaviors, more reasonably utilizes the burst characteristics of microblog network words, and is more suitable for detecting the microblog network region emergency.

Description

A detection method for regional emergencies in Weibo network

technical field

The invention relates to an information mining technology, in particular to a method for detecting regional emergencies in a microblog network.

Background technique

As a real-time and highly interactive social media, Weibo provides a platform for users to freely publish content and exchange information. Many events that happened in reality were first revealed on Weibo, and then the traditional mainstream media reported them, such as the Boston bombing in 2013, the death of Margaret Thatcher, and so on. Weibo-oriented event detection has become a research hotspot in the field of event detection recently.

Since many contents of Weibo contain regional information, including the location mentioned in the blog post, the registration location of the user who published the blog post, and the geotags attached to the blog post, etc., localized event detection for Weibo has become an emerging trend. research direction. This type of event detection has a basic assumption, that is, when no events occur in the local domain, users rarely discuss such events, and once they occur, there will be a lot of discussions, such as fires, explosions, floods, and traffic accidents in the region. , pollution, disease transmission, etc. This is very different from the global event detection of social media. The global event detection does not consider regional characteristics, and faces the entire information flow of the media, which not only requires a large amount of analysis work, but also may ignore local For regional hot events, the existing event detection methods are difficult to directly apply to regional event detection.

Proceedings published in the United States in 2010: 19th International World Wide Web Conference, 2010, titled: Earthquake shakes Twitter users: real- time eventdetection by social sensors), the authors are Takeshi Sakaki, Makoto Okazaki, YutakaMatsuo, this article simulates each Twitter user as a node in the wireless sensor network, and the process of users publishing earthquake-related blog posts is abstracted into the wireless sensor network. The node publishes the information behavior it has collected, and then confirms whether the earthquake occurs through the time and space model of the blog post and subsequent filtering processing. However, this method needs to manually design some query input items, which is difficult to apply to the detection of unconventional emergencies.

The journal published in China in 2016: Modern Library and Information Technology, with the title: Detection and Analysis of Weibo Events Based on Geographic Coordinates, by Li Jinhua and An Zhongjie The characteristics of Weibo are constructed from five indicators: number, user activity and mobile intensity. When this method detects microblog emergencies, the characteristics of social media such as microblogs are not comprehensive, including the frequency of sudden words, regional suddenness, etc., and no specific information is given when calculating each indicator. The calculation method (including formal formulas, etc.).

Proceedings published in the United States in 2016: 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, titled: GeoBurst: Real-time Monitoring of Regional Events from Geo-Tagged Twitter Streams (GeoBurst: Real-Time Local Event Detection in Geo-Tagged Tweet Streams), the authors are Zhang Chao, ZhouGuangyu, Yuan Quan, Zhuang Honglei, Zheng Yu, Kaplan Lance, Wang Shaowen, HanJiawei, this article first identifies some important microblogs in the query window As the central pivot point (Pivots), emergent events are further obtained by comparing with historical data in space and time. This method is based on the microblog text information. Because microblogs are relatively short and the language is not standardized, it is difficult to extract effective features directly from some short single microblog texts.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to aim at the deficiencies of the prior art, and to provide a new detection method of microblog network regional emergencies, which more reasonably utilizes the sudden characteristics of microblog network words and is more suitable for microblog Detection of regional emergencies in the blog network.

The technical problem to be solved by the present invention is achieved through the following technical solutions. The invention provides a detection method for microblog network regional emergencies, which is characterized in that the specific steps are as follows:

A. Collect regional microblogs from the microblog network, obtain the microblog set PLMB, and obtain the microblog set LMB after preprocessing the microblog;

B. Extract the burst words from the microblog set LMB to obtain the burst word set EW;

C. Cluster the emergent words in EW to obtain the emergent event word cluster EWC={ewc ₁ ,ewc ₂ ,...,ewc _q }, assuming that there are q word clusters;

In the step A of the method of the present invention, collecting regional microblogs from the microblog network, and obtaining the microblog set LMB after preprocessing, preferably adopts the following specific steps:

A1. Use the collection tool to obtain the localized microblog information set PLMB={plmb ₁ , plmb ₂ , ..., plmb _m }, where plmb _i (1≤i≤m) is each regional microblog; m represents the regional microblog the number of bars;

A2. Preprocess the microblog set PLMB, remove the link URL and emoticon information in the microblog, remove the microblogs whose length is less than 5 words, and obtain the preprocessed microblog set LMB, LMB={lmb ₁ , lmb ₂ , ..., lmb _n }, where lmb _i (1≤i≤n) is each regional microblog.

Extracting burst words from the microblog set LMB as described in step B of the method of the present invention to obtain burst word set EW, the preferred specific steps are as follows:

B1. Perform word segmentation on each microblog lmb _i (1≤i≤n) in LMB, n represents the number of microblogs, remove stop words, keep nouns, verbs, place names, person names, proper nouns, and get the final The set of words is LMBW={w ₁ , w ₂ ..., w _r }, assuming there are r words;

其中，分子

为词w_i在k时间点出现的频率，分母中的

B2. Calculate the frequency burstiness of word _wi (1≤i≤r), assuming that the time point of the current emergency event detection is k, select the historical data of the previous p moments as a reference, and the word _wi is at the k time point The frequency burstiness of is defined as:

Among them, the molecule

is the frequency of word _wi at time k, where in the denominator

其中，分子

为k时间点，提及到词w_i的不同用户数量，分母中的

B3. Calculate the associated user emergencies of word _wi (1≤i≤r), assuming that the current emergency detection time point is k, select the historical data of the previous p moments as a reference, and the word _wi is at time k The associated user burstiness of a point is defined as:

Among them, the molecule

is the k time point, the number of different users who mentioned the word _wi , in the denominator

其中，分子

为k时间点，提及到词w_i的不同地理标签的数量，分母中的

B4. Calculate the regional burstiness of word _wi (1≤i≤r), and the regional burstiness of word _wi at time k is defined as:

Among them, the molecule

for k time points, the number of distinct geotags that mention word _wi , in the denominator

其中，分子

为k时间点，提及到词w_i的微博的转发数、评论数和阅读数之和，分母中的

B5. Calculate the suddenness of social behavior of word _wi (1≤i≤r). The suddenness of social behavior of word _wi at time k is defined as:

Among them, the molecule

is the sum of the number of retweets, comments and readings of the microblogs mentioning the word _wi at time point k, the denominator

B6. Synthesize the four bursts of steps B2, B3, B4, and B5, and finally obtain the burst value of a word _wi at time k: BurstyScore( _wi )=α*F( _wi )+β* U(u|w _i )+χ*GT(gt|w _i )+δ*SB(sb|w _i ), where α, β, χ, and δ are adjustment coefficients, which are used to adjust the weights of the four types of indicators, α+β+χ+δ=1, α≥0, β≥0, χ≥0, δ≥0;

B7. After calculating the burst value of each word, use the quartile difference to select n burst words to form a burst word set EW. The distance calculation method of the quartile difference is: IQS(EW)=Q ₃ (EW)-Q ₁ (EW). When the burst value of a word is greater than a certain threshold, it is regarded as a burst word, and the calculation method of the threshold is: maxima(EW)=Q ₃ (EW)+1.5×IQS(EW).

In the method of the present invention, in the method for detecting emergencies in microblog network regions, in the step C, the emergent words in the EW are clustered to obtain the emergent event word cluster EWC={ewc ₁ ,ewc ₂ , ..., ewc _q }, the preferred specific steps are as follows:

C1. Based on the burst feature set EW obtained in step B, construct a burst word association network EWN=(V, E), where V is the burst word set EW, and E represents the strength of association between burst words. The association strength of sudden words ew _i and ew _j is to count the number of co-occurrences of the two words in the same Weibo post;

C2. After the construction of the EWN of the burst word association network is completed, use the open source CLUTO toolkit to cluster the EWN to obtain the burst word cluster EWC={ewc ₁ ,ewc ₂ ,...,ewc _q }, assuming that there are q words cluster.

Compared with the prior art, the present invention proposes a comprehensive index for event detection using the characteristics of the microblog network, and proposes four types of indicators, including word frequency, word-related users, word distribution area, and word social behavior, to calculate the prominence of words. It makes more reasonable use of the sudden characteristics of Weibo network words, and is more suitable for the detection of regional emergencies in Weibo network. And gives the specific calculation method, which has great practical value.

Description of drawings

Fig. 1 is a kind of flow chart of the microblog network regional emergency detection method of the present invention;

Fig. 2 is the flow chart of collecting regional microblogs from the microblog network described in step 101 in Fig. 1, obtaining the microblog set PLMB, and obtaining the microblog set LMB after preprocessing the microblog;

Fig. 3 is described in step 102 in Fig. 1 from micro-blog set LMB to extract burst word, obtains the flow chart of burst word set EW;

FIG. 4 is a flowchart of clustering the emergent words in the EW described in step 103 in FIG. 1 to obtain the emergent event word cluster EWC={ewc ₁ , ewc ₂ , . . . , ewc _q }.

Detailed ways

The implementation process of the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

Referring to Fig. 1, a detection method of microblog network regional emergencies, the method comprises the following steps:

Step 101: Collect regional microblogs from the microblog network, obtain a microblog set PLMB, and obtain a microblog set LMB after preprocessing the microblogs. Referring to FIG. 2, the specific steps are as follows:

Step 201 , using a collection tool to obtain a localized microblog information set PLMB={plmb ₁ , plmb ₂ , . . . , plmb _m }, where plmb _i (1≤i≤m) is each regional microblog. After applying for developer permission on Weibo, you can obtain dynamic Weibo information around a certain location by calling different interfaces in the API. Calling the location service interface can obtain the returned Weibo content, number of retweets, number of comments, number of likes, user information, check-in location, etc.

Step 202: Preprocess the microblog set PLMB, remove the link URL and emoticon information in the microblog, remove the microblogs whose length is less than 5 words, and obtain the preprocessed microblog set LMB, LMB={lmb ₁ , lmb ₂ , ..., lmb _n }, where lmb _i (1≤i≤n) is each regional microblog. In the collected regional microblogs, although targeted screening has been carried out from a large number of microblogs, there is still some interference information, which needs to be filtered to reduce the complexity of later calculation.

Step 102, extract burst words from the microblog set LMB, and obtain burst word set EW, with reference to FIG. 3, the specific steps are as follows:

Step 301, perform word segmentation on each microblog lmb _i (1≤i≤n) in the LMB, remove stop words, retain nouns, verbs, place names, personal names, proper nouns, and obtain the final word set as LMBW={ w ₁ , w ₂ , ..., w _r }, suppose there are r words. Because some verbs do not have actual meaning, such as "hold, carry out, carry out, meet", etc., further remove the stop verbs;

Step 302: Calculate the frequency burstiness of the word _wi (1≤i≤r), assuming that the time point of the current emergency event detection is k, select the historical data of the previous p moments as a reference, and the word _wi is at time k. The frequency burstiness of a point is defined as:

为词w_i在k时间点出现的频率，分母中的Among them, the molecule

is the frequency of word _wi at time k, where in the denominator

The larger F( _wi ) is, it means that at the current k time point, the frequency of word _wi appears more frequently, and it is more likely to be a burst word;

其中，分子

为k时间点，提及到词w_i的不同用户数量，分母中的

越大，说明k时间点，提及到词w_i的用户数量增势越大，词w_i越有可能是突发词；Step 303: Calculate the associated user emergencies of word _wi (1≤i≤r), assuming that the current emergency detection time point is k, select the historical data of the previous p moments as a reference, and the word _wi is at k. The associated user burstiness at a point in time is defined as:

Among them, the molecule

is the k time point, the number of different users who mentioned the word _wi , in the denominator

The larger the value, the more likely it is that the word _wi is a sudden word at the _k time point.

Step 304: Calculate the regional burstiness of word _wi (1≤i≤r), and the regional burstiness of word _wi at time point k is defined as:

为k时间点，提及到词w_i的不同地理标签的数量，分母中的

GT(w_i)越大，说明k时间点，提及到词w_i的地理标签数量增势越大，词w_i越有可能是突发词；Among them, the molecule

for k time points, the number of distinct geotags that mention word _wi , in the denominator

The larger the GT( _wi ), the greater the increase in the number of geotags referring to the word _wi at time k, and the more likely the word _wi is a sudden word;

其中，分子

为k时间点，提及到词w_i的微博的转发数、评论数和阅读数之和，分母中的

SB(w_i)越大，说明k时间点，提及到词w_i的社交行为数量增势越大，词w_i越有可能是突发词；Step 305: Calculate the suddenness of social behavior of word _wi (1≤i≤r). The suddenness of social behavior of word _wi at time k is defined as:

Among them, the molecule

is the sum of the number of retweets, comments and readings of the microblogs mentioning the word _wi at time point k, the denominator

The larger the SB( _wi ), the greater the increase in the number of social behaviors referring to the word _wi at time k, and the more likely the word _wi is a sudden word;

Step 306 , synthesizing the four bursts of the above words, finally obtain the burst value of a word _wi at time k: BurstyScore( _wi )=α*F( _wi )+β*U(u|w _i )+χ*GT(gt|w _i )+δ*SB(sb|w _i ), where α, β, χ, and δ are adjustment coefficients, which are used to adjust the weights of the four types of indicators, α+β+χ +δ=1, α≥0, β≥0, χ≥0, δ≥0. The larger the BurstyScore( _wi ), the greater the burstiness of the word _wi at time k, and the more likely the word _wi is a burst word;

Step 307: After calculating the burst value of each word, select n burst words by using the quartile difference to form a burst word set EW. The distance calculation method of the quartile difference is: IQS(EW)=Q ₃ (EW)-Q ₁ (EW). When the burst value of a word is greater than a certain threshold, it is regarded as a burst word, and the calculation method of the threshold is: maxima(EW)=Q ₃ (EW)+1.5×IQS(EW).

Step 103: Cluster the emergent words in the EW to obtain the emergent event word cluster EWC={ewc ₁ , ewc ₂ ,..., ewc _q }, referring to FIG. 4 , the specific steps are as follows:

Step 401 , based on the burst feature set EW, construct a burst word association network EWN=(V, E), where V is the burst word set EW, and E represents the strength of association between burst words. The association strength of sudden words ew _i and ew _j is to count the number of co-occurrences of the two words in the same Weibo post;

Step 402: After the construction of the emergency word association network EWN is completed, use the open source CLUTO toolkit to cluster the EWN to obtain the emergency word cluster EWC={ewc ₁ ,ewc ₂ ,...,ewc _q }, assuming that there are q word clusters. CLUTO provides three clustering algorithms, which can either directly cluster on the feature space of the clustered objects, or cluster according to the similar space of the objects. These algorithms are slice-based, agglomerative-based, and graph-slice-based. In practical applications, agglomeration-based hierarchical clustering methods are often used, so the present invention selects agglomerative hierarchical clustering methods.

Comparative Example: Using three different Weibo network regional emergency detection methods to compare the effectiveness of regional emergency detection. The three methods are as follows:

(1) Method 1-HBED, select the hashtag contained in the microblog, and represent the hashtag as a vector pattern, the weight of the word is calculated by TF-IDF, and the number of microblogs contained in a cluster is considered when calculating the heat of the cluster. .

(2) Method 2-GeoBurst, firstly identify some important microblogs as central axis points in the query window, and further obtain emergencies by comparing with historical data in terms of time and space. The ordering of emergent events is based on the temporal and spatial emergencies of the words in the word cluster. Four main parameters are set: kernel function width h = 0.01, restart probability α = 0.2, random walk similarity threshold δ = 0.02, and parameter η = 0.5 for balancing spatiotemporal abruptness.

(3) Method 3-LocTBED, the method proposed by the present invention is mainly the sudden calculation of the proposed word, using the agglomerative clustering method bagglo provided by CLUTO for clustering, the number of clusters is designated as 10, and the clusters are similar The degree function is specified as the cosine function Cos. When calculating the burst value of a word, the historical investigation time of the word is set to one week (7 days), and the adjustment parameter α=β=χ=δ=0.25 when the four types of indicators are accumulated.

The present invention takes the real social media-Sina Weibo as an example, and collects Weibo with geographic tags in two cities of Beijing and Lianyungang City, Jiangsu Province. The information collection time in Beijing is from December 1st to December 30th, 2016. A total of 346,863 microblogs with geotags were collected from 2016 (1 month data), and the information collection time of Lianyungang City was from May 1, 2016 to October 31, 2016 (half-year data), and a total of 63,744 microblogs were collected. Geo-tagged tweets. Verifies the effectiveness of various event detection methods on a daily basis, that is, detects regional emergencies on a specified day.

Since the daily regional emergencies in each city are unknown, the accuracy rate P@n is used as the evaluation index with reference to the existing mainstream research methods. For Top-k emergencies detected every day, it is manually determined whether the detected events are regional emergencies. Since the number of events detected by Top-k is small, the workload of manual evaluation is not complicated.

The results obtained by the three methods on the five evaluation indicators are shown in Table 1.

Table 1. The detection results of 5 methods on 5 evaluation indicators

Methods P@1 P@2 P@3 P@4 P@5 Average HBED 0.20 0.30 0.20 0.30 0.24 0.24 GeoBurst 0.80 0.70 0.80 0.75 0.72 0.72 LocTBED 0.80 0.80 0.87 0.80 0.76 0.76

Comparing the three methods, the method proposed in this paper, LocTBED, has the most ideal effect, and the average value obtained on the five evaluation indicators is 0.76. Next is GeoBurst, with an average of 0.72 on the five evaluation metrics. Although the values obtained by these two methods are relatively close, there is a big difference in the ordering of the emergent events in the detection results obtained by the two methods. The method LocTBED considers the number of regional words contained in the cluster when calculating the popularity of emergency event clusters, which is of great help in detecting regional emergencies.

The main reason for the deviation of the effect of the method HBED is that in the obtained geo-tagged micro-blogs, the number of micro-blogs with hashtags is relatively small, and most of them are wide-area events, which are not applicable to the detection of regional events.

The method described in the present invention is not limited to the examples described in the specific implementation manner, and other embodiments obtained by those skilled in the art according to the technical solutions of the present invention also belong to the technical innovation scope of the present invention.

Claims (3)

1. a detection method for microblog network regional emergencies, is characterized in that, its concrete steps are as follows: A. Collect regional microblogs from the microblog network, obtain the microblog set PLMB, and obtain the microblog set LMB after preprocessing the microblog; B. Extract the burst words from the microblog set LMB to obtain the burst word set EW; C. Cluster the emergent words in EW, assuming that there are q word clusters, and obtain the emergent event word cluster EWC={ewc ₁ ,ewc ₂ ,...,ewc _q }; The specific steps of the step B are as follows: B1. Perform word segmentation on each microblog lmb _i (1≤i≤n) in LMB, n represents the number of microblogs, remove stop words, keep nouns, verbs, place names, person names, proper nouns, and get the final The word set of is LMBW={w ₁ , w ₂ , ..., w _r }, assuming there are r words; B2. Calculate the frequency burstiness of word _wi (1≤i≤r), assuming that the time point of the current emergency event detection is k, select the historical data of the previous p moments as a reference, and the word _wi is at the k time point The frequency burstiness of is defined as:

Among them, the molecule

is the frequency of word _wi at time k, where in the denominator

B3. Calculate the associated user emergencies of word _wi (1≤i≤r), assuming that the current emergency detection time point is k, select the historical data of the previous p moments as a reference, and the word _wi is at time k The associated user burstiness of a point is defined as:

Among them, the molecule

is the k time point, the number of different users who mentioned the word _wi , in the denominator

B4. Calculate the regional burstiness of word _wi (1≤i≤r), and the regional burstiness of word _wi at time k is defined as:

Among them, the molecule

for k time points, the number of distinct geotags that mention word _wi , in the denominator

B5. Calculate the suddenness of social behavior of word _wi (1≤i≤r). The suddenness of social behavior of word _wi at time k is defined as:

Among them, the molecule

is the sum of the number of retweets, comments and readings of the microblogs mentioning the word _wi at time point k, the denominator

B6. Synthesize the four bursts of steps B2, B3, B4, and B5, and finally obtain the burst value of a word _wi at time k: BurstyScore( _wi )=α*F( _wi )+β* U(u|w _i )+χ*GT(gt|w _i )+δ*SB(sb|w _i ), where α, β, χ, and δ are adjustment coefficients, which are used to adjust the weights of the four types of indicators, α+β+χ+δ=1, α≥0, β≥0, χ≥0, δ≥0; B7. After calculating the burst value of each word, use the quartile difference to select n burst words to form the burst word set EW; the distance calculation method of the quartile difference is: IQS(EW)=Q ₃ ( EW)-Q ₁ (EW); when the burst value of a word is greater than a certain threshold, it is regarded as a burst word, and the calculation method of the threshold is: maxima(EW)=Q ₃ (EW)+1.5×IQS(EW) . 2. the detection method of a kind of microblog network regional emergency according to claim 1, is characterized in that: the concrete steps of above-mentioned step A are as follows: A1. Use the collection tool to obtain the localized microblog information set PLMB={plmb ₁ , plmb ₂ , ..., plmb _m } where plmb _i (1≤i≤m) is each regional microblog; m represents the number of regional microblogs number of bars; A2. Preprocess the microblog set PLMB, remove the link URL and emoticon information in the microblog, remove the microblogs whose length is less than 5 words, and obtain the preprocessed microblog set LMB, LMB={lmb ₁ , lmb ₂ , ..., lmb _n } where lmb _i (1≤i≤n) is each regional microblog. 3. the detection method of a kind of micro-blog network regional emergencies according to claim 1, is characterized in that, the concrete steps of described step C are as follows: C1. Based on the burst feature set EW obtained in step B, construct a burst word association network EWN=(V, E), where V is the burst word set EW, and E represents the association strength between burst words; burst word ew The association strength of _i and ew _j is to count the number of co-occurrences of two words in the same Weibo post; C2. After the construction of the EWN of the burst word association network is completed, use the open source CLUTO toolkit to cluster the EWN to obtain the burst word cluster EWC={ewc ₁ ,ewc ₂ ,...,ewc _q }, assuming that there are q words cluster.

Images