CN111966786A - A Weibo Rumor Detection Method - Google Patents

Abstract

The invention provides a microblog rumor detection method, which considers the attention mechanism and comprises the following steps: collecting microblog events and corresponding comment data sets as sample data; preprocessing the sample data, and respectively extracting text contents of the original microblog and the comment; pre-training the text by adopting a BERT pre-training model, and generating a sentence vector with a fixed length for each sentence of the text; constructing a dictionary, and extracting an original microblog and a plurality of corresponding comments to form a microblog event vector matrix; training the vector matrix by adopting a deep learning method Text CNN-Attention, and constructing a multi-level training model; and carrying out classification detection on the vector matrix according to the multi-level training model to obtain a rumor detection result corresponding to the social network data. Compared with the traditional rumor detection method, the method improves the accuracy.

Description

A Weibo Rumor Detection Method

technical field

The invention belongs to the technical field of natural language processing, and in particular relates to a microblog rumor detection method.

Background technique

Rumors generally refer to unverified statements or statements, often related to an event. With the rapid development of social media, rumors can spread rapidly through social media at the speed of nuclear fission. Microblog, one of the social media, is a new type of open Internet social service in the Web2.0 era. Users can update their Weibo with short text anytime and anywhere with the help of communication media such as the Internet or mobile phones, and share information with more users. Compared with traditional blogs, microblogs have the following characteristics: instant blog sharing, innovative interactive methods, and lively on-site interpretation. In terms of communication effect, it shows: popularity accumulation, economical and fast brand marketing. However, in the diversified communication, liberalized communication content, popular disseminators and wide audiences, and diversified communication channels have promoted the spread and spread of rumors on Weibo. The spread of rumors on Weibo is mostly carried out through comments and forwarding of information between users. If false rumors are widely spread, it will have a certain negative impact on society.

The methods of rumor detection are generally divided into two categories: one is the method of machine learning based on traditional manual extraction of features, which mines features from the three aspects of rumor content, rumor users, and rumor dissemination, with factors such as emotional polarity and user influence. Rumor detection is carried out through classifiers such as Bayesian and decision trees; the other is based on deep learning methods by constructing neural networks and matching nonlinear functions to learn potential features in texts, and through neural network models such as CNN, RNN and other neural network models. Feature representation learning, and finally rumor detection through a nonlinear classifier. At present, most of the pre-training models in the research on rumor detection by constructing neural networks through deep learning use word2vec word vectors or ELMo, but the word vectors obtained in the former cannot solve the problem of polysemy, so that each word trained can only correspond to one Vector representation, which can dynamically adjust word embeddings according to context, but uses LSTM for feature extraction instead of Transformer, and ELMo uses context vector splicing as the current vector, so the fused vector features are poor. Most of the training models use CNN or RNN networks, but although the CNN network can extract semantic features, it ignores the contextual word order features, and the CNN network cannot distinguish the more obvious features when the pooled features are spliced after the full connection operation. In view of the existing challenges, the present invention proposes a new rumor detection model that considers the attention mechanism, selects the BERT pre-training model that can extract the latent features of the text in the text preprocessing, and introduces the attention mechanism into the CNN model in the training model. , which can automatically assign different weights according to the influence of events, and finally use the Softmax classifier to detect rumors.

In view of this, it is necessary to design a microblog rumor detection method to solve the above problems.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a microblog rumor detection method with high accuracy.

In order to achieve the above purpose, the present invention provides a microblog rumor detection method, comprising the following steps:

A. Collect Weibo events and corresponding comment datasets as sample data;

B. Preprocess the sample data, and extract the text content of the original Weibo and comments respectively;

C. The BERT pre-training model is used to pre-train the text, and each sentence of text generates a fixed-length sentence vector;

D. Construct a dictionary, extract the original microblog and the corresponding comments to form a microblog event vector matrix;

E. Use the deep learning method Text CNN-Attention to train the vector matrix to build a multi-level training model;

F. Classify and detect the vector matrix according to the multi-level training model, and obtain the rumor detection result corresponding to the social network data.

As a further improvement of the present invention, the sample data includes rumor sample data and non-rumor sample data.

As a further improvement of the present invention, in the step B, regular expressions are used to remove noise in the json file.

As a further improvement of the present invention, all the pre-trained texts are used for subsequent model processing according to the training data and the test data in a ratio of 4:1.

As a further improvement of the present invention, the pre-trained BERT model and code can implement the embedding of word vectors.

As a further improvement of the present invention, as a word vector model, the BERT model can fully describe the character-level, word-level, sentence-level and even the relationship between sentences, and gradually move the NLP task to the sentence vector generated by pre-training.

As a further improvement of the present invention, the BERT model proposes a pre-training target: a masked language model (MLM), which overcomes the traditional one-way limitation. The MLM target allows the representation of the context of fusion of the left and right sides, so that pre-training is possible A deep bidirectional Transformer.

As a further improvement of the present invention, the BERT model introduces the task of "predicting the next sentence", which can jointly train the representation of text pairs with MLM.

As a further improvement of the present invention, the BERT model uses sentence-level negative sampling to predict whether the texts at both ends of the input BERT are continuous; during the training process, the second paragraph of the input model will be randomly selected from all texts with a probability of 50% , the remaining 50% will select the subsequent text of the first paragraph.

As a further improvement of the present invention, the construction of the multi-level training model is composed of Text CNN and an attention mechanism; wherein, the Text CNN model uses three convolution kernels with convolution sizes of 3, 4, and 5 to be tested. The matrix performs the convolution operation to obtain different feature representations based on the vector matrix for different convolution kernels. Through the pooling operation, each convolution kernel corresponds to the input matrix to generate only one maximum feature, and then the convolution kernels of different sizes are connected through the full connection operation. The obtained feature representations are connected; the attention mechanism assigns different weights to the feature representations generated after full connection according to the influence of each feature on the output, so that the features with greater influence will have greater influence in rumor detection. .

The beneficial effects of the present invention are as follows: the microblog rumor detection method of the present invention uses the BERT pre-training model in the text preprocessing stage, using the Transformer can more efficiently capture longer-distance dependencies, and can mine deep context information, so that pre-training The resulting sentence vector has better latent features; the training model introduces an attention mechanism by assigning different weights to different features according to their influence, so that features that have a greater impact on the output result will be given more weights, Thus, it has a more important impact on the results, which is conducive to the detection of rumors and improves the accuracy of detection.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention.

in:

Figure 1 is a general flow chart of rumor detection;

Figure 2 is a schematic diagram of the structure of the BERT model;

Fig. 3 is the flow chart of the microblog rumor detection method considering the attention mechanism of the present invention;

Figure 4 is a schematic diagram of the structure of the neural network Text CNN model;

Figure 5 is a schematic structural diagram of introducing an attention mechanism;

Fig. 6 is the MATLAB simulation diagram of the experimental result of embodiment one;

FIG. 7 is a MATLAB simulation diagram of the experimental result of the second embodiment.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

A microblog rumor detection method of the present invention takes into account the attention mechanism. The overall flow of the method is shown in Figure 1, which mainly includes the following steps:

Step 1, collect Weibo events and corresponding comment data as sample data;

The sample data here includes rumor sample data and non-rumor sample data;

The label of rumor sample data is "1", and the label of non-rumor sample data is "0".

Step 2: Preprocess the sample data, and use regular expressions to extract corresponding text content;

The main purpose of preprocessing is to remove noise in the text, including non-Chinese characters, punctuation, stop words, etc. Since the sample data is stored in the json format file; the json file stores the data in the form of "key-value pairs", the data name is used as the key in the json file, and the crawled data value is used as the value in the json file. For example "text: breakfast. Do not associate, so as not to cross the province.";

All the data of a single Weibo original event is a json file, and all the data of all comments of a single event is a json file;

Use regular expressions to remove noise in the json file, and store the text content corresponding to the original microblog event and all its comments;

All texts are used for subsequent model processing according to the ratio of training data to test data 4:1.

Step 3, download the BERT pre-training model, and convert the text into the corresponding sentence vector;

The BERT model can be obtained by downloading Google's BERT pre-training model. The pre-trained Chinese BERT model and code are all from Google Research's BERT, which can implement word vector embedding. The basic structure model is shown in Figure 2;

BERT: The full name is Bidirectional Encoder Representation from Transformers, which is the bidirectional encoding representation of Transformer to improve methods based on architecture fine-tuning. As a word vector model, the BERT model can fully describe the character-level, word-level, sentence-level and even the relationship between sentences. The purpose is to gradually move the downstream NLP tasks to the sentence vectors generated by pre-training;

The BERT model includes the following features: The BERT model proposes a new pre-training objective: a masked language model (MLM), which overcomes the traditional one-way limitation. The MLM objective allows the representation to fuse the left and right contexts, thereby A deep bidirectional Transformer can be pre-trained; the BERT model introduces the "next sentence prediction" task, which can jointly train the representation of text pairs with MLM; the BERT model uses sentence-level negative sampling, and for sentence-level continuity prediction, it is the prediction input Whether the text at both ends of BERT is continuous. During training, the second paragraph of the input model will be randomly selected from all texts with a probability of 50%, and the remaining 50% will choose the subsequent text of the first paragraph.

Step 4, construct the corresponding input matrix according to the selected sentence length and sentence vector dimension;

This paper adopts the BERT base model, the number of network layers is 12, and the dimension of the trained sentence vector is 768 dimensions;

A fixed number of sentence vectors are selected from the original text of Weibo and sentence vectors corresponding to all comments to form an input matrix.

In step 5, a deep learning method is used to construct a multi-level training model of Text CNN-Attention.

Figure 3 shows a detailed flow chart of the rumor detection method considering the attention mechanism proposed by the present invention. The first layer of the model is the input layer, which is mainly composed of sentence vectors generated by the BERT pre-training model. The composition of the blog event is the corresponding random number of comments extracted from the original Weibo; followed by the convolution layer, where filters of different sizes are used to perform convolution to learn the sentence vectors of the input layer, and the results can be obtained based on different filters. feature representation. By splicing the features belonging to the same window, the feature vector of the window can be obtained, and the feature sequence can be obtained according to the different order. Each feature is given a different weight, so that the features that have a greater impact on the output result will be given more weight, which will have a more important impact on the result, and finally the output will be passed to the classifier to judge whether the event is rumored or not. .

Figure 4 shows the structure of the Text CNN model. The detailed process is as follows:

(1) For all rumor and non-rumor events in the dataset and their corresponding comments, the BERT preprocessing model is trained into sentence vectors. For each microblog event, select the corresponding number of comments under the event and the original microblog as input to the input layer, where the input layer is an m×n matrix, m is the total number of selected events, and n is The length of a single sentence vector.

(2) By using three types of filters with different sizes for convolution, the features corresponding to different filters are obtained respectively. The filter will keep sliding in the m×n input matrix. In order to facilitate feature extraction, set the filter The length of is k, and the width is the same as the width of the input matrix. The feature extracted by a filter can be expressed as h∈R ^k×n , then the feature obtained corresponding to any u in m is:

w _u =(x _u ,x _u+1 ,...,x _u-k+1 )

After the input matrix is convolved, a feature list c will be generated, and the features generated by each convolution will correspond to c: c _u =f(w _u *h+b), where f is the ReLU function, and b is Bias term.

(3) When the filter slides over the input of length m, the length of the feature list is (m-k+1). Assuming that there are q filters, q feature lists will be generated, and q is spliced to obtain a matrix :

W ₁ =[c ₁ ,c ₂ ,...,c _q ]

c _q represents the list of features produced by the qth filter. In this paper, a total of three filters of different sizes are used, and the final total matrix is:

W=[W ₁ ,W ₂ ,W ₃ ]=[c ₁ ,c ₂ ,...,c _q ,c _q+1 ,...,c _2q ,c _2q+1 ,...,c _3q ]

(4) The maximum pooling operation is performed on the features obtained by each filter to obtain the output features, and the output features of different filters are fully connected to obtain the CNN output:

W'=[c ₁₁ ,c ₂₂ ,...,c _kk ].

(5) The output of the CNN layer is weighted and summed by the attention layer to obtain the hidden layer representation of the microblog sequence. The structure diagram of introducing the attention mechanism is shown in Figure 5. Introducing the attention mechanism to the CNN network can give different weights to the hidden state sequence W' output by the CNN network, so that the model can focus on the use of the microblog sequence information when learning the representation of the microblog sequence. The attention layer takes the output c _kk of the CNN network as input, and outputs the corresponding representation v _kk of the microblog sequence,

h _i =tanh(W _A *c _kk +b _A )

Composition matrix V=[v ₁₁ , v ₂₂ ,...,v _kk ], W _A is the weight matrix, b _A is the bias value, _hi is the hidden layer representation of c _kk , α _i is the relationship between _hi and context h _A Similarity, v _i is the output vector.

(6) The output is sent to the fully connected layer, and the probability output of rumor and non-rumor is obtained through Softmax, so as to achieve the purpose of judging rumor events.

Step 6: Use the multi-level training model to train and test the input matrix to obtain the corresponding rumor detection result.

Example 1:

In order to demonstrate the effectiveness of the present invention, we selected a series of event data based on the microblogging platform organized by Ma et al. In response, we also grabbed general topic posts that were not reported as rumors and collected a similar number of rumor incidents. The detailed statistics are listed in the following table:

We divide all data according to the ratio of training set and test set 4:1. The specific division is listed in the following table:

The evaluation indicators we use to evaluate the effectiveness of the model are the accuracy rate, precision rate, recall rate and F1 value. The predicted results and the actual results are listed in the following table:

There are four baseline methods we use for comparison, namely SVM-TS, CNN-1, CNN-2, and CNN-GRU. The detailed data about the comparison between our method and the baseline method in rumor detection are listed in the following table. The resulting MATLAB simulation diagram is shown in Figure 6:

It can be seen from the table that the final accuracy rate of the traditional SVM-TS method using the classifier for rumor detection is only 85.7%, and the effect is not particularly good. Comparing the final results of the three models of GRU-1, GRU-2, and CNN-GRU, it can be found that After adding the convolutional neural network to the training model, because different latent features in the input can be extracted through the filter, it has a better performance of 95.7% in accuracy. After introducing the attention mechanism, our model will The output of CNN is given different weights as input, so that the features that have a greater impact on the output results will be given more weights, which will have a more important impact on the results, which is helpful for rumor detection. The results show that the accuracy of our model is It reaches 96.8%, and also has a good improvement in recall and F1 value.

Embodiment 2:

In order to demonstrate the feasibility of our method, we also selected another microblog dataset CED_Data set [23] for experiments, and compared the accuracy obtained by training on different training models using sentence vectors obtained from the same pre-training model. The data set contains 1538 rumor events and 1849 non-rumor events. We conduct experiments according to the ratio of training set to test set of 4:1. The experimental data is listed in the following table. The MATLAB simulation diagram of the experimental results is shown in Figure 7:

The experimental results show that the sentence vectors obtained by the BERT pre-training model will still have deviations in accuracy when trained on different training models, but the deviation is smaller than that of different pre-training models. Through experiments, it can be concluded that the accuracy rate of SVM-TS is about 86.7%, followed by GRU-1, CNN-GRU, and GRU-2 models. The best effect is our proposed CNN-Attention model, and the accuracy rate has reached 95.3%, and the effect on recall and F1 value is also the best among many models.

In summary, our model has shown the best results on two different datasets. By using the BERT pre-training model, the feature performance of the preprocessed sentence vectors can be greatly improved. The CNN model of the force mechanism can more effectively extract the latent features in the text, which is of great significance to the task of rumor detection.

The present invention mainly expounds the problem of microblog rumor event detection from the two aspects of the pre-training model and the training model, and mainly illustrates that the pre-training model will also affect the experimental results, and when some downstream NLP tasks are transferred to the pre-training model, it can be obtained Better effect; in the training model, based on the traditional Text CNN model, this paper proposes a new rumor detection model that introduces an attention mechanism, which can assign different weights to the input sentence vector according to its influence on the input, This will have a positive impact on predicting whether the event is rumored or not. This method has been experimentally verified on the real Weibo dataset and has a good rumor detection effect.

The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced. Without departing from the spirit and scope of the technical solutions of the present invention.

Claims (10)

1. a microblog rumor detection method, is characterized in that, comprises the steps: A. Collect Weibo events and corresponding comment datasets as sample data; B. Preprocess the sample data, and extract the text content of the original microblog and comments respectively; C. The BERT pre-training model is used to pre-train the text, and each sentence of text generates a fixed-length sentence vector; D. Construct a dictionary, extract the original microblog and the corresponding comments to form a microblog event vector matrix; E. Use the deep learning method Text CNN-Attention to train the vector matrix to build a multi-level training model; F. Classify and detect the vector matrix according to the multi-level training model, and obtain the rumor detection result corresponding to the social network data. 2 . The microblog rumor detection method according to claim 1 , wherein the sample data includes rumor sample data and non-rumor sample data. 3 . 3. microblog rumor detection method according to claim 1, is characterized in that: in step B, use regular expression to clear the noise in json file. 4 . The microblog rumor detection method according to claim 3 , wherein the pre-trained all texts are used for subsequent model processing according to the training data and the test data in a ratio of 4:1. 5 . 5. The microblog rumor detection method according to claim 4, wherein the pre-trained BERT model and code can realize the embedding of word vectors. 6. The microblog rumor detection method according to claim 5, characterized in that: the BERT model, as a word vector model, can fully describe character level, word level, sentence level and even the relationship features between sentences, and the NLP task is gradually Move to pre-training to generate sentence vectors. 7. microblog rumor detection method according to claim 1, is characterized in that: described BERT model proposes pre-training target: masked language model (masked language model, MLM), overcomes traditional one-way limitation, MLM target allows The representation fuses the left and right contexts so that a deep bidirectional Transformer can be pre-trained. 8 . The microblog rumor detection method according to claim 7 , wherein the BERT model introduces a task of “predicting the next sentence”, which can jointly train the representation of text pairs with MLM. 9 . 9. microblog rumor detection method according to claim 8, is characterized in that: described BERT model uses sentence-level negative sampling, predicts whether the two ends text of input BERT is continuous; In training process, the second paragraph of input model It will be randomly selected from all texts with a probability of 50%, and the remaining 50% will choose the subsequent text of the first paragraph. 10. microblog rumor detection method according to claim 1, is characterized in that: described building multi-level training model is made up of Text CNN and attention mechanism two parts; Wherein, Text CNN model uses three convolution sizes to be respectively. The convolution kernels of 3, 4, and 5 perform the convolution operation on the vector matrix to be tested, and obtain different feature representations based on the vector matrix for different convolution kernels. Through the pooling operation, each convolution kernel corresponding to the input matrix only produces one maximum feature. , and then connect the feature representations obtained by convolution kernels of different sizes through the full connection operation; the attention mechanism assigns different weights to the feature representations generated after full connection according to the influence of each feature on the output, so that the features with large influence Will have more influence when conducting rumor detection.

Images