CN111259141A - Social media corpus emotion analysis method based on multi-model fusion - Google Patents

Abstract

The invention discloses a social media corpus sentiment analysis method based on multi-model fusion. The pyspide crawler framework is used to obtain from social media, and the data set obtained by the crawler is processed, and the data set is divided into three categories: only contains text information , only contains image information and text and image information, the present invention uses a cross-media method to process corpus, for text information in corpus, uses SO-PMI algorithm to construct an emotional dictionary, and analyzes the positive, neutral and negative of point-by-point mutual information sex. Use similarity distance to replace PMI between words and build a new formula; for image or video corpus, use visual text joint modeling method to get and analyze the meaning of the image, and then get the meaning of the image or video. Using the analysis results of the plain text and the visual analysis results, weighted fusion is performed to obtain the final sentiment analysis result.

Description

A sentiment analysis method for social media corpus based on multi-model fusion

technical field

The invention belongs to the field of sentiment analysis, and relates to a social media corpus sentiment analysis method based on multi-model fusion.

Background technique

In recent years, a large number of social platforms and software have emerged, such as Weibo, WeChat, QQ, etc. These social platforms have greatly enriched people's lives. More and more people are actively sharing information with others and expressing their opinions and feelings on social platforms, so each social platform will gradually appear a large amount of corpus information such as: images, texts, videos, etc. People analyzing the emotions hidden in this information can benefit online marketing, crisis PR, monitoring public opinion, illegal behavior, and spotting signs of suicide such as potential depression. Sentiment analysis is a trend of platform social information, that is, according to the classification of users' corpus information, it can be divided into positive, negative and neutral, three emotional tendencies. Prior to this, various methods have achieved many results for single-recognition analysis of images or texts. However, sentiment analysis of a single feature has many limitations. For example, social platforms such as Weibo, Facebook, and Twitter, which have a large number of users, all support the method of publishing images and texts at the same time. However, most of the current methods cannot comprehensively analyze users’ social media platforms. Misjudgment caused by publishing multiple corpora. For the various corpus information of social platforms, improving the accuracy and comprehensiveness of sentiment analysis needs to be improved.

The present invention is based on the multi-model fusion social media corpus sentiment analysis method, avoids the insufficiency of a single feature for sentiment analysis, and analyzes sentiment by combining images and texts, thereby being more accurate and having wider application range. Semantic analysis of social media information through dual corpus improves the accuracy and comprehensiveness of sentiment analysis.

SUMMARY OF THE INVENTION

The purpose of the present invention is to propose a social media corpus sentiment analysis method based on multi-model fusion. The experimental related data is obtained from social media using the pyspide crawler framework, and the data set obtained by the crawler is processed, and the data set is divided into three categories: only containing text information, only containing image information, and containing both text and image information. The present invention focuses on The processing of text and image information includes situations, and the corpus of the other two situations can be used as a verification of the robustness of the present invention. First of all, to identify the information in the corpus, the identified corpus information can be divided into three categories: only text information, only image information, and both text and image information, regardless of whether the corpus information is one of the above three categories, All of them are processed with corpus containing graphic and text information. The advantage of this is that no matter what the user corpus is, sentiment analysis can be carried out reasonably and the robustness of the model can be guaranteed. First, for the text information in the corpus, use the SO-PMI algorithm (emotional tendency point mutual information algorithm) to build a sentiment dictionary to analyze the positivity, neutrality and negativity of the corpus, but the SO-PMI algorithm cannot flexibly use Chinese words and phrases , so use similarity distance to replace between words and build a new sentiment dictionary. Second, for images (including a collection of pictures and videos), the visual text joint modeling algorithm is used to parse the meaning of the images, so as to obtain the emotional tendencies of the images. Finally, use the results of text corpus analysis and image corpus analysis to perform weighted fusion to the final sentiment analysis results.

In order to achieve the above purpose, the technical solution adopted in the present invention is a social media corpus sentiment analysis method based on multi-model fusion, and the method comprises the following steps:

Step 1 Data preprocessing:

The data used is obtained from social platforms such as Sina Weibo through crawlers, and irrelevant data such as advertisements are filtered, only the blog post data with user subjectivity is retained, and the filtered text data is segmented using the jieba tokenizer. There is a lot of meaningless data in the data. In order to improve the difficulty of later model training, we use a stop word list to filter it, and use the stop word list of Harbin Institute of Technology to get the text after data preprocessing; Processing, the image data is processed into a 256-pixel*256-pixel image in a normalized manner.

Step 2: Perform SO-PMI model training on the text corpus:

The text obtained in step (1) is marked with sentiment of words, which are also divided into three categories: positive, negative and neutral. Text data for model training accounts for 70% of the total data, and test validation data accounts for 30%. First, using 70% of the processed sentiment words for the Word2vec tool for data that has been tokenized and filtered for stop words, an extended sentiment dictionary is obtained. The semantic localization-based point mutual information algorithm (SO-PMI) uses the distance between words and the sentiment dictionary to determine which category they belong to. Then consider the influence of negative words, degree adverbs, interjections, rhetorical sentences and sentiment graphs, weigh all factors, and calculate the sentiment tendency of the text content to get the classification result.

Step 3: Perform CNN+LSTM model training on the image data:

为0-1损失，然后定义函数F的兼容性

使用特性可学的编码器的功能θ(V)图像和文本Φ(t)函数，其中，N表示数据维度，V表示图像集合，T表示文本集合，Y表示映射空间。下述三个公式是从数学角度对多模型融合的社交媒体语料情感分析方法的解释，其中公式(1.3)为图文融合函数，且F(v，t)为图文融合结果，θ(v)^T为图像函数，Φ(t)为文本函数；公式(1.1)为图像最大期望平均函数，其中F(v，t)为公式(1.3)，y为图像语料，t为文本语料，Y为y的映射空间；公式(1.2)为文本最大期望平均函数，其中F(v，t)为公式(1.3)，y为图像语料，v为图像语料，Y为映射空间。On the basis of the picture data set, the emotional description text for the picture is added, and the data of these two modalities is used to provide higher-precision fine-grained classification convolution for image classification, CNN+LSTM for text classification, and the two classification results are combined. up to get an explanation of the emotional meaning of the images after the group. Image and text classification uses a CNN model, which consists of convolutional layers and fully connected layers; for text, a deep structured joint embedding method is used to jointly embed images and fine-grained visual descriptions. The method learns compatibility functions for images and texts, viewed as an extension of multimodal structure stitching embeddings. Instead of using a bilinear compatibility function, a finite element inner product generated by a deep neural encoder is used, maximizing the compatibility between the description and matching images, while minimizing the compatibility with other classes of images. Given data D = (v _n , t _n , _yn ), n = 1, ..., N, where v ∈ V represents visual information, t ∈ T represents text type, and y ∈ Y represents class label, then minimized by Empirical risk to learn image and text classifier functions f _v : V → Y and f _t : T → Y where

for the 0-1 loss, then define the compatibility of the function F

Use feature-learnable encoder functions θ(V) image and text Φ(t) functions, where N represents the data dimension, V represents the image set, T represents the text set, and Y represents the mapping space. The following three formulas are mathematical explanations of the multi-model fusion social media corpus sentiment analysis method, in which formula (1.3) is the image-text fusion function, and F(v, t) is the image-text fusion result, θ(v ) ^T is the image function, Φ(t) is the text function; formula (1.1) is the maximum expected average function of the image, where F(v, t) is the formula (1.3), y is the image corpus, t is the text corpus, Y is The mapping space of y; formula (1.2) is the maximum expected average function of the text, where F(v, t) is formula (1.3), y is the image corpus, v is the image corpus, and Y is the mapping space.

f _v (v) = argmax _y E _{t ~ T(y)} [F(v, t)], yεY (1.1)

f _t (t)=argmax _y E _v～T(y) [F(v, t)], yεY (1.2)

F(v,t)=θ(v) ^T Φ(t) (1.3)

Step 4 Multi-model fusion:

Through steps 2 and 3, the classification results of the final text sentiment of the two texts can be obtained, and then the final classification results are judged by processing the two parts in a weighted manner. The final classification result y=am+bn, where m is the category distance similarity determined by the plain text, and n is the category distance similarity determined by the text obtained from the image, and then the thresholds a and b are obtained according to the GeneticAlgorithm genetic algorithm of the MATLB tool.

Step 5 Final sentiment analysis results:

After step 4, the values of a and b in y=am+bn can be obtained, input the text category similarity and image text similarity, and output the image and text classification value y, whose values are 1, -1 and 0, and 1 is positive, -1 is negative, 0 is a neutral classification result.

Compared with the prior art, the technical advantages of the present invention are mainly reflected in:

(1) The present invention uses a cross-media method for corpus processing. First, for the text information in the corpus, the SO-PMI algorithm is used to construct a sentiment dictionary, and the positive, neutral and negative points of the point-by-point mutual information are analyzed. But this method cannot flexibly use Chinese words and phrases. So use similarity distance to replace PMI between words and build new formula.

(2) Secondly, for the corpus of images or videos (video can be regarded as a collection of images), the visual-text joint modeling method is used to obtain and analyze the meaning of the image, and then the meaning of the image or video is obtained.

(3) Finally, use the analysis results of the plain text and the analysis results obtained by vision to perform weighted fusion to obtain the final sentiment analysis result.

Description of drawings

FIG. 1 is a sample diagram of the corpus used in the present invention.

Figure 2 is a general structure diagram of sentiment analysis of social media corpus based on multi-model fusion.

Fig. 3 is the result diagram after word segmentation is completed in the present invention.

Figure 4 is a stop word table diagram.

Figure 5 is a sample image obtained by processing in step 1.

Figure 6 is a diagram of the SO-PMI model training process.

FIG. 7 is a sub-graph of the training CNN+LSTM model of the present invention.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

The technical solution adopted in the present invention is a social media corpus sentiment analysis method based on multi-model fusion. The specific analysis process of the present invention is as follows

(1) Chinese word segmentation

Chinese word segmentation is the process of recombining consecutive word sequences into word sequences according to certain specifications. According to the Chinese understanding method, it is divided into individual words. During the implementation process, the jieba word segmentation tool can be used to segment the text. After the word segmentation is completed The sentence is shown in Figure 2. You can see that the sentence is divided into individual words.

(2) Remove stop words

A paragraph or sentence of normal Chinese text usually contains special symbols such as commas, periods, and semicolons. These punctuation marks do not need to continue to exist after the word segmentation is complete. Secondly, the sentence contains some words that have little effect on the importance of the sentence, such as words such as , not only, and, and so on, which do not need to be used in the subsequent steps, so they are deleted in the preprocessing.

(3) Constructing word vectors

After a large amount of data processed in (1) and (2), word vectors are extracted by the Word2Vec tool, which reduces the data dimension and obtains an expanded data dictionary.

(4) Training the SO-PMI model

After (1), (2), (3) steps of processing the text data information to obtain an extended sentiment dictionary, and then using the SO-PMI algorithm to determine which category belongs to by the distance between words, and construct the SO-PMI model .

(5) Image normalization processing

The image data obtained by the crawler has the characteristics of inconsistent size, which is more complicated to process. Therefore, the size is normalized according to the selected algorithm, and the size is processed into a picture with a size of 256 pixels*256 pixels.

(6) Training CNN+LSTM model

The image data (labeled data) processed by (5) is used to train the CNN+LSTM model.

(7) Multi-model fusion

After (4) and (6) training, the SO-PMI model and the CNN+LSTM model are obtained. Two processing results will be obtained from the input of graphic data. The two parts are processed in a weighted way to judge the final classification result. The social media corpus sentiment analysis method experiment verifies the effectiveness and accuracy of the method. Compared with the single model and only the text sentiment analysis, the accuracy is significantly improved, and the results show that the proposed method of the present invention has higher accuracy when the microblog sentiment analysis is performed.

Claims (2)

1. A social media corpus emotion analysis method based on multi-model fusion is characterized by comprising the following steps: the method comprises the following steps in total,

step 1, data preprocessing:

the used data is obtained from a social platform through a crawler, irrelevant advertisement data is filtered, only the blog data with user subjectivity is reserved, the filtered text data is subjected to word segmentation by using a jieba word segmentation device, the segmented data has a plurality of meaningless data, a stop word list is used for filtering the data, and a stop word list with a great work size is adopted to obtain the text subjected to data preprocessing; in order to facilitate the processing of the picture data, the picture data is processed into a picture with 256 pixels by 256 pixels in a normalization mode;

step 2, performing SO-PMI model training on the text corpus:

performing emotion marking on words on the text obtained in the step (1), and dividing the words into positive, negative and neutral categories; text data used for model training accounts for 70% of total data, and test verification data accounts for 30%; firstly, for data which are segmented and stop words are filtered, 70% of processed emotion vocabularies are used for a Word2vec tool to obtain an expanded emotion dictionary; judging which type the words belong to by using the distance between the words and an emotion dictionary based on a point mutual information algorithm SO-PMI of semantic positioning; then considering the influence of negative words, degree adverbs, exclamation words, paraphrases and emotion diagrams, balancing all factors, and calculating the emotional tendency of the text content to obtain a classification result;

step 3, CNN + LSTM model training is carried out on the picture data:

adding emotion description texts for pictures on the basis of a picture data set, providing high-precision fine-grained classification convolution for image classification by using data of the two modes, performing text classification by using CNN + LSTM, and combining the two classification results to obtain emotion meaning explanation of the combined image; the classification in the aspect of image texts uses a CNN model, wherein the CNN model consists of a convolution layer and a full connection layer; for the aspect of text, a deep structured joint embedding method is adopted to jointly embed images and fine-grained visual description; the method learns the compatible function of the image and the text, and is regarded as the extension of multi-modal structure splicing embedding; the compatibility between description and matching images is improved to the maximum extent and the compatibility with other images is minimized by using a finite element inner product generated by a deep neural encoder instead of a bilinear compatibility function;

step 4, multi-model fusion:

the final classification result of the text emotion of the two texts can be obtained through the steps 2 and 3, and then the two parts are processed in a weighting mode to judge the final classification result; obtaining a final classification result y which is am + bn, wherein m is the category distance similarity determined by the plain text, n is the category distance similarity determined by the text obtained by the image, and then solving according to a genetic algorithm of an MATLB tool to obtain threshold values a and b;

step 5, final emotion analysis results:

and 4, obtaining the values of a and b in y ═ am + bn, inputting the text type similarity and the image text similarity, and outputting a graph-text classification value y, wherein the values are 1, -1 and 0, 1 is positive, -1 is negative, and 0 is a neutral classification result.

2. The method for analyzing social media corpus emotion based on multi-model fusion as claimed in claim 1, wherein:

given data D ═ v_n，t_n，y_n) N1, …, N, where V e V denotes visual information, T e T denotes text type, Y e Y denotes class labels, and then the image and text classifier function f is learned by minimizing empirical risk_υ: v → Y and f_t: v → Y wherein

For 0-1 penalty, then define the compatibility of the function F

A function θ (V) image and text Φ (T) function of an encoder using characteristics, where N represents a data dimension, V represents a set of images, T represents a set of texts, and Y represents a mapping space; the following three formulas are derived fromThe interpretation of the social media corpus emotion analysis method of multi-model fusion from the learning perspective, wherein the formula (1.3) is a graph-text fusion function, F (v, t) is a graph-text fusion result, and theta (v)^TPhi (t) is an image function and phi (t) is a text function; the formula (1.1) is a maximum expected average function of the image, wherein F (v, t) is the formula (1.3), Y is the image corpus, t is the text corpus, and Y is the mapping space of Y; the formula (1.2) is a maximum expected average function of the text, wherein F (v, t) is the formula (1.3), Y is an image corpus, v is the image corpus, and Y is a mapping space;

f_v(v)＝argmax_yE_t～T(y)[F(v，t)]，yεY (1.1)

f_t(t)＝argmax_yE_v～T(y)[F(v，t)]，yεY(1.2)

F(v，t)＝θ(v)^TΦ(t) (1.3)。

Images