CN110287320A - A deep learning multi-category sentiment analysis model combined with attention mechanism - Google Patents

A deep learning multi-category sentiment analysis model combined with attention mechanism Download PDF

Info

Publication number
CN110287320A
CN110287320A CN201910553755.7A CN201910553755A CN110287320A CN 110287320 A CN110287320 A CN 110287320A CN 201910553755 A CN201910553755 A CN 201910553755A CN 110287320 A CN110287320 A CN 110287320A
Authority
CN
China
Prior art keywords
word
cnn
text
model
feature extraction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910553755.7A
Other languages
Chinese (zh)
Other versions
CN110287320B (en
Inventor
刘磊
孙应红
陈浩
李静
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN201910553755.7A priority Critical patent/CN110287320B/en
Publication of CN110287320A publication Critical patent/CN110287320A/en
Application granted granted Critical
Publication of CN110287320B publication Critical patent/CN110287320B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35Clustering; Classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/30Semantic analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Machine Translation (AREA)

Abstract

本发明涉及一种结合注意力机制的深度学习多分类情感分析模型,属于自然语言处理技术领域,本发明分析了现有CNN网络和LSTM网络在文本情感分析方面的弱点,提出了一种结合注意力机制的深度学习多分类情感分析模型。该模型运用注意力机制将CNN网络提取的局部特征和LSTM模型提取的语序特征相融合,并在分类层采用集成模型的思想,分别将CNN网络和LSTM网络提取的情感特征拼接,作为模型最终提取的情感特征。通过对比实验,发现该模型的准确率有了显著的提高。

The invention relates to a deep learning multi-category sentiment analysis model combined with an attention mechanism, which belongs to the technical field of natural language processing. The invention analyzes the weaknesses of the existing CNN network and LSTM network in text sentiment analysis, and proposes a combination of attention A deep learning multi-category sentiment analysis model based on force mechanism. The model uses the attention mechanism to integrate the local features extracted by the CNN network and the word order features extracted by the LSTM model, and adopts the idea of an integrated model at the classification layer to splicing the emotional features extracted by the CNN network and the LSTM network respectively as the final extraction of the model. emotional characteristics. Through comparative experiments, it is found that the accuracy of the model has been significantly improved.

Description

一种结合注意力机制的深度学习多分类情感分析模型A deep learning multi-category sentiment analysis model combined with attention mechanism

技术领域technical field

本发明属于文本信息处理领域,涉及一种结合注意力机制的深度学习多分类情感分析模型。The invention belongs to the field of text information processing, and relates to a deep learning multi-category sentiment analysis model combined with an attention mechanism.

背景技术Background technique

随着微博、Twitter等社交网络的不断兴起,互联网已不仅仅是人们获取日常信息的来源,同时也成为人们表达自己观点不可或缺的平台。人们在网络社区评论热点事件、抒写影评观点以及描述产品体验等,都会产生大量的带有情感色彩(如:喜怒哀乐等)的文本信息,而对这些文本信息进行有效的情感分析,可以更好地了解用户的兴趣倾向和关注程度。但随着人们对网络信息关注度的增加,网络社区每天都有海量的带有情感色彩的文本产生,如果仅仅依靠人工标记,已经远远无法完成这一任务,这就使得文本情感分析成为自然语言处理领域的一个研究热点。With the continuous rise of social networks such as Weibo and Twitter, the Internet is not only a source for people to obtain daily information, but also an indispensable platform for people to express their opinions. People comment on hot events in online communities, express opinions on film reviews, and describe product experience, etc., which will generate a large amount of text information with emotional color (such as: emotions, etc.), and effective sentiment analysis of these text information can be more effective. Get a better understanding of the user's interest tendencies and attention levels. However, with the increase of people's attention to network information, a large number of texts with emotional color are produced in the network community every day. If only relying on manual marking, it is far from being able to complete this task, which makes text sentiment analysis a natural A research hotspot in the field of language processing.

随着深度学习方法在计算机视觉方向的成功应用,越来越多的深度学习技术也被应用于自然语言处理方向。深度学习的优势在于,不仅可以自动提取文本的特征,而且对大数据有较强的表达能力。目前主流的基于深度学习的文本情感分析方法主要包括卷积神经网络(Convolutional Neural Network,CNN)和循环神经网络(Recurrent NeuralNetwork,RNN)两种,基于这两种方法的情感分析模型的准确率都较低,主要有以下几个方面的原因:With the successful application of deep learning methods in the direction of computer vision, more and more deep learning techniques have also been applied in the direction of natural language processing. The advantage of deep learning is that it can not only automatically extract the features of text, but also has a strong ability to express big data. The current mainstream text sentiment analysis methods based on deep learning mainly include Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). The accuracy of the sentiment analysis model based on these two methods is very low. lower, mainly due to the following reasons:

首先,在文本的情感分析过程中,卷积神经网络通过扩大卷积核尺寸,有效地捕捉到不同位置的情感信息,进而获取到文本的局部情感特征。但卷积的过程中,往往忽略了文本中语序间的前后关系。但在文本情感分析过程中,语序的先后关系又十分重要,没有语序的特征信息必导致结果有一定的偏差。First of all, in the process of text sentiment analysis, the convolutional neural network can effectively capture the emotional information of different positions by expanding the size of the convolution kernel, and then obtain the local emotional features of the text. However, in the process of convolution, the contextual relationship between the word order in the text is often ignored. However, in the process of text sentiment analysis, the sequence of word order is very important, and the lack of characteristic information of word order will lead to certain deviations in the results.

其次,循环神经网络网络利用前后依赖关系,有效模拟文本数据的先后顺序,能够提取到文本的语序关系和语义信息,因此在文本的情感分析中能达到很好的效果。但当样本数据较长或者语言场景较复杂时,有用的情感信息的间隔有大有小,长短不一,长短记忆网络(Long Short-Term Memory,LSTM)的性能也因此受到限制。Secondly, the cyclic neural network network uses the front and back dependencies to effectively simulate the sequence of text data, and can extract the word order relationship and semantic information of the text, so it can achieve good results in text sentiment analysis. However, when the sample data is long or the language scene is complex, the intervals of useful emotional information vary in size and length, and the performance of Long Short-Term Memory (LSTM) is therefore limited.

本发明充分利用了注意力机制、CNN网络、LSTM网络,提出并实现了一种结合注意力机制的深度学习多分类情感分析模型。此模型能够有效的提高文本情感分析的准确率。The present invention makes full use of attention mechanism, CNN network and LSTM network, proposes and implements a deep learning multi-category sentiment analysis model combined with attention mechanism. This model can effectively improve the accuracy of text sentiment analysis.

发明内容Contents of the invention

本发明提出了一种基于注意力机制的深度学习多分类情感分析模型。该模型结合CNN网络和LSTM网络进行情感特征融合。首先利用CNN网络的多尺度卷积核提取出待分析文本的局部特征,然后利用注意力机制,将CNN网络提取的局部特征融入到LSTM网络中。最后利用集成模型的思想,将CNN网络的池化层结果和LSTM网络的特征提取结果进行拼接,作为最终的模型输出。实验表明,在文本情感分析中,该模型的准确率有了显著的提高。The present invention proposes a deep learning multi-category sentiment analysis model based on an attention mechanism. This model combines CNN network and LSTM network for emotional feature fusion. First, the multi-scale convolution kernel of the CNN network is used to extract the local features of the text to be analyzed, and then the attention mechanism is used to integrate the local features extracted by the CNN network into the LSTM network. Finally, using the idea of the integrated model, the pooling layer results of the CNN network and the feature extraction results of the LSTM network are spliced as the final model output. Experiments show that the accuracy of the model has been significantly improved in text sentiment analysis.

为实现上述目的,本发明采用如下的技术方案:To achieve the above object, the present invention adopts the following technical solutions:

1.一种结合注意力机制的深度学习多分类情感分析方法,其特征在于包括以下步骤:1. a deep learning multi-category sentiment analysis method combined with attention mechanism, is characterized in that comprising the following steps:

步骤(1)数据预处理Step (1) data preprocessing

设情感数据集表示为:G=[(segtxt1,y1),(segtxt2,y2),......,(segtxtN,yN)],其中,segtxti表示第i个样本,yi则为对应的情感类别标签,N表示数据集G中样本个数,对G中样本进行数据预处理,Let the emotional data set be expressed as: G=[(segtxt 1 ,y 1 ),(segtxt 2 ,y 2 ),...,(segtxt N ,y N )], where segtxt i represents the i-th sample, y i is the corresponding emotional category label, N represents the number of samples in the data set G, and data preprocessing is performed on the samples in G,

数据集G经预处理后,表示为G′=[(seg1,y1),(seg2,y2),...,(segM,yM)],其中:segi表示为数据集G′中第i个样本,yi则为对应的情感类别标签,M表示数据集G′中样本个数;After the data set G is preprocessed, it is expressed as G′=[(seg 1 ,y 1 ),(seg 2 ,y 2 ),...,(seg M ,y M )], where: seg i is expressed as data The i-th sample in the set G′, y i is the corresponding emotional category label, and M represents the number of samples in the data set G′;

步骤(2)构建模型的输入Step (2) Build the input of the model

对于数据集G′中任意一个待分析样本数据(seg,y),将其进一步细化表示为:For any sample data (seg, y) to be analyzed in the data set G′, it is further refined as:

seg=[w1,w2,w3,...,wd]T (1)seg=[w 1 ,w 2 ,w 3 ,...,w d ] T (1)

y=[0,0,1,...,0] (2)y=[0,0,1,...,0] (2)

其中:wi∈Rε是指依据词表wordList对待分析文本中第i词语的one-hot编码,ε为词表wordList的大小,d表示该文本的句长。y∈Rp是依据情感类别的one-hot编码,p表示模型待分的类别数目。则该样本的词向量嵌入矩阵可表示为:Among them: w i ∈ R ε refers to the one-hot encoding of the i-th word in the text to be analyzed according to the vocabulary wordList, ε is the size of the vocabulary wordList, and d represents the sentence length of the text. y∈R p is the one-hot encoding based on the emotional category, and p represents the number of categories to be classified by the model. Then the word vector embedding matrix of this sample can be expressed as:

X=seg*ET (3)X=seg*E T (3)

其中:X∈Rd×m,X=[x1,x2,...,xd]T为待分析文本的词向量矩阵表示,m为词向量的维度,xi∈Rm为该文本中第i个词汇的词向量表示,E为词向量嵌入层表示;Among them: X∈R d×m , X=[x 1 ,x 2 ,...,x d ] T is the word vector matrix representation of the text to be analyzed, m is the dimension of the word vector, x i ∈ R m is the The word vector representation of the i-th vocabulary in the text, E is the word vector embedding layer representation;

步骤(3)构建深度学习多分类情感分析模型Step (3) Build a deep learning multi-category sentiment analysis model

深度学习多分类情感分析模型包括基于CNN网络的局部特征提取阶段和基于LSTM网络的语序关系特征提取阶段,将基于CNN网络的局部特征提取阶段的池化层结果CCnn和基于LSTM网络的语序关系特征提取阶段的结果C'Rnn拼接,即向量[CCnn;C'Rnn]作为模型最终提取的特征向量。然后将特征向量[CCnn;C'Rnn]经过全连接层得到最终的模型输出向量其中p表示模型待分的类别数目。The deep learning multi-category sentiment analysis model includes the stage of local feature extraction based on CNN network and the stage of word order relationship feature extraction based on LSTM network. The results of the feature extraction stage C' Rnn splicing, that is, the vector [C Cnn ; C' Rnn ] is used as the feature vector finally extracted by the model. Then pass the feature vector [C Cnn ; C' Rnn ] through the fully connected layer to get the final model output vector where p represents the number of categories to be classified by the model.

所述的基于CNN网络的局部特征提取阶段,包括以下内容:The described local feature extraction stage based on the CNN network includes the following:

局部特征提取阶段输入为公式3的待分析文本的词向量矩阵表示X;In the local feature extraction stage, the input is the word vector matrix representation X of the text to be analyzed in formula 3;

局部特征提取阶段基于CNN网络,一共包括两层,即一层卷积层、一层池化层,其中:The local feature extraction stage is based on the CNN network, which includes two layers, namely a convolutional layer and a pooling layer, in which:

卷积层采用n种不同尺度的卷积核对待分析文本进行卷积,且同一尺度卷积核的滤波器即神经元各k个;The convolutional layer uses n types of convolution kernels of different scales to convolve the text to be analyzed, and the filters of the same scale convolution kernels are k neurons;

池化层采用最大池化层的方法将卷积所得的向量做下采样,选出局部最优特征,因此每个滤波器通过最大池化层变为一个标量,该标量代表着该滤波器中最优的情感特征;The pooling layer uses the method of the maximum pooling layer to down-sample the vector obtained by convolution and select the local optimal features, so each filter becomes a scalar through the maximum pooling layer, which represents the Optimal emotional characteristics;

局部特征提取模块的输出为CCnn=[c1,c2,...,cnk],即将池化层中不同尺寸的多个滤波器选取的最优特征拼接到一起CCnn=[c1,c2,...,cnk]作为本模块的输出,其中,CCnn∈Rnk,nk为卷积层中所有滤波器的个数;The output of the local feature extraction module is C Cnn =[c 1 ,c 2 ,...,c nk ], that is, the optimal features selected by multiple filters of different sizes in the pooling layer are spliced together C Cnn =[c 1 ,c 2 ,...,c nk ] as the output of this module, where C Cnn ∈ R nk , nk is the number of all filters in the convolutional layer;

所述的基于LSTM网络的语序关系特征提取阶段,包括以下内容:Described word order relation feature extraction stage based on LSTM network, comprises the following contents:

多尺度CNN网络局部特征提取:将基于CNN网络的局部特征提取阶段中卷积层同一卷积尺度的k个滤波器的卷积结果拼接,得到集合ZCnn,然后将集合ZCnn中的每个向量Zi输入到GLU机制中,即门控卷积网络,得到的结果记为{π12,...,πn},完成多尺度CNN网络局部特征的提取。Multi-scale CNN network local feature extraction: Concatenate the convolution results of k filters of the same convolution scale in the convolutional layer in the local feature extraction stage based on the CNN network to obtain a set Z Cnn , and then combine each of the set Z Cnn The vector Z i is input into the GLU mechanism, that is, the gated convolutional network, and the obtained results are recorded as {π 12 ,...,π n } to complete the extraction of local features of the multi-scale CNN network.

其中,ZCnn={Z1,Z2,...,Zn},Zi为尺度为i的多个滤波器卷积结果的拼接;Among them, Z Cnn = {Z 1 , Z 2 ,..., Z n }, Z i is the concatenation of multiple filter convolution results with a scale of i;

其中,Zi代表某一尺度的k个滤波器卷积结果的拼接,W1,W2∈Rλ×q为权重矩阵,λ表示对应权重矩阵的维度,的b1,b2∈Rq为偏置量,σ表示sigmoid函数,πi∈Rq,q为LSTM网络的输出维度;in, Z i represents the concatenation of k filter convolution results of a certain scale, W 1 , W 2 ∈ R λ×q is the weight matrix, λ represents the dimension of the corresponding weight matrix, and b 1 , b 2 ∈ R q is the bias Set amount, σ represents the sigmoid function, π i ∈ R q , q is the output dimension of the LSTM network;

然后,利用注意力机制,将多尺度CNN网络局部特征提取结果{π12,...,πn}融入到LSTM网络中,得到基于LSTM网络的语序关系特征提取阶段的输出结果C'Rnn,即 Then, using the attention mechanism, the multi-scale CNN network local feature extraction results {π 12 ,...,π n } are integrated into the LSTM network, and the output result C of the word order relationship feature extraction stage based on the LSTM network is obtained ' Rnn , ie

其中,表示待分析文本中最后一个词语所对应的LSTM模块的输出,表示待分析文本中第一个词语所对应的LSTM模块的输出,本发明采用双向LSTM模型,即BiLSTM模型,in, Indicates the output of the LSTM module corresponding to the last word in the text to be analyzed, Indicates the output of the LSTM module corresponding to the first word in the text to be analyzed. The present invention adopts a bidirectional LSTM model, namely the BiLSTM model,

采用正向传播,具体计算过程如下: Using forward propagation, the specific calculation process is as follows:

d为待分析文本的长度,该文本中每一个词语顺序对应一个LSTM模块,,d is the length of the text to be analyzed, each word sequence in the text corresponds to an LSTM module,,

正向传播过程中,第t-1个LSTM模块的输出为则第t个LSTM模块的输出计算公式如下:During forward propagation, the output of the t-1th LSTM module is Then the output of the tth LSTM module Calculated as follows:

其中:是两个向量的点乘,也称打分函数,是用来计算前一词语的LSTM的输出和当前局部特征向量的相似度,in: Is the dot product of two vectors, also known as the scoring function, which is used to calculate the output of the LSTM of the previous word The similarity with the current local feature vector,

其中:αt,i∈R代表特征πi的权重,Among them: α t,i ∈ R represents the weight of feature π i ,

其中:st-1∈Rq是多个卷积特征的加权结果,利用st-1代替结合当前词语的词向量xt求得当前LSTM模块的输出公式如下:Among them: s t-1 ∈ R q is the weighted result of multiple convolutional features, using s t-1 instead Combine the word vector x t of the current word to obtain the output of the current LSTM module The formula is as follows:

采用反向传播,具体计算过程与正向传播一样,此处不再赘述; Using backpropagation, the specific calculation process is the same as that of forward propagation, and will not be repeated here;

步骤(4)模型训练:将训练数据输入多分类情感分析模型,采用交叉熵损失函数,结合反向传播BP算法调整参数,利用softmax回归作为分类算法,完成训练;Step (4) Model training: Input the training data into the multi-category sentiment analysis model, use the cross-entropy loss function, adjust the parameters in combination with the backpropagation BP algorithm, and use softmax regression as the classification algorithm to complete the training;

步骤(5)模型分析:将待分析文本输入训练完成的模型,最终输出对该文本分析后的情感分类结果。Step (5) Model analysis: Input the text to be analyzed into the trained model, and finally output the sentiment classification result after analyzing the text.

所述的预处理过程包括以下步骤:Described pretreatment process comprises the following steps:

1)分词、去除停用、英文大写转小写、繁体转简体。1) Word segmentation, deactivation, uppercase to lowercase, and traditional to simplified.

2)选取数据集G中频率大于等于σ的词语,构造词汇表wordList={word1,word2,...wordε},其中,wordi表示词汇表wordlist中第i个词语,ε表示数据集G中词频超过σ的词语总数。2) Select words whose frequency is greater than or equal to σ in the data set G, and construct a vocabulary wordList={word 1 ,word 2 ,...word ε }, where word i represents the i-th word in the vocabulary wordlist, and ε represents the data The total number of words in the set G whose word frequency exceeds σ.

3)对数据集G中每一个样本,若长度大于d,则删除该样本,若长度小于d,则用符号</>补齐。3) For each sample in the data set G, if the length is greater than d, delete the sample, and if the length is less than d, fill it with the symbol </>.

所述的基于CNN网络的局部特征提取模块的卷积层计算公式如下:The convolutional layer calculation formula of the local feature extraction module based on the CNN network is as follows:

z=f(∑WT*xi:i+s-1+b) (8)z=f(∑W T *x i:i+s-1 +b) (8)

其中:z表示一个神经元对待分析文本的卷积所得的特征向量,f(·)表示激活函数,W∈Rs×m表示神经元的权重矩阵,同一个神经元参数共享,s×m表示卷积核尺寸的大小,b表示阈值,xi:i+s-1表示由文本句子中的第i个词到i+s-1个词语的词向量。Among them: z represents the feature vector obtained by the convolution of a neuron to analyze the text, f(·) represents the activation function, W∈R s×m represents the weight matrix of the neuron, the same neuron parameters are shared, s×m represents The size of the convolution kernel, b represents the threshold, x i:i+s-1 represents the word vector from the i-th word to the i+s-1 word in the text sentence.

所述的训练数据为经过预处理之后的数据。The training data is preprocessed data.

所述的基于CNN网络的局部特征提取阶段的卷积层采用4种不同尺度的卷积核。所述的训练结束条件是准确率不再改变或达到设置迭代次数。The convolution layer of the local feature extraction stage based on the CNN network adopts four convolution kernels of different scales. The training end condition is that the accuracy rate does not change or reaches the set number of iterations.

附图说明Description of drawings

图1本发明的方法流程图;Fig. 1 method flowchart of the present invention;

图2结合注意力机制的深度学习多分类情感分析模型结构示意图。Figure 2 Schematic diagram of the structure of the deep learning multi-category sentiment analysis model combined with the attention mechanism.

具体实施方式Detailed ways

下面结合图表和实施例,对本发明的具体实施方式作进一步的详细描述。以下实施例用于说明本发明,但不用来限制本发明的范围。The specific implementation of the present invention will be further described in detail below in conjunction with the diagrams and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

本发明提出的方法是依次按以下步骤实现的:The method that the present invention proposes is to realize by following steps successively:

步骤(1)数据预处理Step (1) data preprocessing

情感语数据集表示为:G=[(segtxt1,y1),(segtxt2,y2),......,(segtxtN,yN)],其中,segtxti表示第i个样本,yi则为对应的情感类别标签。N表示数据集G中样本个数,情感标签取“喜悦”、“愤怒”、“厌恶”、“低落”四大类,N取80000,其中,四类情感样本各20000条。对G中样本进行数据预处理包括以下几个步骤:The emotional language data set is expressed as: G=[(segtxt 1 ,y 1 ),(segtxt 2 ,y 2 ),...,(segtxt N ,y N )], where segtxt i represents the i-th samples, and y i is the corresponding emotion category label. N represents the number of samples in the data set G, and the emotion labels are divided into four categories: "joy", "anger", "disgust", and "depression". N is 80,000, and each of the four types of emotion samples is 20,000. Data preprocessing for samples in G includes the following steps:

1)分词、去除停用、英文大写转小写、繁体转简体。1) Word segmentation, deactivation, uppercase to lowercase, and traditional to simplified.

2)选取数据集G中频率大于等于σ的词语,构造词汇表wordList={word1,word2,...wordε},其中,wordi表示数据集G中第i个词语,ε表示数据集G中词频超过σ的词语总数。σ取2,最终得到的数据集G中,词频大于等于2的词语共41763个,即ε为41763。2) Select the words whose frequency is greater than or equal to σ in the data set G, and construct a vocabulary wordList={word 1 , word 2 ,...word ε }, where word i represents the i-th word in the data set G, and ε represents the data The total number of words in the set G whose word frequency exceeds σ. If σ is set to 2, in the final data set G, there are 41763 words with a word frequency greater than or equal to 2, that is, ε is 41763.

3)将上述处理后,对数据集G中每一个样本,若长度大于d,则删除该样本,若长度小于d,则用符号</>补齐。d取64。3) After the above processing, for each sample in the data set G, if the length is greater than d, delete the sample, and if the length is less than d, fill it with the symbol </>. d takes 64.

数据集G经预处理后,表示为G′=[(seg1,y1),(seg2,y2),...,(segM,yM)]。其中:segi表示为数据集G′中第i个样本,yi则为对应的情感类别标签,M表示数据集G′中样本个数。最终数据集G′中样本的个数为73150条,各情感类别的样本条数如表1所示:After the data set G is preprocessed, it is expressed as G′=[(seg 1 ,y 1 ),(seg 2 ,y 2 ),...,(seg M ,y M )]. Among them: seg i represents the i-th sample in the data set G′, y i represents the corresponding emotion category label, and M represents the number of samples in the data set G′. The number of samples in the final data set G′ is 73150, and the number of samples of each emotion category is shown in Table 1:

表1预处理后各类别样本数量Table 1 Number of samples in each category after preprocessing

步骤(2)模型的输入Step (2) The input of the model

对于数据集G′中任意一个待分析样本数据(seg,y),将其进一步细化表示为:For any sample data (seg, y) to be analyzed in the data set G′, it is further refined as:

seg=[w1,w2,w3,...,wd]T (1)seg=[w 1 ,w 2 ,w 3 ,...,w d ] T (1)

y=[0,0,1,...,0] (2)y=[0,0,1,...,0] (2)

其中:wi∈Rε是指依据词表wordList对待分析文本中第i词语的one-hot编码,ε为词表wordList的大小,文本的句长d为64。y∈Rp是依据情感类别的one-hot编码,p表示模型待分的类别数目,p为4。则该样本的词向量嵌入矩阵可表示为:Among them: w i ∈ R ε refers to the one-hot encoding of the i-th word in the analyzed text according to the vocabulary wordList, ε is the size of the vocabulary wordList, and the sentence length d of the text is 64. y∈R p is the one-hot encoding based on the emotional category, p represents the number of categories to be classified by the model, and p is 4. Then the word vector embedding matrix of this sample can be expressed as:

X=seg*ET (3)X=seg*E T (3)

其中:X∈Rd×m,X=[x1,x2,...,xd]T为待分析文本的词向量矩阵表示,词向量维度m取256。xi∈Rm为该文本中第i个词汇的词向量表示,词向量嵌入层表示E,采用维基百科开源word2vec词向量,接下来将X作为网络模型的输入。Where: X∈R d×m , X=[x 1 ,x 2 ,...,x d ] T is the word vector matrix representation of the text to be analyzed, and the word vector dimension m is 256. x i ∈ R m is the word vector representation of the i-th vocabulary in the text, the word vector embedding layer represents E, using Wikipedia's open source word2vec word vector, and then X is used as the input of the network model.

步骤(3)构建深度学习多分类情感分析模型Step (3) Build a deep learning multi-category sentiment analysis model

深度学习多分类情感分析模型包括基于CNN网络的局部特征提取阶段和基于LSTM网络的语序关系特征提取阶段,将基于CNN网络的局部特征提取阶段的池化层结果CCnn和基于LSTM网络的语序关系特征提取阶段的结果C'Rnn拼接,即向量[CCnn;C'Rnn]作为模型最终提取的特征向量。然后将特征向量[CCnn;C'Rnn]经过全连接层得到最终的模型输出向量其中p表示模型待分的类别数目。The deep learning multi-category sentiment analysis model includes the stage of local feature extraction based on CNN network and the stage of word order relationship feature extraction based on LSTM network. The results of the feature extraction stage C' Rnn splicing, that is, the vector [C Cnn ; C' Rnn ] is used as the feature vector finally extracted by the model. Then pass the feature vector [C Cnn ; C' Rnn ] through the fully connected layer to get the final model output vector where p represents the number of categories to be classified by the model.

所述的基于CNN网络的局部特征提取阶段,包括以下内容:The described local feature extraction stage based on the CNN network includes the following:

局部特征提取阶段输入为公式3的待分析文本的词向量矩阵表示X;In the local feature extraction stage, the input is the word vector matrix representation X of the text to be analyzed in formula 3;

局部特征提取阶段基于CNN网络,一共包括两层,即一层卷积层、一层池化层,其中:The local feature extraction stage is based on the CNN network, which includes two layers, namely a convolutional layer and a pooling layer, in which:

卷积层采用n种不同尺度的卷积核对待分析文本进行卷积,且同一尺度卷积核的滤波器即神经元各k个,本发明n取4,k取128。The convolution layer uses n types of convolution kernels of different scales to convolve the text to be analyzed, and there are k filters of the same scale convolution kernels, that is, neurons. In the present invention, n is 4, and k is 128.

池化层采用最大池化层的方法将卷积所得的向量做下采样,选出局部最优特征,因此每个滤波器通过最大池化层变为一个标量,该标量代表着该滤波器中最优的情感特征;The pooling layer uses the method of the maximum pooling layer to down-sample the vector obtained by convolution and select the local optimal features, so each filter becomes a scalar through the maximum pooling layer, which represents the Optimal emotional characteristics;

局部特征提取模块的输出为CCnn=[c1,c2,...,cnk],即将池化层中不同尺寸的多个滤波器选取的最优特征拼接到一起CCnn=[c1,c2,...,cnk]作为本模块的输出,其中,CCnn∈Rnk,nk为卷积层中所有滤波器的个数,共512个;The output of the local feature extraction module is C Cnn =[c 1 ,c 2 ,...,c nk ], that is, the optimal features selected by multiple filters of different sizes in the pooling layer are spliced together C Cnn =[c 1 ,c 2 ,...,c nk ] as the output of this module, where C Cnn ∈ R nk , nk is the number of all filters in the convolutional layer, a total of 512;

所述的基于LSTM网络的语序关系特征提取阶段,包括以下内容:Described word order relation feature extraction stage based on LSTM network, comprises the following contents:

多尺度CNN网络局部特征提取:将基于CNN网络的局部特征提取阶段中卷积层同一卷积尺度的k个滤波器的卷积结果拼接,得到集合ZCnn,然后将集合ZCnn中的每个向量Zi输入到GLU机制中,即门控卷积网络,得到的结果记为{π12,...,πn},完成多尺度CNN网络局部特征的提取。Multi-scale CNN network local feature extraction: Concatenate the convolution results of k filters of the same convolution scale in the convolutional layer in the local feature extraction stage based on the CNN network to obtain a set Z Cnn , and then combine each of the set Z Cnn The vector Z i is input into the GLU mechanism, that is, the gated convolutional network, and the obtained results are recorded as {π 12 ,...,π n } to complete the extraction of local features of the multi-scale CNN network.

其中,ZCnn={Z1,Z2,...,Zn},Zi为尺度为i的多个滤波器卷积结果的拼接;Among them, Z Cnn = {Z 1 , Z 2 ,..., Z n }, Z i is the concatenation of multiple filter convolution results with a scale of i;

其中,Zi代表某一尺度的k个滤波器卷积结果的拼接,W1,W2∈Rλ×q为权重矩阵,λ表示对应权重矩阵的维度,b1,b2∈Rq为偏置量,σ表示sigmoid函数,πi∈Rq,q为LSTM网络的输出维度,q取256;in, Z i represents the concatenation of k filter convolution results of a certain scale, W 1 , W 2 ∈ R λ×q is the weight matrix, λ represents the dimension of the corresponding weight matrix, b 1 , b 2 ∈ R q is the bias Quantity, σ represents the sigmoid function, π i ∈ R q , q is the output dimension of the LSTM network, and q is 256;

然后,利用注意力机制,将多尺度CNN网络局部特征提取结果{π12,...,πn}融入到LSTM网络中,得到基于LSTM网络的语序关系特征提取阶段的输出结果C'Rnn,即 Then, using the attention mechanism, the multi-scale CNN network local feature extraction results {π 12 ,...,π n } are integrated into the LSTM network, and the output result C of the word order relationship feature extraction stage based on the LSTM network is obtained ' Rnn , ie

其中,表示待分析文本中最后一个词语所对应的LSTM模块的输出,表示待分析文本中第一个词语所对应的LSTM模块的输出,本发明采用双向LSTM模型,即BiLSTM模型,in, Indicates the output of the LSTM module corresponding to the last word in the text to be analyzed, Indicates the output of the LSTM module corresponding to the first word in the text to be analyzed. The present invention adopts a bidirectional LSTM model, namely the BiLSTM model,

采用正向传播,具体计算过程如下: Using forward propagation, the specific calculation process is as follows:

d为待分析文本的长度,该文本中每一个词语顺序对应一个LSTM模块,,d is the length of the text to be analyzed, each word sequence in the text corresponds to an LSTM module,,

正向传播过程中,第t-1个LSTM模块的输出为则第t个LSTM模块的输出计算公式如下:During forward propagation, the output of the t-1th LSTM module is Then the output of the tth LSTM module Calculated as follows:

其中:是两个向量的点乘,也称打分函数,是用来计算前一词语的LSTM的输出和当前局部特征向量的相似度,in: Is the dot product of two vectors, also known as the scoring function, which is used to calculate the output of the LSTM of the previous word The similarity with the current local feature vector,

其中:αt,i∈R代表特征πi的权重,Among them: α t,i ∈ R represents the weight of feature π i ,

其中:st-1∈Rq是多个卷积特征的加权结果,利用st-1代替结合当前词语的词向量xt求得当前LSTM模块的输出公式如下:Among them: s t-1 ∈ R q is the weighted result of multiple convolutional features, using s t-1 instead Combine the word vector x t of the current word to obtain the output of the current LSTM module The formula is as follows:

采用反向传播,具体计算过程与正向传播一样,此处不再赘述; Using backpropagation, the specific calculation process is the same as that of forward propagation, and will not be repeated here;

步骤(4)模型训练:将训练数据输入多分类情感分析模型,采用交叉熵损失函数,结合反向传播BP算法调整参数,利用softmax回归作为分类算法,完成训练。Step (4) Model training: Input the training data into the multi-category sentiment analysis model, use the cross-entropy loss function, combine the backpropagation BP algorithm to adjust the parameters, and use softmax regression as the classification algorithm to complete the training.

步骤(5)模型分析:将待分析文本输入训练完成的模型,最终输出对该文本分析后的情感分类结果。Step (5) Model analysis: Input the text to be analyzed into the trained model, and finally output the sentiment classification result after analyzing the text.

所述的基于CNN网络的局部特征提取模块的卷积层计算公式如下:The convolutional layer calculation formula of the local feature extraction module based on the CNN network is as follows:

z=f(∑WT*xi:i+s-1+b) (8)z=f(∑W T *x i:i+s-1 +b) (8)

其中:z表示一个神经元对待分析文本的卷积所得的特征向量,f(·)表示激活函数,W∈Rs×m表示神经元的权重矩阵,同一个神经元参数共享,s×m表示卷积核尺寸的大小,b表示阈值,xi:i+s-1表示由文本句子中的第i个词到i+s-1个词语的词向量,s取[2,3,4,5]四种不同的卷积尺寸,f(·)采用RELU激活函数。Among them: z represents the feature vector obtained by the convolution of a neuron to analyze the text, f(·) represents the activation function, W∈R s×m represents the weight matrix of the neuron, the same neuron parameters are shared, s×m represents The size of the convolution kernel, b represents the threshold, x i:i+s-1 represents the word vector from the i-th word in the text sentence to the i+s-1 word, and s takes [2,3,4, 5] Four different convolution sizes, f(·) using the RELU activation function.

所述的训练数据为经过预处理之后的数据。The training data is preprocessed data.

所述的基于CNN网络的局部特征提取阶段的卷积层采用4种不同尺度的卷积核。所述的训练结束条件是准确率不再改变或达到设置迭代次数。The convolution layer of the local feature extraction stage based on the CNN network adopts four convolution kernels of different scales. The training end condition is that the accuracy rate does not change or reaches the set number of iterations.

1.实验分析1. Experimental analysis

测试阶段,选取喜悦、愤怒、厌恶、低落各类别情感语料各占2000条。使用准确率Acc(Accuracy)作为评价指标,测试阶段模型的参数保持不变,测试集结果如表2所示:In the test phase, 2,000 emotion corpora were selected for each category of joy, anger, disgust, and depression. Using the accuracy rate Acc (Accuracy) as the evaluation index, the parameters of the model in the test phase remain unchanged, and the results of the test set are shown in Table 2:

表2情感分析结果对比Table 2 Comparison of sentiment analysis results

表2中给出了几种模型的测试结果对比,其中,实验1是通用的卷积核尺寸为3的单尺度CNN网络模型,实验2是通用的LSTM网络,实验3则是本文提出的基于注意力机制的文本情感分析模型。Table 2 shows the comparison of the test results of several models. Among them, Experiment 1 is a general-purpose single-scale CNN network model with a convolution kernel size of 3, Experiment 2 is a general-purpose LSTM network, and Experiment 3 is based on Text Sentiment Analysis Model with Attention Mechanism.

通过实验的对比性分析可见,相较通常的CNN网络和LSTM网络,本文提出的基于注意力机制的情感分析模型的准确率都明显提高,说明了本发明提出的方法可以有效的提取CNN网络的局部特征信息和LSTM网络的语序特征信息,说明了该方法的有效性。Through the comparative analysis of the experiment, it can be seen that compared with the usual CNN network and LSTM network, the accuracy of the emotional analysis model based on the attention mechanism proposed in this paper is significantly improved, which shows that the method proposed by the present invention can effectively extract the content of the CNN network. The local feature information and the word order feature information of the LSTM network illustrate the effectiveness of the method.

Claims (6)

1. A deep learning multi-classification emotion analysis method combined with an attention mechanism is characterized by comprising the following steps:
step (1) data preprocessing
Let the emotion data set be expressed as: g ═ segtxt1,y1),(segtxt2,y2),...,(segtxtN,yN)]Wherein segtxtiDenotes the ith sample, yiThen the corresponding emotion type label is obtained, N represents the number of samples in the data set G, and the samples in G are processedThe pre-processing of the data is carried out,
the dataset G was preprocessed and denoted as G' ═ seg [ ("seg1,y1),(seg2,y2),...,(segM,yM)]Wherein: segiExpressed as the ith sample, y, in the data set GiThen the label is a corresponding emotion category label, and M represents the number of samples in the data set G';
step (2) input of the constructed model
For any sample data to be analyzed (seg, y) in the data set G', it is further detailed as:
seg=[w1,w2,...,wi,...,wd]T (1)
y=[0,0,1,...,0] (2)
wherein: w is ai∈RεOne-hot coding of the ith word in a text to be analyzed is carried out according to a word list wordList, epsilon is the size of the word list wordList, d represents the sentence length of the text, and y belongs to RpAccording to one-hot encoding of emotion categories, p represents the number of categories to be classified by the model, and the word vector embedding matrix of the sample can be represented as follows:
X=seg*ET (3)
wherein: x is formed by Rd×m,X=[x1,x2,...,xd]TFor a word vector matrix representation of the text to be analyzed, m is the dimension of the word vector, xi∈RmRepresenting the word vector of the ith vocabulary in the text, and E representing the word vector embedded layer;
step (3) constructing a deep learning multi-classification emotion analysis model
The deep learning multi-classification emotion analysis model comprises a CNN network-based local feature extraction stage, an LSTM network-based word order relation feature extraction stage and a pooling layer result C of the CNN network-based local feature extraction stageCnnAnd result C 'of language sequence relation feature extraction stage based on LSTM network'RnnSplicing, i.e. vector [ CCnn;C'Rnn]As the feature vector of the final extraction of the model, and then the feature vector [ C ]Cnn;C'Rnn]Obtaining the final model through the full connection layerOutput vectorWhere p represents the number of classes to be assigned to the model,
the local feature extraction stage based on the CNN network comprises the following contents:
inputting a word vector matrix representation X of the text to be analyzed of a formula 3 in a local feature extraction stage;
the local feature extraction stage is based on a CNN network and comprises two layers in total, namely a convolutional layer and a pooling layer, wherein:
the convolution layer adopts n convolution kernels with different scales to convolute the text to be analyzed, and the number of filters of the convolution kernel with the same scale is k, namely, k of each neuron;
in the pooling layer, the vector obtained by convolution is down-sampled by adopting a maximum pooling layer method, and local optimal features are selected, so that each filter becomes a scalar through the maximum pooling layer, and the scalar represents the optimal emotional features in the filter;
the output of the local feature extraction module is CCnn=[c1,c2,...,cnk]Splicing the optimal features selected by a plurality of filters with different sizes in the pooling layer together CCnn=[c1,c2,...,cnk]As output of this module, where CCnn∈RnkAnd nk is the number of all filters in the convolutional layer;
the language order relation feature extraction stage based on the LSTM network comprises the following contents:
multi-scale CNN network local feature extraction: splicing convolution results of k filters with the same convolution scale of convolution layers in the local feature extraction stage based on the CNN network to obtain a set ZCnnThen set ZCnnEach vector Z in (a)iInputting the result into a GLU mechanism, namely, gating a convolution network, and marking the obtained result as { pi12,...,πnFinishing the extraction of the local features of the multi-scale CNN network,
wherein Z isCnn={Z1,Z2,...,Zn},ZiSplicing convolution results of a plurality of filters with the scale i;
wherein,Zistitching of convolution results of k filters representing a certain scale, W1,W2∈Rλ×qFor the weight matrix, λ represents the dimension of the corresponding weight matrix, b1,b2∈RqFor the offset, σ denotes the sigmoid function, πi∈RqQ is the output dimension of the LSTM network;
then, extracting the local feature extraction result { pi ] of the multi-scale CNN network by using an attention mechanism12,...,πnIntegrating the language sequence relationship feature extraction stage into an LSTM network to obtain an output result C 'of the language sequence relationship feature extraction stage based on the LSTM network'RnnI.e. by
Wherein,represents the output of the LSTM module corresponding to the last word in the text to be analyzed,representing the output of the LSTM module corresponding to the first word in the text to be analyzed, the invention uses a bi-directional LSTM model, i.e. a BiLSTM model,
the forward propagation is adopted, and the specific calculation process is as follows:
d is the length of the text to be analyzed, each word in the text sequentially corresponds to an LSTM module,
in the forward propagation process, the output of the t-1 th LSTM module isThen the output of the tth LSTM moduleThe calculation formula is as follows:
wherein:is a dot product of two vectors, also known as a scoring function, and is the output of the LSTM used to calculate the previous wordAnd the similarity of the current local feature vector,
wherein αt,iEpsilon R represents characteristic piiThe weight of (a) is determined,
wherein: st-1∈RqIs a weighted result of a plurality of convolution features, using st-1Instead of the formerWord vector x in combination with current wordtEvaluating the output of a current LSTM moduleThe formula is as follows:
the backward propagation is adopted, the specific calculation process is the same as the forward propagation, and the details are not repeated here;
step (4), model training: inputting training data into a multi-classification emotion analysis model, adjusting parameters by adopting a cross entropy loss function and combining a back propagation BP algorithm, and finishing training by using softmax regression as a classification algorithm;
and (5) analyzing a model: and inputting the text to be analyzed into the trained model, and finally outputting the emotion classification result after the text is analyzed.
2. The method for deep learning multi-classification emotion analysis combined with attention mechanism as claimed in claim 1, wherein the preprocessing process comprises the following steps:
1) word segmentation, stop removal, conversion from English capitals to lowercases, conversion from traditional Chinese to simplified Chinese,
2) selecting words with frequency more than or equal to sigma in the data set G, and constructing a vocabulary table word List ═ word1,word2,...wordεTherein, wordiRepresenting the ith word in the vocabulary word, epsilon represents the total number of words in the data set G with a word frequency exceeding sigma,
3) for each sample in the data set G, if the length is larger than d, deleting the sample, and if the length is smaller than d, filling the sample with symbols </>.
3. The method for deep learning multi-classification emotion analysis combined with attention mechanism as claimed in claim 1, wherein the convolutional layer calculation formula of the CNN network-based local feature extraction module is as follows:
z=f(∑WT*xi:i+s-1+b) (8)
wherein z represents a feature vector obtained by convolution of a neuron with a text to be analyzed, f (-) represents an activation function, and W is equal to Rs×mWeight matrix representing neurons, shared by the same neuron parameter, sxm represents the size of convolution kernel, b represents threshold, x representsi:i+s-1A word vector representing the i-th word to the i + s-1 words in the text sentence.
4. The method as claimed in claim 1, wherein the training data is preprocessed data.
5. The method for deep learning multi-classification emotion analysis combined with attention mechanism as claimed in claim 1, wherein the convolutional layer of the local feature extraction stage based on the CNN network employs 4 convolutional kernels with different scales.
6. The method as claimed in claim 1, wherein the end condition of the training is that the accuracy is not changed or the set number of iterations is reached.
CN201910553755.7A 2019-06-25 2019-06-25 A Deep Learning Multi-Class Sentiment Analysis Model Combined with Attention Mechanism Active CN110287320B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910553755.7A CN110287320B (en) 2019-06-25 2019-06-25 A Deep Learning Multi-Class Sentiment Analysis Model Combined with Attention Mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910553755.7A CN110287320B (en) 2019-06-25 2019-06-25 A Deep Learning Multi-Class Sentiment Analysis Model Combined with Attention Mechanism

Publications (2)

Publication Number Publication Date
CN110287320A true CN110287320A (en) 2019-09-27
CN110287320B CN110287320B (en) 2021-03-16

Family

ID=68005491

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910553755.7A Active CN110287320B (en) 2019-06-25 2019-06-25 A Deep Learning Multi-Class Sentiment Analysis Model Combined with Attention Mechanism

Country Status (1)

Country Link
CN (1) CN110287320B (en)

Cited By (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110855474A (en) * 2019-10-21 2020-02-28 广州杰赛科技股份有限公司 Network feature extraction method, device, equipment and storage medium of KQI data
CN110866113A (en) * 2019-09-30 2020-03-06 浙江大学 A text classification method based on sparse self-attention mechanism fine-tuning Burt model
CN111079547A (en) * 2019-11-22 2020-04-28 武汉大学 A method of pedestrian moving direction recognition based on mobile phone inertial sensor
CN111079985A (en) * 2019-11-26 2020-04-28 昆明理工大学 A BERT-based Criminal Case Sentence Prediction Method Integrating Distinguished Attribute Features
CN111291832A (en) * 2020-03-11 2020-06-16 重庆大学 Sensor data classification method based on Stack integrated neural network
CN111339768A (en) * 2020-02-27 2020-06-26 携程旅游网络技术(上海)有限公司 Sensitive text detection method, system, electronic device and medium
CN111402953A (en) * 2020-04-02 2020-07-10 四川大学 Protein sequence classification method based on hierarchical attention network
CN111582397A (en) * 2020-05-14 2020-08-25 杭州电子科技大学 CNN-RNN image emotion analysis method based on attention mechanism
CN111881262A (en) * 2020-08-06 2020-11-03 重庆邮电大学 Text emotion analysis method based on multi-channel neural network
CN111914084A (en) * 2020-01-09 2020-11-10 北京航空航天大学 Deep learning-based emotion label text generation and evaluation system
CN112598065A (en) * 2020-12-25 2021-04-02 天津工业大学 Memory-based gated convolutional neural network semantic processing system and method
CN112597279A (en) * 2020-12-25 2021-04-02 北京知因智慧科技有限公司 Text emotion analysis model optimization method and device
CN112818123A (en) * 2021-02-08 2021-05-18 河北工程大学 Emotion classification method for text
CN113177111A (en) * 2021-05-28 2021-07-27 中国人民解放军国防科技大学 Chinese text emotion analysis method and device, computer equipment and storage medium
CN113239199A (en) * 2021-05-18 2021-08-10 重庆邮电大学 Credit classification method based on multi-party data set
CN113268592A (en) * 2021-05-06 2021-08-17 天津科技大学 Short text object emotion classification method based on multi-level interactive attention mechanism
CN113377901A (en) * 2021-05-17 2021-09-10 内蒙古工业大学 Mongolian text emotion analysis method based on multi-size CNN and LSTM models
WO2021174922A1 (en) * 2020-03-02 2021-09-10 平安科技(深圳)有限公司 Statement sentiment classification method and related device
CN113379818A (en) * 2021-05-24 2021-09-10 四川大学 Phase analysis method based on multi-scale attention mechanism network
CN113742401A (en) * 2020-05-27 2021-12-03 阿里巴巴集团控股有限公司 Data display method, device, equipment and storage medium
CN114298025A (en) * 2021-12-01 2022-04-08 国家电网有限公司华东分部 Emotion analysis method based on artificial intelligence
CN114385813A (en) * 2022-01-11 2022-04-22 北京工业大学 Water environment text aspect-level viewpoint mining method based on multi-feature fusion
CN114547299A (en) * 2022-02-18 2022-05-27 重庆邮电大学 Short text sentiment classification method and device based on composite network model
CN114662547A (en) * 2022-04-07 2022-06-24 天津大学 MSCRNN emotion recognition method and device based on EEG signal
CN114897078A (en) * 2022-05-19 2022-08-12 辽宁大学 Short text similarity calculation method based on deep learning and topic model
CN115116448A (en) * 2022-08-29 2022-09-27 四川启睿克科技有限公司 Voice extraction method, neural network model training method, device and storage medium
US20230160942A1 (en) * 2020-04-22 2023-05-25 Qingdao Topscomm Communication Co., Ltd Fault arc signal detection method using convolutional neural network

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108460089A (en) * 2018-01-23 2018-08-28 哈尔滨理工大学 Diverse characteristics based on Attention neural networks merge Chinese Text Categorization
CN109670169A (en) * 2018-11-16 2019-04-23 中山大学 A kind of deep learning sensibility classification method based on feature extraction
CN109710761A (en) * 2018-12-21 2019-05-03 中国标准化研究院 Sentiment analysis method based on attention-enhanced bidirectional LSTM model

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108460089A (en) * 2018-01-23 2018-08-28 哈尔滨理工大学 Diverse characteristics based on Attention neural networks merge Chinese Text Categorization
CN109670169A (en) * 2018-11-16 2019-04-23 中山大学 A kind of deep learning sensibility classification method based on feature extraction
CN109710761A (en) * 2018-12-21 2019-05-03 中国标准化研究院 Sentiment analysis method based on attention-enhanced bidirectional LSTM model

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
MING-HSIANG SU.ETL: "LSTM-based Text Emotion Recognition Using Semantic and Emotional Word Vectors", 《 2018 FIRST ASIAN CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION》 *
THITITORN SENEEWONG NA AYUTTHAYA.ETL: "Thai Sentiment Analysis via Bidirectional LSTM-CNN Model with Embedding Vectors and Sentic Features", 《2018 INTERNATIONAL JOINT SYMPOSIUM ON ARTIFICIAL INTELLIGENCE AND NATURAL LANGUAGE PROCESSING》 *
关鹏飞等: "注意力增强的双向LSTM情感分析", 《中文信息学报》 *

Cited By (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110866113A (en) * 2019-09-30 2020-03-06 浙江大学 A text classification method based on sparse self-attention mechanism fine-tuning Burt model
CN110866113B (en) * 2019-09-30 2022-07-26 浙江大学 Text classification method based on sparse self-attention mechanism fine-tuning burt model
CN110855474A (en) * 2019-10-21 2020-02-28 广州杰赛科技股份有限公司 Network feature extraction method, device, equipment and storage medium of KQI data
CN110855474B (en) * 2019-10-21 2022-06-17 广州杰赛科技股份有限公司 Network feature extraction method, device, equipment and storage medium of KQI data
CN111079547A (en) * 2019-11-22 2020-04-28 武汉大学 A method of pedestrian moving direction recognition based on mobile phone inertial sensor
CN111079985A (en) * 2019-11-26 2020-04-28 昆明理工大学 A BERT-based Criminal Case Sentence Prediction Method Integrating Distinguished Attribute Features
CN111914084A (en) * 2020-01-09 2020-11-10 北京航空航天大学 Deep learning-based emotion label text generation and evaluation system
CN111339768A (en) * 2020-02-27 2020-06-26 携程旅游网络技术(上海)有限公司 Sensitive text detection method, system, electronic device and medium
CN111339768B (en) * 2020-02-27 2024-03-05 携程旅游网络技术(上海)有限公司 Sensitive text detection method, system, electronic equipment and medium
WO2021174922A1 (en) * 2020-03-02 2021-09-10 平安科技(深圳)有限公司 Statement sentiment classification method and related device
CN111291832A (en) * 2020-03-11 2020-06-16 重庆大学 Sensor data classification method based on Stack integrated neural network
CN111402953A (en) * 2020-04-02 2020-07-10 四川大学 Protein sequence classification method based on hierarchical attention network
CN111402953B (en) * 2020-04-02 2022-05-03 四川大学 A protein sequence classification method based on hierarchical attention network
US20230160942A1 (en) * 2020-04-22 2023-05-25 Qingdao Topscomm Communication Co., Ltd Fault arc signal detection method using convolutional neural network
US11860216B2 (en) * 2020-04-22 2024-01-02 Qingdao Topscomm Communication Co., Ltd Fault arc signal detection method using convolutional neural network
CN111582397A (en) * 2020-05-14 2020-08-25 杭州电子科技大学 CNN-RNN image emotion analysis method based on attention mechanism
CN111582397B (en) * 2020-05-14 2023-04-07 杭州电子科技大学 CNN-RNN image emotion analysis method based on attention mechanism
CN113742401A (en) * 2020-05-27 2021-12-03 阿里巴巴集团控股有限公司 Data display method, device, equipment and storage medium
CN111881262A (en) * 2020-08-06 2020-11-03 重庆邮电大学 Text emotion analysis method based on multi-channel neural network
CN111881262B (en) * 2020-08-06 2022-05-20 重庆邮电大学 Text sentiment analysis method based on multi-channel neural network
CN112598065B (en) * 2020-12-25 2023-05-30 天津工业大学 Memory-based gating convolutional neural network semantic processing system and method
CN112597279A (en) * 2020-12-25 2021-04-02 北京知因智慧科技有限公司 Text emotion analysis model optimization method and device
CN112598065A (en) * 2020-12-25 2021-04-02 天津工业大学 Memory-based gated convolutional neural network semantic processing system and method
CN112818123A (en) * 2021-02-08 2021-05-18 河北工程大学 Emotion classification method for text
CN113268592A (en) * 2021-05-06 2021-08-17 天津科技大学 Short text object emotion classification method based on multi-level interactive attention mechanism
CN113377901A (en) * 2021-05-17 2021-09-10 内蒙古工业大学 Mongolian text emotion analysis method based on multi-size CNN and LSTM models
CN113377901B (en) * 2021-05-17 2022-08-19 内蒙古工业大学 Mongolian text emotion analysis method based on multi-size CNN and LSTM models
CN113239199A (en) * 2021-05-18 2021-08-10 重庆邮电大学 Credit classification method based on multi-party data set
CN113379818A (en) * 2021-05-24 2021-09-10 四川大学 Phase analysis method based on multi-scale attention mechanism network
CN113379818B (en) * 2021-05-24 2022-06-07 四川大学 A Phase Resolution Method Based on Multiscale Attention Mechanism Network
CN113177111A (en) * 2021-05-28 2021-07-27 中国人民解放军国防科技大学 Chinese text emotion analysis method and device, computer equipment and storage medium
CN114298025A (en) * 2021-12-01 2022-04-08 国家电网有限公司华东分部 Emotion analysis method based on artificial intelligence
CN114385813A (en) * 2022-01-11 2022-04-22 北京工业大学 Water environment text aspect-level viewpoint mining method based on multi-feature fusion
CN114547299A (en) * 2022-02-18 2022-05-27 重庆邮电大学 Short text sentiment classification method and device based on composite network model
CN114662547A (en) * 2022-04-07 2022-06-24 天津大学 MSCRNN emotion recognition method and device based on EEG signal
CN114897078A (en) * 2022-05-19 2022-08-12 辽宁大学 Short text similarity calculation method based on deep learning and topic model
CN114897078B (en) * 2022-05-19 2024-11-08 辽宁大学 Short text similarity calculation method based on deep learning and topic model
CN115116448A (en) * 2022-08-29 2022-09-27 四川启睿克科技有限公司 Voice extraction method, neural network model training method, device and storage medium
CN115116448B (en) * 2022-08-29 2022-11-15 四川启睿克科技有限公司 Voice extraction method, neural network model training method, device and storage medium

Also Published As

Publication number Publication date
CN110287320B (en) 2021-03-16

Similar Documents

Publication Publication Date Title
CN110287320B (en) A Deep Learning Multi-Class Sentiment Analysis Model Combined with Attention Mechanism
CN107608956B (en) Reader emotion distribution prediction algorithm based on CNN-GRNN
CN108334605B (en) Text classification method and device, computer equipment and storage medium
CN109284506B (en) User comment emotion analysis system and method based on attention convolution neural network
CN109241255B (en) An Intent Recognition Method Based on Deep Learning
CN110502753A (en) A Deep Learning Sentiment Analysis Model Based on Semantic Enhancement and Its Analysis Method
CN109299253A (en) A Construction Method of Chinese Social Text Emotion Recognition Model Based on Deep Fusion Neural Network
CN108537624B (en) A travel service recommendation method based on deep learning
CN110083700A (en) A kind of enterprise&#39;s public sentiment sensibility classification method and system based on convolutional neural networks
CN108009148B (en) Text sentiment classification and representation method based on deep learning
CN110287323B (en) Target-oriented emotion classification method
CN108763216A (en) A kind of text emotion analysis method based on Chinese data collection
CN107818084B (en) Emotion analysis method fused with comment matching diagram
CN109376242A (en) Text Classification Algorithms Based on Variants of Recurrent Neural Networks and Convolutional Neural Networks
CN108614875A (en) Chinese emotion tendency sorting technique based on global average pond convolutional neural networks
CN110543242A (en) expression input method based on BERT technology and device thereof
CN110188195B (en) Text intention recognition method, device and equipment based on deep learning
CN107247703A (en) Microblog emotional analysis method based on convolutional neural networks and integrated study
CN111985247A (en) Microblog user interest identification method and system based on multi-granularity text feature representation
CN110472052A (en) A kind of Chinese social platform sentiment analysis method based on deep learning
CN112749274A (en) Chinese text classification method based on attention mechanism and interference word deletion
CN112364638A (en) Personality identification method based on social text
CN113505226B (en) Text emotion classification system fusing graph convolution neural network
CN113094502A (en) Multi-granularity takeaway user comment sentiment analysis method
CN110046223B (en) Sentiment analysis method of movie reviews based on improved convolutional neural network model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant