CN110287320A - A deep learning multi-category sentiment analysis model combined with attention mechanism - Google Patents
A deep learning multi-category sentiment analysis model combined with attention mechanism Download PDFInfo
- Publication number
- CN110287320A CN110287320A CN201910553755.7A CN201910553755A CN110287320A CN 110287320 A CN110287320 A CN 110287320A CN 201910553755 A CN201910553755 A CN 201910553755A CN 110287320 A CN110287320 A CN 110287320A
- Authority
- CN
- China
- Prior art keywords
- word
- cnn
- text
- model
- feature extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 37
- 238000013135 deep learning Methods 0.000 title claims abstract description 22
- 230000007246 mechanism Effects 0.000 title claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 50
- 230000002996 emotional effect Effects 0.000 claims abstract description 18
- 239000013598 vector Substances 0.000 claims description 52
- 238000000034 method Methods 0.000 claims description 28
- 230000008451 emotion Effects 0.000 claims description 19
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000011176 pooling Methods 0.000 claims description 17
- 238000012549 training Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 13
- 210000002569 neuron Anatomy 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000007635 classification algorithm Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims 2
- 238000002474 experimental method Methods 0.000 abstract description 6
- 238000003058 natural language processing Methods 0.000 abstract description 2
- 230000000052 comparative effect Effects 0.000 abstract 1
- 238000013527 convolutional neural network Methods 0.000 description 29
- 238000012360 testing method Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000009849 deactivation Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 238000010835 comparative analysis Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000002354 daily effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Machine Translation (AREA)
Abstract
本发明涉及一种结合注意力机制的深度学习多分类情感分析模型,属于自然语言处理技术领域,本发明分析了现有CNN网络和LSTM网络在文本情感分析方面的弱点,提出了一种结合注意力机制的深度学习多分类情感分析模型。该模型运用注意力机制将CNN网络提取的局部特征和LSTM模型提取的语序特征相融合,并在分类层采用集成模型的思想,分别将CNN网络和LSTM网络提取的情感特征拼接,作为模型最终提取的情感特征。通过对比实验,发现该模型的准确率有了显著的提高。
The invention relates to a deep learning multi-category sentiment analysis model combined with an attention mechanism, which belongs to the technical field of natural language processing. The invention analyzes the weaknesses of the existing CNN network and LSTM network in text sentiment analysis, and proposes a combination of attention A deep learning multi-category sentiment analysis model based on force mechanism. The model uses the attention mechanism to integrate the local features extracted by the CNN network and the word order features extracted by the LSTM model, and adopts the idea of an integrated model at the classification layer to splicing the emotional features extracted by the CNN network and the LSTM network respectively as the final extraction of the model. emotional characteristics. Through comparative experiments, it is found that the accuracy of the model has been significantly improved.
Description
技术领域technical field
本发明属于文本信息处理领域,涉及一种结合注意力机制的深度学习多分类情感分析模型。The invention belongs to the field of text information processing, and relates to a deep learning multi-category sentiment analysis model combined with an attention mechanism.
背景技术Background technique
随着微博、Twitter等社交网络的不断兴起,互联网已不仅仅是人们获取日常信息的来源,同时也成为人们表达自己观点不可或缺的平台。人们在网络社区评论热点事件、抒写影评观点以及描述产品体验等,都会产生大量的带有情感色彩(如:喜怒哀乐等)的文本信息,而对这些文本信息进行有效的情感分析,可以更好地了解用户的兴趣倾向和关注程度。但随着人们对网络信息关注度的增加,网络社区每天都有海量的带有情感色彩的文本产生,如果仅仅依靠人工标记,已经远远无法完成这一任务,这就使得文本情感分析成为自然语言处理领域的一个研究热点。With the continuous rise of social networks such as Weibo and Twitter, the Internet is not only a source for people to obtain daily information, but also an indispensable platform for people to express their opinions. People comment on hot events in online communities, express opinions on film reviews, and describe product experience, etc., which will generate a large amount of text information with emotional color (such as: emotions, etc.), and effective sentiment analysis of these text information can be more effective. Get a better understanding of the user's interest tendencies and attention levels. However, with the increase of people's attention to network information, a large number of texts with emotional color are produced in the network community every day. If only relying on manual marking, it is far from being able to complete this task, which makes text sentiment analysis a natural A research hotspot in the field of language processing.
随着深度学习方法在计算机视觉方向的成功应用,越来越多的深度学习技术也被应用于自然语言处理方向。深度学习的优势在于,不仅可以自动提取文本的特征,而且对大数据有较强的表达能力。目前主流的基于深度学习的文本情感分析方法主要包括卷积神经网络(Convolutional Neural Network,CNN)和循环神经网络(Recurrent NeuralNetwork,RNN)两种,基于这两种方法的情感分析模型的准确率都较低,主要有以下几个方面的原因:With the successful application of deep learning methods in the direction of computer vision, more and more deep learning techniques have also been applied in the direction of natural language processing. The advantage of deep learning is that it can not only automatically extract the features of text, but also has a strong ability to express big data. The current mainstream text sentiment analysis methods based on deep learning mainly include Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). The accuracy of the sentiment analysis model based on these two methods is very low. lower, mainly due to the following reasons:
首先,在文本的情感分析过程中,卷积神经网络通过扩大卷积核尺寸,有效地捕捉到不同位置的情感信息,进而获取到文本的局部情感特征。但卷积的过程中,往往忽略了文本中语序间的前后关系。但在文本情感分析过程中,语序的先后关系又十分重要,没有语序的特征信息必导致结果有一定的偏差。First of all, in the process of text sentiment analysis, the convolutional neural network can effectively capture the emotional information of different positions by expanding the size of the convolution kernel, and then obtain the local emotional features of the text. However, in the process of convolution, the contextual relationship between the word order in the text is often ignored. However, in the process of text sentiment analysis, the sequence of word order is very important, and the lack of characteristic information of word order will lead to certain deviations in the results.
其次,循环神经网络网络利用前后依赖关系,有效模拟文本数据的先后顺序,能够提取到文本的语序关系和语义信息,因此在文本的情感分析中能达到很好的效果。但当样本数据较长或者语言场景较复杂时,有用的情感信息的间隔有大有小,长短不一,长短记忆网络(Long Short-Term Memory,LSTM)的性能也因此受到限制。Secondly, the cyclic neural network network uses the front and back dependencies to effectively simulate the sequence of text data, and can extract the word order relationship and semantic information of the text, so it can achieve good results in text sentiment analysis. However, when the sample data is long or the language scene is complex, the intervals of useful emotional information vary in size and length, and the performance of Long Short-Term Memory (LSTM) is therefore limited.
本发明充分利用了注意力机制、CNN网络、LSTM网络,提出并实现了一种结合注意力机制的深度学习多分类情感分析模型。此模型能够有效的提高文本情感分析的准确率。The present invention makes full use of attention mechanism, CNN network and LSTM network, proposes and implements a deep learning multi-category sentiment analysis model combined with attention mechanism. This model can effectively improve the accuracy of text sentiment analysis.
发明内容Contents of the invention
本发明提出了一种基于注意力机制的深度学习多分类情感分析模型。该模型结合CNN网络和LSTM网络进行情感特征融合。首先利用CNN网络的多尺度卷积核提取出待分析文本的局部特征,然后利用注意力机制,将CNN网络提取的局部特征融入到LSTM网络中。最后利用集成模型的思想,将CNN网络的池化层结果和LSTM网络的特征提取结果进行拼接,作为最终的模型输出。实验表明,在文本情感分析中,该模型的准确率有了显著的提高。The present invention proposes a deep learning multi-category sentiment analysis model based on an attention mechanism. This model combines CNN network and LSTM network for emotional feature fusion. First, the multi-scale convolution kernel of the CNN network is used to extract the local features of the text to be analyzed, and then the attention mechanism is used to integrate the local features extracted by the CNN network into the LSTM network. Finally, using the idea of the integrated model, the pooling layer results of the CNN network and the feature extraction results of the LSTM network are spliced as the final model output. Experiments show that the accuracy of the model has been significantly improved in text sentiment analysis.
为实现上述目的,本发明采用如下的技术方案:To achieve the above object, the present invention adopts the following technical solutions:
1.一种结合注意力机制的深度学习多分类情感分析方法,其特征在于包括以下步骤:1. a deep learning multi-category sentiment analysis method combined with attention mechanism, is characterized in that comprising the following steps:
步骤(1)数据预处理Step (1) data preprocessing
设情感数据集表示为:G=[(segtxt1,y1),(segtxt2,y2),......,(segtxtN,yN)],其中,segtxti表示第i个样本,yi则为对应的情感类别标签,N表示数据集G中样本个数,对G中样本进行数据预处理,Let the emotional data set be expressed as: G=[(segtxt 1 ,y 1 ),(segtxt 2 ,y 2 ),...,(segtxt N ,y N )], where segtxt i represents the i-th sample, y i is the corresponding emotional category label, N represents the number of samples in the data set G, and data preprocessing is performed on the samples in G,
数据集G经预处理后,表示为G′=[(seg1,y1),(seg2,y2),...,(segM,yM)],其中:segi表示为数据集G′中第i个样本,yi则为对应的情感类别标签,M表示数据集G′中样本个数;After the data set G is preprocessed, it is expressed as G′=[(seg 1 ,y 1 ),(seg 2 ,y 2 ),...,(seg M ,y M )], where: seg i is expressed as data The i-th sample in the set G′, y i is the corresponding emotional category label, and M represents the number of samples in the data set G′;
步骤(2)构建模型的输入Step (2) Build the input of the model
对于数据集G′中任意一个待分析样本数据(seg,y),将其进一步细化表示为:For any sample data (seg, y) to be analyzed in the data set G′, it is further refined as:
seg=[w1,w2,w3,...,wd]T (1)seg=[w 1 ,w 2 ,w 3 ,...,w d ] T (1)
y=[0,0,1,...,0] (2)y=[0,0,1,...,0] (2)
其中:wi∈Rε是指依据词表wordList对待分析文本中第i词语的one-hot编码,ε为词表wordList的大小,d表示该文本的句长。y∈Rp是依据情感类别的one-hot编码,p表示模型待分的类别数目。则该样本的词向量嵌入矩阵可表示为:Among them: w i ∈ R ε refers to the one-hot encoding of the i-th word in the text to be analyzed according to the vocabulary wordList, ε is the size of the vocabulary wordList, and d represents the sentence length of the text. y∈R p is the one-hot encoding based on the emotional category, and p represents the number of categories to be classified by the model. Then the word vector embedding matrix of this sample can be expressed as:
X=seg*ET (3)X=seg*E T (3)
其中:X∈Rd×m,X=[x1,x2,...,xd]T为待分析文本的词向量矩阵表示,m为词向量的维度,xi∈Rm为该文本中第i个词汇的词向量表示,E为词向量嵌入层表示;Among them: X∈R d×m , X=[x 1 ,x 2 ,...,x d ] T is the word vector matrix representation of the text to be analyzed, m is the dimension of the word vector, x i ∈ R m is the The word vector representation of the i-th vocabulary in the text, E is the word vector embedding layer representation;
步骤(3)构建深度学习多分类情感分析模型Step (3) Build a deep learning multi-category sentiment analysis model
深度学习多分类情感分析模型包括基于CNN网络的局部特征提取阶段和基于LSTM网络的语序关系特征提取阶段,将基于CNN网络的局部特征提取阶段的池化层结果CCnn和基于LSTM网络的语序关系特征提取阶段的结果C'Rnn拼接,即向量[CCnn;C'Rnn]作为模型最终提取的特征向量。然后将特征向量[CCnn;C'Rnn]经过全连接层得到最终的模型输出向量其中p表示模型待分的类别数目。The deep learning multi-category sentiment analysis model includes the stage of local feature extraction based on CNN network and the stage of word order relationship feature extraction based on LSTM network. The results of the feature extraction stage C' Rnn splicing, that is, the vector [C Cnn ; C' Rnn ] is used as the feature vector finally extracted by the model. Then pass the feature vector [C Cnn ; C' Rnn ] through the fully connected layer to get the final model output vector where p represents the number of categories to be classified by the model.
所述的基于CNN网络的局部特征提取阶段,包括以下内容:The described local feature extraction stage based on the CNN network includes the following:
局部特征提取阶段输入为公式3的待分析文本的词向量矩阵表示X;In the local feature extraction stage, the input is the word vector matrix representation X of the text to be analyzed in formula 3;
局部特征提取阶段基于CNN网络,一共包括两层,即一层卷积层、一层池化层,其中:The local feature extraction stage is based on the CNN network, which includes two layers, namely a convolutional layer and a pooling layer, in which:
卷积层采用n种不同尺度的卷积核对待分析文本进行卷积,且同一尺度卷积核的滤波器即神经元各k个;The convolutional layer uses n types of convolution kernels of different scales to convolve the text to be analyzed, and the filters of the same scale convolution kernels are k neurons;
池化层采用最大池化层的方法将卷积所得的向量做下采样,选出局部最优特征,因此每个滤波器通过最大池化层变为一个标量,该标量代表着该滤波器中最优的情感特征;The pooling layer uses the method of the maximum pooling layer to down-sample the vector obtained by convolution and select the local optimal features, so each filter becomes a scalar through the maximum pooling layer, which represents the Optimal emotional characteristics;
局部特征提取模块的输出为CCnn=[c1,c2,...,cnk],即将池化层中不同尺寸的多个滤波器选取的最优特征拼接到一起CCnn=[c1,c2,...,cnk]作为本模块的输出,其中,CCnn∈Rnk,nk为卷积层中所有滤波器的个数;The output of the local feature extraction module is C Cnn =[c 1 ,c 2 ,...,c nk ], that is, the optimal features selected by multiple filters of different sizes in the pooling layer are spliced together C Cnn =[c 1 ,c 2 ,...,c nk ] as the output of this module, where C Cnn ∈ R nk , nk is the number of all filters in the convolutional layer;
所述的基于LSTM网络的语序关系特征提取阶段,包括以下内容:Described word order relation feature extraction stage based on LSTM network, comprises the following contents:
多尺度CNN网络局部特征提取:将基于CNN网络的局部特征提取阶段中卷积层同一卷积尺度的k个滤波器的卷积结果拼接,得到集合ZCnn,然后将集合ZCnn中的每个向量Zi输入到GLU机制中,即门控卷积网络,得到的结果记为{π1,π2,...,πn},完成多尺度CNN网络局部特征的提取。Multi-scale CNN network local feature extraction: Concatenate the convolution results of k filters of the same convolution scale in the convolutional layer in the local feature extraction stage based on the CNN network to obtain a set Z Cnn , and then combine each of the set Z Cnn The vector Z i is input into the GLU mechanism, that is, the gated convolutional network, and the obtained results are recorded as {π 1 ,π 2 ,...,π n } to complete the extraction of local features of the multi-scale CNN network.
其中,ZCnn={Z1,Z2,...,Zn},Zi为尺度为i的多个滤波器卷积结果的拼接;Among them, Z Cnn = {Z 1 , Z 2 ,..., Z n }, Z i is the concatenation of multiple filter convolution results with a scale of i;
其中,Zi代表某一尺度的k个滤波器卷积结果的拼接,W1,W2∈Rλ×q为权重矩阵,λ表示对应权重矩阵的维度,的b1,b2∈Rq为偏置量,σ表示sigmoid函数,πi∈Rq,q为LSTM网络的输出维度;in, Z i represents the concatenation of k filter convolution results of a certain scale, W 1 , W 2 ∈ R λ×q is the weight matrix, λ represents the dimension of the corresponding weight matrix, and b 1 , b 2 ∈ R q is the bias Set amount, σ represents the sigmoid function, π i ∈ R q , q is the output dimension of the LSTM network;
然后,利用注意力机制,将多尺度CNN网络局部特征提取结果{π1,π2,...,πn}融入到LSTM网络中,得到基于LSTM网络的语序关系特征提取阶段的输出结果C'Rnn,即 Then, using the attention mechanism, the multi-scale CNN network local feature extraction results {π 1 ,π 2 ,...,π n } are integrated into the LSTM network, and the output result C of the word order relationship feature extraction stage based on the LSTM network is obtained ' Rnn , ie
其中,表示待分析文本中最后一个词语所对应的LSTM模块的输出,表示待分析文本中第一个词语所对应的LSTM模块的输出,本发明采用双向LSTM模型,即BiLSTM模型,in, Indicates the output of the LSTM module corresponding to the last word in the text to be analyzed, Indicates the output of the LSTM module corresponding to the first word in the text to be analyzed. The present invention adopts a bidirectional LSTM model, namely the BiLSTM model,
采用正向传播,具体计算过程如下: Using forward propagation, the specific calculation process is as follows:
d为待分析文本的长度,该文本中每一个词语顺序对应一个LSTM模块,,d is the length of the text to be analyzed, each word sequence in the text corresponds to an LSTM module,,
正向传播过程中,第t-1个LSTM模块的输出为则第t个LSTM模块的输出计算公式如下:During forward propagation, the output of the t-1th LSTM module is Then the output of the tth LSTM module Calculated as follows:
其中:是两个向量的点乘,也称打分函数,是用来计算前一词语的LSTM的输出和当前局部特征向量的相似度,in: Is the dot product of two vectors, also known as the scoring function, which is used to calculate the output of the LSTM of the previous word The similarity with the current local feature vector,
其中:αt,i∈R代表特征πi的权重,Among them: α t,i ∈ R represents the weight of feature π i ,
其中:st-1∈Rq是多个卷积特征的加权结果,利用st-1代替结合当前词语的词向量xt求得当前LSTM模块的输出公式如下:Among them: s t-1 ∈ R q is the weighted result of multiple convolutional features, using s t-1 instead Combine the word vector x t of the current word to obtain the output of the current LSTM module The formula is as follows:
采用反向传播,具体计算过程与正向传播一样,此处不再赘述; Using backpropagation, the specific calculation process is the same as that of forward propagation, and will not be repeated here;
步骤(4)模型训练:将训练数据输入多分类情感分析模型,采用交叉熵损失函数,结合反向传播BP算法调整参数,利用softmax回归作为分类算法,完成训练;Step (4) Model training: Input the training data into the multi-category sentiment analysis model, use the cross-entropy loss function, adjust the parameters in combination with the backpropagation BP algorithm, and use softmax regression as the classification algorithm to complete the training;
步骤(5)模型分析:将待分析文本输入训练完成的模型,最终输出对该文本分析后的情感分类结果。Step (5) Model analysis: Input the text to be analyzed into the trained model, and finally output the sentiment classification result after analyzing the text.
所述的预处理过程包括以下步骤:Described pretreatment process comprises the following steps:
1)分词、去除停用、英文大写转小写、繁体转简体。1) Word segmentation, deactivation, uppercase to lowercase, and traditional to simplified.
2)选取数据集G中频率大于等于σ的词语,构造词汇表wordList={word1,word2,...wordε},其中,wordi表示词汇表wordlist中第i个词语,ε表示数据集G中词频超过σ的词语总数。2) Select words whose frequency is greater than or equal to σ in the data set G, and construct a vocabulary wordList={word 1 ,word 2 ,...word ε }, where word i represents the i-th word in the vocabulary wordlist, and ε represents the data The total number of words in the set G whose word frequency exceeds σ.
3)对数据集G中每一个样本,若长度大于d,则删除该样本,若长度小于d,则用符号</>补齐。3) For each sample in the data set G, if the length is greater than d, delete the sample, and if the length is less than d, fill it with the symbol </>.
所述的基于CNN网络的局部特征提取模块的卷积层计算公式如下:The convolutional layer calculation formula of the local feature extraction module based on the CNN network is as follows:
z=f(∑WT*xi:i+s-1+b) (8)z=f(∑W T *x i:i+s-1 +b) (8)
其中:z表示一个神经元对待分析文本的卷积所得的特征向量,f(·)表示激活函数,W∈Rs×m表示神经元的权重矩阵,同一个神经元参数共享,s×m表示卷积核尺寸的大小,b表示阈值,xi:i+s-1表示由文本句子中的第i个词到i+s-1个词语的词向量。Among them: z represents the feature vector obtained by the convolution of a neuron to analyze the text, f(·) represents the activation function, W∈R s×m represents the weight matrix of the neuron, the same neuron parameters are shared, s×m represents The size of the convolution kernel, b represents the threshold, x i:i+s-1 represents the word vector from the i-th word to the i+s-1 word in the text sentence.
所述的训练数据为经过预处理之后的数据。The training data is preprocessed data.
所述的基于CNN网络的局部特征提取阶段的卷积层采用4种不同尺度的卷积核。所述的训练结束条件是准确率不再改变或达到设置迭代次数。The convolution layer of the local feature extraction stage based on the CNN network adopts four convolution kernels of different scales. The training end condition is that the accuracy rate does not change or reaches the set number of iterations.
附图说明Description of drawings
图1本发明的方法流程图;Fig. 1 method flowchart of the present invention;
图2结合注意力机制的深度学习多分类情感分析模型结构示意图。Figure 2 Schematic diagram of the structure of the deep learning multi-category sentiment analysis model combined with the attention mechanism.
具体实施方式Detailed ways
下面结合图表和实施例,对本发明的具体实施方式作进一步的详细描述。以下实施例用于说明本发明,但不用来限制本发明的范围。The specific implementation of the present invention will be further described in detail below in conjunction with the diagrams and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
本发明提出的方法是依次按以下步骤实现的:The method that the present invention proposes is to realize by following steps successively:
步骤(1)数据预处理Step (1) data preprocessing
情感语数据集表示为:G=[(segtxt1,y1),(segtxt2,y2),......,(segtxtN,yN)],其中,segtxti表示第i个样本,yi则为对应的情感类别标签。N表示数据集G中样本个数,情感标签取“喜悦”、“愤怒”、“厌恶”、“低落”四大类,N取80000,其中,四类情感样本各20000条。对G中样本进行数据预处理包括以下几个步骤:The emotional language data set is expressed as: G=[(segtxt 1 ,y 1 ),(segtxt 2 ,y 2 ),...,(segtxt N ,y N )], where segtxt i represents the i-th samples, and y i is the corresponding emotion category label. N represents the number of samples in the data set G, and the emotion labels are divided into four categories: "joy", "anger", "disgust", and "depression". N is 80,000, and each of the four types of emotion samples is 20,000. Data preprocessing for samples in G includes the following steps:
1)分词、去除停用、英文大写转小写、繁体转简体。1) Word segmentation, deactivation, uppercase to lowercase, and traditional to simplified.
2)选取数据集G中频率大于等于σ的词语,构造词汇表wordList={word1,word2,...wordε},其中,wordi表示数据集G中第i个词语,ε表示数据集G中词频超过σ的词语总数。σ取2,最终得到的数据集G中,词频大于等于2的词语共41763个,即ε为41763。2) Select the words whose frequency is greater than or equal to σ in the data set G, and construct a vocabulary wordList={word 1 , word 2 ,...word ε }, where word i represents the i-th word in the data set G, and ε represents the data The total number of words in the set G whose word frequency exceeds σ. If σ is set to 2, in the final data set G, there are 41763 words with a word frequency greater than or equal to 2, that is, ε is 41763.
3)将上述处理后,对数据集G中每一个样本,若长度大于d,则删除该样本,若长度小于d,则用符号</>补齐。d取64。3) After the above processing, for each sample in the data set G, if the length is greater than d, delete the sample, and if the length is less than d, fill it with the symbol </>. d takes 64.
数据集G经预处理后,表示为G′=[(seg1,y1),(seg2,y2),...,(segM,yM)]。其中:segi表示为数据集G′中第i个样本,yi则为对应的情感类别标签,M表示数据集G′中样本个数。最终数据集G′中样本的个数为73150条,各情感类别的样本条数如表1所示:After the data set G is preprocessed, it is expressed as G′=[(seg 1 ,y 1 ),(seg 2 ,y 2 ),...,(seg M ,y M )]. Among them: seg i represents the i-th sample in the data set G′, y i represents the corresponding emotion category label, and M represents the number of samples in the data set G′. The number of samples in the final data set G′ is 73150, and the number of samples of each emotion category is shown in Table 1:
表1预处理后各类别样本数量Table 1 Number of samples in each category after preprocessing
步骤(2)模型的输入Step (2) The input of the model
对于数据集G′中任意一个待分析样本数据(seg,y),将其进一步细化表示为:For any sample data (seg, y) to be analyzed in the data set G′, it is further refined as:
seg=[w1,w2,w3,...,wd]T (1)seg=[w 1 ,w 2 ,w 3 ,...,w d ] T (1)
y=[0,0,1,...,0] (2)y=[0,0,1,...,0] (2)
其中:wi∈Rε是指依据词表wordList对待分析文本中第i词语的one-hot编码,ε为词表wordList的大小,文本的句长d为64。y∈Rp是依据情感类别的one-hot编码,p表示模型待分的类别数目,p为4。则该样本的词向量嵌入矩阵可表示为:Among them: w i ∈ R ε refers to the one-hot encoding of the i-th word in the analyzed text according to the vocabulary wordList, ε is the size of the vocabulary wordList, and the sentence length d of the text is 64. y∈R p is the one-hot encoding based on the emotional category, p represents the number of categories to be classified by the model, and p is 4. Then the word vector embedding matrix of this sample can be expressed as:
X=seg*ET (3)X=seg*E T (3)
其中:X∈Rd×m,X=[x1,x2,...,xd]T为待分析文本的词向量矩阵表示,词向量维度m取256。xi∈Rm为该文本中第i个词汇的词向量表示,词向量嵌入层表示E,采用维基百科开源word2vec词向量,接下来将X作为网络模型的输入。Where: X∈R d×m , X=[x 1 ,x 2 ,...,x d ] T is the word vector matrix representation of the text to be analyzed, and the word vector dimension m is 256. x i ∈ R m is the word vector representation of the i-th vocabulary in the text, the word vector embedding layer represents E, using Wikipedia's open source word2vec word vector, and then X is used as the input of the network model.
步骤(3)构建深度学习多分类情感分析模型Step (3) Build a deep learning multi-category sentiment analysis model
深度学习多分类情感分析模型包括基于CNN网络的局部特征提取阶段和基于LSTM网络的语序关系特征提取阶段,将基于CNN网络的局部特征提取阶段的池化层结果CCnn和基于LSTM网络的语序关系特征提取阶段的结果C'Rnn拼接,即向量[CCnn;C'Rnn]作为模型最终提取的特征向量。然后将特征向量[CCnn;C'Rnn]经过全连接层得到最终的模型输出向量其中p表示模型待分的类别数目。The deep learning multi-category sentiment analysis model includes the stage of local feature extraction based on CNN network and the stage of word order relationship feature extraction based on LSTM network. The results of the feature extraction stage C' Rnn splicing, that is, the vector [C Cnn ; C' Rnn ] is used as the feature vector finally extracted by the model. Then pass the feature vector [C Cnn ; C' Rnn ] through the fully connected layer to get the final model output vector where p represents the number of categories to be classified by the model.
所述的基于CNN网络的局部特征提取阶段,包括以下内容:The described local feature extraction stage based on the CNN network includes the following:
局部特征提取阶段输入为公式3的待分析文本的词向量矩阵表示X;In the local feature extraction stage, the input is the word vector matrix representation X of the text to be analyzed in formula 3;
局部特征提取阶段基于CNN网络,一共包括两层,即一层卷积层、一层池化层,其中:The local feature extraction stage is based on the CNN network, which includes two layers, namely a convolutional layer and a pooling layer, in which:
卷积层采用n种不同尺度的卷积核对待分析文本进行卷积,且同一尺度卷积核的滤波器即神经元各k个,本发明n取4,k取128。The convolution layer uses n types of convolution kernels of different scales to convolve the text to be analyzed, and there are k filters of the same scale convolution kernels, that is, neurons. In the present invention, n is 4, and k is 128.
池化层采用最大池化层的方法将卷积所得的向量做下采样,选出局部最优特征,因此每个滤波器通过最大池化层变为一个标量,该标量代表着该滤波器中最优的情感特征;The pooling layer uses the method of the maximum pooling layer to down-sample the vector obtained by convolution and select the local optimal features, so each filter becomes a scalar through the maximum pooling layer, which represents the Optimal emotional characteristics;
局部特征提取模块的输出为CCnn=[c1,c2,...,cnk],即将池化层中不同尺寸的多个滤波器选取的最优特征拼接到一起CCnn=[c1,c2,...,cnk]作为本模块的输出,其中,CCnn∈Rnk,nk为卷积层中所有滤波器的个数,共512个;The output of the local feature extraction module is C Cnn =[c 1 ,c 2 ,...,c nk ], that is, the optimal features selected by multiple filters of different sizes in the pooling layer are spliced together C Cnn =[c 1 ,c 2 ,...,c nk ] as the output of this module, where C Cnn ∈ R nk , nk is the number of all filters in the convolutional layer, a total of 512;
所述的基于LSTM网络的语序关系特征提取阶段,包括以下内容:Described word order relation feature extraction stage based on LSTM network, comprises the following contents:
多尺度CNN网络局部特征提取:将基于CNN网络的局部特征提取阶段中卷积层同一卷积尺度的k个滤波器的卷积结果拼接,得到集合ZCnn,然后将集合ZCnn中的每个向量Zi输入到GLU机制中,即门控卷积网络,得到的结果记为{π1,π2,...,πn},完成多尺度CNN网络局部特征的提取。Multi-scale CNN network local feature extraction: Concatenate the convolution results of k filters of the same convolution scale in the convolutional layer in the local feature extraction stage based on the CNN network to obtain a set Z Cnn , and then combine each of the set Z Cnn The vector Z i is input into the GLU mechanism, that is, the gated convolutional network, and the obtained results are recorded as {π 1 ,π 2 ,...,π n } to complete the extraction of local features of the multi-scale CNN network.
其中,ZCnn={Z1,Z2,...,Zn},Zi为尺度为i的多个滤波器卷积结果的拼接;Among them, Z Cnn = {Z 1 , Z 2 ,..., Z n }, Z i is the concatenation of multiple filter convolution results with a scale of i;
其中,Zi代表某一尺度的k个滤波器卷积结果的拼接,W1,W2∈Rλ×q为权重矩阵,λ表示对应权重矩阵的维度,b1,b2∈Rq为偏置量,σ表示sigmoid函数,πi∈Rq,q为LSTM网络的输出维度,q取256;in, Z i represents the concatenation of k filter convolution results of a certain scale, W 1 , W 2 ∈ R λ×q is the weight matrix, λ represents the dimension of the corresponding weight matrix, b 1 , b 2 ∈ R q is the bias Quantity, σ represents the sigmoid function, π i ∈ R q , q is the output dimension of the LSTM network, and q is 256;
然后,利用注意力机制,将多尺度CNN网络局部特征提取结果{π1,π2,...,πn}融入到LSTM网络中,得到基于LSTM网络的语序关系特征提取阶段的输出结果C'Rnn,即 Then, using the attention mechanism, the multi-scale CNN network local feature extraction results {π 1 ,π 2 ,...,π n } are integrated into the LSTM network, and the output result C of the word order relationship feature extraction stage based on the LSTM network is obtained ' Rnn , ie
其中,表示待分析文本中最后一个词语所对应的LSTM模块的输出,表示待分析文本中第一个词语所对应的LSTM模块的输出,本发明采用双向LSTM模型,即BiLSTM模型,in, Indicates the output of the LSTM module corresponding to the last word in the text to be analyzed, Indicates the output of the LSTM module corresponding to the first word in the text to be analyzed. The present invention adopts a bidirectional LSTM model, namely the BiLSTM model,
采用正向传播,具体计算过程如下: Using forward propagation, the specific calculation process is as follows:
d为待分析文本的长度,该文本中每一个词语顺序对应一个LSTM模块,,d is the length of the text to be analyzed, each word sequence in the text corresponds to an LSTM module,,
正向传播过程中,第t-1个LSTM模块的输出为则第t个LSTM模块的输出计算公式如下:During forward propagation, the output of the t-1th LSTM module is Then the output of the tth LSTM module Calculated as follows:
其中:是两个向量的点乘,也称打分函数,是用来计算前一词语的LSTM的输出和当前局部特征向量的相似度,in: Is the dot product of two vectors, also known as the scoring function, which is used to calculate the output of the LSTM of the previous word The similarity with the current local feature vector,
其中:αt,i∈R代表特征πi的权重,Among them: α t,i ∈ R represents the weight of feature π i ,
其中:st-1∈Rq是多个卷积特征的加权结果,利用st-1代替结合当前词语的词向量xt求得当前LSTM模块的输出公式如下:Among them: s t-1 ∈ R q is the weighted result of multiple convolutional features, using s t-1 instead Combine the word vector x t of the current word to obtain the output of the current LSTM module The formula is as follows:
采用反向传播,具体计算过程与正向传播一样,此处不再赘述; Using backpropagation, the specific calculation process is the same as that of forward propagation, and will not be repeated here;
步骤(4)模型训练:将训练数据输入多分类情感分析模型,采用交叉熵损失函数,结合反向传播BP算法调整参数,利用softmax回归作为分类算法,完成训练。Step (4) Model training: Input the training data into the multi-category sentiment analysis model, use the cross-entropy loss function, combine the backpropagation BP algorithm to adjust the parameters, and use softmax regression as the classification algorithm to complete the training.
步骤(5)模型分析:将待分析文本输入训练完成的模型,最终输出对该文本分析后的情感分类结果。Step (5) Model analysis: Input the text to be analyzed into the trained model, and finally output the sentiment classification result after analyzing the text.
所述的基于CNN网络的局部特征提取模块的卷积层计算公式如下:The convolutional layer calculation formula of the local feature extraction module based on the CNN network is as follows:
z=f(∑WT*xi:i+s-1+b) (8)z=f(∑W T *x i:i+s-1 +b) (8)
其中:z表示一个神经元对待分析文本的卷积所得的特征向量,f(·)表示激活函数,W∈Rs×m表示神经元的权重矩阵,同一个神经元参数共享,s×m表示卷积核尺寸的大小,b表示阈值,xi:i+s-1表示由文本句子中的第i个词到i+s-1个词语的词向量,s取[2,3,4,5]四种不同的卷积尺寸,f(·)采用RELU激活函数。Among them: z represents the feature vector obtained by the convolution of a neuron to analyze the text, f(·) represents the activation function, W∈R s×m represents the weight matrix of the neuron, the same neuron parameters are shared, s×m represents The size of the convolution kernel, b represents the threshold, x i:i+s-1 represents the word vector from the i-th word in the text sentence to the i+s-1 word, and s takes [2,3,4, 5] Four different convolution sizes, f(·) using the RELU activation function.
所述的训练数据为经过预处理之后的数据。The training data is preprocessed data.
所述的基于CNN网络的局部特征提取阶段的卷积层采用4种不同尺度的卷积核。所述的训练结束条件是准确率不再改变或达到设置迭代次数。The convolution layer of the local feature extraction stage based on the CNN network adopts four convolution kernels of different scales. The training end condition is that the accuracy rate does not change or reaches the set number of iterations.
1.实验分析1. Experimental analysis
测试阶段,选取喜悦、愤怒、厌恶、低落各类别情感语料各占2000条。使用准确率Acc(Accuracy)作为评价指标,测试阶段模型的参数保持不变,测试集结果如表2所示:In the test phase, 2,000 emotion corpora were selected for each category of joy, anger, disgust, and depression. Using the accuracy rate Acc (Accuracy) as the evaluation index, the parameters of the model in the test phase remain unchanged, and the results of the test set are shown in Table 2:
表2情感分析结果对比Table 2 Comparison of sentiment analysis results
表2中给出了几种模型的测试结果对比,其中,实验1是通用的卷积核尺寸为3的单尺度CNN网络模型,实验2是通用的LSTM网络,实验3则是本文提出的基于注意力机制的文本情感分析模型。Table 2 shows the comparison of the test results of several models. Among them, Experiment 1 is a general-purpose single-scale CNN network model with a convolution kernel size of 3, Experiment 2 is a general-purpose LSTM network, and Experiment 3 is based on Text Sentiment Analysis Model with Attention Mechanism.
通过实验的对比性分析可见,相较通常的CNN网络和LSTM网络,本文提出的基于注意力机制的情感分析模型的准确率都明显提高,说明了本发明提出的方法可以有效的提取CNN网络的局部特征信息和LSTM网络的语序特征信息,说明了该方法的有效性。Through the comparative analysis of the experiment, it can be seen that compared with the usual CNN network and LSTM network, the accuracy of the emotional analysis model based on the attention mechanism proposed in this paper is significantly improved, which shows that the method proposed by the present invention can effectively extract the content of the CNN network. The local feature information and the word order feature information of the LSTM network illustrate the effectiveness of the method.
Claims (6)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910553755.7A CN110287320B (en) | 2019-06-25 | 2019-06-25 | A Deep Learning Multi-Class Sentiment Analysis Model Combined with Attention Mechanism |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910553755.7A CN110287320B (en) | 2019-06-25 | 2019-06-25 | A Deep Learning Multi-Class Sentiment Analysis Model Combined with Attention Mechanism |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110287320A true CN110287320A (en) | 2019-09-27 |
CN110287320B CN110287320B (en) | 2021-03-16 |
Family
ID=68005491
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910553755.7A Active CN110287320B (en) | 2019-06-25 | 2019-06-25 | A Deep Learning Multi-Class Sentiment Analysis Model Combined with Attention Mechanism |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110287320B (en) |
Cited By (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110855474A (en) * | 2019-10-21 | 2020-02-28 | 广州杰赛科技股份有限公司 | Network feature extraction method, device, equipment and storage medium of KQI data |
CN110866113A (en) * | 2019-09-30 | 2020-03-06 | 浙江大学 | A text classification method based on sparse self-attention mechanism fine-tuning Burt model |
CN111079547A (en) * | 2019-11-22 | 2020-04-28 | 武汉大学 | A method of pedestrian moving direction recognition based on mobile phone inertial sensor |
CN111079985A (en) * | 2019-11-26 | 2020-04-28 | 昆明理工大学 | A BERT-based Criminal Case Sentence Prediction Method Integrating Distinguished Attribute Features |
CN111291832A (en) * | 2020-03-11 | 2020-06-16 | 重庆大学 | Sensor data classification method based on Stack integrated neural network |
CN111339768A (en) * | 2020-02-27 | 2020-06-26 | 携程旅游网络技术(上海)有限公司 | Sensitive text detection method, system, electronic device and medium |
CN111402953A (en) * | 2020-04-02 | 2020-07-10 | 四川大学 | Protein sequence classification method based on hierarchical attention network |
CN111582397A (en) * | 2020-05-14 | 2020-08-25 | 杭州电子科技大学 | CNN-RNN image emotion analysis method based on attention mechanism |
CN111881262A (en) * | 2020-08-06 | 2020-11-03 | 重庆邮电大学 | Text emotion analysis method based on multi-channel neural network |
CN111914084A (en) * | 2020-01-09 | 2020-11-10 | 北京航空航天大学 | Deep learning-based emotion label text generation and evaluation system |
CN112598065A (en) * | 2020-12-25 | 2021-04-02 | 天津工业大学 | Memory-based gated convolutional neural network semantic processing system and method |
CN112597279A (en) * | 2020-12-25 | 2021-04-02 | 北京知因智慧科技有限公司 | Text emotion analysis model optimization method and device |
CN112818123A (en) * | 2021-02-08 | 2021-05-18 | 河北工程大学 | Emotion classification method for text |
CN113177111A (en) * | 2021-05-28 | 2021-07-27 | 中国人民解放军国防科技大学 | Chinese text emotion analysis method and device, computer equipment and storage medium |
CN113239199A (en) * | 2021-05-18 | 2021-08-10 | 重庆邮电大学 | Credit classification method based on multi-party data set |
CN113268592A (en) * | 2021-05-06 | 2021-08-17 | 天津科技大学 | Short text object emotion classification method based on multi-level interactive attention mechanism |
CN113377901A (en) * | 2021-05-17 | 2021-09-10 | 内蒙古工业大学 | Mongolian text emotion analysis method based on multi-size CNN and LSTM models |
WO2021174922A1 (en) * | 2020-03-02 | 2021-09-10 | 平安科技(深圳)有限公司 | Statement sentiment classification method and related device |
CN113379818A (en) * | 2021-05-24 | 2021-09-10 | 四川大学 | Phase analysis method based on multi-scale attention mechanism network |
CN113742401A (en) * | 2020-05-27 | 2021-12-03 | 阿里巴巴集团控股有限公司 | Data display method, device, equipment and storage medium |
CN114298025A (en) * | 2021-12-01 | 2022-04-08 | 国家电网有限公司华东分部 | Emotion analysis method based on artificial intelligence |
CN114385813A (en) * | 2022-01-11 | 2022-04-22 | 北京工业大学 | Water environment text aspect-level viewpoint mining method based on multi-feature fusion |
CN114547299A (en) * | 2022-02-18 | 2022-05-27 | 重庆邮电大学 | Short text sentiment classification method and device based on composite network model |
CN114662547A (en) * | 2022-04-07 | 2022-06-24 | 天津大学 | MSCRNN emotion recognition method and device based on EEG signal |
CN114897078A (en) * | 2022-05-19 | 2022-08-12 | 辽宁大学 | Short text similarity calculation method based on deep learning and topic model |
CN115116448A (en) * | 2022-08-29 | 2022-09-27 | 四川启睿克科技有限公司 | Voice extraction method, neural network model training method, device and storage medium |
US20230160942A1 (en) * | 2020-04-22 | 2023-05-25 | Qingdao Topscomm Communication Co., Ltd | Fault arc signal detection method using convolutional neural network |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108460089A (en) * | 2018-01-23 | 2018-08-28 | 哈尔滨理工大学 | Diverse characteristics based on Attention neural networks merge Chinese Text Categorization |
CN109670169A (en) * | 2018-11-16 | 2019-04-23 | 中山大学 | A kind of deep learning sensibility classification method based on feature extraction |
CN109710761A (en) * | 2018-12-21 | 2019-05-03 | 中国标准化研究院 | Sentiment analysis method based on attention-enhanced bidirectional LSTM model |
-
2019
- 2019-06-25 CN CN201910553755.7A patent/CN110287320B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108460089A (en) * | 2018-01-23 | 2018-08-28 | 哈尔滨理工大学 | Diverse characteristics based on Attention neural networks merge Chinese Text Categorization |
CN109670169A (en) * | 2018-11-16 | 2019-04-23 | 中山大学 | A kind of deep learning sensibility classification method based on feature extraction |
CN109710761A (en) * | 2018-12-21 | 2019-05-03 | 中国标准化研究院 | Sentiment analysis method based on attention-enhanced bidirectional LSTM model |
Non-Patent Citations (3)
Title |
---|
MING-HSIANG SU.ETL: "LSTM-based Text Emotion Recognition Using Semantic and Emotional Word Vectors", 《 2018 FIRST ASIAN CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION》 * |
THITITORN SENEEWONG NA AYUTTHAYA.ETL: "Thai Sentiment Analysis via Bidirectional LSTM-CNN Model with Embedding Vectors and Sentic Features", 《2018 INTERNATIONAL JOINT SYMPOSIUM ON ARTIFICIAL INTELLIGENCE AND NATURAL LANGUAGE PROCESSING》 * |
关鹏飞等: "注意力增强的双向LSTM情感分析", 《中文信息学报》 * |
Cited By (39)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110866113A (en) * | 2019-09-30 | 2020-03-06 | 浙江大学 | A text classification method based on sparse self-attention mechanism fine-tuning Burt model |
CN110866113B (en) * | 2019-09-30 | 2022-07-26 | 浙江大学 | Text classification method based on sparse self-attention mechanism fine-tuning burt model |
CN110855474A (en) * | 2019-10-21 | 2020-02-28 | 广州杰赛科技股份有限公司 | Network feature extraction method, device, equipment and storage medium of KQI data |
CN110855474B (en) * | 2019-10-21 | 2022-06-17 | 广州杰赛科技股份有限公司 | Network feature extraction method, device, equipment and storage medium of KQI data |
CN111079547A (en) * | 2019-11-22 | 2020-04-28 | 武汉大学 | A method of pedestrian moving direction recognition based on mobile phone inertial sensor |
CN111079985A (en) * | 2019-11-26 | 2020-04-28 | 昆明理工大学 | A BERT-based Criminal Case Sentence Prediction Method Integrating Distinguished Attribute Features |
CN111914084A (en) * | 2020-01-09 | 2020-11-10 | 北京航空航天大学 | Deep learning-based emotion label text generation and evaluation system |
CN111339768A (en) * | 2020-02-27 | 2020-06-26 | 携程旅游网络技术(上海)有限公司 | Sensitive text detection method, system, electronic device and medium |
CN111339768B (en) * | 2020-02-27 | 2024-03-05 | 携程旅游网络技术(上海)有限公司 | Sensitive text detection method, system, electronic equipment and medium |
WO2021174922A1 (en) * | 2020-03-02 | 2021-09-10 | 平安科技(深圳)有限公司 | Statement sentiment classification method and related device |
CN111291832A (en) * | 2020-03-11 | 2020-06-16 | 重庆大学 | Sensor data classification method based on Stack integrated neural network |
CN111402953A (en) * | 2020-04-02 | 2020-07-10 | 四川大学 | Protein sequence classification method based on hierarchical attention network |
CN111402953B (en) * | 2020-04-02 | 2022-05-03 | 四川大学 | A protein sequence classification method based on hierarchical attention network |
US20230160942A1 (en) * | 2020-04-22 | 2023-05-25 | Qingdao Topscomm Communication Co., Ltd | Fault arc signal detection method using convolutional neural network |
US11860216B2 (en) * | 2020-04-22 | 2024-01-02 | Qingdao Topscomm Communication Co., Ltd | Fault arc signal detection method using convolutional neural network |
CN111582397A (en) * | 2020-05-14 | 2020-08-25 | 杭州电子科技大学 | CNN-RNN image emotion analysis method based on attention mechanism |
CN111582397B (en) * | 2020-05-14 | 2023-04-07 | 杭州电子科技大学 | CNN-RNN image emotion analysis method based on attention mechanism |
CN113742401A (en) * | 2020-05-27 | 2021-12-03 | 阿里巴巴集团控股有限公司 | Data display method, device, equipment and storage medium |
CN111881262A (en) * | 2020-08-06 | 2020-11-03 | 重庆邮电大学 | Text emotion analysis method based on multi-channel neural network |
CN111881262B (en) * | 2020-08-06 | 2022-05-20 | 重庆邮电大学 | Text sentiment analysis method based on multi-channel neural network |
CN112598065B (en) * | 2020-12-25 | 2023-05-30 | 天津工业大学 | Memory-based gating convolutional neural network semantic processing system and method |
CN112597279A (en) * | 2020-12-25 | 2021-04-02 | 北京知因智慧科技有限公司 | Text emotion analysis model optimization method and device |
CN112598065A (en) * | 2020-12-25 | 2021-04-02 | 天津工业大学 | Memory-based gated convolutional neural network semantic processing system and method |
CN112818123A (en) * | 2021-02-08 | 2021-05-18 | 河北工程大学 | Emotion classification method for text |
CN113268592A (en) * | 2021-05-06 | 2021-08-17 | 天津科技大学 | Short text object emotion classification method based on multi-level interactive attention mechanism |
CN113377901A (en) * | 2021-05-17 | 2021-09-10 | 内蒙古工业大学 | Mongolian text emotion analysis method based on multi-size CNN and LSTM models |
CN113377901B (en) * | 2021-05-17 | 2022-08-19 | 内蒙古工业大学 | Mongolian text emotion analysis method based on multi-size CNN and LSTM models |
CN113239199A (en) * | 2021-05-18 | 2021-08-10 | 重庆邮电大学 | Credit classification method based on multi-party data set |
CN113379818A (en) * | 2021-05-24 | 2021-09-10 | 四川大学 | Phase analysis method based on multi-scale attention mechanism network |
CN113379818B (en) * | 2021-05-24 | 2022-06-07 | 四川大学 | A Phase Resolution Method Based on Multiscale Attention Mechanism Network |
CN113177111A (en) * | 2021-05-28 | 2021-07-27 | 中国人民解放军国防科技大学 | Chinese text emotion analysis method and device, computer equipment and storage medium |
CN114298025A (en) * | 2021-12-01 | 2022-04-08 | 国家电网有限公司华东分部 | Emotion analysis method based on artificial intelligence |
CN114385813A (en) * | 2022-01-11 | 2022-04-22 | 北京工业大学 | Water environment text aspect-level viewpoint mining method based on multi-feature fusion |
CN114547299A (en) * | 2022-02-18 | 2022-05-27 | 重庆邮电大学 | Short text sentiment classification method and device based on composite network model |
CN114662547A (en) * | 2022-04-07 | 2022-06-24 | 天津大学 | MSCRNN emotion recognition method and device based on EEG signal |
CN114897078A (en) * | 2022-05-19 | 2022-08-12 | 辽宁大学 | Short text similarity calculation method based on deep learning and topic model |
CN114897078B (en) * | 2022-05-19 | 2024-11-08 | 辽宁大学 | Short text similarity calculation method based on deep learning and topic model |
CN115116448A (en) * | 2022-08-29 | 2022-09-27 | 四川启睿克科技有限公司 | Voice extraction method, neural network model training method, device and storage medium |
CN115116448B (en) * | 2022-08-29 | 2022-11-15 | 四川启睿克科技有限公司 | Voice extraction method, neural network model training method, device and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN110287320B (en) | 2021-03-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110287320B (en) | A Deep Learning Multi-Class Sentiment Analysis Model Combined with Attention Mechanism | |
CN107608956B (en) | Reader emotion distribution prediction algorithm based on CNN-GRNN | |
CN108334605B (en) | Text classification method and device, computer equipment and storage medium | |
CN109284506B (en) | User comment emotion analysis system and method based on attention convolution neural network | |
CN109241255B (en) | An Intent Recognition Method Based on Deep Learning | |
CN110502753A (en) | A Deep Learning Sentiment Analysis Model Based on Semantic Enhancement and Its Analysis Method | |
CN109299253A (en) | A Construction Method of Chinese Social Text Emotion Recognition Model Based on Deep Fusion Neural Network | |
CN108537624B (en) | A travel service recommendation method based on deep learning | |
CN110083700A (en) | A kind of enterprise's public sentiment sensibility classification method and system based on convolutional neural networks | |
CN108009148B (en) | Text sentiment classification and representation method based on deep learning | |
CN110287323B (en) | Target-oriented emotion classification method | |
CN108763216A (en) | A kind of text emotion analysis method based on Chinese data collection | |
CN107818084B (en) | Emotion analysis method fused with comment matching diagram | |
CN109376242A (en) | Text Classification Algorithms Based on Variants of Recurrent Neural Networks and Convolutional Neural Networks | |
CN108614875A (en) | Chinese emotion tendency sorting technique based on global average pond convolutional neural networks | |
CN110543242A (en) | expression input method based on BERT technology and device thereof | |
CN110188195B (en) | Text intention recognition method, device and equipment based on deep learning | |
CN107247703A (en) | Microblog emotional analysis method based on convolutional neural networks and integrated study | |
CN111985247A (en) | Microblog user interest identification method and system based on multi-granularity text feature representation | |
CN110472052A (en) | A kind of Chinese social platform sentiment analysis method based on deep learning | |
CN112749274A (en) | Chinese text classification method based on attention mechanism and interference word deletion | |
CN112364638A (en) | Personality identification method based on social text | |
CN113505226B (en) | Text emotion classification system fusing graph convolution neural network | |
CN113094502A (en) | Multi-granularity takeaway user comment sentiment analysis method | |
CN110046223B (en) | Sentiment analysis method of movie reviews based on improved convolutional neural network model |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |