CN113342971B - Chinese text emotion analysis method based on fusion of multiple text features - Google Patents

Chinese text emotion analysis method based on fusion of multiple text features Download PDF

Info

Publication number
CN113342971B
CN113342971B CN202110554330.5A CN202110554330A CN113342971B CN 113342971 B CN113342971 B CN 113342971B CN 202110554330 A CN202110554330 A CN 202110554330A CN 113342971 B CN113342971 B CN 113342971B
Authority
CN
China
Prior art keywords
text
information
sequence
output
sequences
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110554330.5A
Other languages
Chinese (zh)
Other versions
CN113342971A (en
Inventor
王丽亚
陈哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN202110554330.5A priority Critical patent/CN113342971B/en
Publication of CN113342971A publication Critical patent/CN113342971A/en
Application granted granted Critical
Publication of CN113342971B publication Critical patent/CN113342971B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Machine Translation (AREA)

Abstract

The invention provides a Chinese text emotion analysis method based on fusion of multiple text features, which comprises the following steps: step1, acquiring Chinese text information, and preprocessing the Chinese text information to obtain a plurality of sequences corresponding to the Chinese text information; step2, inputting the sequences into a BiGRU network to extract text features of each sequence and generating a plurality of text feature information corresponding to the sequences; step3, fusing the text characteristic information, and inputting the fused text characteristic information into a BiLSTM network for learning; step4, screening the characteristics by using a self-attention mechanism; step5, inputting the feature vectors screened by the self-attention mechanism into a sigmoid classifier for classification to obtain a final emotion analysis result.

Description

Chinese text emotion analysis method based on fusion of multiple text features
Technical Field
The invention relates to the technical field of natural language processing, in particular to a Chinese text emotion analysis method based on fusion of multiple text features.
Background
Text emotion analysis (Sentiment Analysis) refers to the process of analyzing, processing and extracting subjective text with emotion colors using natural language processing and text mining techniques. The related fields include natural language processing, text mining, information retrieval, information extraction, machine learning, and the like. The traditional two-way long-short-Term memory neural network (BidirectiLnalLLng ShLrt-Term MemLry, biLSTM) has a good effect on text emotion analysis, but does not learn enough about characteristic information contained in the text. In order to solve the problem, a Chinese text emotion analysis method integrating multiple text features is provided.
In summary, the method for analyzing Chinese text emotion based on fusion of multiple text features, which can solve the problem of insufficient learning of characteristic information contained in a text by a traditional two-way long-short-term memory neural network (BiLSTM) and can effectively improve the accuracy of Chinese text emotion analysis, is a problem which needs to be solved by those skilled in the art.
Disclosure of Invention
Aiming at the problems and the demands, the scheme provides a Chinese text emotion analysis method based on fusion of multiple text features, which can solve the technical problems due to the adoption of the following technical scheme.
In order to achieve the above purpose, the present invention provides the following technical solutions: a Chinese text emotion analysis method based on fusion of multiple text features comprises the following steps: step1, acquiring Chinese text information, and preprocessing the Chinese text information to obtain a plurality of sequences corresponding to the Chinese text information;
step2, inputting the sequences into a BiGRU network to extract text features of each sequence and generating a plurality of text feature information corresponding to the sequences;
step3, fusing the text characteristic information, and inputting the fused text characteristic information into a BiLSTM network for learning;
step4, screening the characteristics by using a self-attention mechanism, and distributing corresponding weights to the characteristic information extracted in the Step3 to obtain the most important emotion information;
and Step5, inputting the feature vectors screened by the self-attention mechanism into a sigmoid classifier for classification to obtain a final emotion analysis result.
Further, the plurality of sequences includes a text word sequence, a part-of-speech sequence, a word part-of-speech sequence, a word-position sequence, and a word-part-of-speech-position sequence.
Still further, the extracting text features of each sequence includes: training the sequences through a word2vec model to obtain a plurality of sequence matrixes corresponding to the sequences, wherein the element vector of each element corresponding to each sequence is as follows，/>Wherein->Is the number of elements->Is a vector dimension, then the entire sequence matrix Uj for each sequence is expressed as:j represents a sequence number; the Uj is input into a trained BiGRU network, the forward and reverse text sequences are processed at the same time, and feature extraction is carried out on the deep text information to obtain corresponding feature vector information。
Further, the BiGRU network consists of a forward GRU, a reverse GRU and an output state connecting layer of the forward GRU, if the hidden state of the forward GRU output at the moment t is recorded asThen->The hidden state of the reverse GRU output is +.>Then->While the semantics of the BiGRU network output are expressed asWherein->Is a weight matrix, < >>For GRU function, ++>For GRU input at time t, +.>Is a bias vector.
Still further, the fusing the plurality of text feature information includes: the corresponding feature vector information is processedFusion is carried out through a matrix splicing method or a dot multiplication method, and fused text features are obtained>。
Further, the fused text featuresInputting the output state of the BiLSTM network at a certain moment t, and connecting the outputs of the forward LSTM network and the reverse LSTM network, wherein if the output state is->The hidden state of the moment forward LSTM output is +.>The hidden state of the reverse LSTM output is +.>Hidden state of BiLSTM output +.>。
Still further, the screening features using the self-attention mechanism includes: generating target attention weights，，/>Is to pay attention to the mechanical learning function tanh，/>Is the characteristic vector output by the BiLSTM network; attention weight is then probabilistic, according to the formula: />Generating a probability vector by means of a softmax function>The method comprises the steps of carrying out a first treatment on the surface of the Finally, attention weight configuration is carried out according to the formula +.>The generated attention weights are allocated to corresponding hidden layer semantic codes ++>Wherein->Is->Is a weighted average of +.>。
Still further, the inputting the sigmoid classifier for classification includes: vectors processed by self-attention mechanismObtaining a characteristic vector through a dropout layer>The method comprises the steps of carrying out a first treatment on the surface of the Feature vector +.>Inputting to a full connection layer, wherein the full connection layer parameter is 1, the activation function is a sigmoid function, and according to a model:training output finalEmotional analysis result, wherein the sample is +.>，/>Is negative 0 or positive 1, ">Is a sample feature vector, ++>Representing a trainable parameter; adopts->Training model parameters as loss function->Model optimization is carried out by adopting an Adam optimization algorithm, wherein +.>For input +.>Is used for the purpose of determining the true class of (c),input +.>Probability of belonging to category 1.
From the technical scheme, the beneficial effects of the invention are as follows: the text emotion analysis method based on the two-way long-short-term memory neural network (BiLSTM) solves the problem that the traditional text emotion analysis method based on the two-way long-term memory neural network (BiLSTM) is insufficient in learning the characteristic information contained in the text, strengthens the representation capability of a model on the text in the text language preprocessing process, and can effectively improve the accuracy rate of Chinese text emotion analysis.
In addition to the objects, features and advantages described above, preferred embodiments for carrying out the present invention will be described in more detail below with reference to the accompanying drawings so that the features and advantages of the present invention can be readily understood.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will briefly describe the drawings that are required to be used in the description of the embodiments of the present invention, wherein the drawings are only for illustrating some embodiments of the present invention, and not limiting all embodiments of the present invention thereto.
FIG. 1 is a schematic diagram showing specific steps of a Chinese text emotion analysis method based on fusion of multiple text features.
FIG. 2 is a schematic diagram of a BiGRU network according to the present invention.
Fig. 3 is a schematic diagram of a process of constructing a text emotion analysis neural network model in this embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the technical solutions of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of specific embodiments of the present invention. Like reference numerals in the drawings denote like parts. It should be noted that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be made by a person skilled in the art without creative efforts, based on the described embodiments of the present invention fall within the protection scope of the present invention.
According to the method, firstly, a plurality of original information of a text is obtained step by step, then the original information is processed through a bi-directional threshold cyclic neural network (BidirectiLnal Gated Recurrent Unit, biGRU) to extract various text features, then the various text features are fused and input into a BiLSTM network for learning, then the self-attention mechanism is utilized to screen the features, and finally the 2 classification is carried out through a sigmoid classifier. As shown in fig. 1 to 3, the method includes: step1, acquiring Chinese text information, and preprocessing the Chinese text information to obtain a plurality of sequences corresponding to the Chinese text information, wherein the sequences comprise text word sequences, part-of-speech sequences, word part-of-speech sequences, word-position sequences and word-part-of-speech-position sequences.
In this embodiment, the steps for specifically acquiring a plurality of sequences corresponding to chinese text information include: 1. and acquiring part-of-speech information while segmenting the words by using a posseg method in the jieba segmentation. Wherein, the Chinese text information comprises: word information (word), part-of-speech information (pos), word information (char), word-part-of-speech information (char_pos), word-position information (char_word_tag), word-part-of-speech-position information (char-pos-tag), as follows:
sample text sentence (sense):
"deserving to read at one glance, the problem presented in the book deserves to think, and the theory is not good. Individuals support the ideas of this book. Because of the relative aggression, many people are uncomfortable, but the "aggression" is lacking in the current mainstream society and elite people, and the folk is surging. The person thinks that the high-rise leader should carefully consider the view in the book, i think that the direction is right, and the specific practice can be more gentle, careful and more operable. The problem of the text is that the logic among sentences is slightly disordered, and the support of the text can be further enhanced. "
The samples are processed to obtain:
Word:
the [ 'deserves to be read', '', 'a' book ',' a 'lining', 'a' proposal ',' a 'problem', 'deserve', 'thinking', 'say', 'do not care'. 'personal', 'support', 'book', 'thinking'. ' because ', ' compare ', ' advance ', ' ', ' lead ', ' many ', ' human ', ' don't ', ' comfort ', ' ', ' but ', ' such ', ' ', ' enter ', ' advance ', ' ', ' apply ', ' but is, ' now ', ' society ', ' mainstream ', ' ', ' elite ', ' people ', ' institute ', ' lack ', ' ', ' folk ', ' positive ', ' surging ', ' wearing ', ' the body. 'personal', 'thought' higher-layer ',' collar ',' should ',' earnest ',' consider ',' book ',' inner ',' ',' view ',' ',' me ',' thought ',' direction ',' is ',' pair ',' ',' concrete ',' ',' practice ',' may ',' better ',' principal ',' some ',' fine ',' some ',' cause ',' better ',' have 'operability'. 'whereas', 'this book', 'comparison', 'big', '', 'problem', 'is', 'concrete', 'is', 'line', 'it', 'sentence', 'inter', 'logic', 'slightly', 'display', 'confusion', 'support', 'still', 'cocoa', 'further', 'reinforcement'. ']
Pos:
['l', 'x', 'n', 'f', 'v', 'uj', 'n', 'v', 'v', 'x', 'v', 'uj', 'i', 'x', 'n', 'v', 'r', 'uj', 'n', 'x', 'c', 'd', 'v', 'x', 'v', 'm', 'n', 'd', 'a', 'x', 'c', 'r', 'x', 'v', 'x', 'd', 't', 'n', 'b', 'x', 'n', 'k', 'u', 'v', 'uj', 'x', 'n', 'd', 'd', 'v', 'uz', 'uj', 'x', 'n', 'v', 'n', 'n', 'v', 'ad', 'v', 'n', 'f', 'uj', 'v', 'x', 'r', 'v', 'n', 'v', 'p', 'uj', 'x', 'a', 'uj', 'v', 'c', 'd', 'a', 'm', 'x', 'a', 'm', 'x', 'v', 'u', 'd', 'v', 'l', 'x', 'c', 'r', 'd', 'a', 'uj', 'n', 'v', 'a', 'uj', 'n', 'x', 'r', 'n', 'f', 'uj', 'n', 'd', 'v', 'a', 'x', 'n', 'v', 'd', 'v', 'p', 'd', 'v', 'x']
2. Each word is converted to a list type (list) and saved as a new list.
Char:
The values of [ 'value', 'get', 'one', 'see', '', 'book', 'inside', 'lifting', 'out', 'right', 'question', 'value', 'get', 'think', 'exam', '', 'say', 'not', 'none', 'channel', 'reason', 'get around'. 'personal', 'human', 'branch', 'hold', 'book', 'think', 'road', 'support' and 'support' are defined. ' because ', ' is ', ' specific ', ' relatively ', ' excited ', ' advanced ', ' ', ' in ', ' cause ', ' very ', ' poly ', ' human ', ' not ', ' comfort ', ' suit ', ' but ', ' is ', ' this ', ' species ', ' enter ', ' advance ', ' ', ' is ', ' cool ', ' is, ' present ', ' in ', ' social ', ' meeting ', ' main ', ' stream ', ' refined ', ' english ', ' people ', ' institute ', ' lack ', ' spent ', ' ', ' people ', ' inter ', ' positive ', ' dark ', ' self ', ' surge ', ' move ', ' wearing '. ' personal ', ' human ', ' cognitive ', ' being ', ' high ', ' layer ', ' collar ', ' sleeve ', ' should ', ' the ', ' cognitive ', ' true ', ' consider ', ' book ', ' inner ', ' ', ' see ', ' method ', ' i ', ' recognize ', ' be ' side ', ' direct ', ' pair ', ' bear ', ' apply ', ' have ', ' body ', ' do ', ' method ', ' can ', ' use ', ' more ', ' delegate ', ' constraint ', ' one ', ' some ', ' ', ' fine ', ' cause ', ' one ', ' some ', ' cause ', ' more ', ' have ', ' do ', ' operate ', ' do ', ' sex '. 'but', 'body', 'book', 'ratio', 'compare', 'big', '', 'ask', 'question', 'is', 'have', 'body', '', 'line', 'text', '', 'it', 'sentence', 'son', 'between', 'logical', 'edit', 'slightly', 'show', 'mix', 'disorder', '', 'theoretical', 'data', 'support', 'hold', 'return', 'cocoa', 'in', 'advance', 'one', 'step', 'add', 'strong'. ']
3. The part of speech of each word is assigned to the words that make up that word.
Char_pos:
['l', 'l', 'l', 'l', 'x', 'n', 'f', 'v', 'v', 'uj', 'n', 'n', 'v', 'v', 'v', 'v', 'x', 'v', 'uj', 'i', 'i', 'i', 'i', 'x', 'n', 'n', 'v', 'v', 'r', 'r', 'uj', 'n', 'n', 'x', 'c', 'c', 'd', 'd', 'v', 'v', 'x', 'v', 'm', 'm', 'n', 'd', 'a', 'a', 'x', 'c', 'c', 'r', 'r', 'x', 'v', 'v', 'x', 'd', 'd', 't', 't', 'n', 'n', 'b', 'b', 'x', 'n', 'n', 'k', 'u', 'v', 'v', 'uj', 'x', 'n', 'n', 'd', 'd', 'd', 'v', 'v', 'uz', 'uj', 'x', 'n', 'n', 'v', 'v', 'n', 'n', 'n', 'n', 'v', 'v', 'ad', 'ad', 'v', 'v', 'n', 'f', 'uj', 'v', 'v', 'x', 'r', 'v', 'v', 'n', 'n', 'v', 'p', 'uj', 'x', 'a', 'a', 'uj', 'v', 'v', 'c', 'c', 'd', 'a', 'a', 'm', 'm', 'x', 'a', 'a', 'm', 'm', 'x', 'v', 'u', 'd', 'v', 'v', 'l', 'l', 'l', 'l', 'x', 'c', 'r', 'r', 'd', 'd', 'a', 'uj', 'n', 'n', 'v', 'a', 'a', 'uj', 'n', 'n', 'x', 'r', 'n', 'n', 'f', 'uj', 'n', 'n', 'd', 'v', 'a', 'a', 'x', 'n', 'n', 'v', 'v', 'd', 'v', 'p', 'd', 'd', 'd', 'v', 'v', 'x']
4. For the position information of the words in each word, the first word is marked B, the middle word is marked M, and the last word is marked E.
Char_word_tag:
['B', 'M', 'M', 'E', 'S', 'S', 'S', 'B', 'E', 'S', 'B', 'E', 'B', 'E', 'B', 'E', 'S', 'S', 'S', 'B', 'M', 'M', 'E', 'S', 'B', 'E', 'B', 'E', 'B', 'E', 'S', 'B', 'E', 'S', 'B', 'E', 'B', 'E', 'B', 'E', 'S', 'S', 'B', 'E', 'S', 'S', 'B', 'E', 'S', 'B', 'E', 'B', 'E', 'S', 'B', 'E', 'S', 'B', 'E', 'B', 'E', 'B', 'E', 'B', 'E', 'S', 'B', 'E', 'S', 'S', 'B', 'E', 'S', 'S', 'B', 'E', 'S', 'B', 'E', 'B', 'E', 'S', 'S', 'S', 'B', 'E', 'B', 'E', 'B', 'E', 'B', 'E', 'B', 'E', 'B', 'E', 'B', 'E', 'S', 'S', 'S', 'B', 'E', 'S', 'S', 'B', 'E', 'B', 'E', 'S', 'S', 'S', 'S', 'B', 'E', 'S', 'B', 'E', 'B', 'E', 'S', 'B', 'E', 'B', 'E', 'S', 'B', 'E', 'B', 'E', 'S', 'S', 'S', 'S', 'B', 'E', 'B', 'M', 'M', 'E', 'S', 'S', 'B', 'E', 'B', 'E', 'S', 'S', 'B', 'E', 'S', 'B', 'E', 'S', 'B', 'E', 'S', 'S', 'B', 'E', 'S', 'S', 'B', 'E', 'S', 'S', 'B', 'E', 'S', 'B', 'E', 'B', 'E', 'S', 'S', 'S', 'B', 'M', 'E', 'B', 'E', 'S']
5. And (3) for each word, splicing the results obtained in the steps 3 and 4 to obtain corresponding word-part-of-speech-position information.
char-pos-tag：
[ (B_l ', (M_l ', (E_l ',) S_x ', (S_n ', (S_f ', (B_v ', (E_v ',) S_uj ', ' B_n ', ' E_n ', ' B_v ', ' E_v ', ' S_x ', ' S_v ', ' S_uj ', ' B_i ', ' M_i ', ' E_i ', ' S_x ', ' B_n ', ' E_n ', ' B_v ', ' E_v ', ' B_r ', ' E_r ', ' S_uj ', ' B_n ', ' E_n ', ' S_x ', ' B_c ', ' E_c ', ' B_d ', ' E_d ', ' B_v ', ' E_v ', ' S_x ', ' S_v ', ' B_m ', ' E_m ', ' S_n ', ' S_d ', ' B_a ', ' E_a ', ' S_x ', ' B_c ', ' E_c ', ' B_r ', ' E_r ', ' S_x ', ' B_v ', ' E_v ', ' S_x ', ' B_d ', ' E_d ', ' B_t ', ' E_t ', ' B_n ', ' E_n ', ' B_b ', ' E_b ', ' S_x ', ' B_n ', ' E_n ', ' S_k ', ' S_u ', ' B_v ', ' E_v ', ' S_uj ', ' S_x ', 'B_n', 'E_n', 'S_d', 'B_d', 'E_d', 'B_v', 'E_v', 'S_uz', 'S_uj', 'S_x', 'B_n', 'E_n', 'B_v', 'E_v', 'B_n', 'E_n', 'B_v', 'E_v', 'B_ad', 'E_ad', 'B_v', 'E_v', 'S_n', 'S_f', 'S_uj', 'B_v', 'E_v', 'S_x', 'S_r', 'B_v', 'E_v', 'B_n', 'E_n', 'S_v', 'S_p', 'S_uj', 'S_x', 'B_a', 'E_a', 'S_uj', 'B_v', 'E_v', 'B_c', 'E_c', 'S_d', 'B_a', 'E_a', 'B_m', 'E_m', 'S_x', 'B_a', 'E_a', 'B_m', 'E_m', 'S_x', 'S_x', 'S_v', 'S_u', 'S_d', 'B_v', 'E_v', 'B_l', 'M_l', 'E_l', 'S_x', 'S_c', 'B_r', 'E_r', 'B_d', 'E_d', 'S_a', 'S_uj', 'b_n', 'e_n', 's_v', 'b_a', 'e_a', 's_uj', 'b_n', 'e_n', 's_x', 's_r', 'b_n', 'e_n', 's_f', 's_uj', 'b_n', 'e_n', 's_d', 's_v', 'b_a', 'e_a', 's_x', 'b_n', 'b_v', 's_d', 's_s', 's_v', 's_d', 'e_d', 's_v', 's_s', 's_d'. After the sequence information is obtained, the sequence information is input into a bi-directional threshold cyclic neural network (Bidirectional Gated Recurrent Unit, biGRU) for processing and extracting text characteristics of each sequence.
Step2, inputting the sequences into a BiGRU network to extract text features of each sequence, and generating a plurality of text feature information corresponding to the sequences, wherein the text feature extraction of each sequence specifically comprises the following steps: training the sequences through a word2vec model to obtain a plurality of sequence matrixes corresponding to the sequences, wherein the element vector of each element corresponding to each sequence is as follows，/>Wherein->Is the number of elements->Is a vector dimension, then the entire sequence matrix Uj for each sequence is expressed as: />J represents a sequence number. The Uj is input into a trained BiGRU network, the forward and reverse text sequences are processed at the same time, and feature extraction is carried out on the deep text information to obtain corresponding feature vector information。
In this embodiment, the multiple sequences are vectorized by the word2vec model specifically as follows, for example, for word information: after splitting sentences into words, a sentence list consisting of single words is obtained, for example: [ …, [ ' bulk ', ' hot ', ' non ', ' normal ', ' rod ', ' | ]! ' ], … ]. And constructing a word vector by adopting a Skip-gram model in word2 vec. If the context window size is 5, and the vector form corresponding to the current word W (t) is V (W (t)), and the vector forms corresponding to the surrounding 4 words are V (W (t+2)), V (W (t+1)), V (W (t-1)), V (W (t-2)), the Skip-gram model predicts surrounding words by the center word and is solved by using the conditional probability value of the center word vector V (W (t)), as shown in the following equation:
where V (W (i) ∈ { V (W (t+2)), V (W (t+1)), V (W (t-1)), V (W (t-2)) }.
As shown in FIG. 2, the BiGRU network is composed of output state connection layers of a forward GRU, a reverse GRU and a forward GRU, if the hidden state of the forward GRU output at t moment is recorded asThen->The hidden state of the reverse GRU output is +.>Then->While the semantics of the BiGRU network output are expressed asWherein->Is a weight matrix, < >>For GRU function, ++>For GRU input at time t, +.>Is a bias vector. Wherein GRU is an improvement over LSTM and there is also a Memory unit (Memory Units) throughout which the original LSTM is replaced with an update gateThe forget gate and the input gate of the model are simpler in network structure than LSTM, and the required parameters are reduced, so that the model training speed is improved. The specific calculation process is as follows:
wherein,: weight matrix,/->: update door->: reset door>: alternative activation function, +.>: activating function->: input of GRU at time t, tanh: tanh activation function, < >>: sigmoid activates a function.
Step2 vectorizes the Chinese text, and respectively processes various Chinese text sequence information obtained in Step1 by using word2vec technology, and simultaneously obtains parameters of an initializing layer: input_dim, output_dim, and input_length, specifically, input_dim is the possible number of values of words in the input text data, i.e., the size of a vocabulary (dictionary). output_dim is the size of the vector space of the output embedded word, i.e., the output dimension specified for the word. input_length is the length of the input sequence. ebeddings_initializer: a weight matrix.
Step2 is performed for 6 kinds of original sequence information: word information, part-of-speech information, word part-of-speech information, word-position information, word-part-of-speech-position information, and corresponding feature vectors are obtainedThen, various text features need to be fused.
Step3, fusing the text characteristic information, and inputting the text characteristic information into a BiLSTM network for learning, wherein the fusing the text characteristic information comprises the following steps: the corresponding feature vector information is processedFusion is carried out through a matrix splicing method or a dot multiplication method, and fused text features are obtained. The text feature after fusion +.>Inputting the output state of the BiLSTM network at a certain moment t, and connecting the outputs of the forward LSTM network and the reverse LSTM network, wherein if the output state is->The hidden state of the moment forward LSTM output is +.>The hidden state of the reverse LSTM output is +.>Hidden state of BiLSTM output +.>。
Matrix stitching may be achieved by a confcate function, such as a formulaShown, symbol->Representing a matrix splice.
Dot multiplication can be achieved by dot functions, e.g. formulasAs shown. For the data sets used in the experiments, the embodiment shows that the stitching mode is better, but the invention is not limited to the embodiment, and different fusion modes can be tried for different data sets.
In the present embodiment, the long and short term Memory network (LSTM) is a variant of the Recurrent Neural Network (RNN) which has a Memory unit (Memory Units) throughout which is forgotten, memorized, and output by hidden states passing the last instantAnd the current input +.>Calculated amnesia door->Memory door->Output door->To control, retain important information, forget unimportant information, eliminate gradient explosion (Gradient Explosion) or gradient disappearance problem of the circulatory neural networkThe calculation method is as follows:
door for calculating forgetfulnessAccording to->And->Calculation, e.g. formula->Shown;
calculation memory doorAccording to->And->Calculation, e.g. formula->Shown;
calculating temporary memory statusAccording to->And->Calculation, e.g. formula->Shown;
calculating the current memory stateAccording to->、/>、/>And memory of the last moment->Calculation, e.g. formulasShown;
calculation output doorAccording to->And->Calculation, e.g. formula->Shown;
calculating the hidden state of the current momentAccording to->And->Calculation, e.g. formula->Shown;
wherein the method comprises the steps ofIs a weight matrix; />Is a bias vector; />Activating a function for the tanh; />The function is activated for logistic.
Step4, screening the features by using a self-attention mechanism, distributing corresponding weights to the feature information extracted in the Step3, and obtaining the most important emotion information, namely obtaining the sentence hiding state through the self-attention mechanismThe attention mechanism mainly comprises three processes of generating attention weights, probability of the attention weights and configuration of the attention weights. The method specifically comprises the following steps: generating a target attention weight +.>，/>，/>Is the attention-mechanics learning function tanh, +.>Is the characteristic vector output by the BiLSTM network; attention weight is then probabilistic, according to the formula: />Generating a probability vector by means of a softmax function>The method comprises the steps of carrying out a first treatment on the surface of the Finally, attention weight configuration is carried out according to the formula +.>The generated attention weights are allocated to corresponding hidden layer semantic codes ++>Wherein->Is->Is a weighted average of (a), the weight is。
Step5, inputting the feature vectors screened by the self-attention mechanism into a sigmoid classifier for classification to obtain a final emotion analysis result, wherein the inputting the sigmoid classifier for classification comprises the following steps: vectors processed by self-attention mechanismObtaining a characteristic vector through a dropout layer>The method comprises the steps of carrying out a first treatment on the surface of the Feature vector +.>Inputting to a full connection layer, wherein the full connection layer parameter is 1, the activation function is a sigmoid function, and according to a model:training to output final emotion analysis result, wherein the sample is +.>，/>Is negative 0 or positive 1, ">Is a sample feature vector, ++>Representing a trainable parameter; the goal of model training is to minimize the loss function, which is used in this embodimentTraining model parameters as loss function->Model optimization is carried out by adopting an Adam optimization algorithm, wherein +.>For input +.>Is (are) true category->Input +.>Probability of belonging to category 1. The Adam optimization algorithm is a method for calculating the self-adaptive learning rate of each parameter, and combines the advantages of two optimization algorithms, namely adaGrad and RMSProp. The updating of the parameters is not affected by the expansion transformation of the gradient, and the change is stable.
In this embodiment, as shown in fig. 3, the process of building a text emotion analysis neural network model includes: a. compiling a multi-input single-output neural network model, wherein a plurality of input layers are used for receiving various Chinese text information vectors, an enabling layer, a BiGRU layer, a linkage/dot layer (i.e. a matrix splicing layer) and a BiLSTM layer are arranged in the middle, then the output of the BiLSTM layer is processed by an attribute layer, and finally a sigmoid function is utilized for classification, so that the probability of predicting text emotion to be positive is output; b. initializing the ebedding layer in a; c. initializing a BiGRU layer, which is used for extracting the characteristics contained in various Chinese text information (the processing is not limited to using a BiGRU network, and the processing can be replaced by a BiLSTM network), d initializing a concate/dot layer, which is used for fusing various text characteristics obtained in c, and initializing a BiLSTM layer. And e, inputting the feature vector screened by the self-attention mechanism into a sigmoid classifier for classification, and obtaining a final emotion analysis result.
Because of the traditional Chinese text emotion analysis method, characters or words are adopted in the text language pre-training process, and are directly used as input of a text emotion analysis neural network model after word2vec processing. Resulting in less utilization of the original text information and poor learning ability of the text characterization.
And the present application constructs the information contained in the text from multiple angles: word information (word), part-of-speech information (pos), word information (char), word part-of-speech information (char_pos), word-position information (char_word_tag), word-part-of-speech-position information (char-pos-tag); respectively adopting a word2 vec-based BiGRU network to perform characteristic construction on the 6 types of original information; then, carrying out feature fusion on the 6 types of feature information, and analyzing dot product and direct splicing for a fusion method; the text characterization model is applied to a text emotion analysis task, a multi-input single-output neural network model is compiled, a plurality of input layers are used for receiving various Chinese text information vectors, an enabling layer, a BiGRU layer, a concate/dot layer and a BiLSTM layer are arranged in the middle, and finally a sigmoid function is utilized for classification, so that the probability of predicting text emotion to be positive is output.
In the embodiment, 21105 Chinese online shopping comment texts are used in the experimental data, and the types of data content objects include hotels, milk, books, mobile phones and the like. Comment emotion tags are classified into two categories [0,1], negative emotion is 0, and positive emotion is 1. For example: the mobile phone system is good, the mobile phone system is bad, the mobile phone system is easy to be damaged, partial keys are inflexible, and the mobile phone system has big problems after half a year and the emotion is negative. The dataset settings are shown in table 1.
Table 1 experimental data set up
The evaluation indexes used in the experiment are Accuracy, precision, recall and F1 value) This is 4 model evaluation indexes commonly used in the NLP field.
Let the total test set number be:. The specific meanings are shown in Table 2.
TABLE 2 evaluation of index-related parameter meanings
Accuracy is a comprehensive evaluation of the correct classification ability of a model, and the higher the Accuracy is, the better the classification ability of the model is, as shown in the following formula.
Precision is the accuracy, as the formula:as shown.
Recall is an evaluation of Recall, as expressed by:as shown.
The F1 value is an evaluation index of the complex Precision, recall. Such as formulaAs shown.
The experiment calculates the Accumey value, the Precision value, the Recall value, the,The values, and the results of comparison are shown in Table 3.
Table 3 model comparison results
The above results illustrate the feasibility and effectiveness of the Chinese text emotion analysis method based on the fusion of multiple text features.
It should be noted that the embodiments of the present invention are only preferred modes for implementing the present invention, and only obvious modifications are included in the overall concept of the present invention, and should be considered as falling within the scope of the present invention.

Claims (5)

1. A Chinese text emotion analysis method based on fusion of multiple text features is characterized by comprising the following steps:
step1, acquiring Chinese text information, and preprocessing the Chinese text information to obtain a plurality of sequences corresponding to the Chinese text information;
step2, inputting the sequences into a BiGRU network to extract text features of each sequence and generating a plurality of text feature information corresponding to the sequences;
step3, fusing the text characteristic information, and inputting the fused text characteristic information into a BiLSTM network for learning;
step4, screening the characteristics by using a self-attention mechanism, and distributing corresponding weights to the characteristic information extracted in the Step3 to obtain the most important emotion information;
step5, inputting the feature vectors screened by the self-attention mechanism into a sigmoid classifier for classification to obtain a final emotion analysis result;
the plurality of sequences include a text word sequence, a part-of-speech sequence, a word-part-of-speech sequence, and a word-part-of-speech sequence;
the fusing the text feature information includes: the corresponding feature vector information is processedBy matrix splicing or dot multiplyingLine fusion to obtain fused text features；
The fused text featuresInputting the output state of the BiLSTM network at a certain moment t, and connecting the outputs of the forward LSTM network and the reverse LSTM network, wherein if the output state is->The hidden state of the moment forward LSTM output is +.>The hidden state of the reverse LSTM output is +.>Hidden state of BiLSTM output +.>。
2. The method for emotion analysis of chinese text based on fusion of multiple text features of claim 1, wherein said extracting text features of each sequence comprises: training the sequences through a word2vec model to obtain a plurality of sequence matrixes corresponding to the sequences, wherein the element vector of each element corresponding to each sequence is as follows，/>Wherein->Is the number of elements->Is a vector dimension, then the entire sequence matrix Uj for each sequence is expressed as: />J represents a sequence number; the Uj is input into a trained BiGRU network, the forward and reverse text sequences are processed at the same time, and feature extraction is carried out on the deep text information to obtain corresponding feature vector information ∈>。
3. The method of claim 2, wherein the biglu network is composed of forward GRU, reverse GRU and output state connection layers of forward GRU, if the hidden state of forward GRU output at time t isThen->The hidden state of the reverse GRU output is +.>ThenWhile the semantics of the BiGRU network output are expressed as +.>Wherein->Is a weight matrix, < >>For GRU function, ++>For GRU input at time t, +.>Is a bias vector.
4. The method for emotion analysis of chinese text based on fusion of multiple text features of claim 1, wherein said screening features using self-attention mechanism comprises: generating target attention weights，/>，/>Is the attention-mechanics learning function tanh, +.>Is the characteristic vector output by the BiLSTM network; attention weight is then probabilistic, according to the formula:generating a probability vector by means of a softmax function>The method comprises the steps of carrying out a first treatment on the surface of the Finally, attention weight configuration is carried out, and the method is based on the formulaThe generated attention weights are allocated to corresponding hidden layer semantic codes ++>Wherein->Is->Is a weighted average of +.>。
5. The method for emotion analysis of chinese text based on fusion of multiple text features of claim 4, wherein said inputting a sigmoid classifier for classification comprises: vectors processed by self-attention mechanismObtaining a characteristic vector through a dropout layer>The method comprises the steps of carrying out a first treatment on the surface of the Feature vector +.>Inputting to a full connection layer, wherein the full connection layer parameter is 1, the activation function is a sigmoid function, and according to a model: />Training to output final emotion analysis result, wherein the sample is +.>，/>Is negative 0 or positive 1, ">Is a sample feature vector, ++>Representing a trainable parameter; adopts->Training model parameters as loss function->Model optimization is carried out by adopting an Adam optimization algorithm, wherein +.>For input +.>Is (are) true category->Input +.>Probability of belonging to category 1.
CN202110554330.5A 2021-05-20 2021-05-20 Chinese text emotion analysis method based on fusion of multiple text features Active CN113342971B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110554330.5A CN113342971B (en) 2021-05-20 2021-05-20 Chinese text emotion analysis method based on fusion of multiple text features

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110554330.5A CN113342971B (en) 2021-05-20 2021-05-20 Chinese text emotion analysis method based on fusion of multiple text features

Publications (2)

Publication Number Publication Date
CN113342971A CN113342971A (en) 2021-09-03
CN113342971B true CN113342971B (en) 2024-03-08

Family

ID=77470303

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110554330.5A Active CN113342971B (en) 2021-05-20 2021-05-20 Chinese text emotion analysis method based on fusion of multiple text features

Country Status (1)

Country Link
CN (1) CN113342971B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110532549A (en) * 2019-08-13 2019-12-03 青岛理工大学 A kind of text emotion analysis method based on binary channels deep learning model
CN111651593A (en) * 2020-05-08 2020-09-11 河南理工大学 Text emotion analysis method based on word vector and word vector mixed model

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11010559B2 (en) * 2018-08-30 2021-05-18 International Business Machines Corporation Multi-aspect sentiment analysis by collaborative attention allocation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110532549A (en) * 2019-08-13 2019-12-03 青岛理工大学 A kind of text emotion analysis method based on binary channels deep learning model
CN111651593A (en) * 2020-05-08 2020-09-11 河南理工大学 Text emotion analysis method based on word vector and word vector mixed model

Also Published As

Publication number Publication date
CN113342971A (en) 2021-09-03

Similar Documents

Publication Publication Date Title
CN111275085B (en) Online short video multi-modal emotion recognition method based on attention fusion
Zhang et al. Modeling both Context-and Speaker-Sensitive Dependence for Emotion Detection in Multi-speaker Conversations.
CN111291181B (en) Representation learning for input classification via topic sparse self-encoder and entity embedding
CN109597891B (en) Text emotion analysis method based on bidirectional long-and-short-term memory neural network
CN111046668B (en) Named entity identification method and device for multi-mode cultural relic data
CN108416065B (en) Hierarchical neural network-based image-sentence description generation system and method
CN108536679A (en) Name entity recognition method, device, equipment and computer readable storage medium
CN108595601A (en) A kind of long text sentiment analysis method incorporating Attention mechanism
CN108460089A (en) Diverse characteristics based on Attention neural networks merge Chinese Text Categorization
CN110083833B (en) Method for analyzing emotion by jointly embedding Chinese word vector and aspect word vector
CN109522548A (en) A kind of text emotion analysis method based on two-way interactive neural network
CN112364638B (en) Personality identification method based on social text
CN106372058A (en) Short text emotion factor extraction method and device based on deep learning
CN109255359B (en) Visual question-answering problem solving method based on complex network analysis method
CN110765769B (en) Clause feature-based entity attribute dependency emotion analysis method
CN110472042A (en) A kind of fine granularity sensibility classification method
Quinta de Castro et al. Portuguese named entity recognition using lstm-crf
CN110580287A (en) Emotion classification method based ON transfer learning and ON-LSTM
CN114021524B (en) Emotion recognition method, device, equipment and readable storage medium
CN112527966A (en) Network text emotion analysis method based on Bi-GRU neural network and self-attention mechanism
Lin et al. Sentiment analysis of low-carbon travel APP user comments based on deep learning
Li et al. Learning to write stories with thematic consistency and wording novelty
Af'idah et al. Long short term memory convolutional neural network for Indonesian sentiment analysis towards touristic destination reviews
Bahuleyan Natural language generation with neural variational models
CN113342971B (en) Chinese text emotion analysis method based on fusion of multiple text features

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant