CN112035661A - Text emotion analysis method and system based on graph convolution network and electronic device - Google Patents
Text emotion analysis method and system based on graph convolution network and electronic device Download PDFInfo
- Publication number
- CN112035661A CN112035661A CN202010856600.3A CN202010856600A CN112035661A CN 112035661 A CN112035661 A CN 112035661A CN 202010856600 A CN202010856600 A CN 202010856600A CN 112035661 A CN112035661 A CN 112035661A
- Authority
- CN
- China
- Prior art keywords
- word
- semantic features
- syntax tree
- dependency
- dependency syntax
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008451 emotion Effects 0.000 title claims abstract description 78
- 238000004458 analytical method Methods 0.000 title claims abstract description 45
- 239000013598 vector Substances 0.000 claims abstract description 61
- 238000000034 method Methods 0.000 claims abstract description 41
- 239000011159 matrix material Substances 0.000 claims abstract description 40
- 230000011218 segmentation Effects 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims description 19
- 230000002457 bidirectional effect Effects 0.000 claims description 14
- 239000000284 extract Substances 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000003780 insertion Methods 0.000 claims 1
- 230000037431 insertion Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 11
- 238000013527 convolutional neural network Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000000605 extraction Methods 0.000 description 5
- 230000002996 emotional effect Effects 0.000 description 4
- 238000013508 migration Methods 0.000 description 4
- 230000005012 migration Effects 0.000 description 4
- 230000001419 dependent effect Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007935 neutral effect Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a text emotion analysis method, a text emotion analysis system and an electronic device based on a graph convolution network, wherein the method comprises the following steps: segmenting the input text sequence; converting each word segmentation into corresponding word embedding according to the text sequence; extracting the forward semantic features and the reverse semantic features of each word embedding, and combining the forward semantic features and the reverse semantic features at the same positions to obtain the context semantic features of each word embedding; calculating a semantic relation value between any two word embeddings according to the context semantic features of each word embeddings to obtain a connection matrix; analyzing a dependency syntax tree of the text sequence according to the connection matrix; performing graph convolution operation by taking the dependency syntax tree as a graph to obtain a dependency vector of a ROOT node of the dependency syntax tree; and carrying out sentiment polarity classification and scoring on the dependency vector at the ROOT node position in the dependency syntax tree, and determining the sentiment polarity type of the text sequence.
Description
Technical Field
The application relates to the technical field of text emotion analysis, in particular to a text emotion analysis method and system based on a graph convolution network and an electronic device.
Background
Emotion analysis technology is gradually emerging with the rapid development of the internet in the early 20 th century, and has gradually expanded from the academic research field to the industrial application field. Text sentiment analysis is used as a text classification task, a dictionary-based method is adopted in the early stage, a sufficiently large sentiment dictionary is constructed in advance, and then the sentiment tendency of the text is judged by using rules. However, the construction process of the emotion dictionary needs to be manually arranged for each type of words, and the emotion dictionary needs to be continuously maintained due to continuous appearance of new words, so that the method has huge manpower investment; meanwhile, the performance of the method is poor because the method ignores the sequence of the text. The second category is based on Machine learning methods, such as Support Vector Machine (SVM), naive bayes (e.g., (m:))
Bayes, NB), etc. The performance of the method depends on the selection of the characteristics, so the method has low portability. The third category is a deep learning-based method, which mainly uses two deep Neural networks, namely, Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) to realize classification. The method can automatically capture deep semantic features from massive texts, does not need to construct and maintain an emotion dictionary, does not need to manually construct features, and realizes an end-to-end text emotion analysis task. The CNN effectively captures emotion information at different positions by enlarging the size of a convolution kernel, and further obtains local emotion characteristics of the text, but the extraction capability of the long-distance semantic relationship is weak. The RNN cannot directly model semantic relationships between non-adjacent words, so that when sample data is long or a language scene is complex, the intervals between effective emotion information are large or small, and different lengths are provided, and the RNN performance is also limited. And in addition, the emotion dictionary (HowNet) is also used in the calculation process of part of methods, so that the flexibility of the methods on the field migration is influenced.
Disclosure of Invention
The method aims to solve the problems that the existing text sentiment analysis technology is weak in extraction capability of long-distance semantic relations, cannot directly model semantic relations between non-adjacent words, and is poor in flexibility in field migration.
In order to achieve the purpose, the application provides a text emotion analysis method, a text emotion analysis system and an electronic device based on a graph convolution network.
In a first aspect, an embodiment of the present application provides a text emotion analysis method based on a graph convolution network, including performing word segmentation on an input text sequence; converting each word segmentation into corresponding word embedding according to the text sequence; extracting the forward semantic features and the reverse semantic features of each word embedding, and combining the forward semantic features and the reverse semantic features at the same positions to obtain the context semantic features of each word embedding; calculating a semantic relation value between any two word embeddings according to the context semantic features of each word embeddings to obtain a connection matrix; analyzing a dependency syntax tree of the text sequence according to the connection matrix; performing graph convolution operation by taking the dependency syntax tree as a graph to obtain a dependency vector of a ROOT node of the dependency syntax tree; and carrying out sentiment polarity classification and scoring on the dependency vector at the ROOT node position in the dependency syntax tree, and determining the sentiment polarity type of the text sequence.
In one possible implementation, the converting each participle into a corresponding word embedding according to the text sequence order includes: converting discrete high-frequency words in each participle into low-dimensional continuous vectors according to the text sequence, converting discrete low-frequency words in each participle into low-dimensional continuous vectors corresponding to special symbols, and embedding the low-dimensional continuous vectors into words corresponding to each participle; the word embedding layer is initialized with the Glove vector with a dimension of 300.
In a possible embodiment, the extracting the forward semantic features and the reverse semantic features embedded in each word, and combining the forward semantic features and the reverse semantic features at the same position to obtain the context semantic features embedded in each word includes: inputting each word embedding sequence into a bidirectional LSTM network; the forward LSTM network in the bidirectional LSTM network extracts semantic features of each word embedding forward; the reverse LSTM network in the bidirectional LSTM network extracts semantic features of each word embedded in the reverse direction; combining the forward semantic features and the reverse semantic features embedded into each word at the same position, and outputting the context semantic features embedded into each word.
In a possible implementation manner, the semantic relation value between any two word embeddings is calculated according to the context semantic features of each word embeddings, so as to obtain a connection matrix; parsing a dependency syntax tree for the text sequence according to the connection matrix, comprising: according to the context semantic features of each word embedding, calculating semantic relation values between any two word embedding one by one through a multilayer perceptron to obtain a connection matrix; and analyzing the relationship between two nodes according to each semantic relationship value of the connection matrix by taking each word embedded as a node, determining whether the two corresponding nodes in the dependency syntax tree are connected, and connecting all the nodes according to the connection line to obtain the dependency syntax tree of the text sequence.
In one possible embodiment, performing a graph convolution operation with the dependency syntax tree as a graph to obtain a dependency vector of the dependency syntax tree ROOT node includes: and inputting a graph convolution network by taking the dependency syntax tree as a graph, wherein the graph convolution network takes the context semantic features embedded in each word as the initial state of each node of the dependency syntax tree, takes the ROOT node in the dependency syntax tree as an end identifier, performs graph convolution operation on the dependency syntax tree, and outputs a dependency vector of the ROOT node of the dependency syntax tree.
In one possible embodiment, the method further comprises a training step: performing word segmentation on the text sequence of the training set, and matching each word segmentation result sequence with the emotion polarity type corresponding to the whole sentence of the word segmentation result sequence to form a form of a pair of the text sequence and the emotion polarity type; and taking a text sequence of the training set as input, taking a corresponding emotion polarity category as output, using the classified cross entropy as a loss function, and using Adam by an optimizer to carry out overall training to obtain a trained text emotion analysis model based on the graph convolution network.
In a possible embodiment, before the performing the overall training, the method further includes: independently training the graph convolution network to obtain a trained graph convolution network; the word embedding layer is initialized with the Glove vector, with 300 dimensions.
In a second aspect, an embodiment of the present application provides a text emotion analysis system based on a graph convolution network, including: the word segmentation module is used for segmenting the input text sequence; the word embedding module is used for converting the word segmentation input sequentially into word embedding corresponding to each word segmentation; the bidirectional LSTM network is used for extracting the forward semantic features and the reverse semantic features embedded into each word, and combining the forward semantic features and the reverse semantic features at the same positions to obtain the context semantic features embedded into each word; the dependency syntax tree analysis module is used for calculating a semantic relation value between any two word embeddings according to the context semantic features of each word embeddings to obtain a connection matrix; analyzing a dependency syntax tree of the text sequence according to the connection matrix; the graph convolution network is used for carrying out graph convolution operation by taking the dependency syntax tree as a graph to obtain a dependency vector of a dependency syntax tree ROOT node; and the Softmax classifier is used for carrying out emotion polarity classification and scoring on the dependency vector of the ROOT node position in the dependency syntax tree and determining the emotion polarity category of the text sequence.
In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor; the processor is configured to execute the computer-executable instructions stored in the memory, and the processor executes the computer-executable instructions to perform the text emotion analysis method based on the graph and volume network according to any one of the above embodiments.
In a fourth aspect, an embodiment of the present application provides a storage medium, which includes a readable storage medium and a computer program stored in the readable storage medium, where the computer program is configured to implement the text emotion analysis method based on a graph volume network according to any one of the above embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments disclosed in the present application, the drawings required to be used in the description of the embodiments will be briefly introduced below, and it is apparent that the drawings in the following description are only embodiments disclosed in the present application, and it is obvious for those skilled in the art that other drawings can be obtained based on the drawings without inventive efforts.
FIG. 1 is a model framework diagram of a text emotion analysis method based on a graph convolution network according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a network module of a bidirectional LSTM network of a text emotion analysis method based on a graph convolution network according to an embodiment of the present application;
FIG. 3 is an analysis diagram of a relationship matrix and a dependency syntax tree of a graph-convolution-network-based text emotion analysis apparatus according to an embodiment of the present application;
FIG. 4 is an example of a dependency syntax tree of a graph-convolution-network-based text emotion analysis apparatus according to an embodiment of the present application;
fig. 5 is a dependency syntax tree diagram and connection matrix comparison diagram of a graph-convolution network-based text emotion analysis device according to an embodiment of the present application.
Detailed Description
The technical solution of the present application is further described in detail by the accompanying drawings and examples.
The embodiment of the application discloses a text emotion analysis method based on a graph convolution network, and the invention concept of the method is firstly introduced below.
When text emotion analysis is performed by using a deep Neural Network such as a Convolutional Neural Network (CNN) or a convolutional Neural Network (RNN), the extraction capability of a long-distance semantic relationship is weak, and a semantic relationship between non-adjacent words cannot be directly modeled, so that the performance of effective emotion information analysis is limited when sample data is long or a language scene is complex, and the flexibility of part of methods in field migration is poor.
Considering that the long-short term memory network LSTM is a network for processing time series, the computing units in LSTM, such as neurons, can memorize previous information and take it as a later input. Due to its time-recursive nature, LSTM networks are used to process sequences with time-sequential characteristics, such as text (sequenced by word or word), sequences of actions that occur in sequence, and so forth. Therefore, by utilizing the characteristic that the LSTM can process time sequence information, when input sample data is long or a language scene is complex and the interval between effective emotion information is large or small, a bidirectional LSTM network can be adopted to effectively capture the context feature information of each participle of the input sample, so that the text emotion analysis obtained according to the context feature information is not influenced by the distance of the position.
Further, a Multi Layer Perceptron (MLP) is a feedforward artificial neural network model, and has very good nonlinear mapping capability. Therefore, the multi-layer perceptron can be used for calculating the semantic relation between adjacent and/or non-adjacent words, and whether the semantic relation between the adjacent and/or non-adjacent words has dependency relationship or not can be determined according to the calculated value. After determining whether each participle has a dependency association therebetween, each participle having a dependency association may be connected to each other according to the semantic relationship value to form a dependency syntax tree.
And performing graph convolution operation by taking the dependency syntax tree as a graph, so that word-to-word dependency vectors can be captured. Graph Convolution Networks (GCNs) are a class of neural networks that employ Convolution operations in the Graph. Similar to the CNN network, by carrying out convolution operation for multiple times, the content in a larger range can be seen, and the receptive field is improved. In each convolution operation, the graph convolution network updates itself by using the information of the node connected with the current node; after convolution operation for many times, the nodes which are farther and farther away in the dependency tree can be gradually seen, the current node is updated by using the information of the nodes, and the influence of the neighbor nodes which are closer is larger. A GCN network at L level will perform L update operations that can see information for all nodes within distance L.
Due to the tree structure of the graph convolution network, the extracted features are excellent even though training is not performed.
And finally, carrying out emotion polarity classification and scoring on the text sequence by adopting a Softmax classifier according to the dependency vector of the ROOT node in the dependency syntax tree. And determining the emotion polarity category of the text sequence according to the score.
The Softmax classifier can convert score values obtained by linear classification into probabilities for multi-classification. The input value of the Softmax classifier is a vector, elements in the vector are credit values of any real number, the output is a vector, each element value is between 0 and 1, and the sum of all the elements is 1, namely the classification probability of 1.
According to the above concept, in a first aspect, an embodiment of the present application provides a text emotion analysis method based on a graph convolution network.
Fig. 1 is a frame diagram of a text emotion analysis method based on a graph convolution network according to an embodiment of the present application, and as shown in fig. 1, the method includes: s1, performing word segmentation on the input text sequence; s2, converting the participles into corresponding words to be embedded according to the text sequence; s3, extracting the forward semantic features and the reverse semantic features embedded in each word, and combining the forward semantic features and the reverse semantic features at the same positions to obtain the context semantic features embedded in each word; s4, calculating a semantic relation value between any two word embeddings according to the context semantic features of each word embeddings to obtain a connection matrix; analyzing a dependency syntax tree of the text sequence according to the connection matrix; s5, performing graph convolution operation by taking the dependency syntax tree as a graph to obtain a dependency vector of a dependency syntax tree ROOT node; and S6, carrying out emotion polarity classification and scoring on the dependency vector at the ROOT node position in the dependency syntax tree, and determining the emotion polarity type of the text sequence.
The following specifically analyzes each step of the method according to the frame diagram of the text emotion analysis method based on the graph convolution network shown in fig. 1.
In the embodiment of the application, a text sequence of chinese, english or other characters is input into the word segmentation module 11 for word segmentation, and the text sequence may be a sentence, a plurality of sentences, or a text paragraph. The method and the device adopt a Chinese academy word segmentation model (NLPIR) to segment words of the Chinese input text sequence.
In the embodiment of the present application, the word segmentation is input into the word embedding module 12 according to the text sequence order, the word embedding module 12 processes each word segmentation into a data vector that can be processed by a computer, and the data vector that can be processed by the computer is embedded into a word corresponding to each word segmentation.
Specifically, the word embedding module 12 includes a word embedding layer, where the word embedding layer stores low-dimensional continuous data vectors corresponding to V high-frequency words, and the word embedding layer can convert discrete high-frequency words in a text sequence into low-dimensional continuous data vectors that can be processed by a computer, and process discrete low-frequency words in the text sequence into low-dimensional continuous vectors corresponding to special symbols.
Embodiments of the present application embed words into a bi-directional LSTM network 13, where the bi-directional LSTM network 13 includes a Forward LSTM network 131(Forward LSTM, F-LSTM) and a Backward LSTM network 132(Backward LSTM, B-LSTM). The forward LSTM network 131 extracts the forward semantic features of each participle, the backward LSTM network 132 extracts the backward semantic features of each participle, combines the forward semantic features and the backward semantic features at the same position, and outputs the context semantic features embedded in each word.
FIG. 2 is a schematic diagram of network modules of a bidirectional LSTM network according to an embodiment of the present application, and the input sequence is a word embedding sequence (x) as shown in FIG. 21，x2，...，xT) And T is the length of the sequence,
representing hidden states at various times of positive LSTM131,
representing hidden states at various times of inverse LSTM132, and finallyCombining the LSTM network outputs in two directions at the same position to obtain the final output vector sequence (y)1，y2，...，yT) The elements of the vector sequence represent the embedded contextual semantic features of each of the words.
In the embodiment of the application, the context semantic features of each participle are input into a Multi Layer Perceptron (MLP) 14, and the MLP 14 calculates a semantic relationship value between any two embedded words to obtain a relationship matrix (relationship matrix). And taking each word embedding as a node, determining whether the nodes are connected according to the semantic relation value between the word embedding, and connecting the nodes according to the determined connection to generate a semantic connection graph. And connecting all the nodes through the determined connecting lines to obtain a complete graph. The maximum spanning tree corresponding to the complete graph is the final dependency syntax tree. The representation of the location of the ROOT node "ROOT" of the dependent syntax tree is a random initialization location.
Wherein, the relation matrix is the connection matrix of the dependency syntax tree. The line between the nodes may be an arc with a point. Because each node has a neighbor node having a semantic relation with the node, a complete graph can be obtained by connecting the nodes accordingly.
FIG. 3 is a parsing diagram of a connection matrix and a dependency syntax tree, and as shown in FIG. 3, the word embedding of each participle of a text sequence "green food but the service is dreadful" is egreat、efood、ebut、eservice、eis、edreadfulThe context semantic feature embedded in each word is output as Vgreat、Vfood、Vbut、Vservice、Vis、Vdreadful. The context semantic features of each word embedding are input into the multi-layer perceptron 14, a semantic relation value between any two word embedding is calculated, the semantic relation value between any two word embedding forms a relation matrix (relationship matrix), and each element of the relation matrix is a semantic relation value between corresponding row word embedding and column word embedding. Taking participles of street, food, but, the, service, is and dreadful as nodes to be divided into pointsThe semantic relation value between words determines the connecting line between two nodes, and a graph can be generated, wherein the maximum spanning tree of the graph is the final dependency syntax tree.
FIG. 4 is an example of a dependency syntax tree constructed from an input text sequence according to an embodiment of the present application. In the figure: the Root node is a Root node of the dependency syntax tree, the HED, the SBV, the ADV and the like respectively represent different semantic relationship connection types, and the different semantic relationship types and the connection types are shown in table 1.
TABLE 1
FIG. 5 is a diagram illustrating a dependency syntax tree diagram and a connection matrix, where, as shown in FIG. 5, the dependency syntax tree shows semantic connections of 1-6 nodes, and each row and column in the connection matrix represents a node in the dependency syntax tree, and if the value of the element in the row 1 and column 2 in the table is 1, this indicates that there is a connection line from node 1 to node 2, and if the value of the element is 0, this indicates that there is no connection line between two nodes.
The embodiment of the present application takes the dependency syntax tree as a graph, and inputs a graph convolution network 15, where the graph convolution network 15 performs a plurality of convolution operations on the dependency syntax tree, and each convolution operation is updated by using the semantic relation value of the node connected to the current node, and the graph convolution network 15 outputs the dependency vector between the current node and the connected node in the dependency syntax tree.
The graph-convolution network 15 uses the context semantic feature of each participle output by the bi-directional LSTM as the initial state of each node in the dependency syntax tree, uses the "ROOT" node identifier in the dependency syntax tree as the dependency syntax tree ending identifier, performs graph-convolution operation on the dependency syntax tree, and outputs the dependency vector of the "ROOT" node position of the dependency syntax tree.
The formula for each update of the nodes in the graph convolution network 15 is:
Hl+1＝f(Hl，A) (1)
Hl+1＝Relu(AHlWl) (2)
The specific calculation process of the function f in the formula 1 is given in a formula 2, wherein Relu is an activation function, and a connection matrix A and the l-th layer hidden layer characteristic H are connectedlMultiplying by the weight parameter matrix W of the l-th layerlPerforming Relu activation operation on the multiplied result to obtain hidden layer characteristic H of the l +1 th layerl+1。Hl+1The hidden layer features of the neighbor nodes equivalent to a certain node are added. And the multilayer hidden layers are superposed, so that the semantic relation information of the multilayer neighbor nodes can be obtained.
The embodiment of the application inputs a dependency vector of a node position of a dependency syntax tree ROOT into a Softmax classifier, each element in the dependency vector is a scoring value of context semantic features of each node, the Softmax classifier outputs a score vector of emotion polarity categories, the score of each emotion polarity category in the score vector is between 0 and 1, and the sum of the scores of all the emotion polarity categories is 1. Emotional polarity is often divided into three categories: positive, negative, neutral. The emotional polarity can also be divided according to semantics such as happiness, anger, sadness, happiness, and the like. And determining the emotion polarity type of the text sequence according to the score of each emotion polarity type.
The text emotion analysis method based on the graph convolution network further comprises the step of training each module.
Firstly, segmenting words of a training set text sequence, and pairing each segmented word with the emotion polarity type of a corresponding label to form a form of a < text sequence, emotion polarity type > pair.
Wherein the word embedding module 12 initializes the word embedding layer using the Glove vector, the embedding layer dimension being 300.
Then, the graph convolution network 15 is trained independently, and after the graph convolution network 15 is trained independently, the parameters of the graph convolution module are not updated. Other network parameters use random initialization.
When all modules are integrally trained, the text sequence of the training set is used as input, the corresponding emotion polarity type is used as output, the classified cross entropy is used as a loss function, and the optimizer performs training by using Adam to obtain a trained text emotion analysis model based on the graph convolution network.
In the testing phase, ACC and Macro-F1 are used to measure the performance of the graph-convolution network based text emotion analysis model.
Because the text emotion analysis method based on the graph convolution network provided by the embodiment of the application adopts the bidirectional long-short term memory network LSTM, the extraction capability of the long-distance semantic relationship is strong, the semantic relationship between adjacent and/or non-adjacent words can be calculated by adopting a multilayer perceptron, whether the semantic relationship between the adjacent and/or non-adjacent words has dependency relationship is determined, and a dependency syntax tree is generated according to the dependency relationship; the graph convolution network 15 has a tree structure and has strong characteristic extraction capability, and the requirement of the training process on the scale of the training sample is greatly reduced; and graph convolution operation is carried out on the dependency syntax tree of the input sentence or text sequence, and dependency vectors among the participles can be introduced, so that the text emotion analysis performance is improved. Meanwhile, the method does not use a knowledge base related to the field, so that the field migration capability is higher.
In a second aspect, an embodiment of the present application provides a text emotion analysis system based on a graph convolution network.
Returning to fig. 1, fig. 1 shows that the system includes a segmentation module 11, a word embedding module 12, a bi-directional LSTM network 13, a dependency syntax tree parsing module 14, a graph convolution network 15, and a Softmax classification module 16.
The word segmentation module 11 performs word segmentation on the text sequence input by Chinese, and sequentially inputs word segmentation results into the word embedding module 12. The text sequence may be a sentence, several sentences, or a text paragraph.
The word embedding module 12 processes each word segmentation input sequentially into a data vector that can be processed by a computer and input sequentially into the bi-directional LSTM network 13. The computer may process a data vector for word embedding for each participle.
Specifically, the word embedding module 12 of the embodiment of the present application includes a word embedding layer, where the word embedding layer stores low-dimensional continuous data vectors corresponding to V high-frequency words, and the word embedding layer can convert discrete high-frequency words in a text sequence into a data vector form that can be processed by a computer, and process discrete low-frequency words in the text sequence into a special symbol < unk >. The word embedding layer is initialized with the Glove vector with a dimension of 300.
The bidirectional LSTM network 13 extracts the forward semantic features and the reverse semantic features embedded for each word, combines the forward semantic features and the reverse semantic features at the same position, and outputs a context semantic feature vector embedded for each word.
Specifically, bidirectional LSTM network 13 includes a Forward LSTM network 131(Forward LSTM, F-LSTM), and a Backward LSTM network 132(Backward LSTM, B-LSTM). The forward LSTM network 131 extracts the semantic feature of each word embedding forward direction, the backward LSTM network 132 extracts the semantic feature of each participle backward direction, combines the forward semantic feature and the backward semantic feature of each word embedding according to the position, and outputs the context semantic feature vector of each word embedding to the dependency syntax tree parsing module.
The dependency syntax tree parsing module 14 calculates a semantic relation value between any two word embeddings according to the context semantic features of each word embeddings to obtain a connection matrix; and resolving the dependency syntax tree of the text sequence according to the connection matrix.
Specifically, the dependent syntax tree parsing module 14 includes a multilayer perceptron 141 and a dependent syntax tree parsing unit 142.
The Multi Layer Perceptron 141 (MLP) calculates a semantic relation value between any two word embeddings according to the context semantic features of each word embeddings, and forms a relation matrix (relationship matrix) by each semantic relation value.
The dependency syntax tree parsing unit 142 uses each word embedding as a node, determines a connection line between every two nodes according to a semantic relation value between the word embedding, connects the nodes according to the determined connection line, and constructs a graph. When all the input nodes are connected according to the semantic relation values in the relation matrix, a complete graph is obtained, and the maximum spanning tree of the complete graph is the final dependency syntax tree.
The initial state of each node in the dependency syntax tree is the context semantic feature vector of each participle output by the bi-directional LSTM, and the ending state of the dependency syntax tree is the location where the "ROOT" flag is located.
The Graph Convolution Network 15 (GCN) performs a Graph Convolution operation on the dependency syntax tree, and outputs a dependency vector of the ROOT node position in the dependency syntax tree.
In the embodiment of the present application, the graph-convolution network 15 takes the context semantic feature vector of each participle output by the bi-directional LSTM as the initial state of each node in the dependency syntax tree, takes the "ROOT" identifier in the dependency syntax tree as the end identifier, performs the graph-convolution operation on the dependency syntax tree, and outputs the dependency vector of the ROOT node in the dependency syntax tree to the Softmax classifier 16.
The Softmax classifier 16 performs emotion polarity classification and scoring on the dependency vectors of the ROOT nodes in the dependency syntax tree, outputs emotion polarity category scores of the text sequences, and determines emotion polarity categories of the text sequences.
In the embodiment of the present application, the Softmax classifier 16 takes a dependency vector of the ROOT point of the dependency syntax tree as an input, each element in the dependency vector is a score value of the contextual semantic features of each node, and outputs a score sequence of emotion polarity categories, the score of each emotion polarity category in the score sequence is between 0 and 1, and the sum of the scores of all emotion polarity categories is 1. Emotional polarity is often divided into three categories: positive, negative, neutral. The emotional polarity can also be divided according to semantics such as happiness, anger, sadness, happiness, and the like. And finally, determining the emotion polarity type of the text sequence according to the score of each emotion polarity type.
In a third aspect, embodiments of the present application further provide an electronic device, including a memory and a processor; the processor is used for executing the computer execution instructions stored in the memory, and when the processor runs the computer execution instructions, the text emotion analysis method based on the graph volume network, which is provided by any of the above embodiments, is executed.
In a fourth aspect, the present application further provides a storage medium, which includes a readable storage medium and a computer program stored in the readable storage medium, where the computer program is configured to implement the text emotion analysis method based on a graph-volume network proposed in any of the above embodiments.
It will be further appreciated by those of ordinary skill in the art that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether these functions are performed in hardware or software depends on the particular application of the solution and design constraints. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, a software module executed by a processor, or a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above-mentioned embodiments, objects, technical solutions and advantages of the present application are described in further detail, it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present application, and are not intended to limit the scope of the present application, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present application should be included in the scope of the present application.
Claims (10)
1. A text emotion analysis method based on a graph convolution network is characterized by comprising the following steps:
segmenting the input text sequence;
converting each word segmentation into corresponding word embedding according to the text sequence;
extracting the forward semantic features and the reverse semantic features of each word embedding, and combining the forward semantic features and the reverse semantic features at the same positions to obtain the context semantic features of each word embedding;
calculating a semantic relation value between any two word embeddings according to the context semantic features of each word embeddings to obtain a connection matrix; analyzing a dependency syntax tree of the text sequence according to the connection matrix;
performing graph convolution operation by taking the dependency syntax tree as a graph to obtain a dependency vector of the ROOT node position of the dependency syntax tree;
and carrying out sentiment polarity classification and scoring on the dependency vector at the ROOT node position in the dependency syntax tree, and determining the sentiment polarity type of the text sequence.
2. The method of claim 1, wherein said converting each of said participles into corresponding word insertions in said text sequence order comprises:
converting discrete high-frequency words in each participle into low-dimensional continuous vectors according to the text sequence, converting discrete low-frequency words in each participle into low-dimensional continuous vectors corresponding to special symbols, and embedding the low-dimensional continuous vectors into words corresponding to each participle; the word embedding layer is initialized with the Glove vector with a dimension of 300.
3. The method according to claim 1, wherein said extracting forward semantic features and reverse semantic features of each word embedding, and combining the forward semantic features and reverse semantic features at the same position to obtain context semantic features of each word embedding, comprises:
inputting each word embedding sequence into a bidirectional LSTM network;
the forward LSTM network in the bidirectional LSTM network extracts semantic features of each word embedding forward;
the reverse LSTM network in the bidirectional LSTM network extracts semantic features of each word embedded in the reverse direction;
combining the forward semantic features and the reverse semantic features embedded into each word at the same position, and outputting the context semantic features embedded into each word.
4. The method according to claim 1, wherein a semantic relation value between any two word embeddings is calculated according to the context semantic features of each word embeddings to obtain a connection matrix; parsing a dependency syntax tree for the text sequence according to the connection matrix, comprising:
according to the context semantic features of each word embedding, calculating semantic relation values between any two word embedding one by one through a multilayer perceptron to obtain a connection matrix;
and analyzing the relationship between two nodes according to each semantic relationship value of the connection matrix by taking each word embedded as a node, determining whether the two corresponding nodes in the dependency syntax tree are connected, and connecting all the nodes according to the connection line to obtain the dependency syntax tree of the text sequence.
5. The method according to claim 1, wherein performing a graph convolution operation with the dependency syntax tree as a graph to obtain a dependency vector of the ROOT node position of the dependency syntax tree comprises:
and inputting a graph convolution network by taking the dependency syntax tree as a graph, wherein the graph convolution network takes the context semantic features embedded in each word as the initial state of each node of the dependency syntax tree, takes the ROOT node in the dependency syntax tree as an end identifier, performs graph convolution operation on the dependency syntax tree, and outputs a dependency vector at the ROOT node position of the dependency syntax tree.
6. The method according to one of claims 1 to 5, characterized in that the method further comprises a training step of:
performing word segmentation on the text sequence of the training set, and matching each word segmentation result sequence with the emotion polarity type corresponding to the whole sentence of the word segmentation result sequence to form a form of a pair of the text sequence and the emotion polarity type;
and taking a text sequence of the training set as input, taking a corresponding emotion polarity category as output, using the classified cross entropy as a loss function, and using Adam by an optimizer to carry out overall training to obtain a trained text emotion analysis model based on the graph convolution network.
7. The method of claim 6, further comprising, prior to said performing ensemble training:
independently training the graph convolution network to obtain a trained graph convolution network;
the word embedding layer is initialized with the Glove vector, with 300 dimensions.
8. A text emotion analysis system based on a graph convolution network is characterized by comprising:
the word segmentation module is used for segmenting the input text sequence;
the word embedding module is used for converting the word segmentation input sequentially into word embedding corresponding to each word segmentation;
the bidirectional LSTM network is used for extracting the forward semantic features and the reverse semantic features embedded into each word, and combining the forward semantic features and the reverse semantic features at the same positions to obtain the context semantic features embedded into each word;
the dependency syntax tree analysis module is used for calculating a semantic relation value between any two word embeddings according to the context semantic features of each word embeddings to obtain a connection matrix; analyzing a dependency syntax tree of the text sequence according to the connection matrix;
the graph convolution network is used for carrying out graph convolution operation by taking the dependency syntax tree as a graph to obtain a dependency vector of the dependency syntax tree ROOT node position;
and the Softmax classifier is used for carrying out emotion polarity classification and scoring on the dependency vector of the ROOT node position in the dependency syntax tree and determining the emotion polarity category of the text sequence.
9. An electronic device comprising a memory and a processor; the processor is used for executing the computer-executable instructions stored in the memory, and the processor executes the computer-executable instructions to execute the method for text emotion analysis based on graph and volume network according to any one of claims 1 to 7.
10. A storage medium comprising a readable storage medium and a computer program stored in the readable storage medium, the computer program being configured to implement the method for text emotion analysis based on a graph and volume network according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010856600.3A CN112035661A (en) | 2020-08-24 | 2020-08-24 | Text emotion analysis method and system based on graph convolution network and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010856600.3A CN112035661A (en) | 2020-08-24 | 2020-08-24 | Text emotion analysis method and system based on graph convolution network and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112035661A true CN112035661A (en) | 2020-12-04 |
Family
ID=73580703
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010856600.3A Pending CN112035661A (en) | 2020-08-24 | 2020-08-24 | Text emotion analysis method and system based on graph convolution network and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112035661A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112287699A (en) * | 2020-12-29 | 2021-01-29 | 南京新一代人工智能研究院有限公司 | Information fusion translation method based on syntax tree traversal |
| CN112800754A (en) * | 2021-01-26 | 2021-05-14 | 浙江香侬慧语科技有限责任公司 | Unsupervised grammar derivation method, unsupervised grammar derivation device and medium based on pre-training language model |
| CN112948541A (en) * | 2021-02-01 | 2021-06-11 | 华南理工大学 | Financial news text emotional tendency analysis method based on graph convolution network |
| CN113158684A (en) * | 2021-04-21 | 2021-07-23 | 清华大学深圳国际研究生院 | Emotion analysis method, emotion reminding method and emotion reminding control device |
| CN113239143A (en) * | 2021-04-28 | 2021-08-10 | 国网山东省电力公司电力科学研究院 | Power transmission and transformation equipment fault processing method and system fusing power grid fault case base |
| CN113254637A (en) * | 2021-05-07 | 2021-08-13 | 山东师范大学 | Grammar-fused aspect-level text emotion classification method and system |
| CN117556787A (en) * | 2024-01-11 | 2024-02-13 | 西湖大学 | Method and system for generating target text sequence for natural language text sequence |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107247702A (en) * | 2017-05-05 | 2017-10-13 | 桂林电子科技大学 | A kind of text emotion analysis and processing method and system |
| CN109543039A (en) * | 2018-11-23 | 2019-03-29 | 中山大学 | A kind of natural language sentiment analysis method based on depth network |
| AU2019100371A4 (en) * | 2019-04-05 | 2019-05-16 | Ba, He Mr | A Sentiment Analysis System Based on Deep Learning |
| CN110799981A (en) * | 2017-06-29 | 2020-02-14 | 罗伯特·博世有限公司 | System and method for domain-independent aspect-level emotion detection |
| CN111160008A (en) * | 2019-12-18 | 2020-05-15 | 华南理工大学 | Entity relationship joint extraction method and system |
| CN111241809A (en) * | 2018-11-29 | 2020-06-05 | 深港产学研基地产业发展中心 | Model establishing method and device, computer equipment and storage medium |
| CN111259142A (en) * | 2020-01-14 | 2020-06-09 | 华南师范大学 | Specific target emotion classification method based on attention coding and graph convolution network |
| CN111563164A (en) * | 2020-05-07 | 2020-08-21 | 成都信息工程大学 | Specific target emotion classification method based on graph neural network |
-
2020
- 2020-08-24 CN CN202010856600.3A patent/CN112035661A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107247702A (en) * | 2017-05-05 | 2017-10-13 | 桂林电子科技大学 | A kind of text emotion analysis and processing method and system |
| CN110799981A (en) * | 2017-06-29 | 2020-02-14 | 罗伯特·博世有限公司 | System and method for domain-independent aspect-level emotion detection |
| CN109543039A (en) * | 2018-11-23 | 2019-03-29 | 中山大学 | A kind of natural language sentiment analysis method based on depth network |
| CN111241809A (en) * | 2018-11-29 | 2020-06-05 | 深港产学研基地产业发展中心 | Model establishing method and device, computer equipment and storage medium |
| AU2019100371A4 (en) * | 2019-04-05 | 2019-05-16 | Ba, He Mr | A Sentiment Analysis System Based on Deep Learning |
| CN111160008A (en) * | 2019-12-18 | 2020-05-15 | 华南理工大学 | Entity relationship joint extraction method and system |
| CN111259142A (en) * | 2020-01-14 | 2020-06-09 | 华南师范大学 | Specific target emotion classification method based on attention coding and graph convolution network |
| CN111563164A (en) * | 2020-05-07 | 2020-08-21 | 成都信息工程大学 | Specific target emotion classification method based on graph neural network |
Non-Patent Citations (1)
| Title |
|---|
| 杨顺成;李彦;赵其峰;: "基于GCN和Bi-LSTM的微博立场检测方法", 重庆理工大学学报(自然科学), vol. 34, no. 06, pages 167 - 173 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112287699A (en) * | 2020-12-29 | 2021-01-29 | 南京新一代人工智能研究院有限公司 | Information fusion translation method based on syntax tree traversal |
| CN112800754A (en) * | 2021-01-26 | 2021-05-14 | 浙江香侬慧语科技有限责任公司 | Unsupervised grammar derivation method, unsupervised grammar derivation device and medium based on pre-training language model |
| CN112948541A (en) * | 2021-02-01 | 2021-06-11 | 华南理工大学 | Financial news text emotional tendency analysis method based on graph convolution network |
| CN112948541B (en) * | 2021-02-01 | 2022-09-20 | 华南理工大学 | Financial news text emotional tendency analysis method based on graph convolution network |
| CN113158684A (en) * | 2021-04-21 | 2021-07-23 | 清华大学深圳国际研究生院 | Emotion analysis method, emotion reminding method and emotion reminding control device |
| CN113239143A (en) * | 2021-04-28 | 2021-08-10 | 国网山东省电力公司电力科学研究院 | Power transmission and transformation equipment fault processing method and system fusing power grid fault case base |
| CN113239143B (en) * | 2021-04-28 | 2022-09-30 | 国网山东省电力公司电力科学研究院 | Power transmission and transformation equipment fault processing method and system fusing power grid fault case base |
| CN113254637A (en) * | 2021-05-07 | 2021-08-13 | 山东师范大学 | Grammar-fused aspect-level text emotion classification method and system |
| CN117556787A (en) * | 2024-01-11 | 2024-02-13 | 西湖大学 | Method and system for generating target text sequence for natural language text sequence |
| CN117556787B (en) * | 2024-01-11 | 2024-04-26 | 西湖大学 | Method and system for generating target text sequence for natural language text sequence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112035661A (en) | Text emotion analysis method and system based on graph convolution network and electronic device | |
| CN109413028B (en) | SQL injection detection method based on convolutional neural network algorithm | |
| CN108388651B (en) | Text classification method based on graph kernel and convolutional neural network | |
| US20170011289A1 (en) | Learning word embedding using morphological knowledge | |
| CN112906392B (en) | Text enhancement method, text classification method and related device | |
| CN112231477B (en) | Text classification method based on improved capsule network | |
| WO2023065859A1 (en) | Item recommendation method and apparatus, and storage medium | |
| CN115393692A (en) | Generation formula pre-training language model-based association text-to-image generation method | |
| CN112101031B (en) | Entity identification method, terminal equipment and storage medium | |
| CN113628059B (en) | Associated user identification method and device based on multi-layer diagram attention network | |
| Benzebouchi et al. | Multi-classifier system for authorship verification task using word embeddings | |
| CN112131886A (en) | Method for analyzing aspect level emotion of text | |
| CN111967267B (en) | XLNET-based news text region extraction method and system | |
| CN111078876A (en) | Short text classification method and system based on multi-model integration | |
| CN112307048B (en) | Semantic matching model training method, matching method, device, equipment and storage medium | |
| CN114780748A (en) | Priori weight enhancement-based completion method of knowledge graph | |
| CN113434858A (en) | Malicious software family classification method based on disassembly code structure and semantic features | |
| CN107908757B (en) | Website classification method and system | |
| CN112925904A (en) | Lightweight text classification method based on Tucker decomposition | |
| CN113535897A (en) | Fine-grained emotion analysis method based on syntactic relation and opinion word distribution | |
| CN111737467B (en) | Object-level emotion classification method based on segmented convolutional neural network | |
| CN110377753B (en) | Relation extraction method and device based on relation trigger word and GRU model | |
| CN112560440A (en) | Deep learning-based syntax dependence method for aspect-level emotion analysis | |
| CN117235216A (en) | Knowledge reasoning method based on heterogeneous knowledge fusion | |
| CN113111136B (en) | Entity disambiguation method and device based on UCL knowledge space |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |