CN109446331B

CN109446331B - Text emotion classification model establishing method and text emotion classification method

Info

Publication number: CN109446331B
Application number: CN201811492975.5A
Authority: CN
Inventors: 王邦; 汪畅; 徐明华
Original assignee: Huazhong University of Science and Technology
Current assignee: Huazhong University of Science and Technology
Priority date: 2018-12-07
Filing date: 2018-12-07
Publication date: 2021-03-26
Anticipated expiration: 2038-12-07
Also published as: CN109446331A

Abstract

The invention discloses a text emotion classification model establishing method and a text emotion classification method, wherein the text emotion classification model establishing method comprises the following steps: performing sentence division operation on a document to be classified, and respectively obtaining word vectors of all words in each sentence; sequentially establishing a word conversion layer, a document vector synthesis layer and an output layer according to the sentence dividing result so as to complete the establishment of a text emotion classification model; the word conversion layer comprises M tree structure neural networks which correspond to M sentences obtained by sentence division one by one and are respectively used for converting word vectors of the words in the sentences into hidden vectors; the document vector synthesis layer is used for obtaining a document vector of the document to be classified according to the hidden vector of the word obtained by the conversion of the word conversion layer; the output layer is used for mapping the document vectors obtained by the document vector synthesis layer into emotion probability distribution and normalizing the emotion probability distribution, so that the probability of the documents to be classified on each emotion category is obtained. The method and the device effectively improve the accuracy of text emotion classification by fusing syntactic information, topic information and semantic information.

Description

Text emotion classification model establishing method and text emotion classification method

Technical Field

The invention belongs to the field of machine learning, and particularly relates to a text emotion classification model establishing method and a text emotion classification method.

Background

With the rapid development and popularization of internet technology, people receive a large number of news articles or blog posts on the internet every day, and it becomes important to analyze the emotional tendency of the articles or the posts, which can help public sentiment managers to quickly monitor and guide public sentiments. In addition, the emotion classification can play a role in article search, article recommendation, and the like. For emotion classification, a variety of techniques have been developed, which can be broadly classified into a topic model-based method, a feature selection-based method, and a neural network-based method.

The theme model-based method is characterized in that theme information of a document is extracted by means of a theme model (for example, an LDA theme model), and then the theme information of the document is associated with corresponding emotion of the document, so that the purpose of emotion classification is achieved. The method idea based on feature selection is to extract useful features in the document by means of feature engineering, such as: the part-of-speech characteristics, the label symbol characteristics, the relationship characteristics among emotions and the like, and then the classification is carried out by utilizing a common classifier in machine learning; in recent years, with the deep application of neural networks in other fields, researchers have begun to solve the problem of emotion classification using neural networks. The method idea based on the neural network is to discover the relation between words in the document by means of the neural network, form a proper document feature vector and then classify by using a classifier. The neural network models commonly used at present are CNN (convolutional neural network) and LSTM (long short term memory network). The CNN is characterized in that local features of the document can be effectively extracted, the LSTM serves as a sequence model, the document is regarded as an ordered sequence, and the context information of the document is good for learning.

The method based on the theme model and based on the feature selection has the advantages of simple model, strong interpretability and capability of effectively utilizing the traditional text features, but has the defects of manpower consumption and low detection rate of feature engineering. The method based on the neural network can deeply mine text semantic information, learn characteristic vectors by self, obtain higher detection rate, but has poor interpretability. In addition, most of the existing work separates and researches and uses the traditional text features (such as topics) and deep semantic information learned by a neural network, and the classification accuracy is not high. The use of syntax information is lacking in the conventional research.

Disclosure of Invention

Aiming at the defects and improvement requirements of the prior art, the invention provides a text emotion classification model establishing method and a text emotion classification method, and aims to improve the accuracy of text emotion classification.

To achieve the above object, according to a first aspect of the present invention, there is provided a method for establishing a text emotion classification model, where the text emotion classification model is used to predict the probability of a document to be classified in each emotion category, the method includes:

(1) performing sentence division operation on a document to be classified, and respectively obtaining word vectors of all words in each sentence;

(2) sequentially establishing a word conversion layer, a document vector synthesis layer and an output layer according to the sentence dividing result so as to complete the establishment of a text emotion classification model;

the word conversion layer comprises M tree structure neural networks, corresponds to M sentences obtained by sentence division one by one and is respectively used for converting word vectors of the words in the sentences into hidden vectors; the document vector synthesis layer is used for obtaining a document vector of the document to be classified according to the hidden vector of the word obtained by the conversion of the word conversion layer; the output layer is used for mapping the document vectors obtained by the document vector synthesis layer into emotion probability distribution and normalizing the emotion probability distribution, so that the probability of the documents to be classified on each emotion category is obtained.

In the method, the word vectors of the words are converted into the corresponding hidden vectors by adopting the tree-structure neural network, so that the deep semantic information and the syntactic information learned by the neural network are fused in the process of text emotion classification, and the accuracy of text emotion classification can be effectively improved.

Further, if the number of sentences M obtained by sentence division is 1, the step (2) includes:

carrying out syntactic analysis on the sentence S obtained by the sentence division to obtain a dependency syntactic tree T of the sentence S; establishing a tree structure neural network TN based on the dependency syntax tree T, thereby obtaining a word conversion layer formed by the tree structure neural network TN, and converting the word vector of each word in the sentence S into a corresponding hidden vector;

obtaining the topic probability distribution p (k | d) of the document to be classified and the topic probability distribution p (k | w) of each word in the document to be classified by using the trained topic model TM_mn) And according to the topic probability distribution p (k | d) and the topic probability distribution p (k | w)_mn) Calculating the attention weight of each word in the document to be classified; establishing a document vector synthesis layer according to the attention weight of the words, and carrying out weighted summation on hidden vectors of the words to obtain a document vector of the document to be classified;

an output layer is constructed according to the dimension of the document vector and the number of emotion categories and is used for mapping the document vector into emotion probability distribution and normalizing the emotion probability distribution, so that the probability of the document to be classified on each emotion category is obtained;

wherein d is a document to be classified, k is a topic number, m is a sentence number, n is a word number, w_mnRepresenting the nth word in the mth sentence.

For a single sentence sub-file, the attention weight of each word in the sentence is obtained by using the topic model, and the hidden vector of the word is subjected to weighted summation according to the attention weight of the word to obtain the document vector of the document, so that the topic information is further fused in the process of text emotion classification on the basis of fusing the deep semantic information and the syntactic information learned by the neural network, and the accuracy of text emotion classification can be effectively improved.

Further, if the sentence number M obtained by sentence segmentation is greater than 1, the document vector synthesis layer sequentially comprises a sentence synthesis layer, a sentence transformation layer and a document synthesis layer, and the step (2) comprises:

m sentences S obtained by separating sentences respectively₁～S_MPerforming syntactic analysis to obtain a sentence S₁～S_MDependency syntax tree T₁～T_M(ii) a Based on dependency syntax tree T, respectively₁～T_MBuilding tree-structured neural network TN₁～TN_MSo as to obtain the neural network TN with the tree structure₁～TN_MA word conversion layer for converting each sentence S₁～S_MConverting the word vector of each word into a corresponding hidden vector;

obtaining the topic probability distribution p (k | d) of the document to be classified and the topic probability distribution p (k | w) of each word in the document to be classified by using the topic model TM_mn) And according to the topic probability distribution p (k | d) and the topic probability distribution p (k | w)_mn) Calculating the attention weight of each word in the document to be classified; establishing a sentence synthesis layer according to the attention weight of the word for carrying out weighted summation on the hidden vectors of the word to respectively obtain S₁～S_MSentence vector xs₁～xs_M；

From sentence vector xs₁～xs_MEstablishing a neural network CN with chain structure, thereby obtaining a sentence transformation layer composed of the neural network CN with chain structure, and using the sentence vector xs₁～xs_MRespectively converted into corresponding hidden vectors hs₁～hs_M；

Establishing a document synthesis layer for obtaining a hidden vector hs of a tail node of a neural network CN with a chain structure_MDocuments as documents to be classifiedVector quantity;

and constructing an output layer according to the dimension of the document vector and the number of emotion categories, and mapping the document vector into emotion probability distribution and normalizing, thereby obtaining the probability of the document to be classified on each emotion category.

m sentences S obtained by separating sentences respectively₁～S_MPerforming syntactic analysis to obtain a sentence S₁～S_MDependency syntax tree T₁～T_MAnd are respectively based on the dependency syntax tree T₁～T_MBuilding tree-structured neural network TN₁～TN_MSo as to obtain the neural network TN with the tree structure₁～TN_MA word conversion layer for converting each sentence S₁～S_MConverting the word vector of each word into a corresponding hidden vector;

building sentence synthesis layers for respectively obtaining dependency syntax trees T₁～T_MAs the sentence S₁～S_MSentence vector xs₁～xs_M；

Obtaining the topic probability distribution p (k | d) of the document to be classified and the probability p (w) of each word under each topic by using the topic model TM_mnI k) and according to the topic probability distribution p (k i d) and the probability p (w)_mnL k) calculating the attention weight of each sentence in the document to be classified; establishing a document synthesis layer according to the attention weight of the sentence, and carrying out weighted summation on the hidden vector of the sentence to obtain a document vector of the document to be classified;

For different documents, the specifically established model structures are different: for a single sentence document, only a tree structure neural network is adopted, and a chain structure neural network is not adopted; for multi-sentence documents, simultaneously adopting a tree structure neural network and a chain structure neural network; therefore, the model training speed can be prevented from being too slow under the condition of effectively improving the accuracy of the text emotion classification.

For multi-sentence documents, in the process of text emotion classification, only the attention weight of a word or the attention weight of a sentence is used for synthesizing a document vector, and the model training speed can be prevented from being too slow under the condition of effectively improving the accuracy of the text emotion classification.

Further, the operation of normalizing the emotion probability distribution by the output layer is softmax normalization.

By using softmax normalization, probability distribution can be balanced, and the situation that the probability is 0 is avoided, so that the model does not need to be subjected to smoothing treatment.

According to a second aspect of the present invention, there is provided a text emotion classification method, including: for the documents to be classified, the trained node parameters are utilized, a text emotion classification model is built according to the text emotion classification model building method provided by the first aspect of the invention, and the built text emotion classification model is utilized to predict the probability of the documents to be classified in each emotion category so as to finish the text emotion classification of the documents to be classified.

Further, the method for acquiring the node parameter comprises the following steps:

(S1) dividing the corpus into training sets and testing sets, and turning in (S4); the emotion classification of each document in the corpus is known;

(S2) for document D_iEstablishing a text emotion classification model by using the text emotion classification model establishing method provided by the first aspect of the invention, and predicting the emotion of the document in each emotion category by using the text emotion classification modelProbability;

(S3) according to the document D_iAdjusting node parameters of the text emotion classification model to minimize a pre-constructed loss function;

(S4) performing the steps (S2) - (S3) on the documents in the training set in sequence, thereby completing the training of the text emotion classification model and obtaining trained node parameters;

wherein i is a document number.

Further, the method for acquiring the node parameter further includes: and (S2) executing steps on the documents in the test set in sequence according to the trained node parameters, and testing the text emotion classification model according to the emotion classification and prediction results of the documents, thereby completing the verification of the node parameters.

Generally, by the above technical solution conceived by the present invention, the following beneficial effects can be obtained:

(1) according to the text emotion classification method provided by the invention, the word vectors of the words are converted into corresponding hidden vectors by adopting the tree-structured neural network, so that the process of text emotion classification is integrated with deep semantic information and syntax information learned by the neural network, and the accuracy of text emotion classification can be effectively improved.

(2) According to the text emotion classification method provided by the invention, for a single sentence document, the attention weight of each word in the document is obtained by using the topic model, for a multi-sentence document, the attention weight of each word in the document or the attention weight of each sentence in the document is obtained by using the topic model, and a document vector is synthesized by using the attention weight of the word or the attention weight of the sentence, so that the text emotion classification process is further fused with topic information on the basis of fusing deep semantic information and syntactic information learned by a neural network, and the accuracy of text emotion classification can be effectively improved.

(3) According to the text emotion classification method provided by the invention, different text emotion classification models are respectively established for single-sentence files and multi-sentence files, and when the text emotion classification models are established for the multi-sentence files, only the attention weight of words or the attention weight of sentences is utilized, so that the model training speed is prevented from being too slow under the condition of effectively improving the accuracy of text emotion classification.

(4) According to the text emotion classification method provided by the invention, as the topic model is adopted, the attention weight of each word or sentence can be visualized, so that the word or sentence which is emphasized can be clearly shown in the text emotion classification process, and the interpretability of the model is improved to a certain extent.

Drawings

FIG. 1 is a flowchart of a first embodiment of a text emotion classification model establishment method according to the present invention;

FIG. 2 is a diagram of a dependency syntax tree provided by an embodiment of the present invention;

FIG. 3 is an internal structure diagram of a conventional Tree-LSTM network;

FIG. 4 is a flowchart of a second embodiment of a text emotion classification model establishment method according to the present invention;

FIG. 5 is an internal structure diagram of a conventional Chain-LSTM network;

FIG. 6 is a flowchart of a third embodiment of a text emotion classification model establishment method of the present invention;

FIG. 7 is a flowchart of a method for classifying emotion in text according to an embodiment of the present invention;

fig. 8 is a schematic diagram of word probability distribution of a part of topics learned by the LDA topic model according to the embodiment of the present invention; (a) - (c) respectively representing the probability distribution diagrams of the words corresponding to different subjects;

FIG. 9 is a schematic diagram of a topic probability distribution corresponding to a document in a corpus according to an embodiment of the present invention;

FIG. 10 is a diagram of a word with a higher weight in a document according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

According to the first aspect of the invention, a text emotion classification model establishing method is provided, and the text emotion classification model established by the method is used for predicting the probability of a document to be classified in each emotion category.

As shown in fig. 1, in the first embodiment of the method, when the document to be classified is a single-sentence document, the method for establishing the text emotion classification model provided by the present invention includes:

(1) carrying out sentence segmentation operation on a document to be classified to obtain a sentence S, and obtaining word vectors of words in the sentence S;

in an alternative embodiment, a trained word vector model can be used to obtain a word vector of each word in a sentence; the used Word vector model can be a Word2vec Word vector model, and other Word vector models can also be used;

the word vector representation of the word is as follows:

where m is the sentence number, n is the word number, xw_mnA word vector representing the nth word of the mth sentence, x representing the value of the word vector, d_wA dimension representing a word vector;

(2) performing syntactic analysis on the sentence S to obtain a dependency syntactic tree T of the sentence S; establishing a tree structure neural network TN based on the dependency syntax tree T, thereby obtaining a word conversion layer formed by the tree structure neural network TN, and converting the word vector of each word in the sentence S into a corresponding hidden vector;

the dependency syntax tree describes the syntax dependence relationship among all words, fig. 2 shows an existing dependency syntax tree example, a tree structure neural network is built based on the dependency syntax tree, words with relationship in syntax can be combined to form a phrase feature vector, and the tree structure neural network plays an important role in mining deep semantic information;

in the embodiment of the present invention, the used Tree-structured neural network is Tree-LSTM, the internal structure of the Tree-LSTM is shown in fig. 3, and the transformation function is as follows:

f₁＝σ(W^(f)x₃+U^(f)h₁+b^(f))

f₂＝σ(W^(f)x₃+U^(f)h₂+b^(f))

i₃＝σ(W⁽ⁱ⁾x₃+U⁽ⁱ⁾(h₁+h₂)+b⁽ⁱ⁾)

o₃＝σ(W^(o)x₃+U^(o)(h₁+h₂)+b^(o))

u₃＝tanh(W^(u)x₃+U^(u)(h₁+h₂)+b^(u))

c₃＝i₃⊙u₃+(f₁⊙c₁+f₂⊙c₂)

h₃＝o₃⊙tanh(c₃)

wherein x is₃A word vector, h, representing the current parent node input₁,h₂Hidden vectors, c, for child 1 and child 2, respectively₁,c₂Cell states of child 1 and child 2, f, i, o are forgetting gate, input gate and output gate, respectively, which indicates dot product operation;

the word vectors of the words can be converted into corresponding hidden vectors by using the conversion function; the hidden vector representation of the word is as follows;

where m is the sentence number, n is the word number, hw_mnA hidden vector representing the nth word of the mth sentence, h represents the value of the hidden vector, d_mA dimension representing a concealment vector;

it should be understood that the present invention may also implement a word conversion layer using neural networks of other tree structures;

(3) obtaining the topic probability distribution p (k | d) of the document to be classified and the topic probability distribution p (k | w) of each word in the document to be classified by using the trained topic model TM_mn) (ii) a In the embodiment of the invention, the adopted theme model TM can be an LDA theme model;

calculating a topic probability distribution p (k | d) and a topic probability distribution p (k | w)_mn) The cosine similarity between them is:

performing softmax normalization on the cosine similarity in each sentence, and taking the normalized value as the attention weight of the word, wherein the calculation formula is as follows;

establishing a document vector synthesis layer according to the attention weight of the words, and performing weighted summation on the hidden vectors of the words to obtain a document vector of the document to be classified:

wherein d is a document to be classified, k is a topic number, m is a sentence number, n and n' are word numbers, w_mnRepresenting the nth word in the mth sentence, wherein N is the total number of words in the sentence;

it should be understood that other topic models can be used to obtain corresponding topic probability distributions;

(4) an output layer is constructed according to the dimensionality of the document vector and the emotion category number and is used for mapping the document vector into emotion probability distribution and normalizing the emotion probability distribution, so that the probability of the document to be classified on each emotion category is obtained; is formulated as follows:

l＝W·yd

wherein l is the emotion probability distribution obtained by document vector mapping, W is the weight parameter matrix, E is the number of emotion categories,

is the probability value of the document in the ith mood class.

As shown in fig. 4, in the second embodiment of the method, when the document to be classified is a multi-sentence document, the method for establishing the text emotion classification model provided by the present invention includes:

(1) carrying out sentence splitting operation on the document to be classified to obtain M sentences S₁～S_MRespectively obtaining word vectors of all words in each sentence;

the specific implementation of obtaining the word vector of the word can refer to the description in the first embodiment of the method;

(2) m sentences S obtained by separating sentences respectively₁～S_MPerforming syntactic analysis to obtain a sentence S₁～S_MDependency syntax tree T₁～T_M；

Based on dependency syntax tree T, respectively₁～T_MBuilding tree-structured neural network TN₁～TN_MSo as to obtain the neural network TN with the tree structure₁～TN_MA word conversion layer for converting each sentence S₁～S_MThe word vector of each word in the word is converted into corresponding hidden wordsVector quantity; in the embodiment of the present invention, the Tree structure neural network used is Tree-LSTM, but it should be understood that other Tree structure neural networks may be used;

(3) obtaining the topic probability distribution p (k | d) of the document to be classified and the topic probability distribution p (k | w) of each word in the document to be classified by using the topic model TM_mn) (ii) a In the embodiment of the present invention, the adopted topic model is an LDA topic model, and it should be understood that other topic models may also be adopted;

establishing a sentence synthesis layer according to the attention weight of the word for carrying out weighted summation on the hidden vectors of the word to respectively obtain S₁～S_MSentence vector xs₁～xs_M(ii) a Wherein, the sentence vector of the mth sentence is:

(4) from sentence vector xs₁～xs_MEstablishing a neural network CN with chain structure, thereby obtaining a sentence transformation layer composed of the neural network CN with chain structure, and using the sentence vector xs₁～xs_MRespectively converted into corresponding hidden vectors hs₁～hs_M；

In the embodiment of the invention, the adopted Chain structure neural network is Chain-LSTM, the internal structure of Chain-LSTM is shown in FIG. 5, and the conversion function is as follows:

f＝σ(W^(f)s_t+U^(f)h_t-1+b^(f))

i＝σ(W⁽ⁱ⁾s_t+U⁽ⁱ⁾h_t-1+b⁽ⁱ⁾)

o＝σ(W^(o)s_t+U^(o)h_t-1+b^(o))

u_t＝tanh(W^(u)s_t+U^(u)h_t-1+b^(u))

c_t＝i⊙u_t+f⊙c_t-1

h_t＝o⊙tanh(c_t)

wherein s is_tSentence vector, h, representing time t_tAnd h_t-1Representing the concealment vector at time t and t-1, respectively, c_tAnd c_t-1Respectively representing the cell states at the t moment and the t-1 moment;

the sentence vectors can be converted into corresponding hidden vectors through the conversion function;

it should be understood that the invention can also adopt other chain structure neural networks to realize the sentence conversion layer;

(5) establishing a document synthesis layer for obtaining a hidden vector hs of a tail node of a neural network CN with a chain structure_MAs a document vector of the document to be classified;

the sentence synthesis layer, the sentence conversion layer and the document synthesis layer jointly form a document vector synthesis layer of the text emotion classification model;

(6) an output layer is constructed according to the dimensionality of the document vector and the emotion category number and is used for mapping the document vector into emotion probability distribution and normalizing the emotion probability distribution, so that the probability of the document to be classified on each emotion category is obtained;

for the specific implementation of establishing the output layer, reference may be made to the description in the first embodiment of the method above.

As shown in fig. 6, in the third embodiment of the method, when the document to be classified is a multi-sentence document, the method for establishing the text emotion classification model provided by the present invention includes:

Based on dependency syntax tree T, respectively₁～T_MBuilding tree-structured neural network TN₁～TN_MSo as to obtain the neural network TN with the tree structure₁～TN_MA word conversion layer for converting each sentence S₁～S_MConverting the word vector of each word into a corresponding hidden vector; in the present invention, the Tree-structured neural network used is Tree-LSTM, it should be understood that other Tree-structured neural networks may be used;

(3) building sentence synthesis layers for respectively obtaining dependency syntax trees T₁～T_MAs the sentence S₁～S_MSentence vector xs₁～xs_M；

(4) From sentence vector xs₁～xs_MEstablishing a neural network CN with chain structure, thereby obtaining a sentence transformation layer composed of the neural network CN with chain structure, and using the sentence vector xs₁～xs_MRespectively converted into corresponding hidden vectors hs₁～hs_M(ii) a In the embodiment of the present invention, the neural network with a Chain structure is Chain-LSTM, and it should be understood that other neural networks with a Chain structure may also be used;

(5) obtaining the topic probability distribution p (k | d) of the document to be classified and the probability p (w) of each word under each topic by using the topic model TM_mn|k)；

From the topic probability distribution p (k | d) and the probability p (w)_mn| k) calculating the topic probability score of each sentence respectivelyCloth; wherein the mth sentence s_mSubject probability distribution p (k | s)_m) Comprises the following steps:

wherein N is_mRepresenting a sentence s_mThe number of words in (1);

calculating a topic probability distribution p (k | s)_m) And the cosine similarity sim(s) between the topic probability distribution p (k | d)_mAnd d) is:

wherein, IR represents the information radius, KL represents the KL distance;

mixing sim(s)_mD) normalizing softmax in the document to obtain a sentence s_mThe attention weight of (a) is:

establishing a document synthesis layer according to the attention weight of each sentence, and performing weighted summation on hidden vectors of the sentences to obtain document vectors of the documents to be classified:

the sentence synthesis layer, the sentence conversion layer and the document synthesis layer form a document vector synthesis layer of the text emotion classification model;

(6) an output layer is constructed according to the dimension of the document vector and the number of emotion categories and is used for mapping the document vector into emotion probability distribution and normalizing the emotion probability distribution, so that the probability of the document to be classified on each emotion category is obtained;

In the above method embodiment, the operation of normalizing the emotion probability distribution by the output layer is softmax normalization; by using softmax normalization, probability distribution can be balanced, and the situation that the probability is 0 is avoided, so that the model does not need to be subjected to smoothing treatment.

Based on the above method embodiment, the present invention further provides a text emotion classification method, for implementing text emotion classification of a document to be classified, as shown in fig. 7, including: for the documents to be classified, the trained node parameters are utilized, a text emotion classification model is built according to the text emotion classification model building method provided by the first aspect of the invention, and the built text emotion classification model is utilized to predict the probability of the documents to be classified in each emotion category so as to finish the text emotion classification of the documents to be classified.

In an optional embodiment, the trained node parameters in the text emotion classification method are obtained through model training, and the method for obtaining the node parameters specifically includes:

(S2) for document D_iEstablishing a text emotion classification model by using the text emotion classification model establishing method provided by the first aspect of the invention, and predicting the probability of the document on each emotion category by using the text emotion classification model;

(S3) according to the document D_iAdjusting node parameters of a text emotion classification model by using a BP algorithm so as to minimize a pre-constructed loss function;

in the embodiment of the invention, the pre-constructed loss function is emotion probability distribution based on prediction

And constructing a KL divergence loss function according to the real emotion probability distribution y, wherein the expression is as follows:

wherein the content of the first and second substances,

in order to predict the result of the event,

is the predicted probability value of the document in the ith emotion category, y_iIs the true probability value of the document in the ith emotion category;

wherein i is a document number;

the method for acquiring the node parameters further comprises the following steps: and (S2) executing steps on the documents in the test set in sequence according to the trained node parameters, and testing the text emotion classification model according to the emotion classification and prediction results of the documents, thereby completing the verification of the node parameters.

Examples of the applications

Taking the Sina news data set as a training corpus, and using the Sina news data set as a training corpus to train a model so as to obtain trained node parameters; the Tree structure neural network adopts a Tree-LSTM network, and the Chain structure neural network adopts a Chain-LSTM network; the Word vector model adopts a Word2vec Word vector model trained by a Chinese Wikipedia database; the topic model employs an LDA topic model trained from the entire new wave news dataset. The newscast data set was collected from a social channel of newscast (https:// news. sina. com. cn /), containing a total of 5258 hot news released from month 1 to month 12 in 2016. After each news, there are 6 types of emotion labels for readers to vote, which are: feeling, anger, sympathy, sadness, surprise, and novelty. Table 1 shows detailed voting statistics, wherein the training set is the first 3109 data and the testing set is the last 2149 data in the data set.

TABLE 1

Table 2 shows the setting results of the relevant parameters in the document emotion classification process.

TABLE 2

Fig. 8 shows the word probability distribution of part of the topics learned by the LDA topic model in the form of word clouds. Wherein a larger word indicates a higher probability of occurrence. As can be seen from the figure, the subject 3 is mainly related to "express", "hospital", "telephone", etc., the subject 9 is mainly related to "volunteer", "hospital", "japan", etc., and the subject 24 is mainly related to "school", "teacher", "student", etc.

Fig. 9 shows a topic probability distribution corresponding to a certain document in the news data set of the new wave, and the document original: the school and the teacher's name are … summer-heat period irrelevant to school, school organizes students to make a lesson for a period of time, school also has a lesson-making … after study, the picture of the daily morning taken by the parents in 402 classrooms is shown by the reporter 12 days after the study is performed on weekends and the study is performed in the evening every day by 10 days, the second and middle school students in the city of China have a third-year class leader who says that the parents receive money, and the money-taking activity of the second and middle school parents is finally dispelled. However, the explanation of Ju Ning and Wu Hui Jun explained that some parents do not hear 'call'. In the course of the year, if parents pay the money, the money is handed to who finds out who to return. The money is not accepted by any teacher in the school. In the afternoon of' 14 days, the Shao family committed, and after returning, would communicate with the members of the other parental committees and return all the received money to the parent. "

As can be seen from fig. 9, the topics of the document are mainly distributed on the topics 9 and the topics 24, and with reference to fig. 8, it is easy to see that the content of the document is mainly related to "students", "schools", "teachers" and "parents", and is consistent with the intuitive feeling obtained by directly reading the document, which means that the LDA topic model can effectively extract the topic information of the document, and this also plays a key role in merging topic information in the text emotion classification process.

FIG. 10 shows a word with a higher attention weight in a document. As can be seen from fig. 10, by fusing topic information, the system effectively captures important topic words in the document, such as: "frustration", "cooling with the mind", and the like. This also explains to some extent the superiority of the proposed method.

In order to verify that the method can effectively improve the accuracy of text emotion classification, the accuracy of text emotion classification by using the following 6 models (a) - (f) is tested respectively. The models (a) - (f) are text emotion classification models provided by the embodiment of the invention or models formed by slightly modifying the text emotion classification models provided by the embodiment of the invention. Models (a) - (f) are:

(a) a model built by method embodiment two shown in fig. 4;

(b) a model built by the method embodiment three shown in fig. 6;

(c) modifying the sentence synthesis layer on the basis of the model (a), so that the sentence synthesis layer directly obtains hidden vectors of root nodes of each Tree-LSTM in the word conversion layer as sentence vectors of corresponding sentences, and the rest structures are unchanged;

(d) modifying a word conversion layer on the basis of the model (a), replacing Tree-LSTM in the word conversion layer with Chain-LSTM, weighting and summing hidden vectors of each node of the Chain-LSTM by using attention weight of a word to obtain a corresponding sentence vector, and keeping the other structures unchanged;

(e) modifying the word conversion layer on the basis of the model (b), replacing Tree-LSTM in the word conversion layer with Chain-LSTM, wherein the sentence vector is a hidden vector corresponding to the last node of the Chain-LSTM network, and the rest structures are unchanged;

(f) modifying the word conversion layer on the basis of the model (c), replacing Tree-LSTM in the word conversion layer with Chain-LSTM, wherein the sentence vector is a hidden vector corresponding to the last node of the Chain-LSTM network, and the rest structures are unchanged.

In addition, two models which are best represented on the Sina news data set are Social Opinion Mining model and Emotion-topic model, which are respectively marked as model (g) and model (h).

Table 3 shows the emotion classification accuracy using the models (a) - (h), and it can be seen from the table that the use of the model (b) has higher emotion classification performance than the use of other models, and has more obvious advantages than the existing best models (g and h). Comparing the model (a), the model (b) and the model (c), or the model (d), the model (e) and the model (f), and proving that the fused topic information can promote the emotion classification accuracy; although the former is slightly lower than the latter in the case of the scheme (a) compared with the scheme (d), the gap is not large, and the scheme using the tree-shaped syntactic neural network is better than the scheme using the chain-structured neural network as a whole (e.g., b is better than e, and c is better than f), so that it can be said that the use of the tree-shaped syntactic neural network is more effective for improving the accuracy of emotion classification of texts than the chain-structured neural network.

Model (model)	Accuracy of
		a	63.64％
b	65.50％
		c	62.57％
d	63.73％
		e	63.31％
f	61.69％
		g	58.59％
h	54.19％

TABLE 3

It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A text emotion classification model building method is used for predicting the probability of a document to be classified in each emotion category, and is characterized by comprising the following steps:

(1) performing sentence segmentation operation on the document to be classified, and respectively obtaining word vectors of all words in each sentence;

(2) sequentially establishing a word conversion layer, a document vector synthesis layer and an output layer according to the sentence dividing result so as to complete the establishment of the text emotion classification model;

the word conversion layer comprises M tree structure neural networks, corresponds to M sentences obtained by sentence division one by one, and is respectively used for converting word vectors of words in the sentences into hidden vectors; the document vector synthesis layer is used for obtaining the document vector of the document to be classified according to the hidden vector of the word obtained by the conversion of the word conversion layer; the output layer is used for mapping the document vectors obtained by the document vector synthesis layer into emotion probability distribution and normalizing the emotion probability distribution so as to obtain the probability of the documents to be classified on each emotion category;

if the number of sentences M obtained by sentence division is 1, the step (2) includes:

carrying out syntactic analysis on a sentence S obtained by the sentence so as to obtain a dependency syntactic tree T of the sentence S; establishing a tree structure neural network TN based on the dependency syntax tree T, thereby obtaining a word conversion layer formed by the tree structure neural network TN, and converting the word vector of each word in the sentence S into a corresponding hidden vector;

obtaining the topic probability distribution p (k | d) of the document to be classified and the topic probability distribution p (k | w) of each word in the document to be classified by using the trained topic model TM_mn) And according to said topic probability distribution p (k | d) and said topic probability distribution p (k | w)_mn) Calculating the attention weight of each word in the document to be classified; establishing the document vector synthesis layer according to the attention weight of the words, and performing weighted summation on hidden vectors of the words to obtain a document vector of the document to be classified;

constructing the output layer according to the dimensionality and emotion category number of the document vector, and mapping the document vector into emotion probability distribution and normalizing the emotion probability distribution, so as to obtain the probability of the document to be classified in each emotion category;

wherein d is the document to be classified, k is a topic number, m is a sentence number, n is a word number, w_mnRepresenting the nth word in the mth sentence.

2. The method according to claim 1, wherein if the number of sentences M obtained by sentence segmentation is greater than 1, the document vector synthesis layer comprises a sentence synthesis layer, a sentence conversion layer and a document synthesis layer in sequence, and the step (2) comprises:

m sentences S obtained by separating sentences respectively₁～S_MPerforming a syntactic analysis to obtain the sentence S₁～S_MDependency syntax tree T₁～T_M(ii) a Based on the dependency syntax tree T, respectively₁～T_MBuilding tree-structured neural network TN₁～TN_MSo as to obtain the TN formed by the tree-structured neural network₁～TN_MA word conversion layer for converting the sentences S₁～S_MConverting the word vector of each word into a corresponding hidden vector;

obtaining the topic probability distribution p (k | d) of the document to be classified and the topic probability distribution p (k | w) of each word in the document to be classified by using the topic model TM_mn) And according to said topic probability distribution p (k | d) and said topic probability distribution p (k | w)_mn) Calculating the attention weight of each word in the document to be classified; establishing the sentence synthesis layer according to the attention weight of the word, and performing weighted summation on hidden vectors of the word to respectively obtain the S₁～S_MSentence vector xs₁～xs_M；

According to the sentence vector xs₁～xs_MEstablishing a neural network CN with a chain structure, thereby obtaining a sentence transformation layer formed by the neural network CN with the chain structure, and using the sentence vector xs₁～xs_MRespectively converted into corresponding hidden vectors hs₁～hs_M；

Establishing said document synthesis layer for obtaining said neural network of chain structure CNHidden vector hs of end node_MAs the document vector of the document to be classified;

and constructing the output layer according to the dimensionality and the emotion category number of the document vector, and mapping the document vector into emotion probability distribution and normalizing the emotion probability distribution, so as to obtain the probability of the document to be classified in each emotion category.

3. The method according to claim 2, wherein if the number of sentences M obtained by sentence segmentation is greater than 1, the document vector synthesis layer comprises a sentence synthesis layer, a sentence conversion layer and a document synthesis layer in sequence, and the step (2) comprises:

m sentences S obtained by separating sentences respectively₁～S_MPerforming a syntactic analysis to obtain the sentence S₁～S_MDependency syntax tree T₁～T_MAnd based on the dependency syntax tree T respectively₁～T_MBuilding tree-structured neural network TN₁～TN_MSo as to obtain the TN formed by the tree-structured neural network₁～TN_MA word conversion layer for converting the sentences S₁～S_MConverting the word vector of each word into a corresponding hidden vector;

establishing the sentence synthesis layer for respectively obtaining the dependency syntax tree T₁～T_MAs the sentence S, the hidden vector of the root node of₁～S_MSentence vector xs₁～xs_M；

Obtaining the topic probability distribution p (k | d) of the document to be classified and the probability p (w | d) of each word under each topic by using the topic model TM_mnI k) and according to the topic probability distribution p (k | d) and the probability p (w)_mnL k) calculating the attention weight of each sentence in the document to be classified; establishing the document synthesis layer according to the attention weight of the sentence, and performing weighted summation on the hidden vector of the sentence to obtain the document vector of the document to be classified;

4. The text emotion classification model creation method of claim 1, wherein the operation of normalizing the emotion probability distribution by the output layer is softmax normalization.

5. A text emotion classification method is characterized by comprising the following steps: for the documents to be classified, establishing a text emotion classification model according to the text emotion classification model establishing method of any one of claims 1 to 4 by using the trained node parameters, and predicting the probability of the documents to be classified in each emotion category by using the established text emotion classification model to complete text emotion classification of the documents to be classified.

6. The textual emotion classification method of claim 5, wherein the node parameter acquisition method includes:

(S2) for document D_iEstablishing a text emotion classification model by using the text emotion classification model establishment method of any one of claims 1 to 4, and predicting the document D by using the established text emotion classification model_iProbability in each emotion category;

(S3) according to the document D_iThe emotion classification and prediction result of the text emotion classification model, and node parameters of the text emotion classification model are adjusted to minimize a pre-constructed loss function；

wherein i is a document number.

7. The textual emotion classification method of claim 6, wherein the method for obtaining the node parameters further comprises: and sequentially executing the steps (S2) on the documents in the test set according to the trained node parameters, and testing the text emotion classification model according to the emotion classification and prediction results of the documents, thereby completing the verification of the node parameters.