CN111858933A - Character-based hierarchical text emotion analysis method and system - Google Patents
Character-based hierarchical text emotion analysis method and system Download PDFInfo
- Publication number
- CN111858933A CN111858933A CN202010659957.2A CN202010659957A CN111858933A CN 111858933 A CN111858933 A CN 111858933A CN 202010659957 A CN202010659957 A CN 202010659957A CN 111858933 A CN111858933 A CN 111858933A
- Authority
- CN
- China
- Prior art keywords
- text
- character
- neural network
- sentence
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000008451 emotion Effects 0.000 title claims abstract description 46
- 238000004458 analytical method Methods 0.000 title claims abstract description 28
- 239000013598 vector Substances 0.000 claims abstract description 57
- 238000003062 neural network model Methods 0.000 claims abstract description 40
- 238000013528 artificial neural network Methods 0.000 claims abstract description 30
- 230000000306 recurrent effect Effects 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 21
- 238000007781 pre-processing Methods 0.000 claims abstract description 18
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 14
- 239000000284 extract Substances 0.000 claims abstract description 5
- 239000010410 layer Substances 0.000 claims description 72
- 230000007246 mechanism Effects 0.000 claims description 15
- 230000002457 bidirectional effect Effects 0.000 claims description 13
- 239000002356 single layer Substances 0.000 claims description 9
- 230000014509 gene expression Effects 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 125000004122 cyclic group Chemical group 0.000 claims description 5
- 238000013461 design Methods 0.000 claims description 4
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000002349 favourable effect Effects 0.000 abstract 1
- 230000008569 process Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000002996 emotional effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 241000590419 Polygonia interrogationis Species 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a character-based hierarchical text emotion analysis method and a system, wherein the method comprises the following steps: preprocessing given text data, including designing a character set, dividing sentences in the text, and obtaining a text representation in a digital form based on the character set; establishing a neural network model at a character level: inputting the preprocessed text data into a neural network model at a character level, sequentially passing through a model embedding layer, a convolutional neural network layer and a decoding layer, and extracting and outputting a feature vector of each sentence in the text; establishing a sentence-level neural network model: and taking the output of the character level network as input, and outputting the probability distribution of the emotion classification of the text sequentially through a recurrent neural network layer, an attention layer and a decoding layer. The invention extracts the initial characteristics of the text from the character level, and the sentence level network not only contains the time sequence information, but also enables the network to tend to the sentences which are favorable for the emotion analysis result, thereby improving the accuracy and the robustness of the model.
Description
Technical Field
The invention relates to the technical field of emotion analysis of natural language processing, in particular to a hierarchical text emotion analysis method and system based on characters.
Background
With the great increase of the information amount of the internet in recent years, people can be exposed to a great amount of text information such as news, blogs, comments and the like through terminals such as mobile phones, computers and the like. Extracting important information from a large amount of texts, such as text summaries, emotional tendencies and the like, has become an urgent need for quickly understanding the information in the era of information explosion. The emotional tendency is used as higher-level abstraction of text information and has important application value. The character-based hierarchical text emotion analysis method with attention mechanism provides an efficient solution for extracting emotion tendencies from a large number of texts, can help master the main attitudes of people on hot events, candidates, commodities, movies and other things, and has great application potential for equiangular colors of consumers, managers and competitors.
The conventional text emotion analysis method based on deep learning mostly analyzes texts on the basis of words, and the methods have the following pain points: 1. in various languages around the world, the number of words is huge, for example, the number of common words in english is as high as three to forty thousand, and the words are flexibly changed along with the development of generations. 2. In order to express the relationship between words, such as near-meaning words, roots, etc., a great amount of word vectorization expressions need to be pre-trained, and the training needs massive texts as training samples, so that the consumed computing resources are more immeasurable. 3. There are low frequency words and oov (out of vocabularies) problems, i.e., some uncommon words may only appear in certain topic-specific articles, resulting in pre-trained word vectors that do not include the vectorized representation of the word (oov problem) or that the representation of the word is not sufficiently trained (low frequency word problem).
Disclosure of Invention
Aiming at the problems that the word-based text emotion analysis method is large in number and has flexibility, the problem of relationship among words, the problem of low frequency words and oov, and the problem that a character-based model is easy to over-fit and poor in model robustness, a character-based network is designed, and a sentence-level network is added on the character-level network, the invention provides a character-based neural network, which is different from the existing similar method.
However, character-based models are prone to problems of overfitting and poor model robustness due to the diversity of combinations between characters and the nature of the convolutional network. In view of the problem, the invention starts from a hierarchical idea, a sentence-level network is added on a character-level network, and the vectorization representation of the sentence is extracted from the character sequence of the sentence through the character-level network. The addition obviously relieves the overfitting problem easily appearing in the character-level network, improves the robustness of the model and enables the model to be more stable in performance.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a character-based hierarchical text emotion analysis method, which comprises the following steps:
text preprocessing: preprocessing given text data, including designing a character set, dividing sentences in the text, and obtaining a text representation in a digital form based on the character set;
establishing a neural network model at a character level: inputting the preprocessed text data into a neural network model at a character level, sequentially passing through a model embedding layer, a convolutional neural network layer and a decoding layer, and extracting and outputting a feature vector of each sentence in the text;
establishing a sentence-level neural network model: and taking the output of the character level network as input, and outputting the probability distribution of the emotion classification of the text sequentially through a recurrent neural network layer, an attention layer and a decoding layer.
As a preferred technical solution, the text preprocessing specifically includes:
designing a character set which comprises basic characters under the language of a given text, and packaging the character set into a dictionary, wherein the dictionary is used for searching for corresponding subscripts through characters and searching for corresponding characters through the subscripts;
dividing sentences in the text: dividing a single text into a set of a plurality of sentences with sentence terminators of the language of the given text as separators;
Deriving a textual representation in numeric form based on a character set: and converting each sentence in each text from the character sequence into a corresponding subscript sequence based on the dictionary, and completing the conversion of the text from the character form to the digital form.
As a preferred technical solution, the obtaining of the text representation in a numeric form based on the character set specifically includes:
character segmentation: dividing each sentence of the text into a plurality of characters and storing the characters in a character type array;
case conversion: replacing all characters forming words in the original text into a lower case form;
text digitization: comparing the dictionary, converting all characters in the text into corresponding subscripts in the dictionary, and converting the text from a character form to a digital form;
unifying sentence length: if the sentence length exceeds the set threshold value, cutting, and discarding the sentence part exceeding the length; if the sentence length does not reach the set threshold value, the subscript 0 is used for filling until the sentence length reaches the threshold value;
unifying text length: if the number of sentences in the text exceeds a set threshold value, cutting, and discarding partial sentences exceeding the number; and if the number of sentences in the text does not reach the set threshold value, filling the text with sentences with uniform length with subscripts of 0 until the number of sentences reaches the threshold value.
As a preferred technical solution, the establishing of the neural network model at the character level specifically includes the steps of:
the model embedding layer takes each sentence of the preprocessed text as an input unit, converts the subscript of each character of the sentence into corresponding unique vectorization expression, and converts the expression form of each sentence from a one-dimensional subscript sequence into a two-dimensional character vector sequence;
the convolutional neural network layer adopts a plurality of one-dimensional convolutional kernels with different sizes, simultaneously carries out convolutional operation and global maximum pooling operation on the two-dimensional character vector sequence, and splices operation results to obtain output results of multi-convolutional-kernel operation;
and the decoding layer takes the output result of the multi-convolution kernel operation as input, extracts the feature vector of the sentence through the full-connection layer, and the feature vector of the sentence is used as the input of the neural network model at the sentence level.
As a preferred technical solution, the method for converting the subscript of each character of the sentence into a corresponding unique vectorization representation specifically adopts the following steps: and carrying out one-hot coding on the character subscript in the sentence.
As a preferred technical solution, the performing convolution operation and global maximum pooling operation on the two-dimensional character vector sequence specifically includes: and performing single-layer convolution operation on the two-dimensional character vector sequence and connecting a nonlinear activation function ReLU, wherein the step length of the convolution operation is set to be 1.
As a preferred technical solution, the establishing of the sentence-level neural network model specifically includes the steps of:
the cyclic neural network layer takes the output of the neural network model at the character level as input, and obtains the output and context vector of each time step through the bidirectional cyclic neural network;
the attention layer adopts an attention mechanism, the output of the bidirectional recurrent neural network is used as a value item, the output of the bidirectional recurrent neural network after being connected with a full connection layer is used as a key item, the following vector is used as a query item, the weight distribution of the output of the recurrent neural network at each time step is obtained, and the output and the weight are multiplied and then added to obtain the vector representation of the whole text;
and outputting the numerical distribution of emotion classification through a full connection layer by using the vector representation of the whole text, and converting the result into probability distribution of emotion classification by adopting softmax operation, wherein the probability with a larger probability is an emotion analysis prediction result.
As a preferred technical solution, the method includes the steps of using the output of the bidirectional recurrent neural network as a value item, using the output after the bidirectional recurrent neural network is connected with a full connection layer as a key item, using a following vector as a query item, obtaining a weight distribution of the output of the recurrent neural network at each time step, multiplying the output and the weight, and then adding the multiplied output and the weight to obtain a vector representation of the whole text, and specifically includes:
The output of each time step passes through a single-layer multi-layer perceptron, Tanh is taken as an activation function to obtain a hidden representation of the output, and the hidden representation is taken as a key item in an attention mechanism;
taking the following vector as a query item of an attention mechanism, sequentially carrying out vector multiplication on key items and the query item to obtain attention distribution output for each time step, then carrying out softmax operation on the attention distribution, and converting the attention distribution into probability distribution so that the sum of the proportions of all the time steps in the attention is 1;
and taking the output of each time step as a value item of an attention mechanism, multiplying the attention proportion occupied by each time step by a corresponding value, and adding the obtained results of all the time steps to obtain the sum of all sentence vectors in the text based on the weight, namely the feature vector of the text.
The invention also provides a hierarchical text emotion analysis system based on characters, which comprises the following components: the system comprises a text preprocessing module, a character-level neural network model establishing module and a sentence-level neural network model establishing module;
the text preprocessing module is used for preprocessing given text data, and comprises a character set design module, a text input module, a text output module and a text display module, wherein the character set design module is used for dividing sentences in the text and obtaining text representation in a digital form based on the character set;
The character-level neural network model establishing module is used for inputting the preprocessed text data into the character-level neural network model, sequentially passing through the model embedding layer, the convolutional neural network layer and the decoding layer, and extracting and outputting a feature vector of each sentence in the text;
the sentence-level neural network model building module is used for outputting probability distribution of emotion classification of the text by taking output of the character-level network as input and sequentially passing through the recurrent neural network layer, the attention layer and the decoding layer.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) the method effectively solves the problems of large quantity of words, flexibility, relationship among words, low frequency words and oov of words faced by the word-based text emotion analysis method, and poor overfitting and model robustness of a character-based model, obviously reduces the storage overhead and calculation resource overhead required by text emotion analysis, and improves the accuracy and robustness of the model.
(2) Firstly, extracting the characteristics of a given text based on a designed character set; then, extracting sentence-level feature vectors of a given text by adopting a single-layer Convolutional Neural Network (CNN) containing a plurality of convolutional kernels; extracting feature vectors of the whole text through a bidirectional Recurrent Neural Network (RNN) based on an attention mechanism; finally, connecting the full connection layers, performing softmax operation on the result to obtain probability distribution of text emotion classification, and performing initial feature extraction on the text from the character level, so that a pre-trained word vector is not needed, the problem of low-frequency words is avoided, and the language universality is good; the sentence-level network not only contains time sequence information, but also enables the network to tend to be beneficial sentences for emotion analysis results, and improves the accuracy and the robustness of the model.
Drawings
FIG. 1 is a schematic diagram of an overall framework of a character-based hierarchical text emotion analysis method according to the present embodiment;
FIG. 2 is a block diagram of the data preprocessing of the present embodiment;
FIG. 3 is a block diagram of a character-level neural network model according to the present embodiment;
fig. 4 is a frame diagram of the sentence-level neural network model according to the embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Examples
As shown in fig. 1, the present embodiment provides a character-based hierarchical text emotion analysis method, which includes the following steps:
s1: text preprocessing: as shown in fig. 2, given text data is preprocessed, including designing a character set, dividing sentences in the text, and obtaining a text representation in a digital form based on the character set;
step S1 is an input data preprocessing, specifically including the following substeps:
s11: designing a set of characters
Designing a character set which comprises basic characters under the language of a given text, packaging the character set into a dictionary, and finding out subscripts of the character set through the characters and finding out corresponding characters through the subscripts;
Generally, the basic characters of a language mainly include characters (such as letters in english), arabic numerals (0-9), punctuation marks (, |;
s12: partitioning sentences in text
Dividing a single text into a set of a plurality of sentences by taking sentence end symbols of a language of a given text as separators, wherein the sentence end symbols are mainly periods, exclamation marks, question marks and the like in general;
s13: deriving a textual representation in digital form based on a character set
Converting each sentence in each text from the character sequence to the corresponding subscript sequence by using the dictionary formed in step S11, thereby completing the conversion of the text from the character form to the number form.
Which comprises the following more detailed steps:
character segmentation. Each sentence of the text is equivalent to a character string, and the character string is divided into a plurality of characters and stored in the character type array. Thus, a text becomes a two-dimensional array, with a first dimension storing a plurality of sentences of text and a second dimension storing a plurality of characters under each sentence.
Case and case transformation. For some languages, the characters that make up a word are case-specific, such as English. The presence of capital and lowercase characters can affect the representation of the model because the capital and lowercase characters do not change the meaning of the words of which the capital and lowercase characters constitute, and the occurrence frequency of capital characters in the text is relatively low compared with that of lowercase characters, and if the capital characters are included in the character set, the overfitting phenomenon of the model can be aggravated. For the above reasons, all the characters constituting a word in the original text are replaced in a lower case form.
Text digitization. All characters in the text are converted into their corresponding subscripts in the dictionary against the dictionary formed in step S11, so that the text is converted from a character form into a number form.
Uniform sentence length. In order to process text data in batch and improve the text processing efficiency of the model, the lengths of all sentences in the text need to be unified. If the sentence length exceeds the set threshold value, cutting, and discarding the sentence part exceeding the length; and if the sentence length does not reach the set threshold value, filling the sentence length with the subscript 0 until the sentence length reaches the threshold value.
Uniform text length. In order to process text data in batch and improve the efficiency of model processing, the text length needs to be unified. Since the sentence length is unified in the previous step, the step only needs to unify the number of sentences in the text. If the number of sentences in the text exceeds a set threshold value, cutting, and discarding partial sentences exceeding the number; and if the number of sentences in the text does not reach the set threshold value, filling the text with sentences with uniform length with subscripts of 0 until the number of sentences reaches the threshold value.
S2: establishing a neural network model at a character level: as shown in fig. 3, inputting the preprocessed text data into a neural network model at a character level, sequentially passing through a model embedding layer, a convolutional neural network layer and a decoding layer, and extracting and outputting a feature vector of each sentence in the text;
step S2 is to establish a neural network model at a character level, which specifically includes the following substeps:
s21: model embedding layer
Taking each sentence of the preprocessed text as an input unit, converting the subscript of each character of the sentence into a corresponding unique vectorization expression, and specifically comprising the following steps:
taking each sentence of the preprocessed text as an input unit, performing one-hot coding on the subscript of the character in the sentence, converting the subscript of each character in the sentence into corresponding unique vectorization expression, and converting the expression form of each sentence into a two-dimensional character vector sequence from a one-dimensional subscript sequence;
s22: convolutional neural network layer
Using a plurality of one-dimensional convolution kernels with different sizes to simultaneously carry out convolution operation and global maximum pooling operation on the sentence sequence output by the step S21, splicing operation results together, and specifically comprising the following steps:
performing single-layer convolution operation on the sentence sequence output in the step S21 by using a plurality of one-dimensional convolution kernels with different sizes, and connecting the single-layer convolution operation with the nonlinear activation function ReLU, wherein the step length of the convolution operation is 1, the number of the convolution kernels, the size of the convolution kernels and the number of output channels are model hyper-parameters, the adjustment is needed according to a specific data set and a training process, then obtaining the maximum value of each channel through a global maximum pooling layer according to the operation result of each convolution kernel, and connecting the output along channel dimensions to obtain the characteristic extracted from the sentence based on the multi-element group idea;
S23: decoding layer
And taking the output result of the multi-convolution kernel operation in the step S22 as input, extracting the feature vector of the sentence through a full connection layer, wherein the feature vector of the sentence is taken as the input of a sentence-level model, the dimension of the sentence is a model hyper-parameter, and the sentence is required to be adjusted according to a specific data set and a training process.
S3: establishing a sentence-level neural network model: as shown in fig. 4, the output of the character-level network is used as input, and the probability distribution of emotion classification of the text is output through the recurrent neural network layer, the attention layer, and the decoding layer in this order.
The step S3 of establishing a sentence-level neural network model specifically includes the following substeps:
s31: recurrent neural network layer
The output of the character-level network, namely the vector representation of each sentence of the text, is used as the input, the output and the context vector (namely the final hidden state) of each time step are obtained through a bidirectional cyclic neural network, the hidden layer dimension of the hidden layer dimension is used as a model hyper-parameter, and the hidden layer dimension needs to be adjusted according to a specific data set and a training process;
s32: attention layer
By adopting an attention mechanism, the output of the recurrent neural network in the step S31 is taken as a value item, the output after the output is connected with a full connection layer is taken as a key item, and a following vector is taken as a query item, so that the weight distribution of the output of the recurrent neural network at each time step is obtained, and the output and the weight are multiplied and then added to obtain the vector representation of the whole text, wherein the method comprises the following more detailed steps:
Passing the output of each time step in step S31 through a single-layer multi-layer perceptron and taking Tanh as an activation function to obtain a hidden representation of the output, and keeping the dimension of the output unchanged before and after conversion, wherein the term is a key term in the attention mechanism;
the attention distribution output for each time step is obtained by vector-multiplying the key item obtained in the previous step and the query item in the present step in sequence, using the context vector in step S31 as the query item of the attention mechanism. Then performing softmax operation on the attention distribution, and converting the distribution into probability distribution so that the sum of the proportions of all time steps in the attention is 1;
the output of each time step in step S31 is used as a value term of the attention mechanism, and the proportion of attention occupied by each time step (refer to the probability distribution of attention in the previous step) is multiplied by the corresponding value. Adding the obtained results of all time steps to obtain the sum of all sentence vectors in the text based on the weight, namely the feature vector of the text;
s33: decoding layer
And (5) expressing the numerical distribution of the emotion classification output through the full connection layer by using the vector of the text output in the step (S32), and converting the result into the probability distribution of the emotion classification by using softmax operation, wherein the higher probability is the emotion analysis prediction result.
The embodiment also provides a hierarchical text emotion analysis system based on characters, which comprises: the system comprises a text preprocessing module, a character-level neural network model establishing module and a sentence-level neural network model establishing module;
in this embodiment, the text preprocessing module is configured to preprocess given text data, including designing a character set, dividing sentences in a text, and obtaining a text representation in a digital form based on the character set;
in this embodiment, the character-level neural network model building module is configured to input the preprocessed text data into the character-level neural network model, sequentially pass through the model embedding layer, the convolutional neural network layer, and the decoding layer, and extract and output a feature vector of each sentence in the text;
in this embodiment, the sentence-level neural network model building module is configured to output the probability distribution of emotion classification of a text by using the output of the character-level network as input and sequentially passing through the recurrent neural network layer, the attention layer, and the decoding layer.
In the embodiment, a hierarchical neural network model from a character level to a sentence level and from the sentence level to a text level is established, and the method can be used for carrying out emotion classification on common texts such as comments, blogs and the like. The model is as follows: 1. firstly, extracting the characteristics of a given text based on a designed character set; 2. then, extracting sentence-level feature vectors of a given text by adopting a single-layer Convolutional Neural Network (CNN) containing a plurality of convolutional kernels; 3. extracting feature vectors of the whole text through a bidirectional Recurrent Neural Network (RNN) based on an attention mechanism; 4. and finally, connecting the full connection layers and performing softmax operation on the result to obtain the probability distribution of the text emotion classification.
The embodiment extracts the initial features of the text from the character level, does not need pre-trained word vectors, does not have the problem of low-frequency words, and has good language universality; the sentence-level network not only contains time sequence information, but also enables the network to tend to be beneficial sentences for emotion analysis results, and improves the accuracy and the robustness of the model.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (9)
1. A hierarchical text emotion analysis method based on characters is characterized by comprising the following steps:
text preprocessing: preprocessing given text data, including designing a character set, dividing sentences in the text, and obtaining a text representation in a digital form based on the character set;
establishing a neural network model at a character level: inputting the preprocessed text data into a neural network model at a character level, sequentially passing through a model embedding layer, a convolutional neural network layer and a decoding layer, and extracting and outputting a feature vector of each sentence in the text;
Establishing a sentence-level neural network model: and taking the output of the character level network as input, and outputting the probability distribution of the emotion classification of the text sequentially through a recurrent neural network layer, an attention layer and a decoding layer.
2. The method for hierarchical emotion analysis of texts based on characters as claimed in claim 1, wherein the specific steps of text preprocessing include:
designing a character set which comprises basic characters under the language of a given text, and packaging the character set into a dictionary, wherein the dictionary is used for searching for corresponding subscripts through characters and searching for corresponding characters through the subscripts;
dividing sentences in the text: dividing a single text into a set of a plurality of sentences with sentence terminators of the language of the given text as separators;
deriving a textual representation in numeric form based on a character set: and converting each sentence in each text from the character sequence into a corresponding subscript sequence based on the dictionary, and completing the conversion of the text from the character form to the digital form.
3. The method for hierarchical emotion analysis of texts based on characters as claimed in claim 2, wherein the text representation in digital form is obtained based on character set, and the specific steps include:
Character segmentation: dividing each sentence of the text into a plurality of characters and storing the characters in a character type array;
case conversion: replacing all characters forming words in the original text into a lower case form;
text digitization: comparing the dictionary, converting all characters in the text into corresponding subscripts in the dictionary, and converting the text from a character form to a digital form;
unifying sentence length: if the sentence length exceeds the set threshold value, cutting, and discarding the sentence part exceeding the length; if the sentence length does not reach the set threshold value, the subscript 0 is used for filling until the sentence length reaches the threshold value;
unifying text length: if the number of sentences in the text exceeds a set threshold value, cutting, and discarding partial sentences exceeding the number; and if the number of sentences in the text does not reach the set threshold value, filling the text with sentences with uniform length with subscripts of 0 until the number of sentences reaches the threshold value.
4. The method for hierarchical emotion analysis of texts based on characters according to claim 1, wherein the establishing of the neural network model at the character level comprises the following specific steps:
the model embedding layer takes each sentence of the preprocessed text as an input unit, converts the subscript of each character of the sentence into corresponding unique vectorization expression, and converts the expression form of each sentence from a one-dimensional subscript sequence into a two-dimensional character vector sequence;
The convolutional neural network layer adopts a plurality of one-dimensional convolutional kernels with different sizes, simultaneously carries out convolutional operation and global maximum pooling operation on the two-dimensional character vector sequence, and splices operation results to obtain output results of multi-convolutional-kernel operation;
and the decoding layer takes the output result of the multi-convolution kernel operation as input, extracts the feature vector of the sentence through the full-connection layer, and the feature vector of the sentence is used as the input of the neural network model at the sentence level.
5. The method according to claim 4, wherein the index of each character in the sentence is converted into a corresponding unique vectorized representation by: and carrying out one-hot coding on the character subscript in the sentence.
6. The method for hierarchical emotion analysis of texts based on characters according to claim 4, wherein the convolution operation and the global maximum pooling operation are performed on the two-dimensional character vector sequence, and the specific steps include: and performing single-layer convolution operation on the two-dimensional character vector sequence and connecting a nonlinear activation function ReLU, wherein the step length of the convolution operation is set to be 1.
7. The method for hierarchical emotion analysis of texts based on characters according to claim 1, wherein the step of establishing the neural network model at sentence level comprises the following specific steps:
the cyclic neural network layer takes the output of the neural network model at the character level as input, and obtains the output and context vector of each time step through the bidirectional cyclic neural network;
the attention layer adopts an attention mechanism, the output of the bidirectional recurrent neural network is used as a value item, the output of the bidirectional recurrent neural network after being connected with a full connection layer is used as a key item, the following vector is used as a query item, the weight distribution of the output of the recurrent neural network at each time step is obtained, and the output and the weight are multiplied and then added to obtain the vector representation of the whole text;
and outputting the numerical distribution of emotion classification through a full connection layer by using the vector representation of the whole text, and converting the result into probability distribution of emotion classification by adopting softmax operation, wherein the probability with a larger probability is an emotion analysis prediction result.
8. The method for analyzing hierarchical text emotion based on characters according to claim 7, wherein the method comprises the following specific steps of taking the output of the bidirectional recurrent neural network as a value item, taking the output after the bidirectional recurrent neural network is connected with a full connection layer as a key item, taking a context vector as a query item, obtaining the weight distribution of the output of each time step of the recurrent neural network, multiplying the output and the weight, and then adding the multiplied output and the weight to obtain the vector representation of the whole text:
The output of each time step passes through a single-layer multi-layer perceptron, Tanh is taken as an activation function to obtain a hidden representation of the output, and the hidden representation is taken as a key item in an attention mechanism;
taking the following vector as a query item of an attention mechanism, sequentially carrying out vector multiplication on key items and the query item to obtain attention distribution output for each time step, then carrying out softmax operation on the attention distribution, and converting the attention distribution into probability distribution so that the sum of the proportions of all the time steps in the attention is 1;
and taking the output of each time step as a value item of an attention mechanism, multiplying the attention proportion occupied by each time step by a corresponding value, and adding the obtained results of all the time steps to obtain the sum of all sentence vectors in the text based on the weight, namely the feature vector of the text.
9. A hierarchical character-based text emotion analysis system, comprising: the system comprises a text preprocessing module, a character-level neural network model establishing module and a sentence-level neural network model establishing module;
the text preprocessing module is used for preprocessing given text data, and comprises a character set design module, a text input module, a text output module and a text display module, wherein the character set design module is used for dividing sentences in the text and obtaining text representation in a digital form based on the character set;
The character-level neural network model establishing module is used for inputting the preprocessed text data into the character-level neural network model, sequentially passing through the model embedding layer, the convolutional neural network layer and the decoding layer, and extracting and outputting a feature vector of each sentence in the text;
the sentence-level neural network model building module is used for outputting probability distribution of emotion classification of the text by taking output of the character-level network as input and sequentially passing through the recurrent neural network layer, the attention layer and the decoding layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010659957.2A CN111858933B (en) | 2020-07-10 | 2020-07-10 | Hierarchical text emotion analysis method and system based on characters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010659957.2A CN111858933B (en) | 2020-07-10 | 2020-07-10 | Hierarchical text emotion analysis method and system based on characters |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111858933A true CN111858933A (en) | 2020-10-30 |
| CN111858933B CN111858933B (en) | 2024-08-06 |
Family
ID=73153005
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010659957.2A Active CN111858933B (en) | 2020-07-10 | 2020-07-10 | Hierarchical text emotion analysis method and system based on characters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111858933B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112686898A (en) * | 2021-03-15 | 2021-04-20 | 四川大学 | Automatic radiotherapy target area segmentation method based on self-supervision learning |
| CN112699679A (en) * | 2021-03-25 | 2021-04-23 | 北京沃丰时代数据科技有限公司 | Emotion recognition method and device, electronic equipment and storage medium |
| CN114297379A (en) * | 2021-12-16 | 2022-04-08 | 中电信数智科技有限公司 | Text binary classification method based on Transformer |
| US11966702B1 (en) * | 2020-08-17 | 2024-04-23 | Alphavu, Llc | System and method for sentiment and misinformation analysis of digital conversations |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107038480A (en) * | 2017-05-12 | 2017-08-11 | 东华大学 | A kind of text sentiment classification method based on convolutional neural networks |
| CN108628823A (en) * | 2018-03-14 | 2018-10-09 | 中山大学 | In conjunction with the name entity recognition method of attention mechanism and multitask coordinated training |
| US20180329886A1 (en) * | 2017-05-15 | 2018-11-15 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Artificial intelligence based method and apparatus for generating information |
| CN108846017A (en) * | 2018-05-07 | 2018-11-20 | 国家计算机网络与信息安全管理中心 | The end-to-end classification method of extensive newsletter archive based on Bi-GRU and word vector |
| CN109543722A (en) * | 2018-11-05 | 2019-03-29 | 中山大学 | A kind of emotion trend forecasting method based on sentiment analysis model |
| US20190188257A1 (en) * | 2016-09-05 | 2019-06-20 | National Institute Of Information And Communications Technology | Context analysis apparatus and computer program therefor |
| CN110134771A (en) * | 2019-04-09 | 2019-08-16 | 广东工业大学 | A kind of implementation method based on more attention mechanism converged network question answering systems |
| CN110147452A (en) * | 2019-05-17 | 2019-08-20 | 北京理工大学 | A kind of coarseness sentiment analysis method based on level BERT neural network |
| CN110210037A (en) * | 2019-06-12 | 2019-09-06 | 四川大学 | Category detection method towards evidence-based medicine EBM field |
| WO2019214145A1 (en) * | 2018-05-10 | 2019-11-14 | 平安科技（深圳）有限公司 | Text sentiment analyzing method, apparatus and storage medium |
| CN110825845A (en) * | 2019-10-23 | 2020-02-21 | 中南大学 | Hierarchical text classification method based on character and self-attention mechanism and Chinese text classification method |
| US20200065384A1 (en) * | 2018-08-26 | 2020-02-27 | CloudMinds Technology, Inc. | Method and System for Intent Classification |
| CN110866117A (en) * | 2019-10-25 | 2020-03-06 | 西安交通大学 | Short text classification method based on semantic enhancement and multi-level label embedding |
-
2020
- 2020-07-10 CN CN202010659957.2A patent/CN111858933B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190188257A1 (en) * | 2016-09-05 | 2019-06-20 | National Institute Of Information And Communications Technology | Context analysis apparatus and computer program therefor |
| CN107038480A (en) * | 2017-05-12 | 2017-08-11 | 东华大学 | A kind of text sentiment classification method based on convolutional neural networks |
| US20180329886A1 (en) * | 2017-05-15 | 2018-11-15 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Artificial intelligence based method and apparatus for generating information |
| CN108628823A (en) * | 2018-03-14 | 2018-10-09 | 中山大学 | In conjunction with the name entity recognition method of attention mechanism and multitask coordinated training |
| CN108846017A (en) * | 2018-05-07 | 2018-11-20 | 国家计算机网络与信息安全管理中心 | The end-to-end classification method of extensive newsletter archive based on Bi-GRU and word vector |
| WO2019214145A1 (en) * | 2018-05-10 | 2019-11-14 | 平安科技（深圳）有限公司 | Text sentiment analyzing method, apparatus and storage medium |
| US20200065384A1 (en) * | 2018-08-26 | 2020-02-27 | CloudMinds Technology, Inc. | Method and System for Intent Classification |
| CN109543722A (en) * | 2018-11-05 | 2019-03-29 | 中山大学 | A kind of emotion trend forecasting method based on sentiment analysis model |
| CN110134771A (en) * | 2019-04-09 | 2019-08-16 | 广东工业大学 | A kind of implementation method based on more attention mechanism converged network question answering systems |
| CN110147452A (en) * | 2019-05-17 | 2019-08-20 | 北京理工大学 | A kind of coarseness sentiment analysis method based on level BERT neural network |
| CN110210037A (en) * | 2019-06-12 | 2019-09-06 | 四川大学 | Category detection method towards evidence-based medicine EBM field |
| CN110825845A (en) * | 2019-10-23 | 2020-02-21 | 中南大学 | Hierarchical text classification method based on character and self-attention mechanism and Chinese text classification method |
| CN110866117A (en) * | 2019-10-25 | 2020-03-06 | 西安交通大学 | Short text classification method based on semantic enhancement and multi-level label embedding |
Non-Patent Citations (4)
| Title |
|---|
| C´ ICERO NOGUEIRA DOS SANTOS ET AL: "Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts", PROCEEDINGS OF COLING 2014, THE 25TH INTERNATIONAL CONFERENCE ON COMPUTATIONAL LINGUISTICS: TECHNICAL PAPERS, 29 August 2014 (2014-08-29), pages 69 - 78, XP055676919 * |
| FEIRAN HUANG ET AL: "Character-level Convolutional Networks for Text Classification", ACM TRANS. MULTIMEDIA COMPUT. COMMUN. APPL., vol. 16, no. 03, 5 July 2020 (2020-07-05), pages 1 - 19 * |
| 宋岩 等: "基于层次特征提取的文本分类研究", 计算机应用与软件, vol. 37, no. 02, 29 February 2020 (2020-02-29), pages 68 - 72 * |
| 王丽亚 等: "CNN-BiGRU网络中引入注意力机制的中文文本情感分析", 计算机应用, vol. 39, no. 10, 10 October 2019 (2019-10-10), pages 2841 - 2846 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11966702B1 (en) * | 2020-08-17 | 2024-04-23 | Alphavu, Llc | System and method for sentiment and misinformation analysis of digital conversations |
| CN112686898A (en) * | 2021-03-15 | 2021-04-20 | 四川大学 | Automatic radiotherapy target area segmentation method based on self-supervision learning |
| CN112686898B (en) * | 2021-03-15 | 2021-08-13 | 四川大学 | Automatic radiotherapy target area segmentation method based on self-supervision learning |
| CN112699679A (en) * | 2021-03-25 | 2021-04-23 | 北京沃丰时代数据科技有限公司 | Emotion recognition method and device, electronic equipment and storage medium |
| CN112699679B (en) * | 2021-03-25 | 2021-06-29 | 北京沃丰时代数据科技有限公司 | Emotion recognition method and device, electronic equipment and storage medium |
| CN114297379A (en) * | 2021-12-16 | 2022-04-08 | 中电信数智科技有限公司 | Text binary classification method based on Transformer |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111858933B (en) | 2024-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111291195B (en) | Data processing method, device, terminal and readable storage medium | |
| KR102155768B1 (en) | Method for providing question and answer data set recommendation service using adpative learning from evoloving data stream for shopping mall | |
| CN111858933B (en) | Hierarchical text emotion analysis method and system based on characters | |
| CN110825845A (en) | Hierarchical text classification method based on character and self-attention mechanism and Chinese text classification method | |
| CN104834747A (en) | Short text classification method based on convolution neutral network | |
| CN103473280A (en) | Method and device for mining comparable network language materials | |
| CN112434535A (en) | Multi-model-based factor extraction method, device, equipment and storage medium | |
| CN111259153B (en) | Attribute-level emotion analysis method of complete attention mechanism | |
| CN112860896A (en) | Corpus generalization method and man-machine conversation emotion analysis method for industrial field | |
| US20230073602A1 (en) | System of and method for automatically detecting sarcasm of a batch of text | |
| Banik et al. | Gru based named entity recognition system for bangla online newspapers | |
| CN113704416A (en) | Word sense disambiguation method and device, electronic equipment and computer-readable storage medium | |
| CN113377953B (en) | Entity fusion and classification method based on PALC-DCA model | |
| CN116258137A (en) | Text error correction method, device, equipment and storage medium | |
| CN113159831A (en) | Comment text sentiment analysis method based on improved capsule network | |
| CN115759119B (en) | Financial text emotion analysis method, system, medium and equipment | |
| Niam et al. | Hate speech detection using latent semantic analysis (lsa) method based on image | |
| Shahade et al. | Multi-lingual opinion mining for social media discourses: An approach using deep learning based hybrid fine-tuned smith algorithm with adam optimizer | |
| CN115269834A (en) | High-precision text classification method and device based on BERT | |
| CN111241273A (en) | Text data classification method and device, electronic equipment and computer readable medium | |
| Kang et al. | Sentiment analysis on Malaysian airlines with BERT | |
| CN115757680A (en) | Keyword extraction method and device, electronic equipment and storage medium | |
| CN114911940A (en) | Text emotion recognition method and device, electronic equipment and storage medium | |
| CN114691836A (en) | Method, device, equipment and medium for analyzing emotion tendentiousness of text | |
| CN115292495A (en) | Emotion analysis method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |