CN114912423A - Method and device for analyzing aspect level emotion based on transfer learning - Google Patents
Method and device for analyzing aspect level emotion based on transfer learning Download PDFInfo
- Publication number
- CN114912423A CN114912423A CN202210302985.8A CN202210302985A CN114912423A CN 114912423 A CN114912423 A CN 114912423A CN 202210302985 A CN202210302985 A CN 202210302985A CN 114912423 A CN114912423 A CN 114912423A
- Authority
- CN
- China
- Prior art keywords
- model
- emotion analysis
- task
- roberta
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/169—Annotation, e.g. comment data or footnotes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses an aspect level emotion analysis method and device based on transfer learning, belonging to the technical field of natural language processing, wherein the method comprises the steps of obtaining an aspect level emotion analysis data set and preprocessing the aspect level emotion analysis data set to obtain a sample sequence containing an aspect level evaluation entity and context as non-tag data; constructing a RoBERTA model, and training the RoBERTA model by using the label-free data as a mask prediction task to obtain the RoBERTA model for completing the training task at the stage; acquiring a document level emotion data set; performing a document level emotion analysis task on a RoBERTA model which completes a mask prediction pre-training task to train the model; constructing sentence pairs fusing comment text information and aspect information as a second sample sequence; and inputting the second sample sequence into a RoBERTA model which finishes a document level emotion analysis task, and outputting an emotion category label, so that the high-accuracy expression with higher accuracy can be obtained.
Description
Technical Field
The invention belongs to the technical field of natural languages, and particularly relates to a method and a device for analyzing aspect-level sentiment based on transfer learning.
Background
Natural Language Processing (NLP) is an artificial intelligence technique that has been developed to allow computing mechanisms to deal with human Language in the real world, and belongs to the interdisciplinary discipline of linguistics and computer disciplines. Emotion analysis is a wide application field in NLP, and is a very close and challenging task. The aspect level sentiment analysis may identify the sentiment polarity corresponding to a particular attribute in the sample data, such as the comment text "this restaurant environment is nice but serves too badly. "more specific emotion polarity discrimination is proposed for the terms of environment and service respectively. In recent years, researchers are increasingly using fine-grained textual sentiment classification methods to make decisions based on the comment data of social networks and e-commerce platforms. Aspect level emotion text classification based on target entities and context comment texts is a fine-grained emotion analysis task, and a large amount of related field knowledge and expert level annotation work is needed when data set annotation is carried out. Due to the lack of high-quality label data in the fine-grained text classification task, the generalized error of the trained deep learning model is large, and the improvement of the accuracy of the emotion polarity judgment of the model in a test set is limited.
The emotion analysis methods based on traditional machine learning, such as a K-nearest neighbor algorithm, naive Bayes, vector support and the like, can realize text classification, but the work of the traditional machine learning method is concentrated on feature engineering, and the performance of the model is difficult to be improved by introducing external domain knowledge. The deep learning model such as the recurrent neural network RNN and the convolutional neural network extracts semantic knowledge in the text sequence, can capture and evaluate deep semantic information of entities and contexts, and effectively improves the accuracy of text sentiment classification. When a deep learning neural network model is constructed, more benefits can not be obtained by deepening the structure due to the memory consumption and gradient attenuation problem model.
The Transformer model issued by Google in 2018 only uses a self-attention mechanism, gets rid of the sequence dependency of a basic neural network when processing NLP tasks, and provides the most basic technical support for a bidirectional encoder model BERT. The Facebook AI follows the neural network structure of the BERT model, removes the pre-training subtask predicted in the next sentence by adjusting the super-parameters and the training set size, and provides a more stable bidirectional encoder model RoBERTA model. The application of the transfer learning in the NLP research is mainly embodied as domain adaptation, and particularly in a text classification task with less training data and high labeling cost, namely aspect level emotion analysis. And (3) based on the transfer learning idea, an auxiliary module data set except for the aspect level emotion analysis is regarded as a source domain, and when a text emotion classification task is trained, external knowledge obtained from related tasks can be used for helping a model to realize a target task.
Disclosure of Invention
In order to solve the defects, the invention provides a method and a device for analyzing aspect level emotion based on transfer learning.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a method for analyzing aspect level emotion based on transfer learning comprises the following steps:
obtaining an emotion analysis data set of an aspect level, preprocessing the emotion analysis data set, removing emotion polarity labels, and obtaining a first sample sequence containing an aspect level evaluation entity and context as label-free data;
constructing a RoBERTA model, and training the RoBERTA model by using the label-free data as a mask prediction task to obtain the RoBERTA model for completing the training task at the stage;
acquiring a document level emotion data set under the same scene with the aspect level emotion analysis;
performing a document level emotion analysis task on a RoBERTA model which completes an automatic supervision mask prediction training task to train the model;
obtaining the aspect level emotion analysis data set again, and constructing a sentence pair fusing comment text information and aspect information as a second sample sequence;
and inputting the second sample sequence into a RoBERTA model which finishes a document level emotion analysis task, performing aspect level emotion analysis, and outputting an emotion category label.
The method is further improved in that: converting each comment text in the first sample sequence into a tabular format.
The method is further improved in that: the mask prediction task training RoBERTA model comprises the following steps:
obtaining comment text sequence { w) without label data 1 ,w 2 ,.....w n-1 ,w n Making a random mask according to the probability of fifteen percent to obtain a mask flag bit; and constructing a RoBERTA model, and training the RoBERTA model by using a pre-training task of mask prediction to obtain the trained RoBERTA model.
The method is further improved in that: the acquiring of the document level emotion data set in the same scene as the aspect level emotion analysis comprises: obtaining a document level emotion analysis data set, dividing emotion polarity categories according to a scoring interval corresponding to the comment, screening out samples of empty labels and comment text messy codes, and obtaining the document level emotion analysis data set in the same scene after data processing.
The method is further improved in that: the step of training the RoBERTA model for completing the self-supervision mask prediction training task by the document level emotion analysis task comprises the following steps: and capturing structural knowledge and semantic information of different levels of comment texts in the document level emotion analysis data through a language model of a Transformer encoder, and setting a layered learning rate in the process of training a RoBERTA model by using a document level emotion analysis task for optimizing model parameters.
The method is further improved in that: inputting the second sample sequence into a RoBERTA model which completes a document level emotion analysis task, performing aspect level emotion analysis, and outputting an emotion category label, wherein the outputting of the emotion category label comprises:
the second sample sequence input process includes fusing semantic information and aspect information of the comment text in the form of sentence pairs as a sample sequence input to the RoBERTa model, and the formula is as follows:
input＝{<s>w 1 ,w 2 ,...,w n-1 ,w n ,</s>t 1 ,t 2 ,...t m }
in the formula (I), the compound is shown in the specification,<s>to classify the flag bits, { w 1 ,w 2 ,.....w n-1 ,w n Is a sequence of the text of the comment,</s>for a delimiter, { t 1 ,t 2 ,.....,t m The terms are sequences of terms;
and (3) accessing a Softmax classifier on the classification flag bit, judging the emotion polarity corresponding to the aspect level evaluation object, calculating the difference between the learning model distribution and the training distribution by using a cross entropy function to obtain a final aspect level emotion analysis model, inputting the sample sequence into the model to predict the emotion polarity, and outputting an emotion category label.
An aspect-level emotion analysis apparatus based on transfer learning, comprising:
the data acquisition module is used for acquiring the emotion analysis data set of the aspect level, preprocessing the emotion analysis data set and removing emotion polarity labels to obtain a first sample sequence containing the evaluation entity and context of the aspect level as non-label data;
the RoBERTA module is used for constructing a RoBERTA model, training the RoBERTA model by using the label-free data as a mask prediction task, and obtaining the RoBERTA model for completing the training task at the stage;
the training module is used for acquiring a document level emotion data set under the same scene with the aspect level emotion analysis, and performing document level emotion analysis task training on a RoBERTA model which completes an automatic supervision mask prediction training task;
and the output module is used for acquiring the aspect level emotion analysis data set again, constructing a sentence pair fusing comment text information and aspect information as a second sample sequence, inputting the second sample sequence into the RoBERTA model which finishes the document level emotion analysis task, performing aspect level emotion analysis and outputting emotion category labels.
Due to the adoption of the technical scheme, the invention has the technical progress that: according to the method, a multi-step transfer learning-based emotion analysis strategy is designed by combining a model RoBERTA and researching a related switch source data set, transferred knowledge is coded into model parameters, dependence on target data is reduced, low resource performance caused by small labeled data amount due to strong target task field is effectively improved, and generalization performance of the model is improved. The method comprises the steps of training a RoBERTA model by using a self-supervision learning mask language prediction task, improving the context understanding ability of the model to emotion texts, sequentially transferring the RoBERTA model, performing a document level emotion analysis task, further training the RoBERTA model, enabling the model to learn emotion classification knowledge in related fields, and finally obtaining the RoBERTA model which can obtain precision expression with higher accuracy on the aspect level emotion analysis task.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a diagram of a model for transfer learning according to the present invention.
Detailed Description
The invention is described in further detail below with reference to the accompanying drawings:
as shown in fig. 1 and fig. 2, the present invention provides a method and an apparatus for analyzing an aspect level emotion based on transfer learning, where the apparatus includes a data acquisition module, a RoBERTa module, a training module, and an output module.
The method of the device comprises the following steps: obtaining an emotion analysis data set of an aspect level, carrying out pretreatment, removing emotion polarity labels, and obtaining a first sample sequence containing an aspect level evaluation entity and context as label-free data; constructing a RoBERTA model, and using unlabeled data as the RoBERTA model for predicting the pre-training task to obtain the RoBERTA model for completing the training task at the stage; acquiring a document level emotion data set under the same scene with the aspect level emotion analysis; performing a document level emotion analysis task on a RoBERTA model which completes an automatic supervision mask prediction training task to train the model; obtaining the aspect level emotion analysis data set again, and constructing a sentence pair fusing comment text information and aspect information as a second sample sequence; and inputting the second sample sequence into a RoBERTA model which finishes a document level emotion analysis task, performing aspect level emotion analysis, and outputting an emotion category label.
The following combined examples are as follows:
TABLE 1
The evaluation object of the aspect level emotion analysis task is an independent target entity, so that a plurality of evaluation objects such as 'Good food but the service is dreadful' aspect words 'food' and 'service' respectively correspond to positive and negative emotion polarities in the appearance of one comment sentence, and the whole comment sentence is taken as the linguistic data for mask language learning by ignoring the corresponding relation between the sample and the label.
Step 2.1: loading a roberta-based model, strictly according to the division of a training set and a test set in SemEval2014 data, only using a training set corpus to perform mask mechanism pre-training in mask language learning, and performing fifteen percent random mask on comment text sequences of input models, for example: "Good Mask but the service was dreadful";
step 2.2: in the process that the model embedding layer converts sentences into Mask language models containing word vector training, the RoBERTA model converts Mask]The flag bit is regarded as noise, and a classification flag bit is inserted at the beginning of the sentence<s>End of sentence use</s>As separator, word vector E is obtained by BPE character encoding token And segment coding E seg Relative position coding E pos And performing feature fusion. The expression is as follows:
E word ＝{x 1 ,x 2 ,...x n }
the embedded layer vector input model, the RoBERTa model, has 12 layers in total, and the coding layer parameters of the l-th layer are represented by the calculation of the self-attention mechanism as follows:
X l ＝Transfomer(X l-1 ),l∈[1,12]
judging whether the words of the [ Mask ] flag bits in the sentences are accurately restored or not according to all hidden states h epsilon [ batch _ size × max _ size × hidden _ size ] in the last layer of the Model, wherein a loss function of a self-supervision learning Mask Language Model (MLM) is as follows:
wherein x is a token sequence { [ s ] of the input pre-training model],x 1 ,...,x n [/s]}, m (x) denotes the set of tokens, x, \ that was dropped by the mask m(x) Is the token that is not replaced in the input sequence, and λ is the L2 regularization parameter for stably adjusting the model parameter θ value, avoiding overfitting.
And step 3: the aspect level emotion analysis data set comprises two scenes, namely a Laptop scene and a Resturant scene, as target domains, and the data set in the related domain is selected to select a Restaurant Review Restaurant _ Review and a smartphone evaluation Cell Phone _ Review as the related domains of the external classification knowledge source for transfer learning. The retaurant _ Review dataset contains, for example; "We are so glad We found this place." the emotion polarity tag is "positive"; comment information on the smart Phone in the Cell Phone _ Review data is similar to main content contained in the Laptop, such as screen display definition, battery endurance and the like, for example: "Sold as new and we're not new but we're directionality and di't work correct", labeled satisfaction score "2"; the label source of the data set Cell Phone _ view is used for a reviewer to score 1-10 aiming at the user experience of the smart Phone, a score interval is used as the division of three categories of emotion analysis, and the observation shows that [0,4) has negative emotion tendencies, [4,7) is divided into neutral evaluation, and [7,10] corresponds to positive emotion tendencies, and the data distribution of relevant domains is such as DSC after Document level emotion analysis (DSC) corpus preprocessing
Table 2:
TABLE 2
And 4, step 4: and (3) using DSC corpora to perform document level emotion analysis pre-training of the migration completion step 3, wherein the database corresponding to the Restaurant scene contains Restaurant _ Review which is used for classifying two categories of emotion polarities (positive and negative), and the document level emotion analysis task corresponding to the Laptop notebook is used for classifying three categories (positive, neutral and negative).
And (3) using labeled data in the related field for further pre-training, which is equivalent to directly fine-tuning the RoBERTA model on a document level emotion analysis task. Based on the language model of the Transformer encoder, different layers of coding can capture different levels of structural knowledge and semantic information. The use of a hierarchical learning rate in the fine tuning process of a document-level sentiment analysis task is inspired by literature, and the formula is as follows:
η l-1 ＝η l /2.6
dividing 4 adjacent layers in 12-layer coding layers of the RoBERTA model into a group of coding layers, and decomposing parameters into { theta [ [ theta ] ] 1 ,θ 2 ,θ 3 }。η l Is the learning rate of the l sets of coding layers,
is the model parameter of the coding layer set L at the time t of the gradient update, L (theta) is the loss function of the model, and the attenuation factor is 2.6. The learning rate of the model decays from the top to the bottom layers with the goal of fully learning to more general semantic features at the lower layers, rather than the top layer knowledge being specific to the pre-training task.
Classification flag bit [ s ] of last layer hidden state of model]Outputting a sentence-level feature matrix h s ∈[batch_size×hidden_size]Mapping conversion by full connection layer, where W i 、W f As a weight matrix, b i 、b f To bias a parameterAnd predicting emotion polarity labels under the restaurant and smartphone scenes by using a sigmoid function and a softmax function:
according to the network structure, the emotion classification fitting task selects a cross entropy loss function to be used for training the whole model, the smaller the difference value between the output value and the target value in the back propagation process is, the better the model training effect is, and the expression is as follows:
where M is the total number of categories, y i Is the true emotion category label to which the sample corresponds,
is the emotional tag, λ, predicted by the model d Is L2 regularization coefficient, theta is the model parameter set, eta l Is the learning rate of the current layer update parameter.
And 5: fusing the aspect word information and comment text in the SemEval2014 data set to construct an input sequence in the form of sentence pairs, such as: the emotion label of the form of the 'Good food but the service water dreadful [/s ] food' after the 'Good food but the service water dreadful [/s ] food' is preprocessed is positive, and the's ] Good food but the service water dreadful [/s ] service' is negative.
Step 6: and (4) migrating the RoBERTA model of the document level emotion analysis task in the step 4, reserving the structural system and the initial weight of the model, and connecting a Softmax classifier as a specific task layer on a classification flag bit < s >. And (5) inputting the preprocessed corpus into the RoBERTA model in the step 5, and updating all layer parameters of the pre-training language model and the classifier by adopting a gradient descent method combined with the layered learning rate.
The linear layer maps hidden layer dimensions of the RoBERTa model class mark output vector to the number of classes of emotion polarity. Calculation of k-th item in probability feature vector by softmax function to predict probability C in 3 types of emotion labels (positive, neutral and negative) k Selecting the class label with the highest probability value as the output of the model classifier:
when the target domain data is finally used for emotion level analysis tasks, a cross entropy function is selected to calculate the difference between real distribution and model output distribution, model parameters are updated by utilizing the difference in a back propagation mode, and the expression is as follows:
where K is the number of emotion classification categories, y i Is the true emotion category label to which the sample corresponds,
is the emotional tag predicted by the model, lambda a Is L2 regularization coefficient, theta is the model parameter set, eta l Is the learning rate of the current layer update parameters.
To prove the effectiveness of the present invention, the following provides a specific experimental case with SemEval2014 as a target task data set:
1. evaluation index
In the technical scheme implementation, the accuracy and the macro F1 are used for evaluating the classification effect of the aspect level emotion analysis task.
2. Hyper-parameter
In the implementation, the same hyper-parameter settings are selected for the restaurant in the data set and the notebook, and AdamW is selected as the optimizer for the auto-convergence tuning parameters during the training process. In order to make the model learn more accurate Classification performance during training, multiple experiments are performed on the hyper-parameter setting of the model, and the most effective hyper-parameter combinations in the mask prediction pre-training MLM and DSC document level emotion analysis pre-training stage and Aspect level emotion Analysis (ASC) are selected as the following table 3:
TABLE 3
3. Comparison test model
(1) TD-LSTM: the target words are extracted twice by the LSTM mechanism from left to right and from right to left, the last hidden state of the LSTM is input into a softmax classifier, and semantic information of an evaluation object is fully utilized on a fine-grained sentiment analysis task.
(2) IAN: the method is characterized in that the characteristics of target words at aspect levels and comment contexts are independently modeled, the hidden state of the mark information and the attention mechanism are output through pooling processing of the hidden layer of the recurrent neural network, the text weight information is calculated, a new layer of attention is added to the original attention to describe the importance of each attention, and contribution of each aspect word to an emotion analysis task is definitely considered through information interaction.
(3) And (4) using a sentence pair of [/s ] separating a comment sentence and an evaluation target word as a sample sequence, inputting the sample sequence into a 12-layer transform-based bidirectional encoder, wherein the hidden layer dimension of the sample sequence is 768 dimensions, and performing softmax classification on hidden vectors output by classification flag bits.
(4) BERT-LSTM utilizes an attention mechanism to interactively learn context and target word knowledge.
(5) RAM: a multi-head attention mechanism and a bidirectional LSTM mechanism are combined, the long-distance dependence relationship between aspect information and context in a complex sentence structure is captured, weighting memory provides different characteristic projects for target words with different emotion polarities in a sentence, and the robustness of a model is enhanced.
(6) AOA: the word embedding matrix and the bidirectional LSTM acquire comment information and hidden vectors of a target BERT middle layer, the capability of a model for capturing aspect-level-specific emotion analysis semantic knowledge is enhanced, and an LSTM mechanism is set to integrate classification mark output vectors of each layer of the self-attention neural network.
Design ablation experiments analysis the role of the training model in each stage in the transfer learning of the invention is as follows: the RoBERTA-bases directly calls a RoBERTA model obtained based on extensive corpus training in the pan-domain to carry out ASC task; the RoBERTA-MLM is a model obtained by extracting label-free data of a training set and performing self-supervision mask language learning pre-training; the RoBERTA-DSC is a model obtained by introducing emotion analysis data sets in related fields and performing supervised learning pre-training of emotion label classification; the RoBERTA-MLM-DSC is a model for carrying out two-stage training tasks on the sequential migration model designed by the invention; the accuracy of each protocol is shown in table 4:
TABLE 4
As can be seen from Table 4, the deep learning model built by the recurrent neural network and the attention mechanism can effectively realize the classification of the emotion of the text at the aspect level, and the basic BERT model can achieve the classification precision of the neural network model combined by various modules. The performance of the bidirectional encoder model with higher robustness in the ASC task is better than the merging and utilization of the hidden vectors in the middle layer of the specific task of the BERT model, and the performance of the model in the downstream task is ensured by the strong feature extraction capability of the model. The model after the migration self-supervision mask learning training can enhance the semantic understanding capability of the RoBERTA model in a specific context when processing a target task, and compared with the method of directly calling the RoBERTA model, the accuracy is respectively improved by 0.07% and 2.34% in a Laptop scene and a Restaurant scene. The model after the pre-training of the document level emotion analysis is migrated, semantic knowledge in related fields and the classification capability of the RoBERTA model are learned, and the accuracy is respectively improved by 1.14% and 2.74% under the Laptop and Restaurant scenes. The model performance of the multi-step migration learning strategy is superior to that of other comparison tests and a RoBERTA model obtained by singly carrying out MLM and DSC pre-training tasks, and the effectiveness of the method is proved.
The above-mentioned embodiments are merely illustrative of the preferred embodiments of the present invention, and do not limit the scope of the present invention, and various modifications and improvements made to the technical solution of the present invention by those skilled in the art without departing from the spirit of the present invention shall fall within the protection scope defined by the claims of the present invention.
Claims (7)
1. An aspect level emotion analysis method based on transfer learning is characterized by comprising the following steps:
obtaining an emotion analysis data set of an aspect level, preprocessing the emotion analysis data set, removing emotion polarity labels, and obtaining a first sample sequence containing an evaluation entity and context of the aspect level as label-free data;
constructing a RoBERTA model, and training the RoBERTA model by using the label-free data as a mask prediction task to obtain the RoBERTA model for completing the training task at the stage;
acquiring a document level emotion data set under the same scene with the aspect level emotion analysis;
performing a document level emotion analysis task on a RoBERTA model which completes an automatic supervision mask prediction training task to train the model;
obtaining the aspect level emotion analysis data set again, and constructing a sentence pair fusing comment text information and aspect information as a second sample sequence;
and inputting the second sample sequence into a RoBERTA model which finishes a document level emotion analysis task, performing aspect level emotion analysis, and outputting an emotion category label.
2. The method for analyzing aspect level sentiment based on transfer learning of claim 1, wherein each comment text in the first sample sequence is converted into a table format.
3. The aspect level emotion analysis method based on transfer learning of claim 1, wherein the mask prediction task training RoBERTa model comprises:
obtaining comment text sequence { w) without label data 1 ,w 2 ,.....w n-1 ,w n Making a random mask according to the probability of fifteen percent to obtain a mask flag bit; and constructing a RoBERTA model, and training the RoBERTA model by using a pre-training task of mask prediction to obtain the trained RoBERTA model.
4. The method of claim 1, wherein the obtaining of the document level emotion data set in the same scene as the aspect level emotion analysis comprises: obtaining a document level sentiment analysis data set, dividing sentiment polarity categories according to a scoring interval corresponding to the comment, screening out samples of empty labels and comment text messy codes, and obtaining the document level sentiment analysis data set in the same scene after data processing.
5. The aspect level emotion analysis method based on transfer learning of claim 1, wherein the performing of the document level emotion analysis task on the RoBERTa model for completing the self-supervision mask prediction training task comprises: and capturing structural knowledge and semantic information of different levels of comment texts in the document level emotion analysis data through a language model of a Transformer encoder, and setting a layered learning rate in the process of training a RoBERTA model by using a document level emotion analysis task for optimizing model parameters.
6. The aspect level emotion analysis method based on transfer learning of claim 1, wherein the second sample sequence is input into the RoBERTa model of completed document level emotion analysis task for aspect level emotion analysis, and outputting emotion category labels comprises:
the second sample sequence input process includes fusing semantic information and aspect information of comment text in the form of sentence pairs as a sample sequence input to the RoBERTa model, as expressed as follows:
input＝{<s>w 1 ,w 2 ,...,w n-1 ,w n ,</s>t 1 ,t 2 ,...t m }
in the formula (I), the compound is shown in the specification,<s>to classify the flag bits, { w 1 ,w 2 ,.....w n-1 ,w n Is a sequence of the text of the comment,</s>for a delimiter, { t 1 ,t 2 ,.....,t m The terms are sequences of terms;
and (3) accessing a Softmax classifier on the classification flag bit, judging the emotion polarity corresponding to the aspect level evaluation object, calculating the difference between the learning model distribution and the training distribution by using a cross entropy function to obtain a final aspect level emotion analysis model, inputting the sample sequence into the model to predict the emotion polarity, and outputting an emotion category label.
7. An aspect-level emotion analysis device based on transfer learning, comprising:
the data acquisition module is used for acquiring an emotion analysis data set of an aspect level, preprocessing the emotion analysis data set, removing emotion polarity labels, and obtaining a first sample sequence containing an aspect level evaluation entity and context as label-free data;
the RoBERTA module is used for constructing a RoBERTA model, using the label-free data as a mask prediction task to train the RoBERTA model and obtaining the RoBERTA model for finishing the training task at the stage;
the training module is used for acquiring a document level emotion data set under the same scene with the aspect level emotion analysis, and performing document level emotion analysis task training on a RoBERTA model which completes an automatic supervision mask prediction training task;
and the output module is used for acquiring the aspect level emotion analysis data set again, constructing a sentence pair fusing comment text information and aspect information as a second sample sequence, inputting the second sample sequence into the RoBERTA model which finishes the document level emotion analysis task, performing aspect level emotion analysis and outputting an emotion category label.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210302985.8A CN114912423A (en) | 2022-03-24 | 2022-03-24 | Method and device for analyzing aspect level emotion based on transfer learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210302985.8A CN114912423A (en) | 2022-03-24 | 2022-03-24 | Method and device for analyzing aspect level emotion based on transfer learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114912423A true CN114912423A (en) | 2022-08-16 |
Family
ID=82763136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210302985.8A Pending CN114912423A (en) | 2022-03-24 | 2022-03-24 | Method and device for analyzing aspect level emotion based on transfer learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114912423A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115563290A (en) * | 2022-12-06 | 2023-01-03 | 广东数业智能科技有限公司 | Intelligent emotion recognition method based on context modeling |
| CN116204642A (en) * | 2023-03-06 | 2023-06-02 | 上海阅文信息技术有限公司 | Intelligent character implicit attribute recognition analysis method, system and application in digital reading |
| CN116527411A (en) * | 2023-07-05 | 2023-08-01 | 安羚科技(杭州)有限公司 | Data security intelligent protection model construction method and device and collaboration platform |
| CN117236338A (en) * | 2023-08-29 | 2023-12-15 | 北京工商大学 | Named entity recognition model of dense entity text and training method thereof |
| CN117574981A (en) * | 2024-01-16 | 2024-02-20 | 城云科技（中国）有限公司 | Training method of information analysis model and information analysis method |
| CN117236338B (en) * | 2023-08-29 | 2024-05-28 | 北京工商大学 | Named entity recognition model of dense entity text and training method thereof |
-
2022
- 2022-03-24 CN CN202210302985.8A patent/CN114912423A/en active Pending
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115563290A (en) * | 2022-12-06 | 2023-01-03 | 广东数业智能科技有限公司 | Intelligent emotion recognition method based on context modeling |
| CN115563290B (en) * | 2022-12-06 | 2023-04-07 | 广东数业智能科技有限公司 | Intelligent emotion recognition method based on context modeling |
| CN116204642A (en) * | 2023-03-06 | 2023-06-02 | 上海阅文信息技术有限公司 | Intelligent character implicit attribute recognition analysis method, system and application in digital reading |
| CN116204642B (en) * | 2023-03-06 | 2023-10-27 | 上海阅文信息技术有限公司 | Intelligent character implicit attribute recognition analysis method, system and application in digital reading |
| CN116527411A (en) * | 2023-07-05 | 2023-08-01 | 安羚科技(杭州)有限公司 | Data security intelligent protection model construction method and device and collaboration platform |
| CN116527411B (en) * | 2023-07-05 | 2023-09-22 | 安羚科技(杭州)有限公司 | Data security intelligent protection model construction method and device and collaboration platform |
| CN117236338A (en) * | 2023-08-29 | 2023-12-15 | 北京工商大学 | Named entity recognition model of dense entity text and training method thereof |
| CN117236338B (en) * | 2023-08-29 | 2024-05-28 | 北京工商大学 | Named entity recognition model of dense entity text and training method thereof |
| CN117574981A (en) * | 2024-01-16 | 2024-02-20 | 城云科技（中国）有限公司 | Training method of information analysis model and information analysis method |
| CN117574981B (en) * | 2024-01-16 | 2024-04-26 | 城云科技（中国）有限公司 | Training method of information analysis model and information analysis method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902145B (en) | Attention mechanism-based entity relationship joint extraction method and system | |
| CN111291185B (en) | Information extraction method, device, electronic equipment and storage medium | |
| CN114912423A (en) | Method and device for analyzing aspect level emotion based on transfer learning | |
| CN109753566A (en) | The model training method of cross-cutting sentiment analysis based on convolutional neural networks | |
| CN110866542B (en) | Depth representation learning method based on feature controllable fusion | |
| CN108038492A (en) | A kind of perceptual term vector and sensibility classification method based on deep learning | |
| CN111985239A (en) | Entity identification method and device, electronic equipment and storage medium | |
| CN112256866B (en) | Text fine-grained emotion analysis algorithm based on deep learning | |
| CN112883714B (en) | ABSC task syntactic constraint method based on dependency graph convolution and transfer learning | |
| Du et al. | Wasserstein based transfer network for cross-domain sentiment classification | |
| CN112687374B (en) | Psychological crisis early warning method based on text and image information joint calculation | |
| CN112784013B (en) | Multi-granularity text recommendation method based on context semantics | |
| CN113254675B (en) | Knowledge graph construction method based on self-adaptive few-sample relation extraction | |
| Sadr et al. | Convolutional neural network equipped with attention mechanism and transfer learning for enhancing performance of sentiment analysis | |
| CN111582506A (en) | Multi-label learning method based on global and local label relation | |
| CN115935991A (en) | Multitask model generation method and device, computer equipment and storage medium | |
| CN113705238A (en) | Method and model for analyzing aspect level emotion based on BERT and aspect feature positioning model | |
| CN116258990A (en) | Cross-modal affinity-based small sample reference video target segmentation method | |
| Ye et al. | A joint-training two-stage method for remote sensing image captioning | |
| CN113988079A (en) | Low-data-oriented dynamic enhanced multi-hop text reading recognition processing method | |
| CN110569355B (en) | Viewpoint target extraction and target emotion classification combined method and system based on word blocks | |
| Zhang et al. | Aspect-based sentiment analysis of user reviews in 5G networks | |
| CN112668633B (en) | Adaptive graph migration learning method based on fine granularity field | |
| CN113869055A (en) | Power grid project characteristic attribute identification method based on deep learning | |
| CN113657473A (en) | Web service classification method based on transfer learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |