CN114416991A - Method and system for analyzing text emotion reason based on prompt - Google Patents
Method and system for analyzing text emotion reason based on prompt Download PDFInfo
- Publication number
- CN114416991A CN114416991A CN202210055865.2A CN202210055865A CN114416991A CN 114416991 A CN114416991 A CN 114416991A CN 202210055865 A CN202210055865 A CN 202210055865A CN 114416991 A CN114416991 A CN 114416991A
- Authority
- CN
- China
- Prior art keywords
- text
- emotion
- reason
- predicted
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008451 emotion Effects 0.000 title claims abstract description 139
- 238000000034 method Methods 0.000 title claims abstract description 60
- 239000013598 vector Substances 0.000 claims abstract description 90
- 238000012549 training Methods 0.000 claims abstract description 43
- 238000004458 analytical method Methods 0.000 claims abstract description 33
- 230000011218 segmentation Effects 0.000 claims abstract description 14
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 46
- 230000002996 emotional effect Effects 0.000 claims description 38
- 238000010276 construction Methods 0.000 claims description 30
- 238000000605 extraction Methods 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 17
- 230000007246 mechanism Effects 0.000 claims description 10
- 230000001364 causal effect Effects 0.000 claims description 8
- 230000008909 emotion recognition Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/353—Clustering; Classification into predefined classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a system for analyzing a text emotion reason based on prompt, wherein the method comprises the following steps: s1: collecting text data and preprocessing the text data; s2: sequentially adding text prompt words and text to-be-predicted words in the preprocessed text, and setting a target candidate word set aiming at the text to-be-predicted words; s3: adding a clause segmentation symbol, a text starting symbol and a text ending symbol to a text; s4: performing feature vector coding on the text and the target candidate word set by using a BERT pre-training model to obtain a text feature vector and a target candidate word set vector; s5: calculating the coding distance between the text characteristic vector and the target candidate word set vector, and calculating the probability of the coding distance vector of each word to be predicted by utilizing a softmax function to obtain the prediction result of the word to be predicted; s6: and (4) performing prediction module combination based on the specific task to obtain a method suitable for the specific text emotion reason analysis task. The invention introduces prompt to solve the difference between the fine tuning task and the pre-training task.
Description
Technical Field
The invention relates to the field of text emotion reason analysis, in particular to a method and a system for text emotion reason analysis based on prompt.
Background
Emotional cause analysis is intended to identify emotional information and causes of emotions in the emotional text. In the field, specific subtasks include emotion cause extraction, emotion cause matching pair extraction, conditional causal relationship classification and the like. Emotional cause extraction is a subtask which is firstly proposed and defined as a word-level sequence tagging problem, and aims to explore the causes behind a certain emotional expression in a clause. However, in some cases, the emotion or reason information of the text may span the entire sequence of clauses. To solve this problem, emotional cause extraction is redefined as a clause-level classification problem. However, in the emotion reason extraction task, the reason extraction is based on the existing emotion information label, so that the application of the emotion reason extraction related technology in a real scene is very limited. For this problem, emotional cause matching is proposed for the extraction task. By the technology and the method in the task, all emotions and corresponding reasons can be directly identified from the text without the emotion marked. In addition, in the field of emotional cause analysis, tasks such as conditional relationship classification are also proposed to further discuss causal relationships in emotional texts.
With the development of the internet, text information such as social public opinions, after-sales comments, friends' circles and leave messages is ubiquitous. Accompanying text is often an inherent emotional expression and underlying cause. By analyzing and utilizing the text information generated on each platform, the method has great significance for positioning and controlling public opinions in the social platform, analyzing and improving the reasons for after-sale evaluation of buyers, or carrying out decision transformation according to different times and reasons. This places great demands on the relevant technology in the field of emotional cause analysis.
The existing emotional cause analysis work mainly obtains good effect under data sets of different tasks by establishing a novel deep neural network model. Most of the methods adopt a uniform fine-tuning structure, the structure firstly obtains word representation of an input text sequence from a pre-training model, and then obtains clause-level characteristics from word-level characteristic codes by using an attention mechanism. The context feature representation of the clause is then generated by the interaction module for final classification. However, these methods have significant drawbacks. Firstly, these methods only use the pre-trained language model as the word embedding layer, and cannot fully exert the capability of the pre-trained language model; secondly, the introduction of the position information enables the methods to have a bias phenomenon and neglects important indication information; moreover, designing a feature fusion module suitable for context and text interactive learning is difficult, and the generality and robustness of these algorithm modules are insufficient.
Aiming at the defects in the existing emotional cause analysis work, a prompt method is introduced to specifically solve the problem of emotional cause analysis. In more detail, the prompt method converts a specific fine tuning task form into the same form as a pre-training task. Aiming at the emotional cause analysis task, the prompt method converts the classification, matching, recognition and other task forms involved in the emotional cause analysis task into a pre-training task form, so that the performance of a pre-training model can be fully exerted in the task training process. Accordingly, the prompt method is also referred to as the fourth training paradigm after the fine tuning paradigm.
In the prior art, a method, an apparatus, a device and a storage medium for analyzing text emotion content are disclosed, and the method includes: analyzing a text to be analyzed through a BERT model to obtain a word vector of the text to be analyzed; adding a global attention mechanism to the clause information corresponding to the word vector to obtain global text information; analyzing the clause information corresponding to the word vector through a preset attention mechanism to obtain clause information combined with attention information; combining the global text information and the clause information combined with the attention information through an attention mechanism to obtain a target text; and analyzing through a classifier according to the relation between the clauses in the target text to obtain an emotion reason pair, and obtaining an emotion analysis result of the text to be analyzed through the emotion reason. The method also aims at the problems that the capability of pre-training the language model cannot be fully exerted and important indication information is ignored, so that the universality and the robustness of the method are insufficient.
Disclosure of Invention
The invention aims to provide a method for analyzing the emotion reason of a text based on prompt, which effectively utilizes the indication information between the emotion and the reason in the text, solves the bias phenomenon in the prior art and has more excellent performance
It is a further object of the present invention to provide a prompt-based text emotional cause analysis system.
In order to solve the technical problems, the technical scheme of the invention is as follows:
a text emotion reason analysis method based on prompt comprises the following steps:
s1: collecting text data and preprocessing the text data;
s2: sequentially adding text prompt words and text to-be-predicted words in the preprocessed text, and setting a target candidate word set aiming at the text to-be-predicted words;
s3: adding a clause segmentation symbol, a text starting symbol and a text ending symbol to the text processed in the step S2;
s4: performing feature vector coding on the text processed in the step S3 and the target candidate word set in the step S2 by using a BERT pre-training model to obtain a text feature vector and a target candidate word set vector;
s5: and calculating the coding distance between the text feature vector and the target candidate word set vector, and calculating the probability of the coding distance vector of each word to be predicted by using a softmax function to obtain a prediction result of the word to be predicted, namely the text emotion reason prediction result.
S6: and (4) performing prediction module combination based on the specific task to obtain a method suitable for the specific text emotion reason analysis task.
Further, the preprocessing in step S1 includes removing punctuation marks, merging text clauses, word segmentation, and removing coding error words.
Further, the text cue words in step S2 include:
when the BERT pre-training model is guided to understand the emotion recognition task, emotion cue words are added to the original text;
when the BERT pre-training model is guided to understand the reason recognition task, reason prompt words are added into the original text;
matching hints words are added to the original text when understanding the matching task between emotional causes in order to guide the BERT pre-trained model.
Further, in step S2, words to be predicted of the text are added, and a target candidate word set is set for the words to be predicted of the text, specifically including:
1) based on the emotion recognition task, setting emotion words to be predicted for emotion clauses recognition to obtain an emotion indication module, wherein a text construction template and a candidate word set of the emotion indication module are as follows:
wherein,
the function represents a text construction template of the emotion indicating module,
the function represents a candidate word set of words to be predicted in the emotion indication module, cIRepresenting the ith clause in the text,<·>representing emotion cues added to the text, [ MASK]emoRepresenting words to be predicted of emotion;
2) based on the requirement of an emotional reason analysis task, identifying and setting a reason to-be-predicted word for a reason clause to obtain a reason indication module, wherein a text construction template and a candidate word set of the reason indication module are as follows:
wherein
The function represents a text construction template for the cause indication module,
function represents a set of candidate words of the word to be predicted in the cause indication module, ciRepresenting the ith clause in the text,<·>indicates a reason cue word, [ MASK ], to be added to the text]cauA word representing a reason to be predicted;
3) based on the matching task requirements among emotional reasons, words to be predicted are matched in the matching work among the emotional reason clauses, a directional constraint module is obtained, and a text construction template and a candidate word set of the directional constraint module are as follows:
wherein
The function represents a text build template that points to a constraint module,
the function represents a set of candidate words pointing to the word to be predicted in the constraint module, ciRepresenting the ith clause in the text,<·>indicating matching hints added to the text, [ MASK ]]dirIndicating a matching word to be predicted, n indicating the number of clauses of the current text, and "None" indicating no other clauses associated with the current clause.
Further, in step S2, in order to make the pre-training model complete the task of learning the sequence features of the clauses, a sequence learning module is constructed by adding a sequence word to be predicted into the text, and a text construction template and a candidate word set of the sequence learning module are:
wherein
The function represents a text construction template of the sequence learning module,
function representation refers to a set of candidate words of the word to be predicted in the sequence learning module, d represents the input text, ciRepresents the ith clause, [ MASK ] in the text]dirRepresenting the word to be predicted of the sequence, and n representing the number of clauses of the current text.
Further, the step S3 specifically includes the following steps:
s3.1: adding a clause segmentation symbol to each constructed clause in the text for the text processed in the step S2;
s3.2: and respectively adding a start symbol and an end symbol at the beginning and the end of the constructed complete text.
Further, the step S4 specifically includes the following steps:
s4.1: converting text characters into corresponding word embedding vectors by searching a dictionary and a corresponding word vector matrix;
s4.2: inputting the word embedding vector into a BERT pre-training model to obtain an output text characteristic vector;
s4.3: and converting the text of the target candidate word set into a corresponding word embedding vector by searching the dictionary and the corresponding word vector matrix to obtain the target candidate word set vector.
Further, the step S5 specifically includes the following steps:
s5.1: multiplying a text characteristic vector of a word to be predicted in a text by a target candidate word set vector matrix to obtain a coding distance between the word vector to be predicted and the corresponding target candidate word set vector matrix;
s5.2: and obtaining a final prediction probability value through a softmax function:
P*＝softmax(V*T*)
in the formula, subscript indicates four prediction modules including emotion indication module, reason indication module, direction constraint module and sequence learning module, V*Text feature vector, T, representing the word to be predicted*And representing a vector matrix of the target candidate word set.
Further, the method also includes step S6: the method comprises the following steps of combining different modules to be predicted according to different specific tasks in the emotion reason analysis direction to complete prediction of the specific tasks, and specifically comprises the following steps:
1) for the emotion reason matching pair extraction task, an emotion indication module, a reason indication module, a pointing constraint module and a sequence learning module are combined at the same time, three subtasks of emotion clause identification, reason clause identification and emotion reason clause matching are completed, and the complete construction mode is shown as the following formula:
the emotion clause and reason clause can be predicted by the indication module, and the specific form is shown as the following formula:
wherein P isemoFunction sum PcauThe functions representing the prediction results of the emotion clause and the reason clause, respectively, femoFunction sum fcauThe function respectively represents the prediction results of the words to be predicted for emotion and the words to be predicted for reason;
meanwhile, the matching prediction between the emotion clause and the reason clause can be completed by the pointing constraint module and the reason indication module together, and the specific form is shown in the following formula.
Wherein (i, j) indicates that the ith clause and the jth clause form an emotional cause matching pair relation, null indicates that no clause is associated with the current clause, and fdirRepresenting a prediction result matching a word to be predicted;
2) for the emotion reason extraction task, a reason indication module, a pointing constraint module and a sequence learning module are combined at the same time, an emotion indication module is set according to prior conditions, the reason clauses are identified on the premise that the emotion clauses are known, and the complete construction mode is shown in the following formula.
Wherein the pair [ MASK]cauThe prediction is the prediction result of the final reason clause, and the corresponding function formula is shown as follows;
3) and for the conditional causal relationship classification task, combining the pointing constraint module and the sequence learning module at the same time, setting an emotion indication module and a reason indication module according to the prior condition, and judging whether the given emotion clause and reason clause group still form causal relationship under the situation of a specific text. The complete construction is shown in the following formula.
Setting a voting mechanism for finally predicting the category of the sample, wherein the voting mechanism is specifically shown as the following function:
wherein c isemoDenotes an emotion clause, and X denotes a set of emotion clauses.
A prompt-based text emotional cause analysis system, comprising:
the data processing module is used for collecting text data and carrying out preprocessing;
the word adding module is used for sequentially adding text prompt words and text to-be-predicted words in the preprocessed text and setting a target candidate word set aiming at the text to-be-predicted words;
the symbol adding module is used for adding a clause segmentation symbol, a text starting symbol and a text ending symbol to the text processed by the word adding module;
the encoding module uses a BERT pre-training model to perform feature vector encoding on the text processed by the symbol adding module and the target candidate word set by the word adding module to obtain a text feature vector and a target candidate word set vector;
and the prediction module is used for calculating the coding distance between the text characteristic vector and the target candidate word set vector, calculating the probability of the coding distance vector of each word to be predicted by utilizing a softmax function, obtaining the prediction result of the word to be predicted, and obtaining the text emotion reason prediction result.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
1. the invention uses a prompt method and a pre-training model BERT to convert the emotion reason analysis task into a pre-training task form, thereby solving the difference of the form of the previous fine tuning task and the pre-training task and fully utilizing the knowledge learned by the pre-training task in the BERT.
2. The difference is that the invention adopts a method of combining a pointing constraint module and a sequence learning module, so that the pointing information between the emotion clause and the reason clause is effectively learned, thereby improving the robustness of the algorithm on a balanced data set
3. The invention designs a general algorithm suitable for various emotion reason analysis tasks, and besides the applicability among different tasks, the algorithm provided by the invention can learn the commonality among different tasks, thereby further improving the efficiency and performance of the algorithm.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
FIG. 2 is a schematic diagram of the model structure of the present invention.
FIG. 3 is a block diagram of the system of the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;
it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Example 1
The embodiment provides a method for analyzing a text emotion reason based on prompt, as shown in fig. 1 and 2, the method includes the following steps:
s1: collecting text data and preprocessing the text data;
s2: sequentially adding text prompt words and text to-be-predicted words in the preprocessed text, and setting a target candidate word set aiming at the text to-be-predicted words;
s3: adding a clause segmentation symbol, a text starting symbol and a text ending symbol to the text processed in the step S2;
s4: performing feature vector coding on the text processed in the step S3 and the target candidate word set in the step S2 by using a BERT pre-training model to obtain a text feature vector and a target candidate word set vector;
s5: and calculating the coding distance between the text feature vector and the target candidate word set vector, and calculating the probability of the coding distance vector of each word to be predicted by using a softmax function to obtain a prediction result of the word to be predicted, namely the text emotion reason prediction result.
S6: and (4) performing prediction module combination based on the specific task to obtain a method suitable for the specific text emotion reason analysis task.
The preprocessing in step S1 includes removing punctuation marks, merging text clauses, word segmentation, and removing coding error words.
The text prompt in step S2 includes:
when the BERT pre-training model is guided to understand the emotion recognition task, emotion cue words are added to the original text;
when the BERT pre-training model is guided to understand the reason recognition task, reason prompt words are added into the original text;
matching hints words are added to the original text when understanding the matching task between emotional causes in order to guide the BERT pre-trained model.
In step S2, words to be predicted of the text are added, and a target candidate word set is set for the words to be predicted of the text, specifically including:
1) based on the emotion recognition task, setting emotion words to be predicted for emotion clause recognition to obtain an emotion indicating module, wherein the emotion indicating module can complete recognition of emotion clauses in a text, and a text construction template and a candidate word set of the emotion indicating module are as follows:
wherein,
the function represents a text construction template of the emotion indicating module,
the function represents a candidate word set of words to be predicted in the emotion indication module, ciRepresenting the ith clause in the text,<·>representing emotion cues added to the text, [ MASK]emoRepresenting words to be predicted of emotion;
2) based on the requirement of an emotional reason analysis task, reason to-be-predicted words are set for reason clause identification to obtain a reason indication module, the reason indication module can complete identification of reason clauses in texts, and a text construction template and a candidate word set of the reason indication module are as follows:
wherein
The function represents a text construction template for the cause indication module,
function represents a set of candidate words of the word to be predicted in the cause indication module, ciRepresenting the ith clause in the text,<·>indicating a reason hint for adding to textWord, [ MASK ]]cauA word representing a reason to be predicted;
3) based on the matching task requirements among emotional reasons, words to be predicted are set and matched for matching work among emotional reason clauses, a directional constraint module is obtained, the directional constraint module can match the reason clauses and the emotional clauses in a text, and a text construction template and a candidate word set of the directional constraint module are as follows:
wherein
The function represents a text build template that points to a constraint module,
the function represents a set of candidate words pointing to the word to be predicted in the constraint module, ciRepresenting the ith clause in the text,<·>indicating matching hints added to the text, [ MASK ]]dirIndicating a matching word to be predicted, n indicating the number of clauses of the current text, and "None" indicating no other clauses associated with the current clause.
In step S2, in order to enable the pre-training model to complete the learning task of the sequence features of the clauses, a sequence learning module is constructed by adding a sequence word to be predicted into a text, the sequence learning module enables the pre-training model to learn sequence information of a specific label in the training process, and a text construction template and a candidate word set of the sequence learning module are:
wherein
The function represents a text construction template of the sequence learning module,
function representation refers to a set of candidate words of the word to be predicted in the sequence learning module, d represents the input text, ciRepresents the ith clause, [ MASK ] in the text]dirRepresenting the word to be predicted of the sequence, and n representing the number of clauses of the current text.
The step S3 specifically includes the following steps:
s3.1: adding a clause segmentation symbol to each constructed clause in the text for the text processed in the step S2;
s3.2: and respectively adding a start symbol and an end symbol at the beginning and the end of the constructed complete text.
The step S4 specifically includes the following steps:
s4.1: converting text characters into corresponding word embedding vectors by searching a dictionary and a corresponding word vector matrix;
s4.2: inputting the word embedding vector into a BERT pre-training model to obtain an output text characteristic vector;
s4.3: and converting the text of the target candidate word set into a corresponding word embedding vector by searching the dictionary and the corresponding word vector matrix to obtain the target candidate word set vector.
The step S5 specifically includes the following steps:
s5.1: multiplying a text characteristic vector of a word to be predicted in a text by a target candidate word set vector matrix to obtain a coding distance between the word vector to be predicted and the corresponding target candidate word set vector matrix;
s5.2: and obtaining a final prediction probability value through a softmax function:
P*＝softmax(V*T*)
in the formula, subscript indicates four prediction modules including emotion indication module, reason indication module, direction constraint module and sequence learning module, V*Text feature vector, T, representing the word to be predicted*And representing a vector matrix of the target candidate word set.
Further comprising step S6: the method comprises the following steps of combining different modules to be predicted according to different specific tasks in the emotion reason analysis direction to complete prediction of the specific tasks, and specifically comprises the following steps:
1) for the emotion reason matching pair extraction task, an emotion indication module, a reason indication module, a pointing constraint module and a sequence learning module are combined at the same time, three subtasks of emotion clause identification, reason clause identification and emotion reason clause matching are completed, and the complete construction mode is shown as the following formula:
the emotion clause and reason clause can be predicted by the indication module, and the specific form is shown as the following formula:
wherein P isemoFunction sum PcauThe functions representing the prediction results of the emotion clause and the reason clause, respectively, femoFunction sum fcauThe function respectively represents the prediction results of the words to be predicted for emotion and the words to be predicted for reason;
meanwhile, the matching prediction between the emotion clause and the reason clause can be completed by the pointing constraint module and the reason indication module together, and the specific form is shown in the following formula.
Wherein (i, j) indicates that the ith clause and the jth clause form an emotional cause matching pair relation, null indicates that no clause is associated with the current clause, and fdirRepresenting a prediction result matching a word to be predicted;
2) for the emotion reason extraction task, a reason indication module, a pointing constraint module and a sequence learning module are combined at the same time, an emotion indication module is set according to prior conditions, the reason clauses are identified on the premise that the emotion clauses are known, and the complete construction mode is shown in the following formula.
Wherein the pair [ MASK]cauThe prediction is the prediction result of the final reason clause, and the corresponding function formula is shown as follows;
3) and for the conditional causal relationship classification task, combining the pointing constraint module and the sequence learning module at the same time, setting an emotion indication module and a reason indication module according to the prior condition, and judging whether the given emotion clause and reason clause group still form causal relationship under the situation of a specific text. The complete construction is shown in the following formula.
Unlike the two tasks mentioned above, which are to determine the relationship between a single emotion clause and multiple reason clauses, a voting mechanism is provided for finally predicting the category of the sample, and the voting mechanism is specifically shown as the following function:
wherein c isemoDenotes an emotion clause, and X denotes a set of emotion clauses.
Example 2
This embodiment provides a specific embodiment of embodiment 1, specifically:
data sets, including ECE data, ECPE data sets, and CCRC data sets, are analyzed using the disclosed emotional causes. The ECE data set is most widely used in affective cause analysis, and it is derived from news in the new wave city and has 2105 text data. Each text data only has one emotion clause and more than one reason clause; the ECPE data set is constructed based on the ECE data set, and text data which contain the same text and have different emotion and reason labels are aggregated in the data set; the CCRC data set is constructed on the basis of the ECPE data set, and the CCRC data set is obtained by manually marking and negatively sampling and employing three experts to mark data. Each data set was randomly divided into ten equal parts for cross validation.
The RTX 3090 graphics card with 24GB memory is adopted, and algorithm training is carried out on the disclosed Chinese BERT pre-training model. More specifically. The training process used an AdamW optimizer, the learning rate was 1e-5 and the number of samples per iteration was 8.
In the prior method, either regularized statement prediction is performed based on a manually designed rule, emotion reason judgment is performed only through keywords in a text, or learning in a mathematical and statistical sense is considered only based on a traditional machine learning method, language information in the text is ignored, or a fine tuning structure is adopted based on deep learning, a pre-training model and a downstream module are trained and optimized aiming at an emotion reason analysis task, and the method can be specifically divided into a method based on a graph structure, a method based on joint learning, a method based on multi-task learning, a method based on sequence labeling and a method based on grammar transfer, so that the robustness and universality of the method are insufficient, and the language knowledge in the pre-training model cannot be fully utilized. The embodiment effectively utilizes the indication information between the emotion and the reason in the text while fully playing the capability of the pre-training model, solves the bias phenomenon in the existing method, and has obviously better robustness and universality than the existing method.
The embodiment is a text emotion reason analysis method based on prompt, and is essentially a deep learning method. As shown in the method flowchart of fig. 1, in the method, the input text is first preprocessed. And then sequentially adding text prompt words and text to-be-predicted words in the text, and setting a target candidate word set and adding clause separation symbols and text starting and ending symbols aiming at the text to-be-predicted words. And then inputting the text into a BERT pre-training model for text feature coding, and then calculating the distance between feature vectors. And finally, predicting and classifying the words to be predicted by using a softmax function. The details are as follows:
1. firstly, reading text data, and carrying out preprocessing such as punctuation removal, text clause combination, word segmentation operation and the like on the text
2. And adding emotion cue words, reason cue words and matching cue words in the text.
3. Adding words to be predicted with emotion, words to be predicted with reason, words to be predicted matched and words to be predicted with sequence in the text, and setting a candidate word set aiming at the four types of words to be predicted.
4. And adding an inter-clause separator into the text, and adding a start character and an end character into the positions of the start and the end of the text respectively.
5. The Chinese pre-training parameters of the BERT model are loaded (the dataset is a Chinese dataset).
6. And (4) inputting the text constructed in the step (4) into the BERT model loaded with the pre-training parameters in the step (5), and coding to obtain a text feature vector to be output.
7. And (4) performing word feature processing on the candidate words, and performing vector distance calculation by using the processed candidate word feature vectors and the text feature vectors output in the step (6) to obtain a distance calculation matrix.
8. And calculating the distance vector of the word to be predicted in the distance calculation matrix through softmax to obtain emotion classification, reason classification and matching classification.
9. And (4) performing 20 epochs of iterative training on the model calculation part in the steps 6 to 8, performing index calculation on the test set, and storing the model with the highest index value for final emotion reason analysis task prediction.
For comparison with the previous methods, the specific results were evaluated by Accuracy (Accuracy), Recall (Recall) and F1 score (F1-score). The specific results on the three tasks of emotion reason matching extraction, emotion reason extraction and conditional emotion reason classification are shown in the following tables:
table 1, experimental comparison results with other models on emotion cause matching extraction task
Table 2, results of experimental comparison with other models on emotion cause extraction task
| Model | F1 | P | R |
| EF-BHA | 0.7868 | 0.7938 | 0.7808 |
| RTHN(Layer3) | 0.7677 | 0.7697 | 0.7662 |
| RHNN | 0.7914 | 0.8112 | 0.7725 |
| MANN | 0.7706 | 0.7843 | 0.7587 |
| FSS-GCN | 0.7861 | 0.7572 | 0.7714 |
| LambdaMART | 0.7608 | 0.7720 | 0.7499 |
| Multi-kernel | 0.6752 | 0.6588 | 0.6972 |
| SVM | 0.4285 | 0.4200 | 0.4375 |
| Our Model | 0.8319 | 0.8274 | 0.8373 |
Table 3 Experimental comparison results with other models on the task of classifying conditional emotional causes
| Model | F1 | P | R |
| BiLSTM+Concatenation | 0.6127 | 0.5412 | 0.7119 |
| BiLSTM+BiLSTM | 0.6976 | 0.6606 | 0.7400 |
| BiLSTM+Self-Attention | 0.6605 | 0.5766 | 0.7770 |
| Our Model | 0.8074 | 0.7707 | 0.8487 |
From the above experimental results, the present embodiment has a significant improvement compared with other methods under multiple tasks of emotional cause analysis, and all of the tasks on the current data set are close to or even exceed the best level. Meanwhile, the method has strong universality on a plurality of tasks, and is suitable for a plurality of tasks of emotion reason analysis.
Example 3
The embodiment provides a prompt-based text emotional cause analysis system, as shown in fig. 3, including:
the data processing module is used for collecting text data and carrying out preprocessing;
the word adding module is used for sequentially adding text prompt words and text to-be-predicted words in the preprocessed text and setting a target candidate word set aiming at the text to-be-predicted words;
the symbol adding module is used for adding a clause segmentation symbol, a text starting symbol and a text ending symbol to the text processed by the word adding module;
the encoding module uses a BERT pre-training model to perform feature vector encoding on the text processed by the symbol adding module and the target candidate word set by the word adding module to obtain a text feature vector and a target candidate word set vector;
and the prediction module is used for calculating the coding distance between the text characteristic vector and the target candidate word set vector, calculating the probability of the coding distance vector of each word to be predicted by utilizing a softmax function, obtaining the prediction result of the word to be predicted, and obtaining the text emotion reason prediction result.
The same or similar reference numerals correspond to the same or similar parts;
the terms describing positional relationships in the drawings are for illustrative purposes only and are not to be construed as limiting the patent;
it should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (10)
1. A method for analyzing text emotion reason based on prompt is characterized by comprising the following steps:
s1: collecting text data and preprocessing the text data;
s2: sequentially adding text prompt words and text to-be-predicted words in the preprocessed text, and setting a target candidate word set aiming at the text to-be-predicted words;
s3: adding a clause segmentation symbol, a text starting symbol and a text ending symbol to the text processed in the step S2;
s4: performing feature vector coding on the text processed in the step S3 and the target candidate word set in the step S2 by using a BERT pre-training model to obtain a text feature vector and a target candidate word set vector;
s5: calculating the coding distance between the text characteristic vector and the target candidate word set vector, and calculating the probability of the coding distance vector of each word to be predicted by utilizing a softmax function to obtain the prediction result of the word to be predicted;
s6: and (4) performing prediction module combination based on the specific task to obtain a method suitable for the specific text emotion reason analysis task.
2. The method for analyzing emotion reason of text based on prompt as recited in claim 1, wherein the preprocessing in step S1 includes removing punctuation, merging text clauses, word segmentation operation and removing wrong words from encoding.
3. The method for analyzing text emotion reason based on prompt according to claim 2, wherein the text prompt in step S2 includes:
when the BERT pre-training model is guided to understand the emotion recognition task, emotion cue words are added to the original text;
when the BERT pre-training model is guided to understand the reason recognition task, reason prompt words are added into the original text;
matching hints words are added to the original text when understanding the matching task between emotional causes in order to guide the BERT pre-trained model.
4. The method for analyzing text emotion reason based on prompt as claimed in claim 3, wherein a text word to be predicted is added in step S2, and a target word candidate set is set for the text word to be predicted, specifically comprising:
1) based on the emotion recognition task, setting emotion words to be predicted for emotion clauses recognition to obtain an emotion indication module, wherein a text construction template and a candidate word set of the emotion indication module are as follows:
wherein,
the function represents a text construction template of the emotion indicating module,
the function represents a candidate word set of words to be predicted in the emotion indication module, ciRepresenting the ith clause in the text,<·>to representEmotional cue words [ MASK ] added to text]emoRepresenting words to be predicted of emotion;
2) based on the requirement of an emotional reason analysis task, identifying and setting a reason to-be-predicted word for a reason clause to obtain a reason indication module, wherein a text construction template and a candidate word set of the reason indication module are as follows:
wherein
The function represents a text construction template for the cause indication module,
function represents a set of candidate words of the word to be predicted in the cause indication module, ciRepresenting the ith clause in the text,<·>indicates a reason cue word, [ MASK ], to be added to the text]cauA word representing a reason to be predicted;
3) based on the matching task requirements among emotional reasons, words to be predicted are matched in the matching work among the emotional reason clauses, a directional constraint module is obtained, and a text construction template and a candidate word set of the directional constraint module are as follows:
wherein
The function represents a text build template that points to a constraint module,
the function represents a set of candidate words pointing to the word to be predicted in the constraint module, ciRepresenting the ith clause in the text,<·>indicating matching hints added to the text, [ MASK ]]dirIndicating a matching word to be predicted, n indicating the number of clauses of the current text, and "None" indicating no other clauses associated with the current clause.
5. The method for analyzing text emotion reason based on prompt as claimed in claim 4, wherein in step S2, in order to make the pre-training model complete the task of learning sequence characteristics of clauses, a sequence learning module is constructed by adding words to be predicted in sequence to the text, and the text construction template and the candidate word set of the sequence learning module are:
wherein
The function represents a text construction template of the sequence learning module,
function representation refers to a set of candidate words of the word to be predicted in the sequence learning module, d represents the input text, ciRepresents the ith clause, [ MASK ] in the text]dirRepresenting the word to be predicted of the sequence, and n representing the number of clauses of the current text.
6. The method for analyzing text emotion reason based on prompt according to claim 5, wherein the step S3 specifically includes the following steps:
s3.1: adding a clause segmentation symbol to each constructed clause in the text for the text processed in the step S2;
s3.2: and respectively adding a start symbol and an end symbol at the beginning and the end of the constructed complete text.
7. The method for analyzing text emotion reason based on prompt according to claim 6, wherein the step S4 specifically includes the following steps:
s4.1: converting text characters into corresponding word embedding vectors by searching a dictionary and a corresponding word vector matrix;
s4.2: inputting the word embedding vector into a BERT pre-training model to obtain an output text characteristic vector;
s4.3: and converting the text of the target candidate word set into a corresponding word embedding vector by searching the dictionary and the corresponding word vector matrix to obtain the target candidate word set vector.
8. The method for analyzing text emotion reason based on prompt according to claim 7, wherein the step S5 specifically includes the following steps:
s5.1: multiplying a text characteristic vector of a word to be predicted in a text by a target candidate word set vector matrix to obtain a coding distance between the word vector to be predicted and the corresponding target candidate word set vector matrix;
s5.2: and obtaining a final prediction probability value through a softmax function:
P*＝softmax(V*T*)
in the formula, subscript indicates four prediction modules including emotion indication module, reason indication module, direction constraint module and sequence learning module, V*Text feature vector, T, representing the word to be predicted*Representing a target waitingAnd selecting a word set vector matrix.
9. The method for analyzing text emotion reason based on prompt according to claim 8, wherein the step S6 specifically includes:
1) for the emotion reason matching pair extraction task, an emotion indication module, a reason indication module, a pointing constraint module and a sequence learning module are combined at the same time, three subtasks of emotion clause identification, reason clause identification and emotion reason clause matching are completed, and the complete construction mode is shown as the following formula:
the emotion clause and reason clause can be predicted by the indication module, and the specific form is shown as the following formula:
wherein P isemoFunction sum PcauThe functions representing the prediction results of the emotion clause and the reason clause, respectively, femoFunction sum fcauThe function respectively represents the prediction results of the words to be predicted for emotion and the words to be predicted for reason;
meanwhile, the matching prediction between the emotion clause and the reason clause can be completed by the pointing constraint module and the reason indication module together, and the specific form is shown in the following formula.
Wherein (i, j) represents the ith clause andj-th clause constitutes emotion reason matching pair relation, null indicates that no clause is associated with the current clause, fdirRepresenting a prediction result matching a word to be predicted;
2) for the emotion reason extraction task, a reason indication module, a pointing constraint module and a sequence learning module are combined at the same time, an emotion indication module is set according to prior conditions, the reason clauses are identified on the premise that the emotion clauses are known, and the complete construction mode is shown in the following formula.
Wherein the pair [ MASK]cauThe prediction is the prediction result of the final reason clause, and the corresponding function formula is shown as follows;
3) and for the conditional causal relationship classification task, combining the pointing constraint module and the sequence learning module at the same time, setting an emotion indication module and a reason indication module according to the prior condition, and judging whether the given emotion clause and reason clause group still form causal relationship under the situation of a specific text. The complete construction is shown in the following formula.
Setting a voting mechanism for finally predicting the category of the sample, wherein the voting mechanism is specifically shown as the following function:
wherein c isemoDenotes an emotion clause, and X denotes a set of emotion clauses.
10. A prompt-based text emotional cause analysis system is characterized by comprising:
the data processing module is used for collecting text data and carrying out preprocessing;
the word adding module is used for sequentially adding text prompt words and text to-be-predicted words in the preprocessed text and setting a target candidate word set aiming at the text to-be-predicted words;
the symbol adding module is used for adding a clause segmentation symbol, a text starting symbol and a text ending symbol to the text processed by the word adding module;
the encoding module uses a BERT pre-training model to perform feature vector encoding on the text processed by the symbol adding module and the target candidate word set by the word adding module to obtain a text feature vector and a target candidate word set vector;
and the prediction module is used for calculating the coding distance between the text characteristic vector and the target candidate word set vector, calculating the probability of the coding distance vector of each word to be predicted by utilizing a softmax function, obtaining the prediction result of the word to be predicted, and obtaining the text emotion reason prediction result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210055865.2A CN114416991A (en) | 2022-01-18 | 2022-01-18 | Method and system for analyzing text emotion reason based on prompt |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210055865.2A CN114416991A (en) | 2022-01-18 | 2022-01-18 | Method and system for analyzing text emotion reason based on prompt |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114416991A true CN114416991A (en) | 2022-04-29 |
Family
ID=81273952
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210055865.2A Pending CN114416991A (en) | 2022-01-18 | 2022-01-18 | Method and system for analyzing text emotion reason based on prompt |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114416991A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114792097A (en) * | 2022-05-14 | 2022-07-26 | 北京百度网讯科技有限公司 | Method and device for determining prompt vector of pre-training model and electronic equipment |
| CN117787267A (en) * | 2023-12-29 | 2024-03-29 | 广东外语外贸大学 | Emotion cause pair extraction method and system based on neural network |
-
2022
- 2022-01-18 CN CN202210055865.2A patent/CN114416991A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114792097A (en) * | 2022-05-14 | 2022-07-26 | 北京百度网讯科技有限公司 | Method and device for determining prompt vector of pre-training model and electronic equipment |
| CN117787267A (en) * | 2023-12-29 | 2024-03-29 | 广东外语外贸大学 | Emotion cause pair extraction method and system based on neural network |
| CN117787267B (en) * | 2023-12-29 | 2024-06-07 | 广东外语外贸大学 | Emotion cause pair extraction method and system based on neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111985239B (en) | Entity identification method, entity identification device, electronic equipment and storage medium | |
| CN107943784B (en) | Relationship extraction method based on generation of countermeasure network | |
| CN110489750A (en) | Burmese participle and part-of-speech tagging method and device based on two-way LSTM-CRF | |
| CN113191148B (en) | Rail transit entity identification method based on semi-supervised learning and clustering | |
| CN113505200A (en) | Sentence-level Chinese event detection method combining document key information | |
| CN111339260A (en) | BERT and QA thought-based fine-grained emotion analysis method | |
| CN111462752A (en) | Client intention identification method based on attention mechanism, feature embedding and BI-L STM | |
| CN114416991A (en) | Method and system for analyzing text emotion reason based on prompt | |
| CN112800184B (en) | Short text comment emotion analysis method based on Target-Aspect-Opinion joint extraction | |
| CN112926345A (en) | Multi-feature fusion neural machine translation error detection method based on data enhancement training | |
| CN115080750B (en) | Weak supervision text classification method, system and device based on fusion prompt sequence | |
| CN117689963B (en) | Visual entity linking method based on multi-mode pre-training model | |
| CN111259153A (en) | Attribute-level emotion analysis method of complete attention mechanism | |
| CN113486178A (en) | Text recognition model training method, text recognition device and medium | |
| CN115952794A (en) | Chinese-Tai cross-language sensitive information recognition method fusing bilingual sensitive dictionary and heterogeneous graph | |
| CN115759119A (en) | Financial text emotion analysis method, system, medium and equipment | |
| CN115098673A (en) | Business document information extraction method based on variant attention and hierarchical structure | |
| CN113486174B (en) | Model training, reading understanding method and device, electronic equipment and storage medium | |
| CN114970536A (en) | Combined lexical analysis method for word segmentation, part of speech tagging and named entity recognition | |
| CN118093689A (en) | Multi-mode document analysis and structuring processing system based on RPA | |
| CN114818711B (en) | Multi-information fusion named entity identification method based on neural network | |
| CN114595329B (en) | System and method for extracting few sample events of prototype network | |
| CN116227486A (en) | Emotion analysis method based on retrieval and contrast learning | |
| CN115563253A (en) | Multi-task event extraction method and device based on question answering | |
| CN114611489A (en) | Text logic condition extraction AI model construction method, extraction method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |