Disclosure of Invention
1. Technical problem to be solved
The invention aims to provide an emotion-reason pair extraction system based on knowledge assistance, which solves the problems in the prior art:
after word-level encoding of a text, the machine may not be able to identify the problem of the emotion words in the clause more accurately.
2. Technical scheme
The system introduces a manually constructed language and psychological characteristic knowledge base and the like for auxiliary coding, strengthens the identification of emotional words and psychological characteristics, and improves the extraction effect of emotional clauses. Meanwhile, part-of-speech labels including entity identification are added, information such as characters and events in the text is captured, and richer features are provided for extracting emotion and emotion reasons. Third, emotion and emotional cause are often co-occurring, meaning that if a clause is identified as an emotional clause with a greater probability, there is also at least one reason clause in its context with a greater probability. Therefore, the external knowledge auxiliary system learning is added, and the emotion-reason pair extraction is facilitated to a certain extent.
A system for emotion-reason pair extraction based on knowledge assistance, comprising the steps of:
s1, extracting emotion clauses;
s2, extracting reason clauses;
s3, emotion-reason pairing;
S1-S3 are all represented by knowledge-assisted word encoding.
Preferably, the knowledge-assisted word encoding representation consists of 3 parts: BERT-based semantic encoding, LIWC linguistic psychology knowledge base-based part-of-speech encoding, and NLPIR-based part-of-speech encoding, wherein,
based on the semantic coding of BERT, each word w in the clause is coded by a BERT BASE model
j Encoding to obtain 768-dimensional word vector representation
Based on word class coding of a language psychological characteristic knowledge base of LIWC, an SC-LIWC dictionary (comprising 71 categories such as human sensory word class, emotional history word class, cognitive history word class and social history word class) constructed by Huanglangen and the like is adopted to code words w in clauses according to one-hot
j Encoding is performed to obtain a 71-dimensional vector representation
Based on the part-of-speech coding of NLPIR, 9 parts-of-speech (including a person name nr, a place name ns, other nouns n, an adjective a, an adverb d, a verb v, a person pronoun rr, other pronouns r and other parts-of-speech other) are reserved, and words w in the clause are subjected to one-hot coding
j To obtain a 9-dimensional vector representation
Preferably, the knowledge-assisted word encoding means that the semantic encoding of the BERT of the current word, the part-of-speech encoding of the LIWC linguistic-psychological characteristic knowledge base, and the part-of-speech encoding of the NLPIR are concatenated, and the calculation formula is as follows:
wherein x j A vector representation of the word.
Preferably, the S1 adopts a two-layer Bi-LSTM model of a word layer and a clause layer to encode and express the clause and carries out binary bounding prediction, namely if the model identifies the emotion clause in the text d, the emotion clause is known
The clauses of (2) are set with the recognition results corresponding to the clauses
1, and the recognition results of the other clauses are 0; if the recognition results of all clauses in the text d are 0, namely
Then will be
Maximum first two clause recognition results
Set to 1 and the recognition results of the remaining clauses to 0.
Preferably, the calculation formula of S1 is as follows:
wherein the content of the first and second substances,
as clause c
i Is used to indicate that the emotion is encoded by the emotion encoding,
is a clause c
i Is indicative of the context of the user,
is the predicted probability of an emotional clause.
Preferably, the S2 adopts a two-layer Bi-LSTM model of a word layer and a clause layer to encode and express the clause, the clause is spliced with the emotion clause encoding and expressing, and then binary bounding prediction is carried out, namely if the model identifies a reason clause in the text d, namely, if the model identifies the reason clause, the reason clause exists
The clauses of (2) are set with the recognition results corresponding to the clauses
1, and the recognition results of the other clauses are 0; if the recognition results of all clauses in the text d are 0, namely
Then will be
Maximum first two clause recognition results
Set to 1 and the recognition results of the remaining clauses to 0.
Preferably, the calculation formula of S2 is as follows:
wherein the content of the first and second substances,
is a clause c
i Is used to indicate that the emotion is encoded by the emotion encoding,
is a clause c
i Is indicative of the context of the user,
the predicted probability of a reason clause.
Preferably, in the step S3, a Bi-LSTM model of a word layer and a clause layer is used to encode and represent the clauses, and prediction probabilities and distance information of emotion clauses and reason clauses are added and then sent to a logistic regression model for prediction.
Preferably, the calculation formula of S3 is as follows:
preferably, the distance information is calculatedThe method is as follows: setting emotion clauses
And reason clause
Is d relative to
i,j = j-i, and the maximum number of clauses in all texts does not exceed M sentences. Initializing a 2M x 50 dimensional array with each row conforming to a normal distribution function, then v
d Represents the (d) th in the array
i,j + M) rows, which are applied to the test dataset by continuous training of the dataset to obtain a final representation of each relative position.
3. Advantageous effects
Compared with the prior art, the invention has the advantages that:
(1) Emotion-cause pair extraction (ECPE) with greater accuracy
The evaluation results of ECPE-KA on the emotion-cause pair extraction task EPCE are shown in Table 1. As can be seen from Table 1, the ECPE-KA is significantly higher than the ECPE-2Steps and RANKCP models in the accuracy P and F1 values, and is respectively higher than the ECPE-2Steps and the RANKCP models in the F1 value by 18.84 percent and 4.59 percent; although ECPE-KA is slightly lower than the TDGC model in the accuracy rate P, the recall rate R is obviously improved, so that the F1 value of the ECPE-KA is also better than that of the TDGC model, and the improvement of the recall rate also indicates that the model obtains more correct clause pairs.
Compared with the ECPE-2D model which is the most advanced currently, ECPE-KA (F1 = 0.6914) achieves better effect than the ECPE-2D model (F1 = 0.6889) on the ECPE task, and the accuracy P is improved by 0.85%, while the recall rate only has a deficiency gap of 0.19%.
TABLE 1 results of the experimental evaluation
(2) Reduced number of candidate affective cause pairs
The binarization process adopted by ECPE-KA ensures that each text has at least one candidate emotion-reason pair to be sent into calculation, and the number of the candidate emotion-reason pairs in the three submodels of ECPE-2step is less than 1, which means that the model has serious defects in emotion clause extraction or reason clause extraction, so that the pairing number is sharply reduced, but a plurality of possible correct candidate emotion-reason pairs are inevitably lost.
Therefore, the ECPE-KA model not only ensures that the emotion clauses and reason clauses are extracted as accurately as possible, but also reduces the number of investigation candidate emotion-reason pairs and improves the identification efficiency.
(3) ECPE-KA is more accurate in Emotion Cause Extraction (ECE)
In a classical ECE task, emotion clauses are manually marked, and an ECPE-KA model does not require manual marking of the emotion clauses in a test set.
Table 2 shows that without sentiment clauses labeling the test dataset, the ECPE-KA model is only lower in accuracy than CANN and PAE-DGL, superior in recall to all reference models, and finally differs only by 2% from the best result (CANN) at 1. This shows that the method proposed herein can overcome the application limitation problem of manual sentiment clause annotation on ECE mission, and certainly there is room for improvement.
This document compares the CANN-E model, which is a label that removes the emotion clauses in the data set from the CANN model that performs better under test. As is clear from Table 2, the performance of the CANN-E model after the emotion labels are removed is reduced linearly, and compared with the CANN model, the performance is reduced by 47.74% in the value of 1. And under the condition that the ECPE-KA also has no emotional clause labels, the value 1 reaches 0.7083, which is improved by 86.54% compared with CANN-E.
TABLE 2 evaluation of emotional cause extraction tasks
Detailed Description
An emotion-reason pair extraction system (ECPE-KA) based on knowledge assistance is provided by combining an external artificial knowledge LIWC (language-mental feature) knowledge base and an NLPIR (nlPIR) part-of-speech analysis platform. The ECPE-KA system structure is shown in FIG. 2.
Example 1: knowledge-assisted clause representation
The knowledge-aided clause representation structure is shown in fig. 3. Given a text d = { c) containing | d | clauses
1 ,c
2 ,…,c
|d| }, each clause
Respectively contain | c
i | words. Each word w
j Is represented by the code x
j The method comprises three parts, namely semantic coding based on BERT, part of speech coding based on an LIWC language psychological characteristic knowledge base and part of speech coding based on NLPIR.
(1) The ECPE-KA model first adopts the BERT BASE model to carry out the operation on each word w in the clause
j Encoding to obtain 768-dimensional word vector representation
(2) Since the text is directed to a chinese dataset, the SC-LIWC dictionary constructed by golden blue et al is employed. The SC-LIWC dictionary contains 71 categories such as human sensory part of speech, emotional part of speech, cognitive part of speech, and social part of speech. One-hot pair of words w in clauses is adopted in the text
j Encoding to obtain a 71-dimensional vector representation
(3) Because only the entities such as names, pronouns and the like need to be identified with emphasisIn order to help extraction of emotion clauses and avoid dimension sparseness caused by excessive part-of-speech types, the ECPE-KA model only adopts one type and partial two types of parts-of-speech, removes three types of parts-of-speech described in detail, and finally retains 8 types of parts-of-speech (including a name nr, a place ns, a noun n, an adjective a, an adverb, a verb v, a name pronoun rr and a pronoun r) after screening, and uniformly merges the other types of parts-of-speech into other parts-of-speech (other). One-hot is adopted in the text to the word w in the clause
j To obtain a 9-dimensional vector representation
In the ECPE-KA model, a word w in a candidate clause
j Is coded by
And
expressed as:
example 2: sentiment clause extraction
The extraction of emotion clauses adopts a two-layer Bi-LSTM model of a word layer and a clause layer:
(1) Word layer Bi-LSTM
One contains | c
i Clause of | words
Is represented by a code
As input, send into Bi-LSTM model to get clause c
i Hidden layer representation of the jth word in
Using self-attention to each wordGenerating clause c by mechanical action
i Coded representation of
Where F represents a Bi-LSTM network using the self-attention mechanism.
(2) Clause layer Bi-LSTM
The purpose of the clause layer Bi-LSTM is to capture semantic dependencies between clauses. For text containing | d | clauses, d = { c
1 ,c
2 ,…,c
2 ,…,c
|d| }, encoding each clause
Sending the hidden state of the Bi-LSTM, namely the clause c, into a Bi-LSTM model
i Is represented by the context of
Finally will
Enter softmax function to get clause c
i Probability of being an emotional clause
Considering that at least one emotion clause exists in the text and most of the text contains at most two emotion clauses, the ECPE-KA model considers that in the binary bounding stageTwo cases are: if the model has recognized an emotion clause in the text d, then
The clauses of (2) are set up with the recognition results corresponding to the clauses
1, and the recognition results of the other clauses are 0; if the recognition results of all clauses in the text d are 0, namely
Then will be
Maximum first two clause recognition results
Set to 1 and the recognition results of the remaining clauses are set to 0.
Thus, a candidate emotion clause set in d is obtained
Example 3: reason clause extraction
The extraction of the reason clauses also adopts a Bi-LSTM with a word layer and a clause layer, wherein the coding structure of the clauses (the coding of the Bi-LSTM with the word layer) is the same as the clause coding structure in the emotion clause extraction stage.
Clause c
i Is represented by a code
And the emotion prediction probability value obtained in the first stage
Splicing to obtain clause c
i Is represented by a code
To capture context information, a vector representation of | d | clauses in text d is presented herein
As an input to the Bi-LSTM model, the hidden state of Bi-LSTM, i.e., clause c, is obtained
i Is represented by the context of
Finally will be
Sending into softmax function to obtain clause c
i Is predicted to have a probability value
Similar to binarization adopted by emotion clause extraction, considering that most texts contain at most two reason clauses, binarization of the reason clause extraction result is also divided into two cases: if the model has identified a reason clause in the text d, then there is
The clauses of (2) are set with the recognition results corresponding to the clauses
1, and the recognition results of the other clauses are 0; if the recognition results of all clauses in the text d are 0, namely
Then will be
Maximum first two clause recognition results
Set to 1 and the recognition results of the remaining clauses are set to 0.
Thus, a candidate reason clause set in d is obtained
Example 4: emotion-reason pairing
For the set of emotion clauses in document d
And reason clause set
Performing a cartesian product to obtain all possible pairing results:
obtaining candidate emotion clauses by adopting text representation method in section 1
Coded representation of
And candidate reason clause
Coded representation of
Distance v for joining two clauses simultaneously
d Prediction probability of candidate emotion clause
And predicted probability of candidate reason clause
As a feature, the five codes are spliced to obtain an input vector of the emotion-reason pair filtering model
Comprises the following steps:
distance feature v
d The calculation of (c) is as follows: setting emotion clauses
And reason clause
Is d relative to
i,j = j-i, and the maximum number of clauses in all texts does not exceed M sentences. Initializing a 2M x 50 dimensional array with each row conforming to a normal distribution function, then v
d Represents the (d) th in the array
i,j + M) rows, applied to the test dataset, through the continued training of the dataset to get a final representation of each relative position.
Then inputting the vector
Sending the sentence into a Logistic regression (Logistic) model to detect whether the two clauses have a causal relationship, and filtering to obtain an emotion-cause pair set:
retention
Emotion-origin ofThe cause pairs are extracted as final emotion-cause pairs.
The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and the preferred embodiments of the present invention are described in the above embodiments and the description, and are not intended to limit the present invention. The scope of the invention is defined by the appended claims and equivalents thereof.