CN114281993B - A text empathy prediction system and method - Google Patents

Abstract

The invention discloses a text co-emotion prediction system and a method, wherein the system comprises the following steps: the public-emotion private feature encoder encodes private features of the public-emotion data into public-emotion private features; the polarity private feature encoder encodes the private feature of the polarity data into a polarity private feature; the public feature encoder encodes public features of the common-case data and the polarity data into public features of the common-case data and the polarity data respectively; the public and private common-situation feature fusion module is used for weighting and fusing public and private common-situation features into final common-situation prediction feature expression; the polarity public and private feature fusion module is used for weighting and fusing the polarity public and private features into final polarity classification feature expression; the co-condition predictor predicts the final co-condition prediction feature expression to obtain a prediction label; the polarity classifier predicts the final polarity classification feature expression to obtain a predictive tag. The system and the method can realize small-scale text co-emotion prediction through large-scale text emotion classification data based on transfer learning, and improve prediction accuracy.

Description

A text empathy prediction system and method

Technical Field

The present invention relates to the field of natural language processing, and in particular to a text empathy prediction system and method.

Background Art

Empathy, as an important component of emotion, reflects the corresponding emotions people have when facing others' experiences or witnessing others' situations. Empathy can reflect people's emotions and feedback on others' experiences, and the field of empathy analysis is also closely related to human-computer interaction, sentiment analysis and other fields. Therefore, it is very necessary to identify the empathy factors contained in the text, which has strong research value.

However, the sample size of text-based empathy datasets is too small. Currently, the recognized open source text empathy datasets only contain more than a thousand data points, and the existing mainstream empathy prediction methods all perform empathy prediction on empathy datasets alone. Obviously, when the amount of data is insufficient, the generalization ability of the trained model will be relatively poor, and its prediction accuracy will not be too high. In sharp contrast, some other sentiment analysis tasks have very sufficient training data. Taking the task of sentiment polarity classification as an example, this task currently corresponds to a large number of open source datasets, which contain tens of thousands or even hundreds of thousands of training samples. Therefore, the poor accuracy of empathy prediction in text processing due to the small amount of data in open source text empathy datasets is a problem that needs to be solved.

In view of this, the present invention is proposed.

Summary of the invention

The purpose of the present invention is to provide a text empathy prediction system and method, which can utilize text sentiment classification data with a large amount of sample data to assist in text empathy prediction, improve the accuracy of empathy prediction in text processing, and thus solve the above-mentioned technical problems existing in the prior art.

The objective of the present invention is achieved through the following technical solutions:

An embodiment of the present invention provides a text empathy prediction system, comprising:

Empathy private feature encoder, polarity private feature encoder, public feature encoder, empathy public-private feature fusion module, polarity public-private feature fusion module, empathy predictor and polarity classifier; wherein,

The input of the empathy private feature encoder is the empathy data in the empathy data set, and the private features of the input empathy data can be encoded to obtain the empathy private features;

The polarity private feature encoder is inputted with polarity data in the polarity data set, and can encode the private features of the inputted polarity data to obtain the polarity private features;

The public feature encoder receives the empathy data in the empathy data set and the polarity data in the polarity data set respectively, and can encode the public features of the empathy data to obtain empathy public features, and encode the public features of the polarity data to obtain polarity public features;

The empathy public-private feature fusion module is connected to the output end of the empathy private feature encoder and the output end of the public feature encoder respectively, and can weightedly fuse the empathy private features output by the empathy private feature encoder and the empathy public features output by the public feature encoder into a final empathy prediction feature expression;

The polarity public-private feature fusion module is connected to the output end of the polarity private feature encoder and the output end of the public feature encoder respectively, and can weightedly fuse the polarity private features output by the polarity private feature encoder and the polarity public features output by the public feature encoder into a final polarity classification feature expression;

The empathy predictor is connected to the output end of the empathy public and private feature fusion module, and can predict the final empathy prediction feature expression to obtain the corresponding empathy label;

The polarity classifier is connected to the output end of the polarity public-private feature fusion module, and can predict the final polarity classification feature expression to obtain the corresponding polarity label.

The embodiment of the present invention further provides a text empathy prediction method, which uses the text empathy prediction system of the present invention as a text empathy prediction model, including:

Step 1, encoding the private features of the empathy data in the input empathy data set to obtain the empathy private features, and encoding the private features of the polarity data in the input polarity data set to obtain the polarity private features;

Encoding the public features of the empathy data in the input empathy data set to obtain empathy public features, and encoding the public features of the polarity data in the input polarity data set to obtain polarity public features;

Step 2, weighting and fusing the empathy private features and the empathy public features obtained in step 1 into a final empathy prediction feature expression; and weighting and fusing the polarity private features and the polarity public features obtained in step 1 into a final polarity prediction feature expression;

Step 3: Perform empathy prediction on the final empathy prediction feature expression obtained in step 2 to obtain a corresponding empathy label as a prediction result; and perform polarity classification on the final polarity classification feature expression to obtain a corresponding polarity label as a prediction result.

Compared with the prior art, the text empathy prediction system and method provided by the present invention have the following beneficial effects:

By using a sentiment polarity dataset containing a large amount of sample data to assist a text empathy dataset with a small amount of sample data to perform empathy prediction, and by performing transfer learning between two tasks with different prediction labels and different dataset fields, the interference caused by different dataset fields and different prediction labels can be eliminated at the same time. The method of the present invention introduces transfer learning into text empathy prediction for the first time, and currently has better performance than other methods in the field of text empathy prediction. Because a large-scale sentiment polarity dataset is used to assist an empathy dataset, a better empathy prediction effect is achieved.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

FIG1 is a schematic diagram of the structure of a text empathy prediction system provided by an embodiment of the present invention;

FIG2 is a schematic diagram of another text empathy prediction system provided by an embodiment of the present invention;

FIG3 is a schematic diagram of the structure of another text empathy prediction system provided by an embodiment of the present invention;

FIG4 is a schematic diagram of the structure of an adversarial classification and anti-interference text empathy prediction system provided by an embodiment of the present invention.

DETAILED DESCRIPTION

The following is a clear and complete description of the technical solutions in the embodiments of the present invention in combination with the specific content of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, which does not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

First, the terms that may be used in the present invention are explained as follows:

The term “and/or” means that either or both of them can be realized at the same time. For example, X and/or Y means both “X” or “Y” and “X and Y”.

The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.

The text empathy prediction system and method provided by the present invention are described in detail below. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in the field. If no specific conditions are specified in the embodiments of the present invention, the conventional conditions in the field or the conditions recommended by the manufacturer are followed. The reagents or instruments used in the embodiments of the present invention, for which the manufacturer is not specified, are all conventional products that can be purchased commercially.

As shown in FIG1 , an embodiment of the present invention provides a text empathy prediction system, including:

Empathy private feature encoder, polarity private feature encoder, public feature encoder, empathy public-private feature fusion module, polarity public-private feature fusion module, empathy predictor and polarity classifier; wherein,

The input of the empathy private feature encoder is the empathy data in the empathy data set, and the private features of the input empathy data can be encoded to obtain the empathy private features;

The polarity private feature encoder is inputted with polarity data in the polarity data set, and can encode the private features of the inputted polarity data to obtain the polarity private features;

The public feature encoder receives the empathy data in the empathy data set and the polarity data in the polarity data set respectively, and can encode the public features of the empathy data to obtain empathy public features, and encode the public features of the polarity data to obtain polarity public features;

The empathy public-private feature fusion module is connected to the output end of the empathy private feature encoder and the output end of the public feature encoder respectively, and can weightedly fuse the empathy private features output by the empathy private feature encoder and the empathy public features output by the public feature encoder into a final empathy prediction feature expression;

The polarity public-private feature fusion module is connected to the output end of the polarity private feature encoder and the output end of the public feature encoder respectively, and can weightedly fuse the polarity private features output by the polarity private feature encoder and the polarity public features output by the public feature encoder into a final polarity classification feature expression;

The empathy predictor is connected to the output end of the empathy public and private feature fusion module, and can predict the final empathy prediction feature expression to obtain the corresponding empathy label;

The polarity classifier is connected to the output end of the polarity public-private feature fusion module, and can predict the final polarity classification feature expression to obtain the corresponding polarity label.

As shown in Figure 2, in the above text empathy prediction system,

The empathy private feature encoder also inputs polarity data in the polarity data set, and can encode the private features of the input polarity data to obtain empathy polarity private features;

The polarity private feature encoder also inputs the empathy data in the empathy data set, and can encode the private features of the input empathy data to obtain the polarity empathy private features;

It also includes: a domain binary classifier, the input end of which is connected to the output end of the empathy private feature encoder, the polarity private feature encoder and the public feature encoder respectively, and can perform binary classification processing on the public feature code output by the public feature encoder in an adversarial classification loss manner;

As shown in Figure 3, in the above text empathy prediction system,

The empathy private feature encoder also inputs polarity data in the polarity data set, and can encode the private features of the input polarity data to obtain empathy polarity private features;

The polarity private feature encoder also inputs the empathy data in the empathy data set, and can encode the private features of the input empathy data to obtain the polarity empathy private features;

It also includes: an empathy hinge loss module and a polarity hinge loss module; wherein,

The empathy hinge loss module is connected to the output end of the empathy predictor, and can perform empathy prediction when the difference between the empathy prediction result _Lem corresponding to the correct splicing result and the empathy prediction result _Lem ' corresponding to the incorrect splicing result is less than a preset difference, the correct splicing result refers to the splicing result of the empathy private feature and the empathy public feature, and the incorrect splicing result refers to the splicing result of the polarity private feature and the empathy private feature;

The polarity hinge loss module is connected to the output end of the polarity classifier, and can perform polarity prediction when the difference between the polarity prediction result _Lem corresponding to the correct splicing result and the polarity prediction result _Lem ' corresponding to the incorrect splicing result is less than a preset difference, the correct splicing result refers to the splicing result of the polarity private feature and the polarity public feature, and the incorrect splicing result refers to the splicing result of the empathy private feature and the polarity private feature.

As shown in Figure 4, in the above text empathy prediction system,

The empathy private feature encoder also inputs polarity data in the polarity data set, and can encode the private features of the input polarity data to obtain empathy polarity private features;

The polarity private feature encoder also inputs the empathy data in the empathy data set, and can encode the private features of the input empathy data to obtain the polarity empathy private features;

It also includes: a domain binary classifier, the input end of which is connected to the output end of the empathy private feature encoder, the polarity private feature encoder and the public feature encoder respectively, and can perform binary classification processing on the public feature code output by the public feature encoder in an adversarial classification loss manner;

and/or, an empathy hinge loss module and a polarity hinge loss module; wherein,

The empathy hinge loss module is connected to the output end of the empathy predictor, and can perform empathy prediction when the difference between the empathy prediction result _Lem corresponding to the empathy correct splicing result and the empathy prediction result _Lem ' corresponding to the empathy incorrect splicing result is less than a preset difference, the empathy correct splicing result refers to the splicing result of the empathy private feature and the empathy public feature, and the empathy incorrect splicing result refers to the splicing result of the polarity private feature and the empathy private feature; this empathy hinge loss module can enlarge the difference between the empathy prediction result _Lem corresponding to the empathy correct splicing result and the empathy prediction result _Lem ' corresponding to the empathy incorrect splicing result;

The polarity hinge loss module is connected to the output end of the polarity classifier, and can perform polarity prediction when the difference between the polarity prediction result _Lem corresponding to the polarity correct splicing result and the polarity prediction result _Lem ' corresponding to the polarity incorrect splicing result is less than a preset difference, the polarity correct splicing result refers to the splicing result of the polarity private feature and the polarity public feature, and the polarity incorrect splicing result refers to the splicing result of the empathy private feature and the polarity private feature. This polarity hinge loss module can expand the difference between the polarity prediction result _Lpo corresponding to the polarity correct splicing result and the polarity prediction result _Lpo ' corresponding to the polarity incorrect splicing result.

In the above text empathy prediction system, the domain binary classifier adopts a fully connected network.

In the above-mentioned text empathy prediction system, the empathy private feature encoder adopts a Bi-LSTM type or a BERT type feature encoder;

The polarity private feature encoder adopts a Bi-LSTM type or a BERT type feature encoder;

The common feature encoder adopts a Bi-LSTM type or a BERT type feature encoder;

The empathy public-private feature fusion module and the polarity public-private feature fusion module both use an attention network module;

The empathy predictor adopts a fully connected network;

The polarity classifier adopts a fully connected network.

The systems shown in Figures 2, 3, and 4 above reduce the interference caused by differences in data set domains by designing adversarial classification losses, and reduce the interference caused by different labels by setting empathy and polarity hinge loss modules.

The embodiment of the present invention further provides a text empathy prediction method, which uses the above-mentioned text empathy prediction system as a text empathy prediction model, including:

Step 1, encoding the private features of the empathy data in the input empathy data set to obtain the empathy private features, and encoding the private features of the polarity data in the input polarity data set to obtain the polarity private features;

Encoding the public features of the empathy data in the input empathy data set to obtain empathy public features, and encoding the public features of the polarity data in the input polarity data set to obtain polarity public features;

Step 2, weighting and fusing the empathy private features and the empathy public features obtained in step 1 into a final empathy prediction feature expression; and weighting and fusing the polarity private features and the polarity public features obtained in step 1 into a final polarity prediction feature expression;

Step 3: Perform empathy prediction on the final empathy prediction feature expression obtained in step 2 to obtain a corresponding empathy label as a prediction result; and perform polarity classification on the final polarity classification feature expression to obtain a corresponding polarity label as a prediction result.

In step 2 of the above method, the final empathy prediction feature expression _fem and the final polarity classification feature expression _fpo obtained by weighted fusion are respectively:

In the above formulas (1) and (2), q is q _k *U(S _em ), where q _k is a hyperparameter vector with a dimension of R ^1*d , q _k is a hyperparameter vector, which is randomly initialized and used to obtain the initial value Q; T is the transposed matrix, that is, U(s _em ) ^T is the transposed matrix of U(s _em ), which is convenient for matrix multiplication operations; d is the dimension of the feature vector; U(s _em ) is the concatenation result of the empathy private feature e _em (s _em ) and the empathy public feature e _c (s _em ), U(s _em )∈R ^2*d ; s _em is the empathy data in the empathy dataset; V(s _po ) is the concatenation result of the polarity private feature e _po (s _po ) and the polarity public feature e _c (s _po ), V(s _po )∈R ^2*d , s _po is the polarity data in the polarity dataset;

In step 3, the final empathy prediction feature expression _fem is input into the empathy predictor _gem () to predict the empathy label according to formula (3). Formula (3) is:

The final polarity prediction feature expression _fpo is input into the polarity classifier _gpo (.) and the predicted polarity label is obtained according to formula (4): Formula (4) is:

In step 3 of the above method, if the empathy task is a regression task, the training loss function of the empathy predictor is formula (5):

In the above formula (5), θ _eem , θ _ec , θ _gem are the parameters of the empathy private feature encoder e _em (), the public feature encoder e _c (), and the empathy predictor g _em () respectively; is the predicted empathy label; em ^* is the true empathy label;

The θ _eem , θ _ec , and θ _gem are parameters set according to the specific model framework of the empathy private feature encoder e _em (), the public feature encoder e _c (), and the empathy predictor g _em (). For example, if the encoder is a Bi-LSTM network, it will correspond to a set of θ _eem , θ _ec , and θ _gem parameters; if the encoder is other network models, it will correspond to another set of θ _eem , θ _ec , and θ _gem parameters.

If the empathy task is a classification task, the training loss function of the empathy predictor is formula (6):

In the above formula (6), θ _eem , θ _ec , θ _gem are the parameters of the empathy private feature encoder e _em (), the public feature encoder e _c (), and the empathy predictor g _em () respectively; is the predicted empathy label; em ^* is the real empathy label; N is the number of empathy label types when the empathy task is a classification task;

In step 3 of the above method, the training loss function of the polarity classifier is:

In the above formula (7), θ _epo , θ _ec , and θ _gpo are the parameters of the polarity private feature encoder e _po (), the public feature encoder e _c (), and the polarity classifier g _po () respectively; is the predicted polarity label; po ^* is the true polarity label.

The θ _epo , θ _ec , θ _gpo are respectively parameters of the polarity private feature encoder e _po (), the public feature encoder e _c (), and the polarity classifier g _po (), which are similar to the determination method of the above-mentioned θ _eem , θ _ec , θ _gem parameters and will not be repeated here.

The method further includes using a domain binary classifier g _l () to discriminate the source domains of the empathy private features obtained by the empathy private feature encoder e _em () and the polarity private features obtained by the polarity private feature encoder e _po () in an adversarial loss manner, and the training loss function of the domain binary classifier g _l () is L _cop :

And the domain binary classifier g _l () is used to discriminate the source domains of the empathy public features and polarity public features obtained by the common feature encoder e _c () in an adversarial loss manner. The training loss function of the domain binary classifier g _l () is L _adv :

And/or, when the difference between the prediction result _Lem corresponding to the empathy correct splicing result output by the empathy prediction and the prediction result _Lem ' corresponding to the empathy incorrect splicing result is less than a preset difference, an empathy prediction is performed by an empathy hinge loss module, wherein the empathy correct splicing result refers to the splicing result of the empathy private feature and the empathy public feature, and the empathy incorrect splicing result refers to the splicing result of the polarity private feature and the empathy private feature; and when the difference between the prediction result _Lem corresponding to the polarity correct splicing result output by the empathy prediction and the prediction result _Lem ' corresponding to the polarity incorrect splicing result is less than a preset difference, a polarity prediction is performed by a polarity hinge loss module, wherein the polarity correct splicing result refers to the splicing result of the polarity private feature and the polarity public feature, and the polarity incorrect splicing result refers to the splicing result of the empathy private feature and the polarity private feature; wherein the training loss function (10) of the empathy hinge loss module is:

In the above formula (10), the meanings of the parameters are as follows: _Lem is the empathy loss value (i.e., empathy loss value) corresponding to the correct empathy splicing result; _Lem ' is the empathy loss value (i.e., empathy loss value) corresponding to the incorrect empathy splicing result; _δ1 is the difference between the two different empathy splicing results that need to be set;

The training loss function (11) of the polar hinge loss module is:

In the above formula (11), the meanings of the parameters are as follows: _Lpo is the polarity loss value (i.e., polarity loss value) corresponding to the polarity correct splicing result; _Lpo ' is the polarity loss value (i.e., polarity loss value) corresponding to the polarity incorrect splicing result; _δ2 is the difference that needs to be achieved between the two different splicing results of the set polarity.

The final hinge loss function (12) of the text empathy prediction model is:

L _hin =L _{hin_1} +L _{hin_2} (12);

The final loss function L _o of the text empathy prediction model is:

L _o =λ ₁ *L _em +λ ₂ *L _po +λ ₃ *L _cop +λ ₄ *L _adv +λ ₅ *L _hin (13);

In the above formula (13), λ ₁ , λ ₂ , λ ₃ , λ ₄ , and λ ₅ are weights of each loss function _Lem , _Lpo , _Lcop , _Ladv , and _Lhin . In specific experiments, the weight values of λ ₁ , λ ₂ , λ ₃ , λ 4 , and λ ₅ are 1, 0.5, 2, _{2, and} 1.5, respectively.

The above-mentioned final loss function L _o is obtained by weighted average of _Lem , _Lpo , _Lcop , _Ladv , and _Lhin , and is the final loss function of each module.

The above method uses adversarial classification loss to reduce the interference caused by differences in dataset domains, and uses hinge loss to reduce the interference caused by different labels. The adversarial classification loss to reduce the differences in dataset domains is reflected in step 5, and the corresponding loss functions are (8) and (9); the hinge loss method to reduce the interference caused by different labels is reflected in the loss functions (10), (11), and (12).

Since polarity and empathy are both sub-attributes of emotion, and the size of the label values of both are relatively dependent on the emotional words that appear in the text, there is a large correlation between text polarity data and text empathy data. The amount of data contained in the data set in the field of text polarity data is often very large, reaching tens of thousands or even hundreds of thousands of data. Compared with the data set of text empathy data that only contains one or two thousand data, it has the advantage of large data volume. The present invention is based on a text empathy prediction model and method assisted by text sentiment classification, and uses polarity data to assist empathy data through transfer learning to achieve better results.

In summary, it can be seen that the text empathy prediction system and method based on the assistance of text sentiment classification in the embodiments of the present invention uses text sentiment classification to assist text empathy analysis. It not only considers the difference in the data set fields between the two tasks, but also considers the difference in the label fields between the two tasks, thereby achieving more accurate text empathy prediction results.

In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, the text empathy prediction model and method based on text sentiment classification assistance provided by the embodiments of the present invention are described in detail with specific embodiments below.

Example

As shown in FIG1 , an embodiment of the present invention provides a text empathy prediction system, which is a model that can assist small-scale text empathy prediction through large-scale text sentiment classification data based on transfer learning. Specifically, since the empathy data set has very little empathy data, it is impossible to support the empathy predictor to train a neural network with strong generalization ability. Therefore, the empathy prediction is assisted by learning transferable common features from polarity classification tasks with a large amount of polarity data. The method for the text empathy prediction model to perform this empathy prediction is as follows:

The data sets and related definitions involved are:

The empathy data set is set as _Dem , _Dem = {( _sem1 , _em1 ), ..., ⁽ _semn , ^emn ), ( _semN , ^emN )}; where _semn ( ¹ <= ⁿ <= N) and ^emn represent the input ^text and empathy label of the nth empathy data respectively;

The polarity data set is set as D _po , D _po ={(s _po ¹ ,po ¹ ),...,(s _po ^m ,po ^m ),...,(s _po ^M ,po ^M )}; where s _po ^m (1<=m<=M) and em ^m ∈{0,1} represent the input text and polarity label of the mth polarity data respectively;

The total number of polarity data M in the polarity data set is much larger than the total number of empathy data N in the empathy data set.

The main network framework of the text empathy prediction model of the present invention is shown in FIG1 . The encoder part mainly includes three feature encoders, namely, e _em (), e _po (), and e _c (), wherein e _em () and e _po () represent the private feature encoders of the two tasks of empathy prediction and polarity classification, which are used to encode the private features of the two tasks, and e _c () represents the public feature encoder, which is used to encode the public features between the two tasks. Assuming that the empathy data in the input sample pair is denoted as s _em and the polarity data is denoted as s _po , the final encoding results are e _em (s _em ), e _po (s _em ), e _c (s _em ), e _em (s _po ), e _po (s _po ), e _c (s _po ).

In the subsequent network architecture, since the contributions of public and private features to empathy prediction may be inconsistent, the present invention first weights the public and private features through the two public and private feature fusion modules of the attention architecture; thereby, the public and private features are fused into the final feature expression through dynamic weighting, and the corresponding labels are predicted; at the same time, in view of the interference caused by the differences in data set domains and label evaluation between the two tasks of empathy prediction and polarity classification, the present invention further reduces the interference caused by the differences in data set domains through adversarial classification loss, and reduces the interference caused by different labels by setting a hinge loss module (i.e., Hinge-loss module, including empathy hinge loss module and polarity hinge loss module). As a result, the transferable features learned by the feature encoder can be applied to different fields and labels, further disentangling the public and private features between the two tasks obtained.

The loss function of the above-mentioned hinge loss module is generally loss _final = max(0,δ-loss), which means: when the original loss is small, loss _final is not 0; when the loss is large, loss _final is 0; corresponding to the present invention, when the gap between the correct splicing result and the wrong combination method is very small, it means that the feature separation is not complete at this time, so loss _final is not 0, and the hinge loss module works; and if the gap between the correct splicing result and the wrong splicing result is relatively large, it means that the feature separation effect is better at this time, loss _final is 0, and the hinge loss module does not work.

Specifically, the detailed description of each module is as follows:

(i) For the input empathy data s _em and polarity data s _po , the empathy private features obtained by the empathy data s _em through the empathy private feature encoder and the empathy public features obtained by the public feature encoder are e _em (s _em ) and e _c (s _em ) respectively; while the polarity private features obtained by the polarity data s _po through the polarity private feature encoder and the polarity public features obtained by the public feature encoder are e _po (s _po ) and e _c (s _po ) respectively. Since the benefits of domain private features and public features for the final label prediction may be different, the present invention dynamically weights the public and private features respectively through two public and private feature fusion modules of the attention architecture, thereby obtaining the final feature expressions _fem and f _po after the fusion of public and private features. The specific process is shown in formulas (1) and (2):

In the above formulas (1) and (2), d is the dimension of the feature vector; U(s _em )∈R ^2*d , which is the concatenation result of e _em (s _em ) and e _c (s _em ); and V(s _po )∈R ^2*d , which is the concatenation result of e _po (s _po ) and e _c (s _po );

After obtaining the final feature expressions of the empathy prediction task and the polarity classification task, the final feature expressions are respectively sent to the empathy predictor and the polarity classifier to obtain the corresponding prediction labels, namely, the empathy label and the polarity label; wherein _fem is input into the empathy predictor _gem (), and _fpo is input into the polarity classifier _gpo (). The specific processing formulas are (3) and (4):

In the above formulas (3) and (4), em is the empathy label; po is the polarity label; em ^* is the true empathy label, and po ^* is the true polarity label.

If the empathy task is a regression task, its training loss function is as shown in formula (5); if it is a classification task, its training loss function is as shown in formula (6):

The training loss function for polarity classification is shown in formula (7):

In the above formulas (5), (6) and (7), θ _eem , θ _epo , θ _ec , θ _gem , θ _gpo are the parameters of e _em (), e _po (), e _c (), g _em (), g _po (), respectively; N is the number of empathy label types when the empathy task is a classification task.

(ii) The present invention separates the learned features into domain common features and domain unique features by means of adversarial learning, and improves the common features encoded by the common feature encoder, which is helpful and universal for any data set domain. Specifically, the present invention uses a domain binary classifier g _l () to discriminate the source domain of the encoded features of e _em (), e _po (), and e _c (). Among them, for the encoded features of the two private feature encoders e _em () and e _po (), g _l () should be able to distinguish whether these features come from the empathy domain or the polarity domain, because these private features are unique features of the two domains. The corresponding training process is shown in formula (8), and the corresponding loss function is L _cop :

As for the encoding features of the common feature encoder e _c (), no matter whether the source domain is the empathy domain or the polarity domain, since the common features should be common features of both tasks, g _l () should not be able to distinguish the sources of these features, that is, it will confuse the sources of these features. The specific training loss function is shown in formula (9), and the corresponding loss function is L _adv :

(3) The present invention proposes a label training strategy to eliminate the label difference between the empathy task and the polarity task. Specifically, taking empathy prediction as an example, some transferable features are useful for empathy prediction, while some features are useless. The present invention extracts useful features through a common feature encoder, while invalid features are extracted by a private feature extractor of another task. Based on this, the present invention proposes to use a hinge loss module (i.e., Hinge-loss module, divided into an empathy hinge loss module and a polarity hinge loss module) to maximize the difference between the experimental results corresponding to the common feature encoder and the empathy private feature encoder and the experimental results corresponding to the polarity private feature encoder and the empathy private feature encoder.

Specifically, in formulas (1) and (2), _fem is the final fusion feature, and the corresponding loss is _Lem , which is the experimental result corresponding to the correct splicing result; in addition, the present invention defines _fem '= _eem ( _sem )+ _epo ( _sem ), and the prediction label based on this is em'= _gem ( _fem '), thereby obtaining a new empathy loss _Lem ', _which is the experimental result corresponding to the wrong splicing result; the empathy hinge loss module maximizes the difference between _Lem and _Lem ' so that the features that are helpful for empathy prediction are more concentrated in the public feature encoder, while those useless features are more concentrated in the private feature encoder of another task; the implementation method of the polarity hinge loss module of the polarity classification task is consistent with that of the empathy prediction, so the training objective and loss function of the final empathy hinge loss module are shown in formula (10):

In the above formula (10), the meanings of the parameters are as follows: _Lem is the empathy loss value (i.e., empathy loss value) corresponding to the correct empathy splicing result; _Lem ' is the empathy loss value (i.e., empathy loss value) corresponding to the incorrect empathy splicing result; _δ1 is the difference between the two different empathy splicing results that need to be set;

The training loss function (11) of the polar hinge loss module is:

In the above formula (11), the meanings of the parameters are as follows: _Lpo is the polarity loss value (i.e., polarity loss value) corresponding to the polarity correct splicing result; _Lpo ' is the polarity loss value (i.e., polarity loss value) corresponding to the polarity incorrect splicing result; _δ2 is the difference that needs to be achieved between the two different splicing results of the set polarity.

The final hinge loss function (12) of the text empathy prediction model is:

L _hin =L _{hin_1} +L _{hin_2} (12);

The final loss function L _o of the text empathy prediction model is the weighted average of _Lem , _Lpo , _Lcop , _Ladv and _Lhin , as shown in formula (13):

L _o =λ ₁ *L _em +λ ₂ *L _po +λ ₃ *L _cop +λ ₄ *L _adv +λ ₅ *L _hin (13);

Among them, λ ₁ , λ ₂ , λ ₃ , λ ₄ , and λ ₅ correspond to the weights of each loss function and are used to control the balance between multiple loss functions. In the specific experiment, these weights λ ₁ , λ ₂ , λ ₃ , λ ₄ , and λ ₅ are 1, 0.5, 2, 2, and 1.5, respectively.

The model and method of the present invention were experimented on two data sets, and preliminary experimental results were obtained. The experimental results proved the effectiveness of the method proposed in the present invention.

In terms of empathy data sets, the empathy data set proposed by Buchel includes 1,860 annotated data, which mainly comes from the annotation personnel's reading experience of various news. Each data sample includes two empathy labels, EC and PD, and the value range of these two labels is 1-7. The empathy data set proposed by Zhou includes 1,000 annotated data, which comes from the Reddit forum. The main content is the user's posting on the forum and the reply to the corresponding post. Each data sample is marked with an empathy label, and the label value range is 1-5.

The former mainly includes various tweets posted by users on Twitter, while the latter mainly includes users' impressions and comments on various types of movies. The former contains 7061 positive samples and 3240 negative samples; the latter includes 25,000 positive samples and 25,000 negative samples.

In order to ensure the uniformity of the evaluation criteria, for the empathy dataset proposed by Buchel, the Pearson correlation coefficient (PCC) is also used as the evaluation annotation according to the original evaluation criteria in the dataset. Since the dataset has two labels, EC and PD, the evaluation criteria are PCC-EC and PCC-PD; and for the empathy dataset proposed by Zhou, MSE loss and R2 are used as the evaluation criteria according to the original evaluation criteria in the dataset. Since the work involved is transfer learning, only the experimental results on the empathy prediction dataset need to be considered, and the experimental results on the polarity classification dataset will not be discussed in the following analysis.

In terms of experimental parameters, two encoders are used for the encoder part, namely Bi-LSTM and BERT. When Bi-LSTM is used as an encoder, the hidden dimension of its forward LSTM and backward LSTM is set to 200. In order to maintain consistency, the bert-base-uncased model in BERT is directly used as the baseline encoding model. When Bi-LSTM is used as an encoder, the learning rate is set to 0.001, and when BERT is used as an encoder, the learning rate is set to 0.00002, and the attenuation coefficient is uniformly set to 0.95. The dropout rate is set to 0.3, the batch size of training is 16, and the regularization method is L2 regularization. During the training process, the samples in the polarity dataset are combined with the samples in the empathy dataset and input into the network architecture in pairs. The experimental framework is Pytorch, and the optimizer uses the Adam optimizer.

The comparison with related works is mainly divided into two categories: the works without transfer learning, that is, the methods that only use empathy data for empathy prediction, such as FNN, CNN, RoBERTa, etc.; the second category of comparison work is the other related works that use empathy data for transfer learning, such as DATNet, ADV-SA, etc., which are also compared. The experimental results are shown in the table.

Table 4 Experimental results based on comparison with related work

It can be found from Table 4 that the experimental results of the model and method of the present invention have achieved the best experimental results in the field at present, whether on Buchel's empathy dataset or Zhou's empathy dataset. Specifically speaking, compared with the work without transfer learning, such as CNN, FNN, RoBERTa, BERT, Random Forest, etc., the performance of the model proposed in this article is significantly better. This is because the transfer learning model proposed in this article can help a small-scale empathy analysis dataset learn better common feature expressions through a large-scale polarity classification dataset, thereby making the performance of empathy prediction better. In addition, compared with the work of transfer learning such as DATNet and ADV-SA, the experimental results of the model proposed in this article are also better. This is because the model proposed in this article not only reduces the interference caused by the domain differences between the two tasks through adversarial learning; it also reduces the interference caused by the label differences between the two tasks by designing hinge-loss, so that the learned transferable common features are effective for different fields and different labels.

The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a technician familiar with the technical field within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or in any form that the information constitutes prior art known to those skilled in the art.

Claims (9)

1. A text empathy prediction system, comprising: Empathy private feature encoder, polarity private feature encoder, public feature encoder, empathy public-private feature fusion module, polarity public-private feature fusion module, empathy predictor and polarity classifier; wherein, The input of the empathy private feature encoder is the empathy data in the empathy data set, and the private features of the input empathy data can be encoded to obtain the empathy private features; The polarity private feature encoder is inputted with polarity data in the polarity data set, and can encode the private features of the inputted polarity data to obtain the polarity private features; The public feature encoder receives the empathy data in the empathy data set and the polarity data in the polarity data set respectively, and can encode the public features of the empathy data to obtain empathy public features, and encode the public features of the polarity data to obtain polarity public features; The empathy public-private feature fusion module is connected to the output end of the empathy private feature encoder and the output end of the public feature encoder respectively, and can weightedly fuse the empathy private features output by the empathy private feature encoder and the empathy public features output by the public feature encoder into a final empathy prediction feature expression; The polarity public-private feature fusion module is connected to the output end of the polarity private feature encoder and the output end of the public feature encoder respectively, and can weightedly fuse the polarity private features output by the polarity private feature encoder and the polarity public features output by the public feature encoder into a final polarity classification feature expression; The empathy predictor is connected to the output end of the empathy public and private feature fusion module, and can predict the final empathy prediction feature expression to obtain the corresponding empathy label; The polarity classifier is connected to the output end of the polarity public-private feature fusion module, and can predict the final polarity classification feature expression to obtain the corresponding polarity label. 2. The text empathy prediction system according to claim 1, characterized in that: The empathy private feature encoder also inputs polarity data in the polarity data set, and can encode the private features of the input polarity data to obtain empathy polarity private features; The polarity private feature encoder also inputs the empathy data in the empathy data set, and can encode the private features of the input empathy data to obtain the polarity empathy private features; It also includes: a domain binary classifier, the input end of which is connected to the output end of the empathy private feature encoder, the polarity private feature encoder and the public feature encoder respectively, and can perform binary classification processing on the public feature code output by the public feature encoder in an adversarial classification loss manner; and/or, an empathy hinge loss module and a polarity hinge loss module; wherein, The empathy hinge loss module is connected to the output end of the empathy predictor, and can perform empathy prediction when the difference between the empathy prediction result _Lem corresponding to the correct splicing result and the empathy prediction result _Lem ' corresponding to the incorrect splicing result is less than a preset difference, the correct splicing result refers to the splicing result of the empathy private feature and the empathy public feature, and the incorrect splicing result refers to the splicing result of the polarity private feature and the empathy private feature; The polarity hinge loss module is connected to the output end of the polarity classifier, and can perform polarity prediction when the difference between the polarity prediction result _Lem corresponding to the correct splicing result and the polarity prediction result _Lem ' corresponding to the incorrect splicing result is less than a preset difference, the correct splicing result refers to the splicing result of the polarity private feature and the polarity public feature, and the incorrect splicing result refers to the splicing result of the empathy private feature and the polarity private feature. 3. The text empathy prediction system according to claim 2, characterized in that: The polarity private feature encoder adopts a Bi-LSTM type or a BERT type feature encoder; The common feature encoder adopts a Bi-LSTM type or a BERT type feature encoder; The empathy public-private feature fusion module and the polarity public-private feature fusion module both use an attention network module; The empathy predictor adopts a fully connected network; The polarity classifier adopts a fully connected network; The domain binary classifier adopts a fully connected network. 4. The text empathy prediction system according to claim 1 or 2, characterized in that the empathy private feature encoder adopts a Bi-LSTM type or a BERT type feature encoder; The polarity private feature encoder adopts a Bi-LSTM type or a BERT type feature encoder; The common feature encoder adopts a Bi-LSTM type or a BERT type feature encoder; The empathy public-private feature fusion module and the polarity public-private feature fusion module both use an attention network module; The empathy predictor adopts a fully connected network; The polarity classifier adopts a fully connected network. 5. A text empathy prediction method, characterized in that the text empathy prediction system according to any one of claims 1 to 4 is used as a text empathy prediction model, comprising: Step 1, encoding the private features of the empathy data in the input empathy data set to obtain the empathy private features, and encoding the private features of the polarity data in the input polarity data set to obtain the polarity private features; Encoding the public features of the empathy data in the input empathy data set to obtain empathy public features, and encoding the public features of the polarity data in the input polarity data set to obtain polarity public features; Step 2, weighting and fusing the empathy private features and the empathy public features obtained in step 1 into a final empathy prediction feature expression; and weighting and fusing the polarity private features and the polarity public features obtained in step 1 into a final polarity prediction feature expression; Step 3: Perform empathy prediction on the final empathy prediction feature expression obtained in step 2 to obtain a corresponding empathy label as a prediction result; and perform polarity classification on the final polarity classification feature expression to obtain a corresponding polarity label as a prediction result. 6. The text empathy prediction method according to claim 5, characterized in that, in step 2, the final empathy prediction feature expression _fem and the final polarity classification feature expression _fpo obtained by weighted fusion are respectively: In the above formulas (1) and (2), q is q _k *U(S _em ), where q _k is a hyperparameter vector with a dimension of R ^1*d , q _k is a hyperparameter vector, and the initial value Q is obtained by random initialization; T is a transposed matrix, that is, U(s _em ) ^T is the transposed matrix of U(s _em ); d is the dimension of the feature vector; U(s _em ) is the concatenation result of the empathy private feature e _em (s _em ) and the empathy public feature e _c (s _em ), U(s _em )∈R ^2*d ; s _em is the empathy data in the empathy dataset; V(s _po ) is the concatenation result of the polarity private feature e _po (s _po ) and the polarity public feature e _c (s _po ), V(s _po )∈R ^2*d , s _po is the polarity data in the polarity dataset; In step 3, the final empathy prediction feature expression _fem is input into the empathy predictor _gem () to predict the empathy label according to formula (3). Formula (3) is: The final polarity prediction feature expression _fpo is input into the polarity classifier _gpo (.) and the polarity label is predicted according to formula (4): Formula (4) is: 7. The text empathy prediction method according to claim 6, characterized in that in step 3, if the empathy task is a regression task, the training loss function of the empathy predictor is formula (5): In the above formula (5), θ _eem , θ _ec , θ _gem are the parameters of the empathy private feature encoder e _em (), the public feature encoder e _c (), and the empathy predictor g _em () respectively; is the predicted empathy label; em ^* is the true empathy label; If the empathy task is a classification task, the training loss function of the empathy predictor is formula (6): In the above formula (6), θ _eem , θ _ec , θ _gem are the parameters of the empathy private feature encoder e _em (), the public feature encoder e _c (), and the empathy predictor g _em () respectively; is the predicted empathy label; em ^* is the actual empathy label; N is the number of empathy label types when the empathy task is a classification task. 8. The text empathy prediction method according to claim 6, characterized in that in step 3, the training loss function of the polarity classifier is: In the above formula (7), θ _epo , θ _ec , and θ _gpo are the parameters of the polarity private feature encoder e _po (), the public feature encoder e _c (), and the polarity classifier g _po () respectively; is the predicted polarity label; po ^* is the true polarity label. 9. The text empathy prediction method according to claim 6, characterized in that the method further comprises using a domain binary classifier g _l () to discriminate the source domains of the empathy private features obtained by the empathy private feature encoder e _em () and the polarity private features obtained by the polarity private feature encoder e _po () in an adversarial loss manner, and the training loss function of the domain binary classifier g _l () is L _cop : And the domain binary classifier g _l () is used to discriminate the source domains of the empathy public features and polarity public features obtained by the common feature encoder e _c () in an adversarial loss manner. The training loss function of the domain binary classifier g _l () is L _adv : And/or, when the difference between the prediction result _Lem corresponding to the empathy correct splicing result output by the empathy prediction and the prediction result _Lem ' corresponding to the empathy incorrect splicing result is less than a preset difference, an empathy prediction is performed by an empathy hinge loss module, wherein the empathy correct splicing result refers to the splicing result of the empathy private feature and the empathy public feature, and the empathy incorrect splicing result refers to the splicing result of the polarity private feature and the empathy private feature; and when the difference between the prediction result _Lem corresponding to the polarity correct splicing result output by the empathy prediction and the prediction result _Lem ' corresponding to the polarity incorrect splicing result is less than a preset difference, a polarity prediction is performed by a polarity hinge loss module, wherein the polarity correct splicing result refers to the splicing result of the polarity private feature and the polarity public feature, and the polarity incorrect splicing result refers to the splicing result of the empathy private feature and the polarity private feature; wherein the training loss function (10) of the empathy hinge loss module is: In the above formula (10), the meanings of the parameters are as follows: _Lem is the empathy loss value corresponding to the correct empathy splicing result; _Lem ' is the empathy loss value corresponding to the incorrect empathy splicing result; _δ1 is the difference between the two different empathy splicing results that need to be set; The training loss function (11) of the polar hinge loss module is: In the above formula (11), the meanings of the parameters are as follows: _Lpo is the polarity loss value corresponding to the polarity correct splicing result; _Lpo ' is the polarity loss value corresponding to the polarity incorrect splicing result; _δ2 is the difference between the two different splicing results of the set polarity that needs to be achieved; The final hinge loss function (12) of the text empathy prediction model is: L _hin =L _{hin_1} +L _{hin_2} (12); The final loss function L _o of the text empathy prediction model is: L _o =λ ₁ *L _em +λ ₂ *L _po +λ ₃ *L _cop +λ ₄ *L _adv +λ ₅ *L _hin (13); In the above formula (13), λ ₁ , λ ₂ , λ ₃ , λ ₄ , and λ ₅ are weights of the respective loss functions _Lem , _Lpo , _Lcop , _Ladv , and _Lhin , and the values of λ ₁ , λ ₂ , λ ₃ , λ ₄ , and λ ₅ are 1, 0.5, 2, 2, and 1.5, respectively.