CN116189064B - Barrage emotion analysis method and system based on joint model - Google Patents

Abstract

The invention discloses a barrage emotion analysis method and system based on a joint model, wherein barrage comments are input into the trained joint model to output emotion tendencies corresponding to the barrage comments, the joint model comprises a coding module and a decoding module, the coding module comprises a video coding module, a text coding module, a gating fusion module and a multi-mode fusion module, the decoding module comprises a barrage reconstruction module and an emotion analysis module, and the decoding module takes the output of the coding module as input to output emotion analysis tendencies corresponding to the barrage comments; the barrage emotion analysis method and system utilize a gating fusion screening mechanism to take surrounding barrage comments as context information of target barrage comments, and utilize a multi-mode fusion mode to take video information into consideration, and fully utilize useful information to strengthen characteristic representation of video barrages so as to accurately identify emotion tendencies of the target barrage comments.

Description

A joint model-based bullet screen sentiment analysis method and system

technical field

The invention relates to the technical field of barrage sentiment analysis, in particular to a joint model-based barrage sentiment analysis method and system.

Background technique

Video barrage sentiment analysis refers to analyzing the emotional polarity of real-time video comments.

Existing video bullet chat sentiment analysis methods tend to extract sentence-level features for sentiment analysis and classification, all of which are based on rule-based grammar and semantics. Special characters represent specific meanings, grammar is extremely irregular, etc., so existing sentiment analysis methods cannot accurately perform proper word segmentation and grammatical analysis on bullet chats, and thus cannot accurately analyze sentiment.

In addition, the existing barrage comments are short, do not have enough context information, have extremely irregular grammar, are related to the topic of the video at that time, are highly interactive, and are highly real-time, making it impossible for existing methods to analyze them in a short period of time. Perform effective and accurate sentiment analysis.

Contents of the invention

Based on the technical problems existing in the background technology, the present invention proposes a joint model-based bullet chat sentiment analysis method and system, which can accurately identify the emotional tendency of target bullet chat comments.

A joint model-based bullet chat sentiment analysis method proposed by the present invention inputs the bullet chat comments into the trained joint model to output the emotional tendency corresponding to the bullet chat comments;

The training process of the joint model is as follows:

S1: Construct a training sample set, the training sample set includes time Barrage comments/> , time /> arrive Inner barrage comments/> surrounding video /> And and barrage comments /> Bullet comments around the video within the same frame of video /> ;

S2: To the video Encode and concatenate to get encoded video features/> , comments on the barrage /> and bullet chatting comments around the video /> Encoding to get the encoded target barrage feature/> and surrounding barrage features/> ;

S3: Based on the characteristics of the target barrage Features on the surrounding barrage /> Filter and connect in series to get all surrounding barrage comments after filtering /> ;

S4: Video features through self-attention layer and cross-attention layer , target barrage features/> , Surrounding barrage comments /> Perform enhancement processing to obtain enhanced video features/> , Enhance the characteristics of the target barrage /> and enhanced surrounding barrage /> ;

S5; Enhance video features based on multi-layer multi-head attention layer pairs , Enhance the characteristics of the target barrage /> , enhance the surrounding barrage /> Perform reconstruction to obtain the reconstructed bullet chat comments, and use cross-entropy to construct a loss function for bullet chat reconstruction by using cross-entropy to construct the reconstructed bullet chat comments and real bullet chat comments. ;

S6: For enhanced video features , Enhance the characteristics of the target barrage /> , enhance the surrounding barrage /> Carry out regularization and normalization operations in sequence, and output the barrage comments/> Corresponding to the predicted barrage emotion/> ;

S7: Using cross-entropy to predict the emotion of the barrage and real barrage emotion/> Constructing a loss function for sentiment prediction /> , the loss function based on barrage reconstruction /> and the loss function for sentiment prediction /> Calculate the overall loss function /> , update the parameters of the joint model based on the overall loss function and the backpropagation algorithm until the performance of the joint model meets the set expectations;

The surrounding barrage comments Calculated as follows:

in, After filtering for the first /> surrounding barrage comments, /> for No. /> Barrage comments around the video /> surrounding features, /> , /> is a learnable gate matrix, /> is a learnable gate offset vector, /> is the ReLU function, Indicates concatenation, /> Indicates the product.

Further, the video feature The calculation formula is as follows:

The characteristics of the target barrage The calculation formula is as follows:

The surrounding barrage features The calculation formula is as follows:

in, , /> , /> Indicates concatenation, /> Indicates a video encoder, /> Represents a long short-term memory network.

Further, in step S4: through the self-attention layer and the cross-attention layer to video features , target barrage features/> , Surrounding barrage comments /> Perform enhancement processing to obtain enhanced video features/> , Enhance the characteristics of the target barrage /> and enhanced surrounding barrage /> , including:

feature video , target barrage features/> , Surrounding barrage comments /> As the first layer input of the self-attention layer and the cross-attention layer and perform L layer iterations, the L layer is the total number of layers of the self-attention layer and the cross-attention layer;

on the layer input video features /> , to get the input video features of the next layer /> as follows:

on the layer input target barrage feature/> , to get the input target barrage features of the next layer/> :

on the Barrage comments around layer input /> , to get the next layer's input surrounding barrage comments /> :

Among them, SA represents the self-attention layer, and CA represents the cross-attention layer.

Further, in step S5, the loss function of the barrage reconstruction The construction formula is as follows:

in, Indicates batch processing, /> represents the cross-entropy loss, /> Represents a refactoring module, /> Indicates the barrage comments generated by the refactoring module, /> Indicate time/> real barrage comments;

Specifically, the barrage comments generated by the reconstruction module are expressed in the following form:

in denotes a multi-layer perceptron, LN denotes a regularization operation, and MHA denotes cross-multiple attention.

Further, in step S6, the predicted barrage emotion Calculated as follows:

in, Is the Softmax function, LN represents the layer regularization operation, /> represents a multi-layer perceptron, /> is a learnable sentiment prediction matrix, /> is a learnable video sentiment matrix, /> is a learnable surrounding barrage emotion matrix, Represents a learnable target barrage emotion matrix, /> denotes a concatenation operation, and · denotes a product.

Further, in step S7, the loss function of the emotion prediction The construction formula is as follows:

The overall loss function The calculation process is as follows:

in, For the predicted barrage emotion, /> represents the cross-entropy loss, /> For the real barrage emotion, /> Indicates the loss balance parameter, /> Indicates batch processing.

A bullet chatting sentiment analysis system based on a joint model, inputting bullet chatting comments into a trained joint model to output the emotional tendency corresponding to the bullet chatting comments;

The analysis system includes a building block, a video encoding module, a text encoding module, a gating fusion module, a multimodal fusion module, a bullet chat reconstruction module, a bullet chat emotion prediction module, and a loss calculation module;

The building block is used to construct a training sample set, and the training sample set includes time Barrage comments/> , time /> to /> Inner barrage comments/> surrounding video /> And and barrage comments /> Bullet comments around the video within the same frame of video /> ;

The video encoding module is used to encode the video Encode and concatenate to get the encoded video features ;

The text encoding module is used to comment on the barrage and bullet chatting comments around the video /> Encoding to get the encoded target barrage feature/> and surrounding barrage features/> ;

The gated fusion module is based on the target barrage feature Features on the surrounding barrage /> Filter and connect in series to get all surrounding barrage comments after filtering /> ;

The multi-modal fusion module is used for video features through self-attention layer and cross-attention layer , target barrage features/> , Surrounding barrage comments /> Perform processing to obtain enhanced video features/> , Enhance the characteristics of the target barrage /> and enhanced surrounding barrage /> ;

The barrage reconstruction module is used to enhance video features based on multi-layer multi-head attention layers , Enhance the characteristics of the target barrage /> , enhance the surrounding barrage /> Perform reconstruction to obtain the reconstructed bullet chat comments, and use cross-entropy to construct a loss function for bullet chat reconstruction by using cross-entropy to construct the reconstructed bullet chat comments and real bullet chat comments. ;

The barrage emotion prediction module is used to enhance video features , Enhance the characteristics of the target barrage /> , enhance the surrounding barrage /> Carry out regularization and normalization operations in sequence, and output the barrage comments/> Corresponding to the predicted barrage emotion/> ;

The loss calculation module is used to use cross entropy to convert the predicted barrage emotion and real barrage emotion/> Constructing a loss function for sentiment prediction /> , the loss function based on barrage reconstruction /> and the loss function for sentiment prediction Calculate the overall loss function /> , update the parameters of the joint model based on the overall loss function and the backpropagation algorithm until the performance of the joint model meets the set expectations;

The surrounding barrage comments Calculated as follows:

in, After filtering for the first /> surrounding barrage comments, /> for No. /> Barrage comments around the video /> surrounding features, /> , /> is a learnable gate matrix, /> is a learnable gate offset vector, /> is the ReLU function, Indicates concatenation, /> Indicates the product.

The advantages of the joint model-based bullet chat sentiment analysis method and system provided by the present invention are: the joint model-based bullet chat sentiment analysis method and system provided in the structure of the present invention, through the multi-modal fusion module, the video information Include it, fully consider the relationship between the video theme and the bullet chat, obtain enhanced feature representation, and improve the performance of the joint model for sentiment analysis of the target bullet chat comment; include video information through the multi-modal fusion module, fully consider the video theme The relationship with the barrage, obtain enhanced feature representation, improve the performance of the joint model for sentiment analysis of target barrage comments; use the barrage reconstruction module to promote the overall learning effect of each module, and improve the performance of the sentiment analysis module.

Description of drawings

Fig. 1 is a structural representation of the present invention;

Fig. 2 is a schematic diagram of the module frame of the present invention.

Detailed ways

In the following, the technical solution of the present invention will be described in detail through specific embodiments, and many specific details are set forth in the following description so as to fully understand the present invention. However, the present invention can be implemented in many other ways different from those described here, and those skilled in the art can make similar improvements without departing from the connotation of the present invention, so the present invention is not limited by the specific implementation disclosed below.

As shown in Figures 1 and 2, a joint model-based bullet-screen sentiment analysis method proposed by the present invention inputs the bullet-screen comments into the trained joint model to output the corresponding emotional tendency of the bullet-screen comments; the joint The model uses an encoding-decoding architecture. The joint model includes an encoding module and a decoding module. The encoding module includes a video encoding module, a text encoding module, a gated fusion module, and a multimodal fusion module. The decoding module includes a barrage reconstruction module and Sentiment analysis module. The sentiment analysis module includes a bullet chat sentiment prediction module and a loss calculation module. The decoding module takes the output of the encoding module as input to output the sentiment analysis tendency corresponding to the bullet chat comments.

This embodiment mainly uses the gating screening mechanism in the joint model to use the surrounding comments as the context information of the target barrage, and uses the multi-modal fusion method to take the video information into account, and fully utilizes useful information to strengthen the feature representation of the video barrage. The joint model is constructed based on residual convolutional neural network, long-term short-term memory network, gated fusion self-attention layer, cross-attention layer, etc. The learnable parameters in the joint model are trained and learned as follows, and the learnable parameters are optimized to achieve pass The joint model accurately identifies the effect of the emotional tendency of the target barrage comments, as follows.

The training process of the joint model is as follows:

S1: Construct a training sample set, the training sample set includes time Barrage comments/> , time /> arrive Inner barrage comments/> surrounding video /> And and barrage comments /> Bullet comments around the video within the same frame of video /> ;

video Yes /> frame video /> , Barrage comments around the video /> Yes /> video barrage comments /> , Barrage comments around the video /> Is in the barrage comment /> Comments around.

For example, in the example shown in Figure 2, the barrage comment y "For yourself, stick to it!" is used as input, and the barrage comments around the video "Beautiful picture" and "good figure" are used as the context content of y, and the video corresponding to the barrage comment y /> together as input.

S2: To the video Encode and concatenate to get encoded video features/> , comments on the barrage /> and bullet chatting comments around the video /> Encoding to get the encoded target barrage feature/> and surrounding barrage features/> ;

Within the video encoding module, using a residual convolutional neural network, encoding frame video /> , and concatenate the obtained encoding vectors to obtain encoded frame-level video features/> :

in, Indicates a video encoder, /> Indicates a series operation;

In the text encoding module, the long short-term memory network ( ), comment on the barrage respectively /> and its surrounding video barrage comments /> Encoding to get the encoded target barrage feature/> and surrounding barrage features/> :

Right now

in, ;

=/> ;

It should be understood that the Barrage comments around the video /> The surrounding features are /> .

S3: Based on the characteristics of the target barrage Features on the surrounding barrage /> Filter and connect in series to get all surrounding barrage comments after filtering /> ;

Based on the characteristics of video bullet chatting, some surrounding useful bullet chatting comments with the same emotion can be used as the context information of the target bullet chatting comments, so the gated fusion module can be used to utilize come on Perform screening and filtering operations to obtain the first /> after processing by the gated fusion module surrounding barrage comments /> :

in, After filtering for the first /> surrounding barrage comments, /> for No. /> Barrage comments around the video /> surrounding features, /> , /> is a learnable gate matrix, /> is a learnable gate offset vector, the activation function /> is the ReLU function, /> Indicates concatenation, /> represents the product, /> and /> Both are learnable parameters, and parameters are optimized during the joint model training process to achieve the expected effect with the lost model;

Put the first to surrounding barrage comments /> concatenated to get all surrounding barrage comments /> :

in, Indicates a concatenation operation.

S4: Video features through self-attention layer and cross-attention layer , target barrage features/> , Surrounding barrage comments /> Perform enhancement processing to obtain enhanced video features/> , Enhance the characteristics of the target barrage /> and enhanced surrounding barrage /> ;

The multi-modal fusion module is composed of L-layer self-attention layer and cross-attention layer, which will combine video features , target barrage features/> , Surrounding barrage comments /> As the input of the first layer of the multi-modal fusion module, after multi-layer iteration (that is, after the L-layer self-attention layer and cross-attention layer processing), the corresponding enhancement of other modalities is obtained in the last layer. Video Features/> , Enhance the characteristics of the target barrage /> and enhanced surrounding barrage /> .

on the layer input video features /> , to get the input video features of the next layer /> as follows:

on the layer input target barrage feature/> , to get the input target barrage features of the next layer/> :

on the Barrage comments around layer input /> , to get the next layer's input surrounding barrage comments /> :

Among them, SA represents the self-attention layer, and CA represents the cross-attention layer.

S5; Enhance video features based on multi-layer multi-head attention layer pairs , Enhance the characteristics of the target barrage /> , enhance the surrounding barrage /> Perform reconstruction to obtain the reconstructed bullet chat comments, and use cross-entropy to construct a loss function for bullet chat reconstruction by using cross-entropy to construct the reconstructed bullet chat comments and real bullet chat comments. ;

The decoding module consists of two parts: the barrage reconstruction module and the sentiment analysis module. The decoding module uses the enhanced video features obtained in the encoding module , Enhance the characteristics of the target barrage /> , enhance the surrounding barrage /> as input;

In the barrage reconstruction module, the reconstruction loss is analyzed and calculated by the barrage reconstruction module, and is added to the closed-loop training to promote the learning effect of the multi-modal fusion module and improve the effect of the sentiment analysis module.

The barrage reconstruction module consists of multi-layer multi-head attention layers, and the loss function of the barrage reconstruction for:

in, Represents batch processing, CE represents cross-entropy loss, /> Represents a refactoring module, /> Indicates the barrage comments generated by the refactoring module, /> Indicate time/> real barrage comments;

Specifically, the barrage comments generated by the reconstruction module are expressed in the following form:

in denotes a multi-layer perceptron, LN denotes a regularization operation, and MHA denotes cross-multiple attention.

S6: For enhanced video features , Enhance the characteristics of the target barrage /> , enhance the surrounding barrage /> Carry out regularization and normalization operations in sequence, and output the barrage comments/> Corresponding to the predicted barrage emotion/> ;

In the sentiment analysis module, there are two parts: the bullet chat emotion prediction module and the loss calculation module; the bullet chat emotion predicted in the bullet chat emotion prediction module Calculated as follows:

in, Is the Softmax function, LN represents the layer regularization operation, /> represents a multi-layer perceptron, /> is a learnable sentiment prediction matrix, /> is a learnable video sentiment matrix, /> is a learnable surrounding barrage emotion matrix, Represents a learnable target barrage emotion matrix, /> Indicates concatenation operation, means product, /> , /> , /> Both are learnable parameters, and parameter optimization is performed during the joint model training process to achieve the expected effect with the lost model.

S7: Using cross-entropy to predict the emotion of the barrage and real barrage emotion/> Constructing a loss function for sentiment prediction /> , the loss function based on barrage reconstruction /> and the loss function for sentiment prediction /> Calculate the overall loss function /> , update the parameters of the joint model based on the overall loss function and the backpropagation algorithm until the performance of the joint model meets the set expectations;

The loss function for sentiment prediction in the loss calculation module The construction formula is as follows:

in, Indicates batch processing, /> is the predicted barrage emotion, the predicted barrage emotion is the barrage comment/> The predicted barrage sentiment output by the joint model, /> For the real barrage emotion, the real barrage emotion is the barrage comment/> the corresponding actual emotion;

overall loss function The calculation process is as follows:

in, Indicates the loss balance parameter, and updates the learnable parameters of the joint model based on the loss and the backpropagation algorithm until the model performance reaches the expected effect.

First: step S3 provides a gating fusion mechanism, using the target bullet chat comments to filter the surrounding bullet chat comments, so that some surrounding useful bullet chat comments with the same emotion can be used as the target bullet chat comments Provide help with context information, solve problems such as short bullet chat comments and insufficient context information, and improve the quality of target bullet chat.

Second: step S4 provides a multi-modal fusion enhancement mechanism, which includes video information through the multi-modal fusion module, fully considers the relationship between video topics and barrage, obtains enhanced feature representation, and improves the joint model for the target Performance of Sentiment Analysis on Bullet Comments.

Third, steps S5 to S7 provide a barrage reconstruction and sentiment analysis mechanism, using the barrage reconstruction module to promote the overall learning effect of each module and improve the performance of the sentiment analysis module.

This embodiment is mainly applied to sentiment analysis of real-time video comments. For example, at a certain moment, a user sends a comment, and the emotional tendency of the comment is judged.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, any person familiar with the technical field within the technical scope disclosed in the present invention, according to the technical solution of the present invention Any equivalent replacement or change of the inventive concepts thereof shall fall within the protection scope of the present invention.

Claims (7)

1. The barrage emotion analysis method based on the joint model is characterized in that barrage comments are input into the trained joint model to output emotion tendencies corresponding to the barrage comments;

the training process of the joint model is as follows:

s1: constructing a training sample set, the training sample set comprising momentsBullet comment>Time->To->Inward bullet comment->Surrounding video->And comment on bullet screen->Video surrounding barrage comment +.>；

S2: for the videoCoding and concatenating to obtain coded video feature ∈ ->Comment on the barrage->And the video surrounding barrage comment +.>EncodingObtaining the coded target barrage characteristic +.>And surrounding barrage features；

S3: based on the target barrage featureFor the surrounding barrage feature->After screening and filtering, connecting in series to obtain all surrounding barrage comments +.>；

S4: video characterization through self-attention layers and cross-attention layersTarget barrage feature->Comment on surrounding barrage->Enhancement processing to obtain enhanced video features>Enhanced target barrage feature->And reinforcing the surrounding barrage->；

S5, performing S5; enhancement of video features based on multi-layered multi-headed pairs of attention layersEnhanced target barrage feature->Reinforcing surrounding barrage->Reconstructing to obtain reconstructed barrage comments, and constructing a barrage reconstructed loss function by using the reconstructed barrage comments and the real barrage comments by using cross entropy>；

S6: for enhanced video featuresEnhanced target barrage feature->Reinforcing surrounding barrage->Sequentially carrying out regularization and normalization operations, and outputting the barrage comment +.>Corresponding to predicted barrage emotion +.>；

S7: predicted barrage emotion using cross entropyAnd true barrage emotion->Constructing a loss function for emotion predictionLoss function based on barrage reconstruction>And loss function of emotion prediction->Calculated overall loss functionBased on the overall loss function->And updating parameters of the joint model by a back propagation algorithm until the performance of the joint model reaches a set expected value;

the surrounding barrage commentsThe calculation formula is as follows:

wherein ,for post-selection->Comment on the surrounding bullet screen->Is->Video surrounding barrage comment->Is (are) peripheral features of->，/>Is a learnable gate matrix +.>Is a learnable gate offset vector, +.>For ReLU function>Representing series connection,/->Representing the product.

2. The method of collaborative model-based barrage emotion analysis of claim 1, wherein the video featuresThe calculation formula of (2) is as follows:

the target barrage featureThe calculation formula of (2) is as follows:

the surrounding barrage featureThe calculation formula of (2) is as follows:

wherein ,，/>，/>representing series connection,/->Representing a video encoder>Representing a long and short term memory network.

3. The barrage emotion analysis method based on joint model as set forth in claim 1, characterized in that in step S4: video characterization through self-attention layers and cross-attention layersTarget barrage feature->Comment on surrounding bullet screenEnhancement processing to obtain enhanced video features>Enhanced target barrage feature->And reinforcing the surrounding barrageSpecifically, the method comprises the following steps:

characterizing videoTarget barrage feature->Comment on surrounding barrage->Inputting as a first layer of the self-attention layer and the cross-attention layer and performing an L-layer iteration, wherein the L-layer is the total layer number of the self-attention layer and the cross-attention layer;

in the first placeLayer input video feature->Obtaining the input video feature of the next layer +.>The following are provided:

in the first placeLayer input target barrage feature->Obtaining the input target barrage feature of the next layer>：

In the first placeLayer input surrounding barrage comment->Obtaining the comment +.>：

Where SA represents the self-attention layer and CA represents the cross-attention layer.

4. A combined model-based barrage emotion analysis method as claimed in claim 3, characterized in that in step S5, the barrage reconstructed loss functionThe construction formula is as follows:

wherein ,indicating batch processing, +.>Representing cross entropy loss, < >>Representing a reconstruction module->Comment of bullet generated by the reconstruction module is represented, < ->Indicating time->Is a true bullet comment;

specifically, the bullet comments generated by the reconstruction module are specifically expressed in the following form:

wherein ,representing a multi-layer perceptron, LN representing regularization operation, MHA representing cross-multi-headed attention.

5. The method of claim 4, wherein in step S6, predicted barrage emotion is predictedThe calculation formula is as follows:

wherein ,is a Softmax function, LN represents a layer regularization operation, +.>Representing a multi-layer sensor->For a learnable emotion prediction matrix, +.>Is a learnable video emotion matrix, +.>Is a surrounding barrage emotion matrix which can be learned, < + >>Representing a learnable target barrage emotion matrix, < ->Representing a tandem operation, representing a product.

6. The method of collaborative model-based barrage emotion analysis according to claim 5, wherein in step S7, the emotion predicted penalty functionThe construction formula is as follows:

the overall loss functionThe calculation process of (2) is as follows:

wherein ,for predicted barrage emotion, +.>Is true barrage emotion +.>Representing cross entropy loss, < >>Representing loss balance parameters, +.>Indicating batch processing.

7. The barrage emotion analysis system based on the joint model is characterized in that barrage comments are input into the trained joint model to output emotion tendencies corresponding to the barrage comments;

the analysis system comprises a construction module, a video coding module, a text coding module, a door control fusion module, a multi-mode fusion module, a barrage reconstruction module, a barrage emotion prediction module and a loss calculation module;

the construction module is used for constructing a training sample set, and the training sample set comprises momentsBullet comment>Time of dayTo->Inward bullet comment->Surrounding video->And comment on bullet screen->Video surrounding barrage comment +.>；

The video coding module is used for coding the videoCoding and concatenating to obtain coded video feature ∈ ->；

The text coding module is used for commenting on the barrageAnd the video surrounding barrage comment +.>Coding to obtain the coded target barrage characteristic +.>And surrounding barrage feature->；

The gating fusion module is based on the target barrage characteristicsFor the surrounding barrage feature->After screening and filtering, connecting in series to obtain all surrounding barrage comments +.>；

The multi-mode fusion module is used for video features through the self-attention layer and the cross-attention layerTarget barrage feature->Comment on surrounding barrage->Processing to obtain enhanced video features->Enhanced target barrage feature->And reinforcing the surrounding barrage->；

The bullet screen reconstruction module is used for enhancing vision based on multi-layer multi-head attention layer pairsFrequency characteristicsEnhanced target barrage feature->Reinforcing surrounding barrage->Reconstructing to obtain reconstructed barrage comments, and constructing a barrage reconstructed loss function by using the reconstructed barrage comments and the real barrage comments by using cross entropy>；

The barrage emotion prediction module is used for enhancing video featuresEnhanced target barrage feature->Reinforcing surrounding barrage->Sequentially carrying out regularization and normalization operations, and outputting the barrage comment +.>Corresponding to predicted barrage emotion +.>；

The loss calculation module is used for predicting bullet screen emotion by using cross entropyAnd true barrage emotion->Construction of a loss function for emotion prediction>Loss function based on barrage reconstruction>And loss function of emotion prediction->Calculating the overall loss function->Updating parameters of the joint model based on the overall loss function and the back propagation algorithm until the performance of the joint model reaches a set expectation;

the surrounding barrage commentsThe calculation formula is as follows:

wherein ,for post-selection->Comment on the surrounding bullet screen->Is->Video surrounding barrage comment->Is (are) peripheral features of->，/>Is a learnable gate matrix +.>Is a learnable gate offset vector, +.>For ReLU function>Representing series connection,/->Representing the product.