CN117014671A

CN117014671A - Bullet screen display method and device, electronic equipment and storage medium

Info

Publication number: CN117014671A
Application number: CN202210898930.8A
Authority: CN
Inventors: 陈小帅
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-07-28
Filing date: 2022-07-28
Publication date: 2023-11-07

Abstract

The embodiment of the application discloses a bullet screen display method, a bullet screen display device, electronic equipment and a storage medium; comprising the following steps: identifying a first number of primary selection barrages with the correlation meeting a preset requirement from all barrages included in the video clip; then, for any two primary selection barrages, determining the upper probability that one primary selection barrage is the upper probability of the other primary selection barrage, and obtaining a plurality of upper probabilities corresponding to the first number of primary selection barrages; then, according to the relative size relation between the numerical value of the probability and the preset probability threshold value, a second number of associated barrage sets are screened out from the primary selected barrages; and displaying at least one associated barrage set of the second number of associated barrage sets. The application can acquire the associated barrage set comprising the upper barrage and the lower barrage and display the associated barrage set, thereby enabling the object watching the video to more intuitively and rapidly know the context of barrage dialogue and reducing the burden of the object to watch the associated barrage.

Description

Bullet screen display method and device, electronic equipment and storage medium

Technical Field

The application relates to the field of computers, in particular to a bullet screen display method, a bullet screen display device, electronic equipment and a storage medium.

Background

A bullet screen refers to a comment that appears directly on a video, can appear on the video in a scrolling, hovering or even more action special effects manner, and is a brief comment by a person watching the video.

Many of the video's barrages have contextual relationships. For example, when an object watching a video watches the video, the object may ask questions about the video content by asking questions; other users often answer the question barrage in the form of a barrage when seeing the question type barrage. The question type barrage can be regarded as the above barrage of the corresponding answer type barrage; accordingly, the answer type of barrage may also be considered a follow-up barrage of the question type of barrage.

However, in the related art, multiple backlashes with context association relationship are often presented in a scattered manner in a video, so that when a third party user wants to find a post backlashes of a certain post backlashes, more effort is required to be spent on a link of viewing backlashes, and the viewing burden of the user is increased.

Disclosure of Invention

The embodiment of the application provides a barrage display method, a barrage display device, electronic equipment and a storage medium, which can solve the problem that watching barrages in related technologies increases watching burden of users.

The embodiment of the application provides a bullet screen display method, which comprises the following steps: identifying a first number of primary selection barrages from all barrages included in the video clips, wherein the primary selection barrages are barrages with correlation with the video clips meeting preset requirements;

determining an upper probability, wherein the upper probability refers to the probability that one primary selection barrage is the upper of another primary selection barrage;

screening a second number of associated barrage sets from the first number of primary selection barrages according to the magnitude relation between the upper probability and a preset probability threshold;

at least one associated barrage set of the second number of associated barrage sets is displayed.

The embodiment of the application also provides a barrage display device, which comprises:

the primary selection barrage identification unit is used for identifying a first number of primary selection barrages from all barrages included in the video clips, wherein the primary selection barrages are barrages with correlation with the video clips meeting preset requirements;

the device comprises an upper probability determining unit, a first-choice screen and a second-choice screen, wherein the upper probability determining unit is used for determining the upper probability, and the upper probability refers to the probability that one first-choice screen is the upper of the other first-choice screen;

The barrage set screening unit is used for screening a second number of associated barrage sets from the first number of initially selected barrages according to the magnitude relation between the above probability and a preset probability threshold;

and the barrage set display unit is used for displaying at least one associated barrage set in the second number of associated barrage sets.

In some embodiments, the primary screen identification unit comprises:

the first coding subunit is used for carrying out first coding processing on each barrage to obtain corresponding barrage text characteristics;

the second coding subunit is used for acquiring a plurality of video key frames in the video clip, and performing second coding processing on the video key frames to obtain a plurality of corresponding key frame representation characteristics;

the first fusion subunit is used for carrying out fusion processing on the barrage text characteristics and the plurality of key frame representation characteristics according to an attention mechanism to obtain first fusion characteristics;

the third coding subunit is used for acquiring a plurality of content information in the video clip, and performing third coding processing on the plurality of content information to obtain a plurality of corresponding content text characteristics;

the second fusion subunit is used for carrying out fusion processing on the barrage text characteristics and the content text characteristics according to an attention mechanism to obtain second fusion characteristics;

The characteristic splicing subunit is used for splicing the first fusion characteristic and the second fusion characteristic to obtain a spliced characteristic;

and the bullet screen identification subunit is used for identifying a first number of primarily selected bullet screens with the correlation with the video clips meeting the preset requirement according to the splicing characteristics corresponding to each bullet screen.

In some embodiments, the bullet screen identification subunit comprises:

the full-connection subunit is used for carrying out full-connection transformation on the splicing characteristics corresponding to each bullet screen to obtain a first two-dimensional vector result;

a first score value subunit, configured to obtain a score value of a first dimension vector in the first two-dimensional vector result, where the score value of the first dimension vector in the first two-dimensional vector result is: probability of correlation of the bullet screen and the video clip;

and the barrage determining subunit is used for determining the barrage as the primary barrage when the score value of the first dimension vector exceeds a preset score threshold value.

In some embodiments, the above probability determination unit comprises:

the bullet screen obtaining subunit is used for obtaining a first primary bullet screen and a second primary bullet screen from the first number of primary bullet screens, wherein the first primary bullet screen is any one of the first number of primary bullet screens, and the second primary bullet screen is any one of the first number of primary bullet screens except the first primary bullet screen;

The first segmentation subunit is used for carrying out segmentation processing on the first primary selection barrage to obtain a third number of first word vectors and position information of each first word vector;

the second segmentation subunit is used for carrying out segmentation processing on the second primary selection barrage to obtain a fourth number of second word vectors and position information of each second word vector;

a feature extraction subunit, configured to perform feature extraction on a global indicator and position information of the global indicator, a third number of first word vectors, and position information of each first word vector, a separator and position information of the separator, a fourth number of second word vectors, and position information of each second word vector, to obtain a sum number of feature vectors; wherein the sum number is a sum of the third number, the fourth number, a number of global indicators, and a number of separators;

and the probability calculation subunit is used for calculating the upper probability of the first primary selection barrage serving as the upper text of the second primary selection barrage according to the added number of the feature vectors.

In some embodiments, the probability computation subunit comprises:

A target vector obtaining subunit, configured to obtain a target feature vector corresponding to the global indicator in the added number of feature vectors;

the pooling processing subunit is used for pooling processing the feature vectors except the target feature vector in the added number of feature vectors to obtain a pooling processing result;

a second result subunit, configured to perform full-connection transformation on the target feature vector and the pooling processing result, to obtain a second two-dimensional vector result;

a second score value subunit, configured to obtain a score value of a first dimension vector in the second two-dimensional vector result, where the score value of the first dimension vector in the second two-dimensional vector result is: the first primary screen acts as the contextual probability of the context of the second primary screen.

In some embodiments, a barrage collection screening unit comprises:

the label giving subunit is used for obtaining two primary selection barrages which form a context relation and have a context probability larger than the association probability threshold value, and determining the two primary selection barrages as a context barrage and a context barrage respectively according to the context association relation of the two primary selection barrages;

The overall probability subunit is used for calculating the overall probability value of the upper barrage if the upper barrage with a plurality of lower barrages exists for all the initial barrages with the upper probability larger than the associated probability threshold;

the alternative bullet screen subunit is used for screening the bullet screens with the overall probability value larger than the overall probability threshold, wherein the bullet screens with the overall probability value larger than the overall probability threshold are alternative bullet screens;

a target upper sub-unit, configured to obtain, for each primary selection barrage whose upper probability is greater than the associated probability threshold, a target upper barrage whose upper probability is the largest;

and the set screening subunit is used for screening a second number of associated barrage sets from all the primary selection barrages with the above probabilities greater than the associated probability threshold according to the subordinate relations of the target previous barrages and the multiple alternative previous barrages.

In some embodiments, the set screening subunit comprises:

the bullet screen retaining sub-unit is used for retaining a primary bullet screen corresponding to the target preceding bullet screen and a context association relation between the primary bullet screen and the target preceding bullet screen when the target preceding bullet screen is one of a plurality of alternative preceding bullet screens;

And the bullet screen discarding subunit is used for discarding the primary bullet screen corresponding to the target preceding bullet screen when the target preceding bullet screen is not any one of the multiple alternative preceding bullet screens.

In some embodiments, the apparatus further comprises:

the similarity calculation unit is used for calculating the text similarity of any two associated barrage sets in the plurality of associated barrage sets;

and the set merging unit is used for merging two associated barrage sets with the text similarity larger than the preset similarity threshold value into one associated barrage set when the text similarity is larger than the preset similarity threshold value.

In some embodiments, the set merge unit comprises:

a top Wen Danmu subunit, configured to obtain a top barrage with a larger overall probability value in the two associated barrage sets, and use the top Wen Danmu with the larger overall probability value as a new top barrage;

a lower Wen Danmu subunit for taking all of the lower sheets in the two associated sets of sheets as lower sheets of the new upper sheet.

In some embodiments, a bullet screen collection display unit comprises:

the fourth coding subunit is used for carrying out fourth coding processing on each associated barrage set to obtain corresponding associated set characteristics;

A fifth coding subunit, configured to obtain a plurality of interest tags corresponding to the object, and perform fifth coding processing on each of the plurality of interest tags to obtain a corresponding plurality of interest tag features;

the third fusion subunit is used for carrying out fusion processing on the association set features and the interest tag features according to an attention mechanism to obtain third fusion features;

the interest degree determining subunit is used for determining the interest degree of the object on the associated barrage set according to the third fusion characteristic;

the set determining subunit is used for determining the first K associated barrage sets with high interestingness values;

and the set display subunit is used for displaying the first K associated barrage sets.

In some embodiments, the interestingness determination subunit comprises:

a third result subunit, configured to perform full-connection transformation on the third fusion feature to obtain a third two-dimensional vector result;

and the third result scoring subunit is used for acquiring the scoring value of the first dimension vector in the third two-dimensional vector result, wherein the scoring value of the first dimension vector in the third two-dimensional vector result is the interestingness.

In some embodiments, the aggregate display subunit comprises:

A distance determining subunit, configured to determine, according to the interestingness, a distance between each associated barrage set of the first K associated barrage sets and a horizontal center line of a display interface;

and the set display subunit is used for displaying the first K associated barrage sets according to the distance between each associated barrage set and the horizontal center line of the display interface.

The embodiment of the application also provides electronic equipment, which comprises a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute steps in any bullet screen display method provided by the embodiment of the application.

The embodiment of the application also provides a computer readable storage medium, which stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps in any bullet screen display method provided by the embodiment of the application.

The embodiment of the application can identify the first number of primary selection barrages with the correlation meeting the preset requirement from all barrages included in the video clip; then, for any two primary selection barrages, determining the upper probability that one primary selection barrage is the upper probability of the other primary selection barrage, and obtaining a plurality of upper probabilities corresponding to the first number of primary selection barrages; then, according to the relative size relation between the numerical value of the probability and the preset probability threshold value, a second number of associated barrage sets are screened out from the primary selected barrages; and displaying at least one associated barrage set of the second number of associated barrage sets.

In the method, a plurality of primary selection barrages related to video clips are firstly identified, and the probability of the upper surface of any two primary selection barrages in the plurality of primary selection barrages is calculated; and screening out a plurality of associated barrage sets according to the magnitude relation between the probability and the preset probability threshold value, and displaying at least one associated barrage set. In the embodiment, the associated barrage set comprising the upper barrage and at least one lower barrage can be obtained and displayed, so that the object watching the video can more intuitively and rapidly know the context of barrage conversation, the burden of the object for watching the associated barrage is reduced, and the watching experience is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1a is a schematic view of a scenario of a barrage display method according to an embodiment of the present application;

FIG. 1b is a schematic flow chart of a barrage display method according to an embodiment of the present application;

FIG. 1c is a schematic diagram of a combined model for calculating the probability of correlation between a bullet screen and a video clip;

FIG. 1d is a schematic diagram of a combined model for calculating the probability of a first primary screen being above a second primary screen;

FIG. 1e is a diagram showing contextual relationships between the barrages;

FIG. 1f is a schematic diagram of a combined model for calculating the interest level of an object in an associated barrage set;

FIG. 2 is a schematic flow chart of a method for displaying a bullet screen according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a bullet screen display device according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.

The embodiment of the application provides a bullet screen display method, a bullet screen display device, electronic equipment and a storage medium.

The bullet screen display device can be integrated in electronic equipment, and the electronic equipment can be a terminal, a server and other equipment. The terminal can be a mobile phone, a tablet personal computer, an intelligent Bluetooth device, a notebook computer, a desktop computer, an intelligent television, a vehicle-mounted terminal and other devices; the server may be a single server, or may be a server cluster or cloud server composed of a plurality of servers.

In some embodiments, the bullet screen display device may also be integrated in a plurality of electronic devices, for example, the bullet screen display device may be integrated in a plurality of servers, and the bullet screen display method of the present application is implemented by the plurality of servers.

In some embodiments, the server may also be implemented in the form of a terminal.

For example, referring to fig. 1a, the electronic device may identify a first number of primary selection screens (i.e., j primary selection screens shown in fig. 1 a) from all the screens included in the video clip, where the primary selection screens are screens whose correlation with the video clip meets a preset requirement. The method comprises the steps of determining the upper probabilities, wherein the upper probabilities refer to the upper probabilities of one primary selection barrage being another primary selection barrage, and the first number of primary selection barrages share a plurality of upper probabilities. Screening a second number of associated barrage sets (namely k associated barrage sets shown in fig. 1 a) from the first number of primary selection barrages according to the magnitude relation between the probability and the preset probability threshold; displaying a second number of associated bullet screen collections is associated with the set of barrages.

The following will describe in detail. The numbers of the following examples are not intended to limit the preferred order of the examples.

In this embodiment, a barrage display method is provided, which may be performed by an electronic device; as shown in fig. 1b, the detailed flow of the barrage display method may be as follows:

110. and identifying a first number of primary selection barrages from all barrages included in the video clips, wherein the primary selection barrages are barrages with correlation with the video clips meeting preset requirements.

The video clip is a video clip with the video duration of t and carrying a barrage. Where t is a positive number, and the specific numerical range of t should not be construed as limiting the application.

For long video clips with the video duration exceeding t, the long video clip can be divided into video clips with the video duration of t, so that a plurality of video clips with the video duration of t are obtained.

The first number is a positive integer and the specific value of the first number should not be construed as limiting the application.

The correlation of the primary selection barrage and the video clips meets the preset requirement, the primary selection barrage is a barrage with higher correlation with the video clips, and when watching video, an object watching the video is generally interested in the barrage with high video correlation, so that the interest of watching the video by the object can be increased through screening the primary selection barrage.

Optionally, in a specific embodiment, step 110 includes the following steps 111 to 117 (not shown in the figure):

111. and carrying out first coding processing on each barrage to obtain corresponding barrage text characteristics.

Optionally, referring to fig. 1c for details, for each bullet screen in all bullet screens included in the video clip, a first coding process may be performed through a trained first ALBERT model, so as to obtain corresponding bullet screen text features.

The ALBERT model is a model with the same structure as the BERT model and fewer parameters. Compared to the BERT model, the ALBERT model has three differences:

the bert model typically sets the size E of the vocabulary emb size and the size H of the hidden size in the network to the same value; the ALBERT model sets the values of E and H to be different.

The bert model typically does not implement cross-layer sharing of parameters, typically only implementing sharing of parameters of the same layer, e.g., sharing parameters of the feed forward layer or sharing parameters of the attention layer; the ALBERT model generally performs cross-layer sharing of parameters, and defaults to share all parameters.

The bert model uses Next Sentence Prediction loss, abbreviated as NSP loss; while ALBERT replaces NSP loss with Sentence Order Prediction loss, SOP loss for short.

112. And acquiring a plurality of video key frames in the video fragment, and performing second coding processing on the plurality of video key frames to obtain a plurality of corresponding key frame representation characteristics.

The video key frames are video frames in the video clip reflecting video episodes. The video key frames can be obtained through the existing key frame extraction method, for example, the video key frames can be selected through the existing 3D-CNN model, the key frame extraction can be performed based on a motion analysis streamer method, the key frame extraction can be performed by using a K-means clustering method, and the like; or may be manually selected from video clips by a worker. It should be understood that the method of selecting video key frames should not be construed as limiting the application.

Optionally, referring to fig. 1c for details, for a plurality of video key frames, a second encoding process may be performed on each of the plurality of video key frames through a trained afflicientnet model to obtain key frame representation features corresponding to each of the plurality of video key frames, so as to obtain a plurality of key frame representation features. The number of key frame representation features is the same as the number of video key frames, and the key frames are in one-to-one correspondence.

The EfficientNet model is a rapid high-precision model, and the construction method of the model mainly comprises the following two steps:

i. The generation of the baseline model EfficientNet-B0 is achieved using a reinforcement learning algorithm.

Adopting a compound scaling method, and under the condition that the memory and the calculated amount are preset, scaling the depth, the width (the channel number of the feature map) and the picture size of the baseline model EfficientNet-B0 simultaneously, and outputting a final EfficientNet model; wherein, the scaling of the three dimensions is obtained by grid search.

113. And according to the attention mechanism, fusing the barrage text features and the plurality of key frame representation features to obtain a first fusion feature.

The bullet screen text feature is not limited to 1 n-dimensional vector, the number of the key frame representation features is m, and each key frame representation feature is also an n-dimensional vector, namely, the key frame representation features have m n-dimensional vectors, wherein m and n are positive integers; the specific implementation process of the fusion processing of the barrage text feature and the plurality of key frame representation features according to the attention mechanism is as follows:

and calculating the inner product of the n-dimensional vector of the barrage text feature and m n-dimensional vectors to obtain m products.

And carrying out softmax transformation on the m products to obtain the weight values respectively corresponding to the n-dimensional vector of the barrage text characteristic and the m key frame representation characteristics.

Based on the weight values corresponding to the m key frame representation features, a weighted sum of the m key frame representation features with respect to the barrage text features is calculated, and the weighted sum is the first fusion feature, as shown in fig. 1c for details.

114. And acquiring a plurality of content information in the video clip, and performing third coding processing on the plurality of content information to obtain a plurality of corresponding content text characteristics.

The content information is information reflecting the content expressed by the video clip, and the content information may be a speech subtitle expressed by a character in the video clip, or a speech corresponding to a speech expressed by a character in the video clip, or may include a speech subtitle and a speech corresponding to a speech. Wherein, the line captions can be converted into texts by pattern recognition (Optical Character Recognition, OCR for short); the sound corresponding to the speech of the speech may be converted into text by an automatic speech recognition (Automatic Speech Recognition, abbreviated ASR) technique, so as to obtain content information in text form.

Optionally, referring to fig. 1c for details, for a plurality of content information in the video segment, a third encoding process may be performed on the trained second ALBERT model to obtain a corresponding plurality of content text features; the number of the text features of the content is the same as the number of the content information, and the text features of the content are in one-to-one correspondence with the content information.

115. And according to the attention mechanism, fusing the barrage text features and the content text features to obtain a second fused feature.

The description proceeds with reference to the examples above: the number of the content text features is p, and each content text feature is also an n-dimensional vector, namely the content text features have p n-dimensional vectors, wherein p is a positive integer; the specific implementation process of fusing the barrage text feature and the plurality of content text features according to the attention mechanism is as follows:

and calculating the inner product of the n-dimensional vector of the barrage text feature and p n-dimensional vectors to obtain p products.

And carrying out softmax transformation on the p products to obtain the weight values respectively corresponding to the n-dimensional vector of the barrage text feature and the p content text features.

Based on the weight values corresponding to the p content text features, a weighted sum of the p content text features with respect to the barrage text features is calculated, and the weighted sum is the second fusion feature, as shown in fig. 1c for details.

116. And performing splicing treatment on the first fusion characteristic and the second fusion characteristic to obtain a splicing characteristic.

The splicing process is a process of splicing two feature vectors with the same dimension or different dimensions into one spliced feature vector, and the dimension of the spliced feature vector obtained by splicing is the sum of the dimensions of the two feature vectors before splicing.

The description proceeds with reference to the examples above: the first fusion feature is an a-dimensional vector, the second fusion feature is a b-dimensional vector, and after the first fusion feature and the second fusion feature are spliced, an (a+b) -dimensional vector can be obtained, wherein the (a+b) -dimensional vector is the splicing feature; wherein a and b are positive integers, and a and b can be the same or different.

117. And identifying a first number of primary selection barrages with the correlation with the video clips meeting the preset requirement according to the splicing characteristics corresponding to each barrage.

The splicing characteristics corresponding to each barrage reflect the correlation of a plurality of video key frames of the barrage and the video clips and the correlation of a plurality of content information of the barrage and the video clips. Therefore, the primary selection barrage with higher correlation with the video clips can be screened from all the barrages included in the video clips according to the splicing characteristics of each barrage. Since the object watching the video is generally more interested in the barrage related to the video clip, the barrage related to the video clip can be screened out and displayed through the above steps 111 to 117, thereby increasing the viewing interest of the object.

Alternatively, in one embodiment, step 117 may include the following steps 1171 to 1173 (not shown):

1171. and carrying out full-connection transformation on the splicing characteristics corresponding to each bullet screen to obtain a first two-dimensional vector result.

Optionally, the full connection layer may be used to perform full connection transformation on the spliced feature to obtain the first two-dimensional vector result. The first two-dimensional vector result comprises a first-dimensional vector and a second-dimensional vector, the first-dimensional vector in the first two-dimensional vector result is used for describing the probability that the barrage has correlation with the video segment, and the second-dimensional vector in the first two-dimensional vector result is used for describing the probability that the barrage has no correlation with the video segment.

1172. A score value for a first one of the first two-dimensional vector results is obtained.

Wherein, the score value of the first dimension vector in the first two-dimension vector result is: probability of correlation of the bullet screen and the video clip exists. The higher the score, the greater the probability that the bullet screen will have a correlation with the video segment.

1173. And if the score value of the first dimension vector exceeds a preset score threshold value, determining the barrage as the primary selection barrage.

The preset score threshold is a preset critical value reflecting the correlation strength between the barrage and the video clips. The specific values of the preset score threshold should not be construed as limiting the application.

If the score value of the first dimension vector exceeds the preset score threshold, the correlation between the barrage corresponding to the first dimension vector and the video clip meets the preset requirement, so that the barrage corresponding to the first dimension vector can be determined as the primary barrage.

In the embodiment, the screen from all the barrages included in the video clip to the primary barrages can be screened, so that the operation amount in the subsequent operation process of the barrages is reduced, the primary barrages which are more interested in the object can be processed by the method provided by the application by concentrating the calculation force, and the working efficiency is improved.

The above-described operations of steps 111 to 117 are all implemented in the trained composite model shown in fig. 1 c. Alternatively, in one embodiment, prior to step 111, the combined model that has not been trained may be trained, resulting in a trained combined model.

The specific process of training the combined model, which is not trained, and is shaped as shown in fig. 1c, is as follows:

And manually marking 1 on the barrage texts of all barrages of the first training video clip with the video duration of t, randomly selecting part of barrage texts from barrage texts which are not related to the first training video clip, marking 0 on the part of barrage texts, enabling the number of the barrage texts marked 1 to be equivalent to that of barrage texts marked 0, forming a training data set, and training the combined model which is not trained by utilizing the training data set until a combined model which is completed in training is obtained.

120. The probability of the context of one primary screen being another primary screen is determined.

The above of a bullet screen refers to a bullet screen that induces the bullet screen to be emitted. The above for the bullet screen z is not limited to: inducing the object watching the video to send out the barrage y of the barrage z, namely after the object watching the video sees the barrage y, generating a desire to respond to the barrage y, and sending out the barrage z based on the desire.

Optionally, the barrage y may be a question barrage for a video plot question, and correspondingly, the barrage z is an answer barrage to the barrage y; for example, when the video clip reflects that the character of the video character is eating, the barrage y may be specifically a "eating instant noodles", and correspondingly, the barrage z may be specifically an answer barrage such as "oil splash noodles", "Shanxi braised noodles" or "water-in-pulp noodles".

Optionally, the bullet screen y and the bullet screen z can also be idioms; for example, the barrage y may be "as desired", and accordingly, the bullet screen z may be a idiom beginning with "as" such as "as desired", "for a human master" and the like.

For any two primary selection barrages in the first number of primary selection barrages, the probability of one primary selection barrage being the previous probability of the other primary selection barrage can be calculated, and the probability is recorded as the previous probability. For the primary selection barrage c and the primary selection barrage d, the above probability that the primary selection barrage c is the above of the primary selection barrage d can be calculated, and the above probability that the primary selection barrage d is the above of the primary selection barrage c can be calculated.

Alternatively, in a specific embodiment, step 120 may specifically include the following steps 121 to 125 (not shown in the figure):

121. first primary selection barrages and second primary selection barrages are obtained from the first quantity of primary selection barrages, wherein the first primary selection barrages are any one of the first quantity of primary selection barrages, and the second primary selection barrages are any one of the first quantity of primary selection barrages except the first primary selection barrages.

122. And dividing the first primary selection barrage to obtain a third number of first word vectors and position information of each first word vector.

The third number is the specific number of words included in the first primary selection barrage, namely: the first primary selection barrage comprises a third number of words, and correspondingly, the first primary selection barrage is divided to obtain a third number of first word vectors.

The first character vector is a character vector obtained by dividing the first primary selection barrage. The position information of the first character vector reflects the relative position relation of the corresponding first character vector and other first character vectors in the first primary selection barrage.

For example, the first primary selection barrage may be set as: the third number of the instant noodles is 4. After the instant noodle is divided, 4 first word vectors of 'eat', 'instant', 'noodle' and 'eat' corresponding position information 1, 'corresponding position information 2,' corresponding position information 3 and 'noodle' corresponding position information 4 can be obtained. The position of the first word vector corresponding to the position information in the original first primary selection barrage can be reflected through the numerical value size difference between the position information, for example, the position information 2 corresponding to the ' is smaller than 3 and larger than 1, and the ' position ' is before the ' what ' corresponds to the position information 3 and after the ' eat ' corresponding to the position information 1.

123. And dividing the second primary selection barrage to obtain a fourth number of second word vectors and position information of each second word vector.

The fourth number is the specific number of words included in the second primary selection screen, namely: the second primary selection barrage comprises a fourth number of words, and correspondingly, the fourth number of second word vectors can be obtained by dividing the second primary selection barrage.

The second character vector is a character vector obtained by dividing the second primary selection barrage. The position information of the second word vector reflects: the relative position of the corresponding second character vector and other second character vectors in the second primary selection barrage.

For example, the second primary selection barrage may be set as: the fourth number of "Shaanxi braised noodles" is 4. After the "Shaanxi stewed noodles" are divided, 4 second word vectors "Shaanxi", "xi", "stewed" and "noodle" can be obtained, and the position information 6 corresponding to "Shaanxi", "xi" and the position information 7 corresponding to "stewed" and the position information 8 corresponding to "braised" and the position information 9 corresponding to "noodle" are obtained.

124. Performing feature extraction on a global indicator, position information of the global indicator, a third number of first word vectors, position information of each first word vector, a separator, position information of the separator, a fourth number of second word vectors and position information of each second word vector to obtain a summation number of feature vectors; wherein the sum number is a sum of the third number, the fourth number, a number of global indicators, and a number of separators.

The global indicator indicates the beginning of the process of processing the first and second primary screens, and thus the global indicator may be followed in sequence by the first primary screen, the separator, and the second primary screen. The separator is used for separating the first primary selection barrage from the second primary selection barrage. Alternatively, the global indicator may be represented by CLS and the separator by SEP, see fig. 1d for details.

Since the global indicator is located before the first initial selection screen, the positional relationship between the global indicator and the first initial selection screen can be embodied by setting the positional information of the global indicator to be smaller than the positional information of the forefront first character variable of the first initial selection screen.

Also, since the separator is used to separate the first and second primary screens, the position information of the separator may be set to be greater than the position information of the last first character variable and less than the position information of the forefront second character variable.

In the step 124, during the feature extraction operation, feature extraction may be specifically performed through the third ALBERT model, so as to obtain feature vectors corresponding to the global indicator, the third number of first word variables, the separator, and the fourth number of second word variables, thereby obtaining a sum number of feature vectors.

125. And calculating the upper probability of the first primary selection barrage serving as the upper of the second primary selection barrage according to the added number of feature vectors.

By performing steps 121 to 125 on any two primary selection screens (e.g., the first primary selection screen and the second primary selection screen) of the first number of primary selection screens, the probability of one primary selection screen of the two primary selection screens being used as the probability of the other primary selection screen can be obtained, so that the probabilities of the plurality of primary selection screens that the first number of primary selection screens correspond together can be obtained. The method is used for acquiring the above probability aiming at the first primary barrage instead of all barrages included in the video clips, so that the operation amount is greatly reduced, and the calculation efficiency of the above probability is improved.

Alternatively, in one embodiment, step 125 may include the following steps 1251 through 1254 (not shown):

1251. and obtaining a target feature vector corresponding to the global indicator in the added number of feature vectors.

For the sum number of feature vectors, a feature vector corresponding to the global indicator therein is acquired, and the feature vector is noted as a target feature vector.

1252. And carrying out pooling treatment on the feature vectors except the target feature vector in the added plurality of feature vectors to obtain a pooling treatment result.

Among the added feature vectors, the feature vectors other than the target feature vector include a third number of feature vectors corresponding to a third number of first word variables, a feature vector corresponding to a separator, and a fourth number of feature vectors corresponding to a fourth number of second word variables, see fig. 1d for details.

The Pooling process may specifically be a Max Pooling process, as shown in fig. 1 d; the Pooling process may also be an average Pooling (Mean-Pooling) process. The particular pooling type of pooling process should not be construed as limiting the application.

1253. And performing full-connection transformation on the target feature vector and the pooling processing result to obtain a second two-dimensional vector result.

Optionally, the full-connection layer may be used to perform full-connection transformation on the target feature vector and the pooling result to obtain a second two-dimensional vector result. The second two-dimensional vector result comprises a first-dimensional vector and a second-dimensional vector, the first-dimensional vector in the second two-dimensional vector result is used for describing the upper probability that the first primary selection barrage is used as the upper text of the second primary selection barrage, and the second-dimensional vector in the second two-dimensional vector result is used for describing the upper text probability that the first primary selection barrage is not used as the upper text of the second primary selection barrage.

1254. Obtaining a score value of a first dimension vector in the second dimension vector result, wherein the score value of the first dimension vector in the second dimension vector result is as follows: the first primary screen acts as the contextual probability of the context of the second primary screen.

Wherein, the score value of the first dimension vector in the second dimension vector result is: the first primary screen acts as the contextual probability of the context of the second primary screen. The higher the score value, the greater the probability of the first primary screen being the context of the second primary screen.

130. And screening a second number of associated barrage sets from the first number of primary selection barrages according to the magnitude relation between the upper probability and a preset probability threshold.

The preset probability threshold is a preset threshold value. The second number is a positive integer and the specific value of the second number should not be construed as limiting the application.

The associated barrage set includes one superordinate barrage and at least one subordinate barrage. Wherein, the upper barrage and the lower barrage are relative concepts, namely, the upper barrage is the upper barrage of any lower barrage; any one of the below curtains is the above curtain. For example, if a set of associated scrims includes an upper scrip g, and a lower scrip h, a lower Wen Danmu i, a lower Wen Danmu j, then the upper scrip g is the upper scrip of the lower scrip h, i, or j; lower Wen Danmu h is the lower Wen Danmu of the upper bullet screen g, lower Wen Danmu i is the lower Wen Danmu of the upper bullet screen g, and lower Wen Danmu k is the lower bullet screen of the upper bullet screen g.

The above concepts set forth above continue to describe the above barrage: lower Wen Danmu h, lower Wen Danmu i, lower Wen Danmu j are each a bullet screen for responding to upper Wen Danmu g that the subject watching the video sends out after seeing the upper bullet screen g. The following bullet screen j, wen Danmu h, wen Danmu i, and the following bullet screen j may be sent by the same object for watching video, or may be sent by different objects for watching video, and whether the same object for watching video sends out or not should not be construed as limiting the present application.

Optionally, in a specific embodiment, the preset probability threshold includes an association probability threshold and an overall probability threshold; accordingly, step 130 may include the following steps 131 to 135 (not shown in the figures):

131. and acquiring two primary selection barrages which form a context relation and have the context probability larger than the association probability threshold, and respectively determining the two primary selection barrages as a context barrage and a context barrage according to the context association relation of the two primary selection barrages.

The association probability threshold is a preset threshold value, which may be 0.05, or may be another value, for example, 0.07, and the specific value of the threshold value should not be construed as a limitation of the present application.

After calculating the plurality of context probabilities in step 120, the context probabilities are compared with the associated probability threshold for each context probability, respectively, to obtain a number of context probabilities having a value greater than the associated probability threshold. Each of the plurality of context probabilities corresponds to two primary selection screens that make up a context. For two primary selection screens constituting a context relationship, the two primary selection screens may be distinguished into a top screen and a bottom screen based on the context association relationship.

132. And for all initially selected backlashes with the upper probability being greater than the associated probability threshold, if the upper backlashes with a plurality of lower backlashes exist, calculating the overall probability value of the upper backlashes.

After the two initially selected scrips that make up the context and have a probability of being greater than the associated probability threshold are distinguished from each other, step 131, there may be a scrip with multiple scrips. For details, referring to fig. 1e, for example, for the context barrage b1, a context barrage with which the context relation is constructed and the context probability is greater than the associated probability threshold may include: b0, b2, b3, b4, b9. Therefore, the overall probability value of the upper Wen Danmu b1 needs to be calculated. The two primary selection screens connected by the arrow shown in fig. 1e form a context, wherein the primary selection screens connected by the arrow ends are the upper screens of the primary selection screens connected by the arrow ends, and the primary selection screens connected by the arrow ends are the lower screens of the primary selection screens connected by the arrow ends.

In calculating the overall probability value above, the sum of the probabilities of the contexts corresponding to the multiple context backlashes belonging to the same context backlashes may be calculated. For example, for the above barrage b1, the above probabilities p0, b2, p3, b3, p4, b9 corresponding to b0, b3, b4, b9 can be obtained, and then the sum of the above five above barrages can be calculated: the sum p0+ p2+ p3+ p4+ p9 is the overall probability value for the bullet screen b1 above.

For the upper barrage with a plurality of lower barrages, the calculation can be carried out through the mode, so that the integral probability value corresponding to the upper barrage is obtained. By repeating the above mode, a plurality of overall probability values can be obtained.

133. And screening out the upper barrages with the overall probability value larger than the overall probability threshold, wherein the upper barrages with the overall probability value larger than the overall probability threshold are alternative upper barrages.

The overall probability threshold is a preset threshold value, the specific value of which should not be construed as limiting the application.

For each of the above scrims for which a global probability value exists, its corresponding global probability value may be compared to a global probability threshold: and if the overall probability value is greater than the overall probability threshold value, marking the overall probability value corresponding to the previous barrage as an alternative previous barrage.

Continuing with the above example, b1, b5 shown in FIG. 1e may be provided as alternative backlashes.

134. And for each primary selection barrage with the probability higher than the associated probability threshold, acquiring the target primary selection barrage with the highest probability.

Alternatively, in the above embodiment, there may be a case where there is a probability that one context screen and a plurality of context screens exist, in which case, the context probability that the value of the context screen is the largest may be calculated, and the context screen corresponding to the context probability may be noted as the target context screen. For details, refer to fig. 1e, for example, for b0, the probabilities of b1 and b3 are both the probabilities of b0 and b1, the probabilities of b0 and b3, and the probabilities of p1 and p3 are obtained and compared. If p1 is greater than p3, b1 is b 0; if p1 is less than p3, b3 is b 0's target previous barrage; if p1 and p3 are equal, then randomly selecting one of b1 and b3 as the target superordinate barrage of b 0.

135. And screening a second number of associated barrages from all the primary selected barrages with the upper probabilities greater than the associated probability threshold according to the subordinate relation between the target upper barrages and the plurality of alternative upper barrages.

In the above embodiment, the preliminary screening may be performed on the primary selection barrages according to the relationship between the above probability and the associated probability threshold, so as to obtain all the primary selection barrages with the above probability greater than the associated probability threshold. And then, the primary selection barrages subjected to primary screening are screened again according to the subordinate relations between the target superelevation barrages and the alternative superelevation barrages, and the primary selection barrages subjected to secondary screening and the primary selection barrages with the contextual relevance relations can form a relevance barrage set. Through the mode of twice screening, the operand can be reduced, and the screening efficiency is improved.

Alternatively, in a specific embodiment, step 135 may specifically include the following steps 1351 to 1352 (not shown in the figure):

1351. and if the target upper bullet screen is one of a plurality of alternative upper bullet screens, reserving a primary bullet screen corresponding to the target upper bullet screen and a context association relation between the primary bullet screen and the target upper bullet screen.

1352. And discarding the primary selection barrage corresponding to the target superordinate barrage if the target superordinate barrage is not any one of the plurality of alternative superordinate barrages.

The above operation is performed for each primary selection barrage that has undergone a preliminary screening with an above probability greater than the associated probability threshold. If the target upper barrage of a primary selected barrage is one of a plurality of alternative upper barrages, taking the primary selected barrage as the lower barrage of the target upper barrage; if the target previous barrage of a primary selected barrage is not any of the multiple alternative previous barrages, the primary selected barrage is directly discarded.

The description proceeds with reference to the examples above:

for details, referring to fig. 1e, b1 and b5 are known as alternative supervisors, for b0, if b0 is b1, then b0 is an alternative supervisors, so b0 can be reserved, and the context association between b0 and b1 can be reserved: b0 is the lower Wen Danmu of b1 and b1 is the upper barrage of b0. If the target superordinate barrage of b0 is b3, b0 may be discarded since b3 is not an alternative superordinate barrage.

For the primary selection barrages which undergo preliminary screening and have the upper probability greater than the associated probability threshold, steps 1351 to 1352 are performed to obtain an upper barrage with a plurality of lower barrages and an upper barrage with one lower barrage, and the upper barrage with the plurality of lower barrages and the plurality of lower barrages thereof are jointly recorded as an associated barrage set.

Optionally, in one embodiment, the screening out the second number of associated barrage sets includes the following steps A1 to A2 (not shown in the figure) in addition to performing the steps 1351 to 1352:

a1, calculating text similarity of any two associated barrage sets in the plurality of associated barrage sets.

Alternatively, the calculation of the text similarity may be performed based on the jaccard similarity, or may be performed in other manners, for example: text similarity is calculated based on cosine distance, and text similarity is calculated based on Euclidean distance. The specific method of calculating text similarity should not be construed as limiting the application.

A2, if the text similarity is greater than a preset similarity threshold, combining the two associated barrage sets with the text similarity greater than the preset similarity threshold into one associated barrage set.

The preset similarity threshold is a preset threshold for measuring the similarity degree of the text, and the magnitude of the threshold should not be construed as limiting the application. If the text similarity is greater than the preset similarity threshold, the likelihood that the above barrages of the two associated barrage sets are semantic approximations is higher.

In the above embodiment, the occurrence of the above barrage with similar semantics can be reduced by merging two associated barrage sets with text similarity greater than the preset similarity threshold, so that the number of associated barrage sets in the viewing field of the object viewing video can be reduced, and the video viewing burden of the object viewing video can be further reduced.

Alternatively, in a specific embodiment, step A2 may specifically include the following steps a21 to a22 (not shown in the figure):

a21, acquiring the upper barrage with the larger overall probability value in the two associated barrage sets, and taking the upper Wen Danmu with the larger overall probability value as a new upper barrage.

A22, taking all the lower scrip in the two associated scrip sets as the lower scrip of the new upper scrip.

The greater the overall probability value, the more pertinent the semantic description of the above barrage will be to the following barrage. Therefore, the upper barrage can be screened according to the overall probability value, the upper barrage with the maximum overall probability value is used as a new upper barrage, and then the lower barrages of the two related barrage sets originally are used as the lower barrages of the newly selected upper barrage. The upper barrage which is more attached to the semantic description of the lower barrage is selected through the integral probability value, so that the watching burden of the object watching the video is further reduced, and the watching experience is improved; and the associated barrage sets are combined, so that the number of the associated barrage sets can be reduced, and the video object is more beneficial to concentrating.

140. At least one associated barrage set of the second number of associated barrage sets is displayed.

The second number of associated barrage sets may be displayed, or a portion of the associated barrage sets may be selected from the second number of associated barrage sets and displayed.

Optionally, in a specific embodiment, "selecting and presenting a portion of the associated barrage set from the second plurality of associated barrage sets" may specifically include the following steps 141 to 146 (not shown in the figure):

141. and carrying out fourth coding processing on each associated barrage set to obtain corresponding associated set characteristics.

Optionally, referring to fig. 1f for details, for each associated barrage set, a fourth coding process may be performed by using a fourth ALBERT model that is trained to obtain a corresponding associated set feature.

142. And obtaining a plurality of interest labels corresponding to the object, and carrying out fifth coding processing on the interest labels to obtain a plurality of corresponding interest label characteristics.

The object is a person watching a video; the object may be an object having an account number and viewing a video in an account number login state, a guest not having an account number, or a person having an account number but still not having an account number and viewing a video in a guest state.

The interest tag is a tag reflecting the preference of the subject to watch the video.

Alternatively, in a specific embodiment, the tag may be a tag of a video, and if the object views a certain video for more than a predetermined period of time, or if the object performs an operation (such as endorsing the video, collecting the video, forwarding the video, etc.) that the object likes, the tag of the video may be used as an interest tag of the object. The tag of the video may be a tag reflecting the video type, for example: action, literature, comedy, etc.; the tag of the video may also be a tag reflecting the scene in which the video is located, for example: campus, bar, etc.; the tags of a video may also reflect the director's role of the video, such as: artist a, artist B, etc.

In another embodiment, interest tags may also be manually added to objects by video editors. It should be understood that the specific manner in which the plurality of interest tags corresponding to the object are obtained should not be construed as limiting the application.

Optionally, referring to fig. 1f for details, for a plurality of interest tags, a fifth encoding process may be performed on the trained fifth ALBERT model to obtain interest tag features corresponding to each of the interest tags, so as to obtain a plurality of interest tag features. The number of the interest tag features is the same as that of the interest tags, and the interest tag features are in one-to-one correspondence.

143. And according to an attention mechanism, fusing the association set feature and the interest tag features to obtain a third fusion feature.

The description proceeds with reference to the examples above: the method comprises the steps that a correlation set feature is not used as 1 e-dimensional vector, the number of interest tag features is f, each interest tag feature is also an e-dimensional vector, and f e-dimensional vectors are shared by the interest tag features, wherein e and f are positive integers; the specific implementation process of fusing the association set feature and the plurality of interest tag features according to the attention mechanism is as follows:

and calculating the inner product of the e-dimensional vector of the association set feature and f e-dimensional vectors to obtain f products.

And carrying out softmax transformation on the f products to obtain the weight values respectively corresponding to the e-dimensional vector of the association set feature and the f interest tag features.

Based on the weight values corresponding to the f interest tag features, a weighted sum of the f interest tag features with respect to the association set features is calculated, where the weighted sum is the third fusion feature, and for details, please refer to fig. 1f.

144. And determining the interest degree of the object in the associated barrage set according to the third fusion characteristic.

The third fusion feature corresponding to each associated barrage set may reflect the relevance of the associated barrage set to the plurality of interest tags corresponding to the object. Therefore, the interest degree of the object to the corresponding associated barrage set can be calculated according to the third fusion characteristic.

Alternatively, in one embodiment, step 144 may include, in particular, the following steps 1441 to 1442 (not shown in the figures):

1441. and carrying out full-connection transformation on the third fusion characteristic to obtain a third two-dimensional vector result.

Optionally, the full-connection layer may be used to perform full-connection transformation on the third fusion feature to obtain a third two-dimensional vector result. The third two-dimensional vector result comprises a first-dimensional vector and a second-dimensional vector, the first-dimensional vector in the third two-dimensional vector result is used for describing the probability that the object is interested in the associated barrage set, and the second-dimensional vector in the third two-dimensional vector result is used for describing the probability that the object is not interested in the associated barrage set.

1442. And obtaining a score value of the first-dimension vector in the third-dimension vector result.

And the score value of the first dimension vector in the third dimension vector result is the interestingness.

Steps 1441 through 1442 may be performed for each of the second number of associated barrage sets, such that a respective interest level for each associated barrage set may be obtained.

Steps 141 through 144 described above are implemented based on the trained combined model shown in fig. 1 f. Optionally, in a specific implementation, before step 141, an embodiment of the present application may further include training the combined model that is not trained, to obtain a training process of the trained combined model.

The specific process of training the combined model, which is not trained, and is shaped as shown in fig. 1f, is as follows:

marking 1 on the barrage texts of all barrages of the second training video segment with the video duration of t, randomly selecting part of barrage texts from the barrage texts which are not praised by the object, marking 0 on the part of barrage texts, enabling the number of the barrage texts marked 1 to be equivalent to that of barrage texts marked 0, forming a training data set, and training the combined model which is not trained by utilizing the training data set until a combined model which is completed in training is obtained.

145. And determining the first K associated barrage sets with high interestingness values.

K is a positive integer, and the specific value of K should not be construed as limiting the application.

Optionally, when determining the first K associated barrage sets, the second number of associated barrage sets may be ordered according to the descending order of the interest degree values, to obtain a sequence composed of the ordered associated barrage sets, and intercept the first K associated barrage sets of the sequence. When the first K associated barrage sets are determined, the associated barrage set with the highest interestingness value can be selected from the second plurality of associated barrage sets each time, and each time one associated barrage set is selected, the selected associated barrage set is deleted from the second plurality of associated barrage sets, and the record of the selection times is carried out until the K associated barrage sets are selected. It should be understood that the specific implementation of the first K associated bullet screen sets that determine a high interestingness value should not be construed as limiting the application.

146. And displaying the first K associated barrage sets.

In the above embodiment, the interest degree of the object on each associated barrage set may be calculated according to a plurality of interest tags of the object, and K associated barrage sets with the front interest degree value may be finally determined, and the first K associated barrage sets are displayed, so that the associated barrage sets displayed for the object watch are more attached to the interest of the object, thereby facilitating the improvement of the interest of the object watching the video.

Alternatively, in one embodiment, step 146 may include steps 1461 through 1462 (not shown) as follows:

1461. and determining the distance between each associated barrage set in the first K associated barrage sets and the horizontal center line of the display interface according to the interestingness.

The calculation rule of the distance can be formulated according to the standard that the higher the interest degree is, the closer the associated barrage set is to the horizontal central line of the display interface.

Alternatively, in one embodiment, the following formula may be used:

distance of the associated barrage set from the horizontal center line of the display interface=0.5×0.5×1-the interest level corresponding to the associated barrage set, the distance of the associated barrage set from the horizontal center line of the display interface is calculated.

1462. And displaying the first K associated barrage sets according to the distance between each associated barrage set and the horizontal center line of the display interface.

The first K associated barrage sets are displayed according to the mode that the higher the interest level is and the closer the associated barrage sets are to the horizontal center line of the display interface, so that an object watching videos can more easily see the associated barrage sets with high interest level, and the watching interest of the object is further improved.

Optionally, in a specific implementation manner, the method provided by the embodiment of the present application may further include:

adjusting the upper barrage and all the lower barrages belonging to the same associated barrage set to the same text format; and displaying the associated barrage set with the text format adjusted.

The same text format may refer to the same font size, font format, font color, or background color as the above and all the below scrims belonging to the same associated set of scrims. Besides adjusting the upper and all lower scrips belonging to the same associated scrip set into the same text format, the upper and all lower scrips can be displayed in the same display interface, and the positions of the upper and all lower scrips are close. By adjusting the barrages in the same associated barrage set to be in the same text format and displaying the barrages in the same display interface, the barrages belonging to the same associated barrage set can be intuitively known by the object watching the video, and the load of watching the barrages of the object watching the video is reduced.

The embodiment of the application can identify the first number of primary selection barrages with the correlation meeting the preset requirement from all barrages included in the video clip; then, for any two primary selection barrages, determining the upper probability that one primary selection barrage is the upper probability of the other primary selection barrage, and obtaining a plurality of upper probabilities corresponding to the first number of primary selection barrages; then, according to the relative size relation between the numerical value of the probability and the preset probability threshold value, a second number of associated barrage sets are screened out from the primary selected barrages; and displaying at least one associated barrage set of the second number of associated barrage sets. In the method, a plurality of primary selection barrages related to video clips are firstly identified, and the probability of the upper surface of any two primary selection barrages in the plurality of primary selection barrages is calculated; and screening out a plurality of associated barrage sets according to the magnitude relation between the probability and the preset probability threshold value, and displaying at least one associated barrage set. In the above embodiment, the associated barrage set including one upper barrage and at least one lower barrage may be obtained and displayed.

The method provided by the embodiment of the application can enable the object watching the video to more intuitively and rapidly know the context of the barrage dialogue, reduce the burden of the object to watch the associated barrage, and promote the watching experience.

The method described in the above embodiments will be described in further detail below.

In this embodiment, a method according to an embodiment of the present application will be described in detail by taking an example of performing a first encoding process on each bullet screen.

As shown in fig. 2, a specific flow of a barrage display method is as follows, where the barrage display method may be executed by an electronic device:

201. and carrying out first coding processing on each barrage to obtain corresponding barrage text characteristics.

202. And acquiring a plurality of video key frames in the video fragment, and performing second coding processing on the plurality of video key frames to obtain a plurality of corresponding key frame representation characteristics.

203. And according to the attention mechanism, fusing the barrage text features and the plurality of key frame representation features to obtain a first fusion feature.

204. And acquiring a plurality of content information in the video clip, and performing third coding processing on the plurality of content information to obtain a plurality of corresponding content text characteristics.

205. And according to the attention mechanism, fusing the barrage text features and the content text features to obtain a second fused feature.

206. And performing splicing treatment on the first fusion characteristic and the second fusion characteristic to obtain a splicing characteristic.

207. And identifying a first number of primary selection barrages with the correlation with the video clips meeting the preset requirement according to the splicing characteristics corresponding to each barrage.

Alternatively, in a specific embodiment, step 207 may specifically include the steps of: for the splicing characteristic corresponding to each bullet screen, carrying out full-connection transformation on the splicing characteristic to obtain a first two-dimensional vector result;

obtaining a score value of a first dimension vector in the first two-dimensional vector result, wherein the score value of the first dimension vector in the first two-dimensional vector result is: probability of correlation of the bullet screen and the video clip;

and if the score value of the first dimension vector exceeds a preset score threshold value, determining the barrage as the primary selection barrage.

208. And acquiring a first primary selection barrage and a second primary selection barrage from the first number of primary selection barrages.

The first primary selection barrage is any primary selection barrage in the first quantity of primary selection barrages, and the second primary selection barrage is any primary selection barrage except the first primary selection barrage in the first quantity of primary selection barrages.

209. And dividing the first primary selection barrage to obtain a third number of first word vectors and position information of each first word vector.

210. And dividing the second primary selection barrage to obtain a fourth number of second word vectors and position information of each second word vector.

211. And extracting features of the global indicator, the position information of the global indicator, the third number of first word vectors, the position information of each first word vector, the separator, the position information of the separator, the fourth number of second word vectors and the position information of each second word vector to obtain a summation number of feature vectors.

Wherein the sum number is a sum of the third number, the fourth number, a number of global indicators, and a number of separators.

212. And calculating the upper probability of the first primary selection barrage serving as the upper of the second primary selection barrage according to the added number of feature vectors.

Optionally, in a specific embodiment, step 212 specifically includes the steps of:

obtaining a target feature vector corresponding to the global indicator in the added plurality of feature vectors; pooling the feature vectors except the target feature vector in the added plurality of feature vectors to obtain a pooling result; performing full-connection transformation on the target feature vector and the pooling processing result to obtain a second two-dimensional vector result; obtaining a score value of a first dimension vector in the second dimension vector result, wherein the score value of the first dimension vector in the second dimension vector result is as follows: the first primary screen acts as the contextual probability of the context of the second primary screen.

213. And acquiring two primary selection barrages which form a context relation and have the context probability larger than the association probability threshold, and respectively determining the two primary selection barrages as a context barrage and a context barrage according to the context association relation of the two primary selection barrages.

214. And for all initially selected backlashes with the upper probability being greater than the associated probability threshold, if the upper backlashes with a plurality of lower backlashes exist, calculating the overall probability value of the upper backlashes.

215. And screening out the upper barrages with the overall probability value larger than the overall probability threshold, wherein the upper barrages with the overall probability value larger than the overall probability threshold are alternative upper barrages.

216. And for each primary selection barrage with the probability higher than the associated probability threshold, acquiring the target primary selection barrage with the highest probability.

217. And screening a second number of associated barrages from all the primary selected barrages with the upper probabilities greater than the associated probability threshold according to the subordinate relation between the target upper barrages and the plurality of alternative upper barrages.

Optionally, in a specific embodiment, step 217 specifically includes the steps of:

if the target upper bullet screen is one of a plurality of alternative upper bullet screens, reserving a primary bullet screen corresponding to the target upper bullet screen and a context association relation between the primary bullet screen and the target upper bullet screen;

And discarding the primary selection barrage corresponding to the target superordinate barrage if the target superordinate barrage is not any one of the plurality of alternative superordinate barrages.

218. At least one associated barrage set of the second number of associated barrage sets is displayed.

Optionally, in a specific embodiment, step 218 specifically includes the steps of:

performing fourth coding processing on each associated barrage set to obtain corresponding associated set features; acquiring a plurality of interest labels corresponding to the object, and performing fifth coding processing on the interest labels to obtain a plurality of corresponding interest label characteristics; according to an attention mechanism, fusing the association set features and the interest tag features to obtain a third fusion feature; according to the third fusion characteristic, determining the interest degree of the object on the associated barrage set; determining the first K associated barrage sets with high interestingness values; and displaying the first K associated barrage sets.

From the above, a first number of primary selection barrages with correlation meeting a preset requirement can be identified from all barrages included in the video clip; then, for any two primary selection barrages, determining the upper probability that one primary selection barrage is the upper probability of the other primary selection barrage, and obtaining a plurality of upper probabilities corresponding to the first number of primary selection barrages; then, according to the relative size relation between the numerical value of the probability and the preset probability threshold value, a second number of associated barrage sets are screened out from the primary selected barrages; and at least one associated barrage set in the second number of associated barrage sets is displayed, when the at least one associated barrage set is displayed, the integrity of the same associated barrage set can be enhanced by means of drawing the position and adjusting the text format, the associated barrages can be found by the objects watching the video quickly, the objects watching the video can understand the video scenes in an auxiliary manner through the associated barrages issued by other objects, and therefore the enthusiasm of the objects watching the video for participating in barrage interaction is improved.

The method provided by the embodiment of the application can enable the object watching the video to more intuitively and rapidly know the context of the barrage dialogue, reduce the burden of the object to watch the associated barrage, promote the barrage interaction atmosphere of the video platform and promote the watching experience.

In order to better implement the method, the embodiment of the application also provides a bullet screen display device which can be integrated in electronic equipment, wherein the electronic equipment can be a terminal, a server and the like.

For example, in this embodiment, a detailed description will be given of a method according to an embodiment of the present application, taking a specific integration of a bullet screen display apparatus in an electronic device as an example.

For example, as shown in fig. 3, the bullet screen display apparatus may include:

the primary selection barrage identification unit 301 is configured to identify a first number of primary selection barrages from all barrages included in the video clip, where the primary selection barrages are barrages whose correlation with the video clip meets a preset requirement;

a context probability determining unit 302, configured to determine a context probability, where the context probability refers to a probability that a first primary selection bullet screen is a context of another first primary selection bullet screen;

a barrage set screening unit 303, configured to screen a second number of associated barrage sets from the first number of first selected barrages according to a magnitude relation between the above probability and a preset probability threshold;

And a barrage set display unit 304, configured to display at least one associated barrage set in the second number of associated barrage sets.

In some embodiments, the primary screen identification unit 301 includes:

In some embodiments, the bullet screen identification subunit comprises:

In some embodiments, the above probability determination unit 302 comprises:

In some embodiments, the probability computation subunit comprises:

In some embodiments, barrage set screening unit 303 includes:

In some embodiments, the set screening subunit comprises:

In some embodiments, the apparatus further comprises:

In some embodiments, the set merge unit comprises:

In some embodiments, the bullet screen set presentation unit 304 includes:

In some embodiments, the interestingness determination subunit comprises:

In some embodiments, the aggregate display subunit comprises:

In the implementation, each unit may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.

As can be seen from the above, the barrage display device of the embodiment can identify the first number of first selection barrages with the correlation meeting the preset requirement from all barrages included in the video clip; then, for any two primary selection barrages, determining the upper probability that one primary selection barrage is the upper probability of the other primary selection barrage, and obtaining a plurality of upper probabilities corresponding to the first number of primary selection barrages; then, according to the relative size relation between the numerical value of the probability and the preset probability threshold value, a second number of associated barrage sets are screened out from the primary selected barrages; and displaying at least one associated barrage set of the second number of associated barrage sets. In the method, a plurality of primary selection barrages related to video clips are firstly identified, and the probability of the upper surface of any two primary selection barrages in the plurality of primary selection barrages is calculated; and screening out a plurality of associated barrage sets according to the magnitude relation between the probability and the preset probability threshold value, and displaying at least one associated barrage set. In the embodiment, the related barrage set comprising the upper barrage and the lower barrage can be obtained and displayed, so that the object watching the video can more intuitively and quickly know the context of the barrage session.

Therefore, the embodiment of the application can reduce the burden of the object to view the associated barrage and improve the viewing experience.

As shown in fig. 4, a schematic structural diagram of an electronic device according to an embodiment of the present application is shown, specifically:

the electronic device may include one or more processor cores 401, one or more computer-readable storage media memory 402, a power supply 403, an input module 404, and a communication module 405, among other components. Those skilled in the art will appreciate that the electronic device structure shown in fig. 4 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or may be arranged in different components. Wherein:

the processor 401 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 402, and calling data stored in the memory 402. In some embodiments, processor 401 may include one or more processing cores; in some embodiments, processor 401 may integrate an application processor that primarily processes operating systems, object interfaces, application programs, and the like, with a modem processor that primarily processes wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the electronic device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

The electronic device also includes a power supply 403 for powering the various components, and in some embodiments, the power supply 403 may be logically connected to the processor 401 by a power management system, such that charge, discharge, and power consumption management functions are performed by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The electronic device may also include an input module 404, which input module 404 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to object settings and function control.

The electronic device may also include a communication module 405, and in some embodiments the communication module 405 may include a wireless module, through which the electronic device may wirelessly transmit over a short distance, thereby providing wireless broadband internet access to the object. For example, the communication module 405 may be used to facilitate objects in sending and receiving emails, browsing web pages, accessing streaming media, and so forth.

Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 401 in the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement various functions as follows:

identifying a first number of primary selection barrages from all barrages included in the video clips, wherein the primary selection barrages are barrages with correlation with the video clips meeting preset requirements; determining an upper probability, wherein the upper probability refers to the probability that one primary selection barrage is the upper of another primary selection barrage; screening a second number of associated barrage sets from the first number of primary selection barrages according to the magnitude relation between the upper probability and a preset probability threshold; at least one associated barrage set of the second number of associated barrage sets is displayed.

The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.

From the above, the embodiment of the application can enable the object watching the video to more intuitively and rapidly know the context of the barrage dialogue, reduce the burden of the object to watch the associated barrage, and promote the watching experience.

Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application provides a computer readable storage medium having stored therein a plurality of instructions capable of being loaded by a processor to perform the steps of any of the bullet screen display methods provided by the embodiments of the present application. For example, the instructions may perform the steps of:

Wherein the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.

According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the methods provided in various alternative implementations of the machine translation aspect or the content generation aspect provided in the above-described embodiments.

It will be appreciated that in the specific embodiment of the present application, related data such as a plurality of interest tags corresponding to objects are related to object information, and when the above embodiments of the present application are applied to specific products or technologies, permission or consent of the objects needs to be obtained, and collection, use and processing of related data need to comply with related laws and regulations and standards of related countries and regions.

The steps in any bullet screen display method provided by the embodiment of the present application can be executed by the instructions stored in the storage medium, so that the beneficial effects of any bullet screen display method provided by the embodiment of the present application can be achieved, and detailed descriptions of the previous embodiments are omitted herein.

The bullet screen display method, the bullet screen display device, the electronic equipment and the computer readable storage medium provided by the embodiment of the application are described in detail, and specific examples are applied to illustrate the principle and the implementation of the application, and the description of the above embodiment is only used for helping to understand the method and the core idea of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, the present description should not be construed as limiting the present application.

Claims

1. A method of displaying a bullet screen, the method comprising:

identifying a first number of primary selection barrages from all barrages included in the video clips, wherein the primary selection barrages are barrages with correlation with the video clips meeting preset requirements;

2. The method of claim 1, wherein identifying a first number of primary selection shots from all shots included in the video clip comprises:

carrying out first coding treatment on each barrage to obtain corresponding barrage text characteristics;

acquiring a plurality of video key frames in the video clip, and performing second coding processing on the plurality of video key frames to obtain a plurality of corresponding key frame representation characteristics;

according to the attention mechanism, fusing the bullet screen text features and the plurality of key frame representation features to obtain a first fusion feature;

acquiring a plurality of content information in the video clip, and performing third coding processing on the plurality of content information to obtain a plurality of corresponding content text characteristics;

according to the attention mechanism, fusing the bullet screen text features and the content text features to obtain a second fused feature;

splicing the first fusion feature and the second fusion feature to obtain a spliced feature;

and identifying a first number of primary selection barrages with the correlation with the video clips meeting the preset requirement according to the splicing characteristics corresponding to each barrage.

3. The method of claim 2, wherein the identifying a first number of primary selection barrages that have a correlation with the video clip that meets a preset requirement according to the stitching characteristic corresponding to each barrage comprises:

for the splicing characteristic corresponding to each bullet screen, carrying out full-connection transformation on the splicing characteristic to obtain a first two-dimensional vector result;

4. The method of claim 1, wherein the determining the probability of the context comprises:

acquiring a first primary selection barrage and a second primary selection barrage from the first number of primary selection barrages, wherein the first primary selection barrage is any primary selection barrage in the first number of primary selection barrages, and the second primary selection barrage is any primary selection barrage except the first primary selection barrage in the first number of primary selection barrages;

Dividing the first primary selection barrage to obtain a third number of first word vectors and position information of each first word vector;

dividing the second primary selection barrage to obtain a fourth number of second word vectors and position information of each second word vector;

performing feature extraction on a global indicator, position information of the global indicator, a third number of first word vectors, position information of each first word vector, a separator, position information of the separator, a fourth number of second word vectors and position information of each second word vector to obtain a summation number of feature vectors; wherein the sum number is a sum of the third number, the fourth number, a number of global indicators, and a number of separators;

and calculating the upper probability of the first primary selection barrage serving as the upper of the second primary selection barrage according to the added number of feature vectors.

5. The method of claim 4, wherein said calculating the context probability of the first preliminary choice barrage as the context of the second preliminary choice barrage based on the sum and the number of feature vectors comprises:

Obtaining a target feature vector corresponding to the global indicator in the added plurality of feature vectors;

pooling the feature vectors except the target feature vector in the added plurality of feature vectors to obtain a pooling result;

performing full-connection transformation on the target feature vector and the pooling processing result to obtain a second two-dimensional vector result;

obtaining a score value of a first dimension vector in the second dimension vector result, wherein the score value of the first dimension vector in the second dimension vector result is as follows: the first primary screen acts as the contextual probability of the context of the second primary screen.

6. The method of claim 1, wherein the pre-set probability threshold comprises an associated probability threshold and an overall probability threshold, the associated set of scrims comprising one contextual scrip and at least one contextual scrip;

and screening a second number of associated barrage sets from the first number of primary selection barrages according to the magnitude relation between the above probability and a preset probability threshold, wherein the screening comprises the following steps:

acquiring two primary selection barrages which form a context relation and have a higher probability than the association probability threshold, and respectively determining the two primary selection barrages as a context barrage and a lower barrage according to the context association relation of the two primary selection barrages;

For all primary selection barrages with the above probability greater than the associated probability threshold, if the upper barrages with a plurality of lower barrages exist, calculating the overall probability value of the upper barrages;

screening out the upper barrages with the overall probability value larger than the overall probability threshold, wherein the upper barrages with the overall probability value larger than the overall probability threshold are alternative upper barrages;

for each primary selection barrage with the probability greater than the associated probability threshold, acquiring a target primary selection barrage with the highest probability of primary selection barrage;

and screening a second number of associated barrages from all the primary selected barrages with the upper probabilities greater than the associated probability threshold according to the subordinate relation between the target upper barrages and the plurality of alternative upper barrages.

7. The method of claim 6, wherein said screening a second number of associated bullet screen sets from said all primary selected bullet screens having a probability of being greater than said associated probability threshold based on a membership of said target superordinate bullet screen to a plurality of alternative superordinate bullet screens, comprises:

8. The method of claim 7, wherein the method further comprises:

for any two associated barrage sets in a plurality of associated barrage sets, calculating the text similarity of the two associated barrage sets;

if the text similarity is greater than the preset similarity threshold, combining the two associated barrage sets with the text similarity greater than the preset similarity threshold into one associated barrage set.

9. The method of claim 8, wherein merging two associated barrage sets having text similarity greater than a preset similarity threshold into one associated barrage set comprises:

acquiring the upper barrage with a larger overall probability value in the two associated barrage sets, and taking the upper Wen Danmu with the larger overall probability value as a new upper barrage;

and taking all the lower barrages in the two associated barrage sets as the lower barrages of the new upper barrages.

10. The method of claim 1, wherein said presenting at least one associated barrage set of said second plurality of associated barrage sets comprises:

Performing fourth coding processing on each associated barrage set to obtain corresponding associated set features;

acquiring a plurality of interest labels corresponding to the object, and performing fifth coding processing on the interest labels to obtain a plurality of corresponding interest label characteristics;

according to an attention mechanism, fusing the association set features and the interest tag features to obtain a third fusion feature;

according to the third fusion characteristic, determining the interest degree of the object on the associated barrage set;

determining the first K associated barrage sets with high interestingness values;

and displaying the first K associated barrage sets.

11. The method of claim 10, wherein determining the interest level of the object in the associated barrage set based on the third fused feature comprises:

performing full-connection transformation on the third fusion feature to obtain a third two-dimensional vector result;

and obtaining a score value of a first dimension vector in the third two-dimension vector result, wherein the score value of the first dimension vector in the third two-dimension vector result is the interestingness.

12. The method of claim 10, wherein said presenting said first K associated bullet screen sets comprises:

Determining the distance between each associated barrage set in the first K associated barrage sets and the horizontal center line of the display interface according to the interestingness;

and displaying the first K associated barrage sets according to the distance between each associated barrage set and the horizontal center line of the display interface.

13. A barrage display apparatus comprising:

14. An electronic device comprising a processor and a memory, the memory storing a plurality of instructions; the processor loads instructions from the memory to perform the steps in the bullet screen display method of any one of claims 1 to 12.

15. A computer readable storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps in the barrage presentation method of any of claims 1 to 12.