CN113542898A

CN113542898A - Bullet screen track generation and bullet screen display method and device

Info

Publication number: CN113542898A
Application number: CN202110777955.8A
Authority: CN
Inventors: 李亚涛; 李国庆
Original assignee: Beijing IQIYI Science and Technology Co Ltd
Current assignee: Beijing IQIYI Science and Technology Co Ltd
Priority date: 2021-07-09
Filing date: 2021-07-09
Publication date: 2021-10-22
Anticipated expiration: 2041-07-09
Also published as: CN113542898B

Abstract

The embodiment of the invention provides a bullet screen track generation method and a bullet screen display method and device, which relate to the technical field of video processing, wherein the bullet screen track generation method is applied to a server, and comprises the following steps: identifying an area where an object is located in a video frame of a video; according to the identified area, determining similar video frames which contain the same object and have continuous playing time as target video frames; determining the outline of the area where the object is located in each target video frame; generating bullet screen display tracks corresponding to the target video frames according to the determined outline, wherein the bullet screen display tracks are characterized in a combined manner: and the track that the characters in the bullet screen move from the preset starting position to the preset ending position in the determined outline. The method and the device for generating the bullet screen track enrich bullet screen display modes.

Description

Bullet screen track generation and bullet screen display method and device

Technical Field

The invention relates to the technical field of video processing, in particular to a bullet screen track generation method and device and a bullet screen display method and device.

Background

With the progress and application of information technologies such as mobile internet, cloud computing, big data and the like, streaming media application has become an inseparable part of people's life. In order to improve the interaction mechanism of the user, the streaming media service provider supports the user to issue the barrage aiming at the played video through the client, and displays the barrages issued by the user and other users at the client, so that the different users can generate the feeling of content interactive chat aiming at the video. Therefore, the content of the barrage becomes an important link for connecting the content of the video and the user.

In the prior art, when a client displays a bullet screen, the bullet screen usually scrolls to the other side of a video playing area along a straight line, and the bullet screen display mode is single.

Disclosure of Invention

The embodiment of the invention aims to provide a bullet screen track generation method and a bullet screen display method and device, which are used for enriching bullet screen display modes. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a bullet screen trajectory generation method, which is applied to a server, and the method includes:

identifying an area where an object is located in a video frame of a video;

according to the identified area, determining similar video frames which contain the same object and have continuous playing time as target video frames;

determining the outline of the area where the object is located in each target video frame;

and generating bullet screen display tracks corresponding to the target video frames according to the determined outline, wherein each bullet screen display track comprises: the display position of the bullet screen Chinese characters and the corresponding information of the target video frame, the number of the bullet screen Chinese characters corresponding to each display position is 1, and each bullet screen display track is characterized in a combined manner: and the track that the characters in the bullet screen move from the preset starting position to the preset ending position in the determined outline.

In an embodiment of the present invention, the generating a bullet screen display track corresponding to each target video frame according to the determined contour includes:

determining a first display position of a bullet screen Chinese character included in a first bullet screen display track according to a preset initial position in a contour corresponding to a first video frame, and obtaining the first bullet screen display track including the first display position and first video frame information, wherein the first video frame is as follows: the target video frame with the earliest playing time is displayed on the first bullet screen in a track manner: the track with the earliest corresponding display time in the bullet screen display tracks corresponding to the target video frames;

generating each second bullet screen display track according to the following mode, wherein the second bullet screen display tracks are as follows: the bullet screen display tracks corresponding to the target video frames except for the first bullet screen display track are as follows:

moving each display position included in the third bullet screen display track by a preset stepping distance along the outline indication direction of the outline corresponding to the second video frame, wherein the second video frame is as follows: the third barrage display track of the video frames except the first video frame in each target video frame is as follows: the corresponding display time is adjacent to the second bullet screen display track and is positioned in front of the second bullet screen display track;

determining a second display position of the bullet screen Chinese characters included in the second bullet screen display track according to the moved display position and the bullet screen character size;

and obtaining a second bullet screen display track comprising the second display position and the second video frame information.

In an embodiment of the present invention, the determining, according to the moved display position and the size of the bullet screen characters, a second display position of the bullet screen characters included in the second bullet screen display track includes:

determining the shortest distance between each moved display position and a preset initial position in the outline corresponding to the second video frame;

judging whether the shortest distance is larger than or equal to the size of the bullet screen characters;

if not, taking the display positions after the movement as second display positions of the characters in the bullet screen included in the second bullet screen display track;

and if so, taking the display position after each movement and a preset initial position in the outline corresponding to the second video frame as the second display position.

In an embodiment of the present invention, after determining the contour of the region where the object is located in each target video frame, the method further includes:

determining a position pair to be filled in each contour, filling a new contour pixel point between two contour pixel points in the position pair to be filled along the contour indication direction of the contour, so that the distance between two adjacent contour pixel points after filling is not greater than a preset stepping distance, wherein the distance between the two contour pixel points in the position pair to be filled is greater than the stepping distance, and no other contour pixel point exists between the two contour pixel points;

generating a bullet screen display track corresponding to each target video frame according to the determined outline, wherein the bullet screen display track comprises:

and generating bullet screen display tracks corresponding to the target video frames according to the outlines filled with the outline pixel points.

In an embodiment of the invention, the preset starting position is different from the preset ending position.

In an embodiment of the present invention, the identifying the area where the object is located in the video frame of the video includes:

identifying an object contained in a video frame of a video, and obtaining a mask image reflecting an area where the object is located in the video frame;

determining similar video frames which contain the same object and have continuous playing time according to the identified area, wherein the determining is used as a target video frame and comprises the following steps:

determining a video frame corresponding to the mask image reflecting the same object according to the similarity between the mask images;

and determining video frames with continuous playing time from the determined video frames as target video frames.

In an embodiment of the present invention, the determining the contour of the region where the object is located in each target video frame includes:

selecting a representative target mask image in the mask images corresponding to the target video frames according to the similarity between the mask images corresponding to the target video frames;

and determining the outline of the area where the object is located reflected by the target mask image as the outline of the area where the object is located in each target video frame.

obtaining an initial contour of an area where an object is located in each target video frame according to the mask image corresponding to each target video frame;

and performing expansion processing on the initial contour corresponding to each target video frame to obtain the contour of the region where the object is located in each target video frame.

In an embodiment of the present invention, the determining a video frame corresponding to a mask image reflecting a same object according to a similarity between the mask images includes:

determining a mask image with the size of the reflected object area larger than a preset size threshold value from each mask image as a candidate mask image;

and determining the video frame corresponding to the mask image reflecting the same object according to the similarity between the alternative mask images.

In a second aspect, an embodiment of the present invention provides a bullet screen display method, which is applied to a client, and the method includes:

obtaining a bullet screen display track from a server, wherein the bullet screen display track is as follows: a trajectory generated according to the method of the first aspect;

determining a target bullet screen display track corresponding to a first video frame to be displayed according to the information of the video frame included in the bullet screen display track, and obtaining a target bullet screen corresponding to the first video frame to be displayed;

determining target characters in the target bullet screen corresponding to each first video frame according to the number of display positions included in the target bullet screen display track corresponding to each first video frame;

for each first video frame, sequentially determining the display positions of the target characters corresponding to the first video frame according to the arrangement sequence of the display positions in the target bullet screen display track corresponding to the first video frame;

and displaying each target character on each first video frame according to the determined display position of each target character corresponding to each first video frame.

In an embodiment of the present invention, the determining the target text in the target bullet screen corresponding to each first video frame according to the number of display positions included in the target bullet screen display track corresponding to each first video frame includes:

taking the characters with the former reference number in the target bullet screen as the target characters in the target bullet screen corresponding to the starting video frame with the earliest playing time in each first video frame, wherein the reference number is as follows: the number of display positions included in a target bullet screen display track corresponding to the starting video frame;

for each other video frame in the first video frames, determining target characters in a target bullet screen corresponding to the other video frames according to first characters in the target bullet screen corresponding to a previous video frame of the other video frames, a first number of display positions included in a target bullet screen display track corresponding to the previous video frame, and a second number of display positions included in the target bullet screen display track corresponding to the other video frames;

the target characters are the same as the first characters, or the target characters and characters adjacent to the first characters in the characters of the target bullet screen are included, and the number of the target characters is the same as the number of display positions included in a target bullet screen display track corresponding to the first video frame.

In an embodiment of the present invention, determining the target text in the target bullet screen to be displayed in each other video frame in the following manner includes:

if the first number of the display positions in the bullet screen display track corresponding to the previous video frame is equal to the second number of the display positions in the target display track corresponding to other video frames and the display positions are different, determining the first characters as target characters in a target bullet screen to be displayed in other video frames;

if the first number is not equal to the second number and the shortest distance is greater than a preset stepping distance, determining the first characters and the second characters as the target characters, wherein the shortest distance is as follows: and the shortest distance between each display position in the target bullet screen display track corresponding to other video frames and the preset termination position of the object outline represented by the target bullet screen display track, wherein the second characters are as follows: a third number of words in the target bullet screen that are adjacent to the first words and located behind the first words, the third number being: a difference between the second number and the first number;

if the first number is not equal to the second number and the shortest distance is not greater than the step distance, determining a fourth number of characters in the first characters, and a fifth number of characters in the target bullet screen, which are adjacent to the first characters and located behind the first characters, as target characters, wherein the fourth number is: a difference between the second number and the third number, the fifth number being: a difference between the second number and the fourth number.

In an embodiment of the present invention, the obtaining the bullet screen display track from the server includes:

obtaining a data packet containing a bullet screen display track from a server;

and analyzing the data packet based on a first preset character representing the bullet screen display track, a second preset character representing the video frame information and a third preset character representing the display position to obtain the character display position included in each bullet screen display track and the corresponding video frame information.

In an embodiment of the present invention, the obtaining of the target barrage corresponding to the first video frame to be displayed includes:

acquiring a bullet screen with a display type of being displayed along the outline of the area where the object is located from a preset bullet screen library, and taking the bullet screen as a target bullet screen corresponding to a first video frame to be displayed;

or

And obtaining a bullet screen which is input by a user on the user interface and is used for displaying along the outline of the area where the object is located, and taking the bullet screen as a target bullet screen corresponding to the first video frame to be displayed.

In a third aspect, an embodiment of the present invention provides a bullet screen trajectory generation device, which is applied to a server, where the device includes:

the area identification module is used for identifying the area where the object is located in the video frame of the video;

the video frame determining module is used for determining similar video frames which contain the same object and are continuous in playing time according to the identified area to serve as target video frames;

the contour determining module is used for determining the contour of the region where the object is located in each target video frame;

and the track generation module is used for generating bullet screen display tracks corresponding to the target video frames according to the determined outline, wherein each bullet screen display track comprises: the display position of the bullet screen Chinese characters and the corresponding information of the target video frame, the number of the bullet screen Chinese characters corresponding to each display position is 1, and each bullet screen display track is characterized in a combined manner: and the track that the characters in the bullet screen move from the preset starting position to the preset ending position in the determined outline.

In an embodiment of the invention, the trajectory generating module includes:

a first trajectory obtaining submodule, configured to determine, according to a preset initial position in a contour corresponding to a first video frame, a first display position of a bullet screen Chinese character included in a first bullet screen display trajectory, and obtain a first bullet screen display trajectory including the first display position and first video frame information, where the first video frame is: the target video frame with the earliest playing time is displayed on the first bullet screen in a track manner: the track with the earliest corresponding display time in the bullet screen display tracks corresponding to the target video frames;

generating each second bullet screen display track according to the following sub-modules, wherein the second bullet screen display track is as follows: the bullet screen display tracks corresponding to the target video frames except for the first bullet screen display track are as follows:

and the position moving submodule is used for moving each display position included in the third bullet screen display track by a preset stepping distance along the contour indication direction of the contour corresponding to the second video frame, wherein the second video frame is as follows: the third barrage display track of the video frames except the first video frame in each target video frame is as follows: the corresponding display time is adjacent to the second bullet screen display track and is positioned in front of the second bullet screen display track;

the position determining submodule is used for determining a second display position of the bullet screen Chinese characters included in the second bullet screen display track according to the moved display position and the bullet screen character size;

and the second track obtaining submodule is used for obtaining a second bullet screen display track comprising the second display position and the second video frame information.

In an embodiment of the present invention, the position determining submodule is specifically configured to determine a shortest distance between each of the moved display positions and a preset starting position in the contour corresponding to the second video frame; judging whether the shortest distance is larger than or equal to the size of the bullet screen characters; if not, taking the display positions after the movement as second display positions of the characters in the bullet screen included in the second bullet screen display track; and if so, taking the display position after each movement and a preset initial position in the outline corresponding to the second video frame as the second display position.

In an embodiment of the present invention, the apparatus further includes: a contour-filling module for filling the contour of the object,

the contour filling module is used for determining a position pair to be filled in each contour after the contour determining module, filling a new contour pixel point between two contour pixel points in the position pair to be filled along the contour indicating direction of the contour, and enabling the distance between two adjacent contour pixel points after filling to be not more than a preset stepping distance, wherein the distance between the two contour pixel points in the position pair to be filled is more than the stepping distance, and other contour pixel points do not exist between the two contour pixel points;

and the track generation module is specifically used for generating bullet screen display tracks corresponding to the target video frames according to the outlines after the outline pixel points are filled.

In an embodiment of the present invention, the area identification module is specifically configured to identify an object included in a video frame of a video, and obtain a mask image reflecting an area where the object is located in the video frame;

the video frame determination module comprises:

the first video frame determining submodule is used for determining a video frame corresponding to the mask image reflecting the same object according to the similarity between the mask images;

and the second video frame determining submodule is used for determining video frames with continuous playing time from the determined video frames as target video frames.

In an embodiment of the present invention, the contour determining module is specifically configured to select a representative target mask image from mask images corresponding to target video frames according to a similarity between the mask images corresponding to the target video frames; and determining the outline of the area where the object is located reflected by the target mask image as the outline of the area where the object is located in each target video frame.

In an embodiment of the present invention, the contour determining module is specifically configured to obtain an initial contour of an area where an object in each target video frame is located according to a mask image corresponding to each target video frame; and performing expansion processing on the initial contour corresponding to each target video frame to obtain the contour of the region where the object is located in each target video frame.

In an embodiment of the present invention, the first video frame determining sub-module is specifically configured to determine, from each mask image, a mask image in which a size of the reflected object region is greater than a preset size threshold, as an alternative mask image; and determining the video frame corresponding to the mask image reflecting the same object according to the similarity between the alternative mask images.

In a fourth aspect, an embodiment of the present invention provides a bullet screen display device, which is applied to a client, where the device includes:

the track acquisition module is used for acquiring a bullet screen display track from the server, wherein the bullet screen display track is as follows: a trajectory generated by the apparatus according to the third aspect;

the information determining module is used for determining a target bullet screen display track corresponding to a first video frame to be displayed according to the information of the video frame included in the bullet screen display track and obtaining a target bullet screen corresponding to the first video frame to be displayed;

the character determining module is used for determining target characters in the target bullet screen corresponding to each first video frame according to the number of display positions included in the target bullet screen display track corresponding to each first video frame;

the position determining module is used for sequentially determining the display positions of the target characters corresponding to each first video frame according to the arrangement sequence of the display positions in the target bullet screen display track corresponding to the first video frame;

and the character display module is used for displaying each target character on each first video frame according to the determined display position of each target character corresponding to each first video frame.

In an embodiment of the present invention, the text determining module includes:

the first character determining submodule is configured to use a front reference number of characters in the target bullet screen as target characters in the target bullet screen corresponding to a start video frame with the earliest playing time in each first video frame, where the reference number is: the number of display positions included in a target bullet screen display track corresponding to the starting video frame;

the second character determining submodule is used for determining target characters in the target bullet screen corresponding to other video frames according to the first characters in the target bullet screen corresponding to the previous video frame of the other video frames, the first number of display positions included in the display track of the target bullet screen corresponding to the previous video frame and the second number of display positions included in the display track of the target bullet screen corresponding to the other video frames aiming at each other video frame;

In an embodiment of the present invention, the second character determining sub-module includes:

the first character determining unit is used for determining the first characters as target characters in target bullet screens to be displayed in other video frames if the first number of display positions in the bullet screen display tracks corresponding to the previous video frame is equal to the second number of display positions in the target display tracks corresponding to the other video frames and the display positions are different;

a second text determining unit, configured to determine the first text and the second text as the target text if the first number is not equal to the second number and a shortest distance is greater than a preset step distance, where the shortest distance is: and the shortest distance between each display position in the target bullet screen display track corresponding to other video frames and the preset termination position of the object outline represented by the target bullet screen display track, wherein the second characters are as follows: a third number of words in the target bullet screen that are adjacent to the first words and located behind the first words, the third number being: a difference between the second number and the first number;

a third text determining unit, configured to determine, as the target text, a fourth number of texts in the first text, and a fifth number of texts in the target bullet screen, which are adjacent to the first text and located behind the first text, if the first number is not equal to the second number and the shortest distance is not greater than the step distance, where the fourth number is: a difference between the second number and the third number, the fifth number being: a difference between the second number and the fourth number.

In an embodiment of the present invention, the trajectory obtaining module is specifically configured to obtain, from a server, a data packet including a bullet screen display trajectory; and analyzing the data packet based on a first preset character representing the bullet screen display track, a second preset character representing the video frame information and a third preset character representing the display position to obtain the character display position included in each bullet screen display track and the corresponding video frame information.

In an embodiment of the present invention, the information determining module is specifically configured to obtain, from a preset bullet screen library, a bullet screen whose display type is to be displayed along an outline of an area where an object is located, as a target bullet screen corresponding to a first video frame to be displayed; or acquiring a bullet screen which is input by a user on the user interface and is used for displaying along the outline of the area where the object is located, and taking the bullet screen as a target bullet screen corresponding to the first video frame to be displayed.

In a fifth aspect, an embodiment of the present invention provides a server, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;

a memory for storing a computer program;

a processor configured to implement the method steps of the first aspect when executing the program stored in the memory.

In a sixth aspect, an embodiment of the present invention provides a client, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;

a memory for storing a computer program;

a processor, configured to implement the method steps of the second aspect when executing the program stored in the memory.

In a seventh aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps of the first aspect or the second aspect.

As can be seen from the above, when the bullet screen trajectory is generated by applying the scheme provided by the embodiment of the present invention, the bullet screen display trajectories corresponding to the generated target video frames can be characterized jointly: the trajectory of the characters in the barrage is moved from the preset initial position to the preset end position in the determined outline, and when the characters are displayed based on the barrage display trajectory corresponding to each target video frame, the characters can be displayed along the outline of the object contained in the video frame, so that the display form of the barrage is enriched.

In addition, since the target video frames are similar video frames which contain the same object and have continuous playing time, when the bullet screen characters are displayed based on the display positions included in the bullet screen display track corresponding to each target video frame, the bullet screen characters can be displayed at the outline of the same object in continuous time, so that the continuous effect of rolling and displaying the same object can be formed when the bullet screen characters are displayed.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flow chart of a first bullet screen trajectory generation method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a first video frame according to an embodiment of the present invention;

fig. 3 is a schematic flowchart of a second bullet screen trajectory generation method according to an embodiment of the present invention;

fig. 4 is a schematic flow chart of a position determination process according to an embodiment of the present invention;

fig. 5 is a schematic flowchart of a third bullet screen trajectory generation method according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a first profile provided by an embodiment of the present invention;

fig. 7 is a schematic flowchart of a fourth bullet screen trajectory generation method according to an embodiment of the present invention;

FIG. 8a is a schematic view of a second video frame according to an embodiment of the present invention;

FIG. 8b is a schematic diagram of a mask image according to an embodiment of the present invention;

FIG. 9 is a diagram of a third video frame according to an embodiment of the present invention;

FIG. 10a is a schematic view of a second profile provided by an embodiment of the present invention;

FIG. 10b is a schematic view of a third profile provided by an embodiment of the present invention;

FIG. 11 is a schematic view of a fourth profile provided by an embodiment of the present invention;

fig. 12 is a schematic flowchart of a fifth bullet screen trajectory generation method according to an embodiment of the present invention;

fig. 13 is a flowchart illustrating a first bullet screen displaying method according to an embodiment of the present invention;

FIG. 14a is a diagram illustrating a fourth video frame according to an embodiment of the present invention;

FIG. 14b is a diagram illustrating a fifth video frame according to an embodiment of the present invention;

fig. 15 is a flowchart illustrating a second bullet screen displaying method according to an embodiment of the present invention;

fig. 16 is a schematic structural diagram of a bullet screen display system according to an embodiment of the present invention;

fig. 17 is a schematic structural diagram of a first bullet screen trajectory generating device according to an embodiment of the present invention;

fig. 18 is a schematic structural diagram of a second bullet screen trajectory generation device according to an embodiment of the present invention;

fig. 19 is a schematic structural diagram of a third bullet screen trajectory generation device according to an embodiment of the present invention;

fig. 20 is a schematic structural diagram of a fourth bullet screen trajectory generation device according to an embodiment of the present invention;

fig. 21 is a schematic structural diagram of a bullet screen display device according to an embodiment of the present invention;

fig. 22 is a schematic structural diagram of a server according to an embodiment of the present invention;

fig. 23 is a schematic structural diagram of a client according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Because the bullet screen display form in the prior art is single, in order to solve the technical problem, the embodiment of the invention provides a bullet screen track generation method and a bullet screen display method and device.

In an embodiment of the present invention, a bullet screen trajectory generation method is provided, which is applied to a server, and the method includes:

identifying an area where an object is located in a video frame of a video;

and generating bullet screen display tracks corresponding to the target video frames according to the determined outline, wherein each bullet screen display track comprises: the display position of the characters in the bullet screen and the information of the corresponding target video frame, and the display track of each bullet screen is characterized in a combined manner: and the track that the characters in the bullet screen move from the preset starting position to the preset ending position in the determined outline.

As can be seen from the above, when the bullet screen trajectory is generated by applying the scheme provided in this embodiment, the bullet screen display trajectory corresponding to each generated target video frame can be represented jointly: the trajectory of the characters in the barrage is moved from the preset initial position to the preset end position in the determined outline, and when the characters are displayed based on the barrage display trajectory corresponding to each target video frame, the characters can be displayed along the outline of the object contained in the video frame, so that the display form of the barrage is enriched.

Referring to fig. 1, fig. 1 is a schematic flowchart of a first bullet screen trajectory generation method provided by an embodiment of the present invention, and is applied to a server, where the method includes the following steps S101 to S104.

Step S101: an area in which an object is located in a video frame of a video is identified.

The subject may be a human, an animal, a vehicle, or the like.

The video frame may be each video frame in the video, or may be a video frame at every preset interval in the video, where the preset interval may be 3, 5, or the like.

Specifically, semantic features of each pixel point in the video frame can be extracted, the region type of the region to which each pixel point belongs is determined according to the extracted semantic features, and a region formed by the pixel points of which the region types are the object region types is determined as the region where the object is located.

After the region type of the region to which each pixel belongs is determined, the pixels belonging to the object region type can be determined as the pixels of the foreground layer, and the pixels belonging to the non-object region type can be determined as the pixels of the background layer, so that the background layer and the foreground layer of the video frame can be determined, and the foreground layer includes the region to which the object belongs.

In addition to determining the area of the object in the above manner, the area of the object may be identified with reference to the embodiment shown in fig. 8.

Step S102: and determining similar video frames which contain the same object and have continuous playing time as target video frames according to the identified areas.

Because the obtained target video frames are similar video frames, that is, the similarity between the target video frames is high, and each target video frame contains the same object, the similarity of the objects contained in each target video frame is high, and the positions of the objects are close to each other.

When determining the target video frame, the video frames including the same object may be determined according to the similarity between the identified regions and the difference between the positions of the regions, and the video frames with continuous playing time may be selected from the determined video frames as similar video frames.

Specifically, it may be determined that the similarity is greater than a preset similarity threshold and the difference between the positions is less than a preset difference threshold, and the video frames corresponding to the areas are used as the video frames containing the same object.

After the video frames containing the same object are determined, the video frames with the continuous playing time length larger than the preset time length threshold value can be selected from the video frames to serve as similar video frames. For example: the preset time period can be 30s, 40s, 1min and the like.

In addition to the above manner, the target video frame may be determined according to the embodiment corresponding to fig. 7, and will not be described in detail herein.

Step S103: and determining the outline of the area where the object is located in each target video frame.

The contour can reflect information such as the shape and size of the region in which the object is located. Taking fig. 2 as an example, the region where the person is located in the target video frame shown in fig. 2 is the region where the object is located, the black curve in fig. 2 is the curve on the edge of the region where the object is located, and the contour formed by the black curve is the contour of the region where the object is located.

Specifically, the contour of each target video frame can be identified to obtain the contour of the region where the object is located in each target video frame.

Step S104: and generating a bullet screen display track corresponding to each target video frame according to the determined outline.

Specifically, the bullet screen display track corresponding to the target video frame may be one or multiple.

When the frame rate of the video is the same as the frame rate of the bullet screen, the bullet screen display track corresponding to the target video frame is a bullet screen display track, for example: when the frame rate of the video is 60FPS, the bullet screen refreshing frame rate is 60FPS, and the bullet screen display track corresponding to the target video frame is a bullet screen display track. When the frame rate of the video is different from the bullet screen refresh frame rate, the bullet screen display track corresponding to the target video frame may include multiple bullet screen display tracks, for example: when the frame rate of the video is 30FPS, the bullet screen refresh frame rate is 60FPS, and the bullet screen display tracks corresponding to the target video frame can be two bullet screen display tracks.

Each bullet screen display track comprises: and displaying the position of the characters in the bullet screen and the corresponding information of the target video frame.

The above bullet screen Chinese characters refer to: the number of the characters contained in the bullet screen, which correspond to the display positions, is 1. For example: assuming that the bullet screen display track comprises three display positions, the number of characters contained in the bullet screen corresponding to each display position is 1, and when the client displays the characters in the bullet screen based on the bullet screen display track, one character contained in the bullet screen is displayed in each display position.

In the bullet screen display track, each display position can be arranged according to the outline indication direction of the outline, the outline indication direction is related to the preset bullet screen display direction, when the preset bullet screen display direction is from right to left, the outline indication direction is along the anticlockwise direction of the outline, when the preset bullet screen display direction is from left to right, the outline indication direction is along the clockwise direction of the outline.

The information of the corresponding target video frame refers to: and displaying the information of the target video frame corresponding to the track by the bullet screen. The information of the target video frame may include a frame sequence number, a playing time, and the like of the target video frame.

And (3) jointly representing the display tracks of all the bullet screens: and the track for moving the characters in the bullet screen from the preset starting position to the preset ending position in the determined outline can be also called as a complete track.

Specifically, each bullet screen display track may include a partial bullet screen display track and a complete bullet screen display track.

The display track of each part of the bullet screen is as follows: and a partial track corresponding to the display time of the partial bullet screen display track in the complete track. The display track of each complete bullet screen is as follows: the complete trajectory. The above complete trajectory is: and the track that the characters in the bullet screen move from the preset starting position to the preset ending position in the determined outline.

Since the partial bullet screen display track is a partial track in the complete track, the number of display positions included in the partial bullet screen display track is less than the maximum number, and the maximum number is: the number of display positions that the complete trajectory includes. Since the complete bullet screen display track is a complete track, the number of display positions included in the complete bullet screen display track is the maximum number.

Because the partial bullet screen display track corresponds to the display time, when the display time of the partial bullet screen display track is earlier, the number of display positions included in the partial bullet screen display track is less, and the display positions included in the partial bullet screen display track with the earlier display time are included; when the display time of the partial bullet screen display track is later, the display positions included in the partial bullet screen display track are more and include the display positions included in the partial bullet screen display track with the display time earlier.

The display track of the second bullet screen is as follows: and the number of the display positions included in the second bullet screen display track is the maximum value of the display positions which can be contained in the complete track, so that the display time of the second bullet screen display track is further back.

And integrating the display tracks of all the bullet screens, wherein the display tracks of all the bullet screens can jointly represent the complete track.

The specific manner of generating the bullet screen display track may refer to the embodiment corresponding to fig. 3, and is not described in detail herein.

In order to accurately generate a bullet screen display track, referring to fig. 3, fig. 3 is a schematic flow diagram of a second bullet screen track generation method provided in the embodiment of the present invention. The step S104 can be implemented as the following steps S1041 to S1044.

Step S1041: and determining a first display position of the characters in the bullet screen included in the first bullet screen display track according to a preset initial position in the outline corresponding to the first video frame, and obtaining the first bullet screen display track including the first display position and the first video frame information.

The first video frame is: and playing the target video frame with the earliest playing time.

The display track of the first bullet screen is as follows: and the track with the earliest corresponding display time in the bullet screen display tracks corresponding to the target video frames. The first video frame information included in the first bullet screen display track may be information such as a frame sequence number and a play time of the first video frame.

The number of the first display positions may be one or more.

When the number of the first display positions is one, the preset starting position can be directly determined as the first display position.

When the number of the first display positions is multiple, in order to avoid the situation of character shielding when characters are displayed at each first display position, the multiple first display positions can be calculated according to a preset initial position and the size of the bullet screen characters.

The bullet screen character size is the default size of the characters in the bullet screen, and the size can be 16 pixel points, 25 pixel points and the like.

Specifically, a preset initial position may be directly determined as a first display position, and then a position obtained by moving the preset initial position forward by the size of the bullet screen text is determined as a second first display position along the contour indication direction of the contour; then, a position obtained by moving the second first display position forward by the bullet screen character size is determined as a third first display position along the direction indicated by the outline of the outline, and in this way, a plurality of first display positions can be obtained.

And generating each second bullet screen display track according to the following steps S1042 to S1044. Specifically, the following steps S1042 to S1044 may be executed for each second video frame according to the playing time sequence of the second video frames.

The display track of the second bullet screen is as follows: and the trajectory except the first bullet screen display trajectory in the bullet screen display trajectories corresponding to the target video frames. The second video frame is: and video frames except the first video frame in each target video frame.

Step S1042: and moving each display position included in the third bullet screen display track by a preset stepping distance along the outline indication direction of the outline corresponding to the second video frame.

The direction of the outline indication may be counterclockwise or clockwise.

The preset stepping distance represents the distance between two adjacent video frames of the same character, and can be calculated in advance according to the frame rate of the video and the display speed of the bullet screen. For example: the preset step distance can be 4 pixel points, 8 pixel points and the like.

The third bullet screen display track is as follows: and the corresponding display time is adjacent to the second bullet screen display track and is positioned in front of the second bullet screen display track.

Specifically, the position obtained by moving each display position included in the third bullet screen display track forward by the step distance may be determined as each display position after the movement along the contour indication direction of the contour corresponding to the second video frame.

Step S1043: and determining a second display position of the bullet screen Chinese characters included in the second bullet screen display track according to the moved display position and the bullet screen character size.

The display position after the movement is only to move each display position included in the third bullet screen display track, and for the second video frame currently processed, no new display position is added, but if a new display position is directly added, the characters displayed at the display position may block other characters, thereby affecting the bullet screen display effect. Therefore, when the second display position is determined based on the moved display position and the size of the bullet screen characters, the occurrence of blocking of the bullet screen characters displayed based on the second display position can be reduced.

In an embodiment of the present invention, the shortest distance between each moved display position and the preset starting position in the contour corresponding to the second video frame may be determined; judging whether the shortest distance is larger than or equal to the size of the bullet screen characters; if not, taking each moved display position as a second display position of the Chinese characters in the bullet screen included in the second bullet screen display track; if so, taking each moved display position and a preset initial position in the outline corresponding to the second video frame as a second display position.

Specifically, the distance between each moved display position and the preset start position may be calculated, and the shortest distance may be determined from the calculated distances.

Therefore, when the shortest distance is larger than or equal to the size of the bullet screen characters, when the characters are displayed at the preset initial position, the characters cannot be shielded, and then the positions after the movement and the preset initial position are used as second display positions, the shielding condition when the bullet screen characters are displayed based on the second display positions is reduced.

Moreover, when the shortest distance is smaller than the size of the bullet screen characters, character shielding can occur when the characters are displayed at the preset initial position, so that the situation that the characters are shielded when the bullet screen characters are displayed based on the second display positions is reduced by only taking the display positions after the characters are moved as the second display positions.

Step S1044: and obtaining a second bullet screen display track comprising a second display position and second video frame information.

The second video frame information may include a frame sequence number, a play time, and the like of the second video frame.

Therefore, the second display position is determined according to the moved display position and the size of the bullet screen characters, so that the determined second display position and the size of the bullet screen characters have an association relation, and the blocking condition of the bullet screen characters displayed on the basis of the second display position is reduced.

The manner of generating the bullet screen display trajectory will be specifically described below with reference to fig. 4. Fig. 4 includes 5 target video frames, and the position of each rectangular frame in each target video frame is the display position included in the bullet screen display track corresponding to each target video frame.

The target video frames are arranged according to the playing time sequence. So fig. 1 is the first video frame and fig. 2-5 are the second video frames.

Directly determining the position of the rectangular frame a1 as a first display position included in the first bullet screen display track corresponding to the diagram (1) when the preset starting position in the outline corresponding to the diagram (1) is the position of the rectangular frame a 1;

when the second bullet screen display track in fig. 2 is generated, the current third bullet screen display track is the bullet screen display track corresponding to fig. 1, and the step distance is moved to the display position included in the third bullet screen display track along the outline indication direction of the outline corresponding to fig. 2.

In fig. 2, the initial position of the arrow is a display position included in the third bullet screen display track, the direction pointed by the arrow is the outline indication direction, the length of the arrow is the above step distance, and the display position after movement is the position of the rectangular frame a 2.

After the moved display positions are obtained, the shortest distance between each moved display position and the preset initial position in the outline corresponding to the graph (2) is as follows: the distance between the position of the a2 and the preset initial position is 4 pixel points, the size of the bullet screen characters is 10 pixel points, 4<10, and the position of the rectangular frame a2 is determined as the second display position of the bullet screen characters included in the second bullet screen display track corresponding to the diagram (2).

When the second bullet screen display track of fig. 3 is generated, the current third bullet screen display track is the bullet screen display track corresponding to fig. 2, and the step distance is moved to the display position included in the third bullet screen display track along the outline indication direction of the outline corresponding to fig. 3.

In fig. 3, the initial position of the arrow is a display position included in the third bullet screen display track, the direction pointed by the arrow is the outline indication direction, the length of the arrow is the above step distance, and the display position after movement is the position of the rectangular frame a 3.

After the moved display positions are obtained, the shortest distance between each moved display position and the preset initial position in the outline corresponding to the graph (3) is as follows: the distance between the position of the a3 and the preset initial position is 8 pixel points, the size of the bullet screen characters is 10 pixel points, 8<10, and the position of the rectangular frame a3 is determined as the second display position of the bullet screen characters included in the second bullet screen display track corresponding to the diagram (3).

When the second bullet screen display track of fig. 4 is generated, the current third bullet screen display track is the bullet screen display track corresponding to fig. 3, and the step distance is moved to the display position included in the third bullet screen display track along the outline indication direction of the outline corresponding to fig. 4.

In fig. 4, the initial position of the arrow is a display position included in the third bullet screen display track, the direction pointed by the arrow is the outline indication direction, the length of the arrow is the above step distance, and the display position after the movement is the position of the rectangular frame a 4.

After the moved display positions are obtained, the shortest distance between each moved display position and the preset initial position in the outline corresponding to the graph (4) is as follows: a3, the distance is 12 pixel points, the bullet screen character size is 10 pixel points, 12>10, and the position of the rectangular frame a4 and the preset initial position a5 are determined as the second display position of the bullet screen characters included in the second bullet screen display track corresponding to fig. 4.

When the second bullet screen display track in fig. 5 is generated, the current third bullet screen display track is the bullet screen display track corresponding to fig. 4, and the step distance is moved to the display position included in the third bullet screen display track along the outline indication direction of the outline corresponding to fig. 5.

In fig. 5, the initial position of the arrow is a display position included in the third bullet screen display track, the direction pointed by the arrow is the outline indication direction, the length of the arrow is the above step distance, and the display position after movement is the position of the rectangular frames a6 and a 7.

After the moved display positions are obtained, the shortest distance between each moved display position and the preset starting position in the outline corresponding to the graph (5) is as follows: a7, the distance is 4 pixel points, the size of the bullet screen characters is 10 pixel points, 4<10, and the positions of the rectangular frame a6 and the rectangular frame a7 are determined as the second display position of the bullet screen characters included in the second bullet screen display track corresponding to the diagram (5).

Referring to fig. 5, fig. 5 is a schematic flow chart of a third bullet screen trajectory generation method provided in the embodiment of the present invention. On the basis of the above embodiment, after the above step S103, the following step S105 may be further included.

Step S105: and determining a position pair to be filled in each contour, filling new contour pixel points between two contour pixel points in the position pair to be filled along the contour indication direction of the contour, and enabling the distance between the filled contour pixel points to be not more than the preset stepping distance.

The distance between two contour pixel points in the position pair to be filled is larger than the stepping distance, and no other contour pixel point exists between the two contour pixel points.

When the contour of the region where the object is located is determined in step S103, the distance between contour pixel points in the determined contour may be relatively large, and it is difficult to accurately characterize the contour. Therefore, pixel filling needs to be performed between the pairs of positions to be filled.

Specifically, the positions of the contour pixel points can be determined in sequence at intervals smaller than or equal to the step distance from the initial contour pixel point in the position pair to be filled along the contour indication direction of the contour until the shortest distance between the determined position and the position of the ending contour pixel point in the position pair to be filled is not greater than the step distance, and a new contour pixel point is filled in each determined position, so that the distance between two adjacent filled pixel points is not greater than the step distance.

Taking fig. 6 as an example, the points P3 that are spaced from the position of the point P1 by a preset distance are determined along the contour indication direction by using the pixels P1 and P2 in fig. 6 as two contour pixels in the pair of positions to be filled, where the preset distance is smaller than the step distance, the point P4 that is spaced from the point P3 by the preset distance is determined, and the point P5 that is spaced from the point P4 by the preset distance is determined, and since the distance between the point P5 and the point P2 is smaller than the preset step distance, pixel filling can be performed at the positions of the points respectively, so that the contour pixels included in the filled contour are: p1, P2, P3, P4 and P5.

Based on this, step S104 can be implemented as step S1045 below.

Step S1045: and generating bullet screen display tracks corresponding to the target video frames according to the outlines filled with the outline pixel points.

Specifically, for each target video frame, the display position of the text in the bullet screen included in the bullet screen display track corresponding to the target video frame may be determined from the position of the contour pixel point of the filled contour corresponding to the target video frame.

Because the distance between the filled contour pixel points is not more than the stepping distance, two display positions with the interval of the preset stepping distance can be determined from the dense points, and therefore each bullet screen display track can be generated.

In step S101, in addition to determining the region where the object is located based on the semantic features of each pixel, the following step S1011 can be performed. As shown in fig. 7.

Step S1011: and identifying the object contained in the video frame of the video, and obtaining a mask image reflecting the area where the object is located in the video frame.

The pixel values of the pixels in the mask image are 0 or 1, and the pixel values of the pixels in the region where the object is located in the mask image are the same and different from the pixel values of the pixels in the region where the non-object is located.

Taking fig. 8a and 8b as examples, fig. 8a is a video frame, the area where a person is located in fig. 8a is the area where an object is located, and fig. 8b is a mask image reflecting the area where the object is located in fig. 8a, where a white area is the area where the object is located, and a black area is the area where the object is not located.

In identifying the object, an object identification algorithm may be employed for identification. The semantic features of the pixel points can be extracted by adopting a deep learning algorithm aiming at each pixel point in the video frame, the object type of the object to which the pixel points belong is determined according to the semantic features of the pixel points, and the object contained in the pixel point composition region of which the object type is the same object type is determined. The above method is also called semantic segmentation technology, in which the object of the video frame is determined based on the semantic features of each pixel point, so as to separate the object of the video frame from the background.

After the object is identified, the first pixel values of the pixel points included in the area where the object is located can be adjusted, so that the adjusted first pixel values are the same, the second pixel values of the pixel points included in the area where the non-object is located are adjusted, so that the adjusted second pixel values are the same and different from the adjusted first pixel values, and the mask image is generated according to a preset mask image format based on the adjusted first pixel values and the adjusted second pixel values.

Specifically, after the object is identified, the generated initial mask image may be post-processed, and the post-processed initial mask image may be used as the mask image reflecting the area of the object in the video frame. The post-processing can adopt image morphological processing algorithms such as 'corrosion' and 'expansion' to process the initial mask image, and during the post-processing, the initial mask image can be finely cut, false detection areas are filtered, holes are filled, and the like.

On the basis of the above embodiment, the above step S102 can be implemented as the following steps S1021 to S1022.

Step S1021: and determining the video frame corresponding to the mask image reflecting the same object according to the similarity between the mask images.

Because the mask images reflect the areas where the objects are located in the video frames, when the similarity between the mask images is higher, the probability that the objects contained in the areas where the objects are located are the same objects is higher; when the similarity between the mask images is lower, the probability that the object included in the region where the object is reflected by the mask images is the same object is lower. Therefore, according to the similarity between the mask images, the accuracy of the determined video frame corresponding to the mask image reflecting the same object is high.

Specifically, the video frame corresponding to the mask image with the similarity between the mask images being greater than the second preset similarity threshold may be determined as the video frame corresponding to the mask image reflecting the same object. The second preset similarity threshold may be 90%, 95%, or the like.

In an embodiment of the present invention, a mask image in which the size of the reflected object region is larger than a preset size threshold may also be determined from each mask image as a candidate mask image; and determining the video frame corresponding to the alternative mask images reflecting the same object according to the similarity between the alternative mask images.

When the size of the object region reflected in the mask image is smaller, it indicates that the region where the object is located in the video frame corresponding to the mask image is smaller, even no object is present, and in order to improve the screening efficiency of the target video frame, the mask image can be removed in advance.

The preset size threshold may be set by a worker according to experience, for example: the preset size threshold may be 64 pixels, 128 pixels, etc.

Specifically, the video frame corresponding to the candidate mask image with the similarity between the candidate mask images being greater than the second preset similarity threshold may be determined as the video frame corresponding to the mask image reflecting the same object. The second preset similarity threshold may be 90%, 95%, or the like.

Therefore, the mask images with the size not larger than the preset size threshold in the mask images are removed, the video frame is determined from the removed mask images, and the complexity of determining the video frame is reduced.

Step S1022: and determining video frames with continuous playing time from the determined video frames as target video frames.

Specifically, a video frame with a duration of continuous playing time greater than a preset duration threshold may be selected from the determined video frames as the similar video frame. The preset time threshold may be 30s, 60s, etc.

As can be seen from the above, since the mask images reflect the areas where the objects are located in the video frames, according to the similarity between the mask images, the accuracy of the determined video frame corresponding to the mask image reflecting the same object is higher, and thus the accuracy of the obtained target video frame is high.

After the mask image is obtained in step S1011, step S103 can be implemented as follows.

Selecting a representative target mask image in the mask images corresponding to the target video frames according to the similarity between the mask images corresponding to the target video frames; and determining the outline of the area where the object is located reflected by the target mask image as the outline of the area where the object is located in each target video frame.

The region where the object is reflected by the representative target mask image can represent the region where the object is reflected by each of the other mask images.

Specifically, each mask image with a similarity greater than a first preset similarity threshold may be determined, and from the determined mask images, a mask image with a similarity greater than a second preset similarity threshold with respect to each mask image may be determined as a representative target mask image.

Taking fig. 9 as an example, fig. 9 includes mask images, wherein the second image, the third image and the fourth image are mask images with a similarity greater than a first preset similarity threshold in order from top to bottom. In the three images, the similarity between the second image and the other two images is larger than a second preset similarity threshold, and the second image is a representative target mask image.

Specifically, each mask image with the similarity greater than the first preset similarity threshold may be compressed. And carrying out contour recognition on the region where the object is located and reflected by the target mask image in each compressed mask image, and determining the recognized contour as the contour of the region where the object is located in each target frame.

The area where the object is reflected by the representative target mask image can represent the area where the object is reflected by other mask images, and the outline corresponding to the target mask image is directly determined as the outline of the area where the object is located in each target video frame, so that the outline of the area where the object is located in each target video frame is not required to be identified, and the outline identification efficiency is improved.

After the mask image is obtained in step S1011, step S103 can be realized as follows.

Obtaining an initial contour of an area where an object is located in each target video frame according to the mask image corresponding to each target video frame; and performing expansion processing on the initial contour corresponding to each target video frame to obtain the contour of the region where the object is located in each target video frame.

The occupied area of the outline obtained after expansion processing is larger than that of the initial outline, and the condition that the bullet screen characters are displayed at each display position included on the basis of the bullet screen display track to shield the object in the video frame is reduced. And the expansion processing is carried out on the initial contour, small bulges and burrs in the initial contour can be removed, so that the contour after the expansion processing is smooth, and when the display position included by each bullet screen display track is determined based on the contour after the expansion processing, the bullet screen display track is determined based on the smooth contour, so that the generation complexity of the bullet screen display track is reduced.

Specifically, the contour of the region where the object is located in the mask image may be identified to obtain an initial contour, and the identified initial contour is expanded by using an expansion algorithm to obtain the contour of the region where the object is located in each target video frame.

Taking fig. 10a and 10b as an example, the boundary curve between the black area and the white area in fig. 10a is an initial contour, and the boundary curve between the black area and the white area in fig. 10b is: the original profile of fig. 10a is subjected to an expansion process.

Comparing fig. 10a and 10b, it can be seen that the lower left portion of the profile shown in fig. 10a has some small bumps or burrs, and the same portion becomes smooth in fig. 10 b.

In this way, the expansion processing is performed on the initial contour, so that the situation that the bullet screen characters shield the object in the video frame based on each display position included in the bullet screen display track can be reduced, and the generation complexity of the bullet screen display track is reduced.

In an embodiment of the invention, the preset starting position and the preset ending position in the step S104 may be different positions, in which case, the trajectory jointly represented by each bullet screen display trajectory is a non-closed trajectory, and when a bullet screen is displayed, the display scroll starts from the preset starting position to the preset ending position along the bullet screen display trajectory and disappears.

Specifically, when determining that the contour of the region where the object is located in each target video frame is obtained in step S103, the contour may be segmented, the segmented contour is non-closed, the preset start position and the preset end position of the non-closed contour are set to different positions, and each bullet screen display track may be generated according to the non-closed contour.

Taking fig. 11 as an example, in fig. 11, the boundary between black and white is the contour recognized in step S103, and the straight line near the lower side of the video frame is used as the dividing line to divide the contour, so that each bullet screen display trajectory can be generated from the contour located above the dividing line.

The preset starting position and the preset ending position can also be the same position, and in this case, the joint represented track of the display tracks of the bullet screens is a closed track.

The whole process from the mask image acquisition to the generation of each bullet screen display track will be described below with reference to fig. 12. The following steps S1201 to S1206 are included in fig. 12.

Step S1201: and obtaining mask images, and determining the mask images of which the sizes of the reflected object areas are larger than a preset size threshold value from the mask images as alternative mask images.

Step S1202: and determining video frames corresponding to the alternative mask images reflecting the same object according to the similarity between the alternative mask images, and determining video frames with continuous playing time from the determined video frames to be used as target video frames.

Step S1203: and performing expansion processing on the initial contour corresponding to each target video frame to obtain the contour of the region where the object is located in each target video frame.

Step S1204: and determining a position pair to be filled in each contour, filling new contour pixel points between two contour pixel points in the position pair to be filled along the contour indication direction of the contour, and enabling the distance between the filled contour pixel points to be not more than the preset stepping distance.

Step S1205: and segmenting the outline of the region where the object is located in each target video frame, wherein the preset starting position and the preset ending position of the segmented outline are different positions, and obtaining the non-closed outline.

Step S1206: and acquiring each display position included in the bullet screen display track corresponding to each target video frame, and generating the bullet screen display track containing each display position and target video frame information.

Corresponding to the bullet screen display method applied to the server, the embodiment of the invention also provides a bullet screen display method applied to the client.

Referring to fig. 13, fig. 13 is a schematic flowchart of a first bullet screen display method according to an embodiment of the present invention, where the method includes the following steps S1301-S1305.

Step S1301: and obtaining the bullet screen display track from the server.

The bullet screen display track is as follows: and generating the trajectory according to the bullet screen trajectory generation method applied to the server.

The bullet screen display track comprises: and displaying the position of the characters in the bullet screen and the corresponding information of the target video frame.

Specifically, the client may send a request to the server, where the request is used to determine whether the currently played video contains bullet screen display track data, and if the server sends information indicating that the currently played video contains bullet screen display track data to the client, the client may obtain a bullet screen display track from the server; if not, the step is not executed.

In one embodiment of the invention, a data packet containing a bullet screen display track can be obtained from a server; analyzing the data packet based on a first preset character representing the bullet screen display track, a second preset character representing the video frame information and a third preset character representing the display position to obtain the character display position included in each bullet screen display track and the corresponding video frame information.

The data packet comprises bullet screen display tracks corresponding to the target video frames, the first preset characters are used for representing the bullet screen display tracks, in each bullet screen display track, the second preset characters represent display positions, and the third preset characters represent video frame information. Therefore, the packet can be analyzed based on the characters.

Specifically, the data packet may be analyzed based on the first preset character to obtain each bullet screen display track, and each bullet screen display track may be analyzed based on the second preset character and the third preset character to obtain a display position and video frame information included in each bullet screen display track.

For example: the characters in the data packet may be: (100,20) |1/(90,30) |2/(80,30), wherein, "/" represents a first preset character of a bullet screen display track, "|" represents a second preset character of video frame information, and "()" represents a third preset character of a display position, and the data packet is analyzed based on the first preset character to obtain each bullet screen display track: (100,20) |1, (90,30) |2, and (80,30) | 3), each bullet screen display track is analyzed based on the second preset character and the third preset character to obtain a display position and video frame information included in each bullet screen display track, the display position included in each bullet screen display track is (100,20) |1, the video frame information is 1, the display position included in each bullet screen display track is (90,30) |2, the video frame information is 2, and the display position included in each bullet screen display track (80,30) |3 is (80,30), and the video frame information is 3.

Therefore, the first preset character represents the bullet screen display track, the second preset character represents each display position, and the third preset character represents the video frame information, so that the data packet can be analyzed based on the characters to obtain the bullet screen display track, the display positions and the video frame information included in the bullet screen display track.

Step S1302: and determining a target bullet screen display track corresponding to the first video frame to be displayed according to the information of the video frame included in the bullet screen display track, and obtaining a target bullet screen corresponding to the first video frame to be displayed.

The information of the video frame may be a frame sequence number, a playing time, and the like of the video frame.

Taking the information of the video frame as a frame serial number of the video frame as an example, when a frame serial number of the video frame included in the bullet screen display track is 20, it may be determined that the bullet screen display track is a target bullet screen display track of the video frame with the frame serial number of 20, and the video frame with the frame serial number of 20 is a first video frame to be displayed.

Because the target bullet screen display track corresponding to the first video frame is determined from the video frames according to the information of the video frames included in the bullet screen display track, namely the first video frame with the target bullet screen display track is matched from the video frames, the process can be a video frame matching process.

The number of the target bullet screens can be a complete bullet screen or a bullet screen formed by a plurality of complete bullet screens, and the bullet screens formed by splicing the heads and the tails of all the bullet screens are the target bullet screens.

When the target barrage is obtained, in an embodiment of the present invention, the barrage whose display type is displayed along the outline of the area where the object is located may be obtained from a preset barrage library, and is used as the target barrage corresponding to the first video frame to be displayed.

In another embodiment of the present invention, a bullet screen input by the user on the user interface and used for displaying along the outline of the area where the object is located may also be obtained as a target bullet screen corresponding to the first video frame to be displayed.

Step S1303: and determining the target characters in the target bullet screen corresponding to each first video frame according to the number of display positions included in the target bullet screen display track corresponding to each first video frame.

Specifically, the above step S1303 may be implemented as the following step a 1-step a 2.

Step A1: and taking the characters with the front reference number in the target bullet screen as the target characters in the target bullet screen corresponding to the initial video frame with the earliest playing time in each first video frame.

The reference numbers are: the number of display positions included in the target bullet screen display track corresponding to the starting video frame.

By way of example, the target bullet screen is: "i find an opening at the bullet screen", the number of the corresponding display positions of the starting video frame is 1, the number of the indication references is 1, and the first 1 characters in the target bullet screen are: therefore, the 'I' is determined as the target character corresponding to the starting video frame.

Step A2; and for each other video frame in the first video frames, determining the target characters in the target bullet screen corresponding to the other video frames according to the first characters in the target bullet screen corresponding to the previous video frame of the other video frames, the first number of display positions included in the display track of the target bullet screen corresponding to the previous video frame and the second number of display positions included in the display track of the target bullet screen corresponding to the other video frames.

The previous video frame is a video frame which is before the other currently processed video frame and is adjacent to the other currently processed video frame in playing time. For example: the other video frames arranged according to the playing time sequence are respectively; video frame SP1, video frame SP2, and video frame SP3, the video frame preceding the other video frame SP2 is video frame SP1, and the video frame preceding the other video frame SP3 is video frame SP 2.

The target characters are the same as the first characters, or the target characters comprise the first characters and characters adjacent to the first characters in the characters of the target bullet screen.

The number of the target characters is the same as the number of display positions included in the target bullet screen display track corresponding to the other video frames.

The specific manner of determining the above target text can be found in the following examples, which are not described in detail herein.

Therefore, the target characters are determined according to the first characters corresponding to the previous video frame, the first number of the display positions and the second number corresponding to other video frames, so that the determined target characters have an association relationship with the first characters and have an association relationship with the first number and the second number, and because the barrage characters displayed in each video frame are a continuous process, when the target characters and the information have an association relationship, the accuracy of the target characters is higher.

Step S1304: and for each first video frame, sequentially determining the display positions of the target characters corresponding to the first video frame according to the arrangement sequence of the display positions in the target bullet screen display track corresponding to the first video frame.

For example: the display positions obtained according to the arrangement sequence of the display positions in the target bullet screen display track corresponding to the first video frame are as follows: the target characters obtained by arranging the position L1, the position L2, the position L3 and the position L4 according to the arrangement sequence of the target characters in the target bullet screen are as follows: for hair, bullet, screen and cheer, the display position of the target character 'hair' can be determined to be the position L1, the display position of the target character 'bullet' is the position L2, the display position of the target character 'screen' is the position L3 and the display position of the target character 'cheer' is the position L4.

Step S1305: and displaying each target character on each first video frame according to the determined display position of each target character corresponding to each first video frame.

Specifically, each target character may be directly displayed at the display position of each target character, or the display position of each target character may be adjusted by using a preset position adjustment coefficient, and each first character may be displayed at the adjusted display position of each target character.

The images shown in fig. 14a and 14b are the first video frames displaying the target characters.

Specifically, when each first video frame displays each character of the target bullet screen, the target bullet screen can be displayed in a circulating manner.

For example: the first character of the target bullet screen can be displayed in the first video frame until the last character disappears in the first video frame, and if the first video frame still has the video frame without the bullet screen, the target bullet screen can be continuously displayed in the first video frame;

when the last character of the target bullet screen is displayed in the first video frame, and between the display position of the character and the preset termination position of the object outline represented by the target bullet screen display track corresponding to the first video frame, and when the display position of the undisplayed character exists, the first character in the target bullet screen can be displayed at the display position of the undisplayed character, and the characters in the target bullet screen are sequentially displayed along the playing sequence of the first video frames.

Of course, each target character in the target bullet screen may be displayed only once without circular display. The above situation can be applicable to both the situation that the target bullet screen is one bullet screen and the situation that the target bullet screen is a bullet screen composed of a plurality of bullet screens.

As can be seen from the above, when the bullet screen is displayed by applying the scheme provided in this embodiment, because the obtained bullet screen display tracks can jointly represent the complete track, and the complete track is the track moving from the preset starting position to the preset ending position in the outline, each target bullet screen display track can jointly represent the outline of the area where the object is located in each first video frame to be displayed. Therefore, when the bullet screen characters are displayed based on the display tracks of the target bullet screens, the displayed bullet screen characters are displayed based on the outline of the object in the first video frame, and the bullet screen display form is enriched.

In addition, the target characters corresponding to each first video frame are determined according to the number of the display positions corresponding to each first video frame, and the determined target characters have a correlation with the number of the display positions, so that the display positions of the target characters can be accurately obtained according to the arrangement sequence of the display positions.

At the step a2, the following three embodiments can be implemented.

In an embodiment of the present invention, if the first number of the display positions in the bullet screen display track corresponding to the previous video frame is equal to the second number of the display positions in the target display track corresponding to the other video frame and the display positions are different, the first character is determined as the target character in the target bullet screen to be displayed in the other video frame.

Specifically, when the first number is equal to the second number and the display is different, it indicates that the display positions corresponding to other video frames are: and each display position corresponding to the previous video frame is moved forwards along the pointing direction of the outline by a preset stepping distance to obtain a display position. In this case, since the target character corresponding to the other video frame is the same as the first character corresponding to the previous video frame, the first character can be directly determined as the target character.

In another embodiment of the present invention, if the first number is not equal to the second number and the shortest distance is greater than the predetermined step distance, the first character and the second character are determined as the target characters.

The shortest distance is as follows: and the shortest distance between each display position in the target bullet screen display track corresponding to other video frames and the preset termination position of the object outline represented by the target bullet screen display track.

The second character is: and a third number of characters which are adjacent to the first characters and behind the first characters in the target bullet screen. The third number is: the difference between the second number and the first number.

When the first number is not equal to the second number and the shortest distance is greater than the preset step distance, the first character in the previous video frame in the target characters corresponding to other video frames is represented, and the characters which are adjacent to the first character and are positioned behind the first character in the target bullet screen are also included, so that the first character and the second character can be determined as the target characters.

By way of example, the target bullet screen is: 'I find an opening at the bullet screen', the first number of display positions corresponding to a previous video frame is 3, and the first character is as follows: "i send", the second quantity of the display position that other video frames correspond is 4, and first quantity does not equal to second quantity, and the shortest distance that other video frames correspond is 10 pixel points, and predetermined step distance is 2 pixel points, and the shortest distance is greater than predetermined step distance, confirms that first characters and second characters are the target characters, and wherein, the second characters are: after "i launch," the adjacent (4-3) ═ 1 character, that is, "screen", so the target character is; "I am on the bullet screen".

In an embodiment of the present invention, if the first number is not equal to the second number and the shortest distance is not greater than the step distance, a fourth number of characters in the first characters and a fifth number of characters adjacent to and behind the first characters in the target bullet screen are determined as the target characters.

The fourth quantity is: the difference between the second number and the third number, the fifth number being: the difference between the second number and the fourth number.

When the first number is not equal to the second number and the shortest distance is not greater than the preset step distance, it indicates that each target character corresponding to another video frame includes a rear part character of the first character in the previous video frame and also includes a character adjacent to the first character and located behind the first character in the target bullet screen, so that the fourth number of characters and the fifth number of characters can be determined as target characters.

By way of example, the target bullet screen is: 'I find an opening at the bullet screen', the first number of display positions corresponding to a previous video frame is 3, and the first character is as follows: the second number of the display positions corresponding to other video frames is 4, the first number is not equal to the second number, the shortest distance corresponding to other video frames is 1 pixel point, the preset stepping distance is 2 pixel points, the shortest distance is not greater than the preset stepping distance, and the fourth number of characters is; after the first letter, [3- (4-3) ], 2 letters, namely, "barrage", the fifth number of letters is: after the first letter, (4-2) ═ 2 letters, i.e., "good".

The bullet screen display process provided by the embodiment of the present invention is fully described below with reference to fig. 15. Fig. 15 includes steps S1501 to S1505.

Step S1501: and starting playing the video, initializing the bullet screen, and configuring parameters related to the bullet screen.

Step S1502: and sending a request to a server, wherein the request is used for judging whether the currently played video contains bullet screen display track data. If the information sent by the server indicates that the bullet screen display trajectory data is included, step S1503 is executed.

Step S1503: and obtaining the bullet screen display track from the server.

Step S1504: and determining a target bullet screen display track corresponding to the first video frame to be displayed according to the information of the video frames included in the bullet screen display track.

Step S1505: and obtaining a target bullet screen corresponding to the first video frame to be displayed, and displaying characters in the target bullet screen in each first video frame.

The following describes a bullet screen display system according to an embodiment of the present invention with reference to fig. 16.

The bullet screen display system comprises a server and a client, wherein the server comprises different functional modules, such as a mask image generation module, a bullet screen track generation module and a bullet screen track storage module. The modules may be integrated in one server or distributed in different servers.

The mask image generation module can generate a mask image corresponding to a video frame of the video by adopting a deep learning algorithm, and generates the mask image to the bullet screen track generation module.

The mask image can reflect the area of the object in the video frame. The bullet screen track generation module can trigger the mask image generation module to generate a mask image.

The bullet screen track generation module firstly analyzes the mask image, then carries out pretreatment such as noise point filtration on the mask image, secondly determines similar and continuous alternative mask images, then extracts and optimizes the outline of each alternative mask image, secondly generates bullet screen display tracks based on the outline, and finally stores each bullet screen display track of the video in the bullet screen track storage module.

And the bullet screen track storage module is used for caching bullet screen display tracks of a large number of videos. When the client plays the video, the client can acquire the bullet screen display track from the bullet screen track storage module.

The bullet screen track storage module can be a Content Delivery Network (CDN) platform, and the CDN platform can dynamically manage online activities according to product operation behaviors and support data download services of clients with a large throughput.

And the client side inquires whether the video contains the bullet screen display track or not from the bullet screen track generation module, and if so, acquires the bullet screen display track from the bullet screen track storage module.

Corresponding to the bullet screen track generation method, the embodiment of the invention also provides a bullet screen track generation device.

Referring to fig. 17, fig. 17 is a schematic structural diagram of a first bullet screen trajectory generating device provided in an embodiment of the present invention, which is applied to a server, and the device includes the following

modules

1701 and 1704.

A region identification module 1701 for identifying a region where an object is located in a video frame of a video;

a video frame determining module 1702, configured to determine, according to the identified region, similar video frames that include the same object and have consecutive playing time as target video frames;

a contour determining module 1703, configured to determine a contour of an area where an object is located in each target video frame;

a track generating module 1704, configured to generate bullet screen display tracks corresponding to the target video frames according to the determined profiles, where each bullet screen display track includes: the display position of the bullet screen Chinese characters and the corresponding information of the target video frame, the number of the bullet screen Chinese characters corresponding to each display position is 1, and each bullet screen display track is characterized in a combined manner: and the track that the characters in the bullet screen move from the preset starting position to the preset ending position in the determined outline.

Referring to fig. 18, fig. 18 is a schematic structural diagram of a second bullet screen trajectory generating device according to an embodiment of the present invention, where the module 1704 further includes the following

sub-modules

17041 and 17044.

A first trajectory obtaining submodule 17041, configured to determine, according to a preset starting position in an outline corresponding to a first video frame, a first display position of a bullet screen Chinese character included in a first bullet screen display trajectory, and obtain a first bullet screen display trajectory including the first display position and the first video frame information, where the first video frame is: the target video frame with the earliest playing time is displayed on the first bullet screen in a track manner: the track with the earliest corresponding display time in the bullet screen display tracks corresponding to the target video frames;

a position moving submodule 17042, configured to move each display position included in the third bullet screen display track by a preset step distance along a contour indication direction of a contour corresponding to the second video frame, where the second video frame is: the third barrage display track of the video frames except the first video frame in each target video frame is as follows: the corresponding display time is adjacent to the second bullet screen display track and is positioned in front of the second bullet screen display track;

the position determining submodule 17043 is configured to determine, according to the moved display position and the size of the bullet screen characters, a second display position of the bullet screen characters included in the second bullet screen display track;

a second trajectory obtaining sub-module 17044, configured to obtain a second bullet screen display trajectory including the second display position and the second video frame information.

In an embodiment of the present invention, the position determining submodule 17043 is specifically configured to determine a shortest distance between each moved display position and a preset starting position in the contour corresponding to the second video frame; judging whether the shortest distance is larger than or equal to the size of the bullet screen characters; if not, taking the display positions after the movement as second display positions of the characters in the bullet screen included in the second bullet screen display track; and if so, taking the display position after each movement and a preset initial position in the outline corresponding to the second video frame as the second display position.

Referring to fig. 19, fig. 19 is a schematic structural diagram of a third bullet screen trajectory generation apparatus according to an embodiment of the present invention, where the apparatus further includes an outline filling module 1705.

The contour filling module 1705 is configured to, after the contour determining module 1703, determine a to-be-filled position pair in each contour, and fill a new contour pixel point between two contour pixel points in the to-be-filled position pair along a contour indication direction of the contour, so that a distance between two adjacent filled contour pixel points is not greater than a preset step distance, where a distance between two contour pixel points in the to-be-filled position pair is greater than the step distance, and no other contour pixel point exists between the two contour pixel points;

the track generating module 1704 is specifically configured to generate a bullet screen display track corresponding to each target video frame according to each contour after the contour pixel points are filled.

Referring to fig. 20, fig. 20 is a schematic structural diagram of a fourth bullet screen trajectory generating device according to an embodiment of the present invention, where the area identifying module 1701 is specifically configured to identify an object included in a video frame of a video, and obtain a mask image reflecting an area where the object is located in the video frame;

the video frame determination module 1702 includes:

a first video frame determining submodule 17021, configured to determine, according to a similarity between mask images, a video frame corresponding to a mask image that reflects the same object;

a second video frame determining submodule 17022, configured to determine video frames with continuous playing time from the determined video frames as target video frames.

Referring to fig. 21, fig. 21 is a schematic structural diagram of a bullet screen display device provided in an embodiment of the present invention, which is applied to a client, and the device includes the following

modules

2101 and 2105.

A track obtaining module 2101, configured to obtain a bullet screen display track from a server, where the bullet screen display track is: generating a track according to the bullet screen track;

the information determining module 2102 is configured to determine a target bullet screen display track corresponding to a first video frame to be displayed according to information of video frames included in the bullet screen display track, and obtain a target bullet screen corresponding to the first video frame to be displayed;

a character determining module 2103, configured to determine, according to the number of display positions included in the target bullet screen display track corresponding to each first video frame, a target character in the target bullet screen corresponding to each first video frame;

a position determining module 2104, configured to, for each first video frame, sequentially determine display positions of target characters corresponding to the first video frame according to an arrangement order of the display positions in the target bullet screen display track corresponding to the first video frame;

the text display module 2105 is configured to display each target text on each first video frame according to the determined display position of each target text corresponding to each first video frame.

In an embodiment of the present invention, the text determining module 2103 includes:

In an embodiment of the present invention, the track obtaining module 2101 is specifically configured to obtain a data packet including a bullet screen display track from a server; and analyzing the data packet based on a first preset character representing the bullet screen display track, a second preset character representing the video frame information and a third preset character representing the display position to obtain the character display position included in each bullet screen display track and the corresponding video frame information.

In an embodiment of the present invention, the information determining module 2102 is specifically configured to obtain, from a preset bullet screen library, a bullet screen whose display type is that the bullet screen is displayed along an outline of an area where an object is located, as a target bullet screen corresponding to a first video frame to be displayed; or acquiring a bullet screen which is input by a user on the user interface and is used for displaying along the outline of the area where the object is located, and taking the bullet screen as a target bullet screen corresponding to the first video frame to be displayed.

Corresponding to the bullet screen track generation method, the embodiment of the invention also provides a server.

As shown in fig. 22, fig. 22 is a schematic structural diagram of a server according to an embodiment of the present invention, and includes a processor 2201, a communication interface 2202, a memory 2203, and a communication bus 2204, where the processor 2201, the communication interface 2202, and the memory 2203 complete communication with each other through the communication bus 2204,

a memory 2203 for storing computer programs;

the processor 2201 is configured to implement the bullet screen trajectory generation method provided by the embodiment of the present invention when executing the program stored in the memory 2203.

Corresponding to the bullet screen display method, the embodiment of the invention also provides a client.

As shown in fig. 23, fig. 23 is a schematic structural diagram of a client according to an embodiment of the present invention, and includes a processor 2301, a communication interface 2302, a memory 2303 and a communication bus 2304, where the processor 2301, the communication interface 2302, and the memory 2303 complete communication with each other through the communication bus 2304,

a memory 2303 for storing computer programs;

the processor 2301 is configured to implement the bullet screen display method provided by the embodiment of the present invention when executing the program stored in the memory 2303.

The communication bus mentioned above for the client/server can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the client/server and other devices.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

In another embodiment provided by the present invention, a computer-readable storage medium is further provided, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method for generating a bullet screen trajectory and displaying a bullet screen provided by an embodiment of the present invention is implemented.

In another embodiment provided by the present invention, a computer program product containing instructions is further provided, which when run on a computer, causes the computer to implement the bullet screen trajectory generation and bullet screen display method provided by the embodiment of the present invention when executed.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the embodiments of the apparatus, the client, the server, and the computer-readable storage medium, since they are substantially similar to the embodiments of the method, the description is simple, and the relevant points can be referred to the partial description of the embodiments of the method.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A bullet screen track generation method is applied to a server, and comprises the following steps:

identifying an area where an object is located in a video frame of a video;

2. The method of claim 1, wherein generating a bullet screen display trajectory corresponding to each target video frame according to the determined contour comprises:

3. The method of claim 2, wherein determining the second display position of the bullet screen Chinese characters included in the second bullet screen display track according to the moved display position and the bullet screen character size comprises:

4. The method according to claim 1, further comprising, after determining the contour of the region in which the object is located in each target video frame:

5. The method according to any of claims 1-4, wherein the preset starting position is different from the preset ending position.

6. The method according to any one of claims 1-4, wherein the identifying the region of the object in the video frame of the video comprises:

7. The method of claim 6, wherein determining the contour of the region in which the object is located in each target video frame comprises:

8. The method of claim 6, wherein determining the contour of the region in which the object is located in each target video frame comprises:

9. The method of claim 6, wherein determining the video frame corresponding to the mask image reflecting the same object according to the similarity between the mask images comprises:

10. A bullet screen display method is applied to a client side, and comprises the following steps:

obtaining a bullet screen display track from a server, wherein the bullet screen display track is as follows: a trajectory generated according to the method of any one of claims 1-9;

11. The method of claim 10, wherein the determining the target text in the target bullet screen corresponding to each first video frame according to the number of display positions included in the target bullet screen display track corresponding to each first video frame comprises:

12. The method of claim 11, wherein determining the target text in the target bullet to be displayed in each of the other video frames comprises:

13. The method according to any one of claims 10-12, wherein the obtaining of the bullet screen display trajectory from the server comprises:

obtaining a data packet containing a bullet screen display track from a server;

14. The method according to any one of claims 10-12, wherein the obtaining of the target bullet screen corresponding to the first video frame to be displayed comprises:

or

15. A bullet screen track generation device is applied to a server, and comprises:

16. A bullet screen display device is characterized in that the bullet screen display device is applied to a client side, and the device comprises:

the track acquisition module is used for acquiring a bullet screen display track from the server, wherein the bullet screen display track is as follows: a trajectory generated by the apparatus of claim 15;

17. A server is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 1-9 when executing a program stored in the memory.

18. The client is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 10 to 14 when executing a program stored in the memory.

19. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method steps of any one of claims 1 to 9 or 10 to 14.