CN111107381A - Live broadcast room bullet screen display method, storage medium, equipment and system - Google Patents
Live broadcast room bullet screen display method, storage medium, equipment and system Download PDFInfo
- Publication number
- CN111107381A CN111107381A CN201811253120.7A CN201811253120A CN111107381A CN 111107381 A CN111107381 A CN 111107381A CN 201811253120 A CN201811253120 A CN 201811253120A CN 111107381 A CN111107381 A CN 111107381A
- Authority
- CN
- China
- Prior art keywords
- live broadcast
- barrage
- bullet screen
- broadcast room
- display area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000009877 rendering Methods 0.000 claims abstract description 30
- 230000011218 segmentation Effects 0.000 claims abstract description 30
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 28
- 238000013135 deep learning Methods 0.000 claims abstract description 27
- 238000012216 screening Methods 0.000 claims abstract description 20
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 230000010365 information processing Effects 0.000 abstract description 2
- 230000003287 optical effect Effects 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000004083 survival effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/488—Data services, e.g. news ticker
- H04N21/4884—Data services, e.g. news ticker for displaying subtitles
Abstract
The invention discloses a live broadcast room barrage display method, a storage medium, equipment and a system, which relate to the field of information processing, wherein the method comprises the steps of obtaining a video stream according to a video stream address of a current live broadcast room, and simultaneously obtaining a barrage of the current live broadcast room from a barrage server; performing image semantic segmentation on an image of a video stream based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas; and screening a bullet screen display area in the target scene area, and rendering the bullet screen on the bullet screen display area. When the barrage is rendered on the barrage display area, based on the payment principle and the principle that the barrage text content is related to the live broadcast content of the current live broadcast room, the barrage is scored, and then according to the principle from high to low, the barrage with high score is preferentially rendered on the barrage display area. The method and the device can effectively avoid the influence of the barrage on live broadcast watching.
Description
Technical Field
The invention relates to the field of information processing, in particular to a live broadcast room bullet screen display method, a storage medium, equipment and a system.
Background
At present, with the progress of network communication technology and the speed increase of broadband networks, live webcasts are developed and applied more and more, wherein barrage (barrage) is an important component element in live webcasts, the barrage refers to commentary subtitles popped up when videos are watched or live webcasts on the internet, and each current large live broadcast platform has own barrage culture and is widely loved by users. Meanwhile, for each large live broadcast platform, since the birth of live broadcast, a color live broadcast room is a widely-sought live broadcast room, a large number of live broadcast rooms of large anchor broadcast are full of people, the bullet screen amount of the live broadcast room is huge, and even the bullet screen is paved on the whole screen seriously.
However, when the barrage is more, the barrage quantity is uniformly distributed on the upper half part or the lower half part of the screen, so that the live broadcast picture is shielded (such as shielding the head of the anchor broadcast), and audiences are difficult to clearly see the content occurring in the current live broadcast room, thereby influencing the use experience of users.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a live broadcast room bullet screen display method which can effectively avoid the influence of bullet screens on live broadcast watching. The invention correspondingly provides equipment, a storage medium and a system.
In order to achieve the above purposes, the technical scheme adopted by the invention is as follows:
the invention provides a live broadcast room barrage display method, which comprises the following steps:
acquiring a video stream of the current live broadcast room according to the video stream address of the current live broadcast room, and acquiring a barrage of the current live broadcast room from a barrage server;
performing image semantic segmentation on the image of the video stream of the current live broadcast room based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and screening a bullet screen display area in the target scene area, and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
With reference to the first aspect, in a first possible implementation manner, the depth learning algorithm based on the PascalContext data set performs image semantic segmentation on an image of a video stream to form a plurality of different target scene areas, and the specific steps are as follows:
s201: acquiring historical live broadcast image data stored in a live broadcast server, and marking target scenes contained in images in the historical live broadcast image data;
s202: training a deep learning algorithm based on a PascalContext data set by using marked historical live image data;
s203: and performing image semantic segmentation on the image of the current live broadcast room video stream by adopting a trained deep learning algorithm, and forming a plurality of regions by taking a target scene contained in the image as a unit.
With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner, the screening out the bullet screen display area in the target scene area specifically includes:
s301: judging the live broadcast type of the current live broadcast room, if the live broadcast is outdoor live broadcast, turning to S302, and if the live broadcast is indoor live broadcast, turning to S303;
s302: the target scene area except the target scene area where the anchor character and the outdoor related prop are located is a bullet screen display area;
s303: and the target scene areas except the target scene areas where the anchor character, the display, the keyboard and the mouse are positioned are bullet screen display areas.
With reference to the first aspect, in a third possible implementation manner, when a barrage is rendered on a barrage display area, based on a payment principle and a principle that a text content of the barrage is related to a live content of a current live broadcast room, the barrage is scored, and then according to a principle that scores are from high to low, the barrage with a high score is preferentially rendered on the barrage display area.
With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, the scoring the bullet screen specifically includes:
s311: judging the attribute of the bullet screen, if the sender of the bullet screen is a VIP user or the bullet screen is a paid bullet screen, scoring the bullet screen as a, otherwise, scoring the bullet screen as 0;
s312: judging the text content of the bullet screen, if the text content is related to the live broadcast content of the current live broadcast room, scoring b, otherwise, scoring 0;
s323: the calculation formula of the bullet screen final score is as follows:
score＝a*p+b*(1-p)
wherein a and b are positive integers, p is an adjustable scale factor, and the value range is that p is more than or equal to 0 and less than or equal to 1.
With reference to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner, the barrage is scored, the barrage with the score of 0 is a general barrage, the barrage with the score of not 0 is a special barrage, and a rendering rule of the general barrage on the barrage display area is as follows: and if and only if no special barrage is rendered on the barrage display area, rendering on the barrage display area in sequence according to the sending time sequence of the common barrage.
A second aspect of the invention provides a storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring a video stream of the current live broadcast room according to the video stream address of the current live broadcast room, and acquiring a barrage of the current live broadcast room from a barrage server;
performing image semantic segmentation on the image of the video stream of the current live broadcast room based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and screening a bullet screen display area in the target scene area, and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
A third aspect of the present invention provides an electronic apparatus, comprising:
the system comprises an acquisition unit, a live broadcast room processing unit and a barrage server, wherein the acquisition unit is used for acquiring a video stream of the current live broadcast room according to a video stream address of the current live broadcast room and acquiring a barrage of the current live broadcast room from the barrage server;
the segmentation unit is used for performing image semantic segmentation on the image of the current live broadcast video stream based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and the rendering unit is used for screening the bullet screen display area in the target scene area and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
A fourth aspect of the present invention provides a live broadcast room bullet screen display system, including:
the acquisition module is used for acquiring the video stream of the current live broadcast room according to the video stream address of the current live broadcast room and acquiring the barrage of the current live broadcast room from the barrage server;
the segmentation module is used for performing image semantic segmentation on the image of the current live broadcast video stream based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and the rendering module is used for screening the bullet screen display area in the target scene area and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
In combination with the fourth aspect, in a first possible implementation manner, when the rendering module renders the barrage on the barrage display area, based on the payment principle and the principle that the barrage text content is related to the live broadcast content of the current live broadcast room, the barrage is scored, and then according to the principle that the score is from high to low, the barrage with the high score is preferentially rendered on the barrage display area.
Compared with the prior art, the invention has the advantages that: through the deep learning algorithm, carry out image semantic segmentation to the image of video stream, form a plurality of different target scene regions, then screen out the barrage display area in the target scene region, when carrying out the barrage demonstration, only carry out the barrage demonstration in the barrage display area, and the barrage display area is for not the region to the influence of live broadcast content, when satisfying the user and watching the demand to live broadcast and barrage, effectively guarantees user's use and experiences.
Drawings
Fig. 1 is a flowchart of a live broadcast room bullet screen display method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a live broadcast room barrage display method, which is characterized in that when the barrage display is carried out in a live broadcast room, an area unsuitable for displaying the barrage is detected through an image detection and recognition technology, and the barrage display is not carried out in the area, so that the use experience of a user is effectively ensured. The embodiment of the invention also provides a corresponding storage medium, equipment and a live broadcast room bullet screen display system.
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, an embodiment of a live broadcast room bullet screen display method provided in an embodiment of the present invention includes:
s1: acquiring a video stream of the current live broadcast room according to the video stream address of the current live broadcast room, and acquiring a barrage of the current live broadcast room from a barrage server;
in the embodiment of the invention, a barrage is displayed on a live broadcast picture of a live broadcast room of a client, the client generally runs in an intelligent mobile device, and when a user watches live broadcast at the client, the client acquires a video stream address of the current live broadcast room from a CDN (Content delivery network) manufacturer. And simultaneously, the client acquires the barrage of the current live broadcast room from the barrage server.
S2: and performing image semantic segmentation on the image of the current live broadcast video stream based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas. There are 400 categories of indoor and outdoor scene objects in the pascallcontext dataset.
In the embodiment of the present invention, the target scene refers to a single object contained in the image, for example, the image contains a character, a computer desk and a wall, and the character, the computer desk and the wall are all target scenes in the image. Each target scene is a region by performing image semantic segmentation on the image of the video stream.
S3: and screening a bullet screen display area in the target scene area, and rendering the bullet screen of the current live broadcast room on the bullet screen display area. When screening is performed, many target scenes are of no particular significance in live scenes, such as target scenes including tables, sky, walls, roads, numbers and the like, and the area where the target scenes are located can be used as a bullet screen display area. The specific screening rule is manually flexibly specified according to the live broadcast scene. Through the bullet screen is rendered in the bullet screen display area, then the bullet screen is displayed, the bullet screen display area is generally an unimportant scene area, and the blocking of important contents in a live-action picture by the bullet screen is avoided through the mode.
Optionally, on the basis of the embodiment corresponding to fig. 1, in a first optional embodiment of the live broadcast room barrage display method provided in the embodiment of the present invention, an image semantic segmentation is performed on an image of a video stream based on a deep learning algorithm of a pascallcontext data set to form a plurality of different target scene areas, and the specific steps are as follows:
s201: historical live broadcast image data stored in a live broadcast server are obtained, and target scenes contained in images in the historical live broadcast image data are marked. Under the normal condition, the specific target scene type contained in the live broadcast picture is generally fixed, and in a live broadcast room under the same category, the target scene style appearing in the anchor live broadcast is fixed, so that historical live broadcast image data stored in a live broadcast server is obtained, and then the target scene contained in the image in the historical live broadcast image data is marked, so that the application of the target scene is facilitated, and the deep learning algorithm after the follow-up training has the targeted recognition capability.
S202: and training a deep learning algorithm based on the PascalContext data set by using the marked historical live image data.
In the embodiment of the invention, a deep learning algorithm is trained, and the process is as follows: inputting the marked image, and completing the training of the deep learning algorithm through a series of operations such as convolution, pooling, up-sampling and down-sampling.
S203: and performing image semantic segmentation on the image of the current live broadcast room video stream by adopting a trained deep learning algorithm, and forming a plurality of regions by taking a target scene contained in the image as a unit.
In the embodiment of the invention, further, after a plurality of regions are formed, the target scene region can be marked, and the marking dimensions include a series of dimensions such as foreground, background, meaningful region, meaningless region, region area and the like, so as to facilitate subsequent screening operation.
Optionally, on the basis of the first optional embodiment corresponding to fig. 1, in a second optional embodiment of the live broadcast room barrage display method provided in the embodiment of the present invention, a barrage display area is screened out from the target scene area, which specifically includes:
s301: judging the live broadcast type of the current live broadcast room, if the live broadcast is outdoor live broadcast, turning to S302, and if the live broadcast is indoor live broadcast, turning to S303; the live broadcast scene types are divided from a large aspect and generally divided into outdoor live broadcast and indoor live broadcast.
S302: the target scene area except the target scene area where the anchor character and the outdoor related prop are located is a bullet screen display area; outdoor live broadcast is generally a live broadcast type of a main broadcast survival class or an activity display class, so main broadcast characters and outdoor related props are more important, and a target scene area except a target scene area where the main broadcast characters and the outdoor related props are located is a bullet screen display area.
S303: and the target scene areas except the target scene areas where the anchor character, the display, the keyboard and the mouse are positioned are bullet screen display areas. The indoor live broadcast is generally game live broadcast and talent live broadcast, so that the area where the anchor character, the display, the keyboard and the mouse are located is an area capable of perfectly showing the operation of the anchor character, and the target scene area except the target scene area where the anchor character, the display, the keyboard and the mouse are located is a bullet screen display area.
Optionally, on the basis of the embodiment corresponding to fig. 1, in a third optional embodiment of the live broadcast room barrage display method provided in the embodiment of the present invention, when a barrage is rendered on a barrage display area, based on a payment principle and a principle that the text content of the barrage is related to the live broadcast content of the current live broadcast room, the barrage is scored, and then, according to a principle that the score is from high to low, the barrage with a high score is preferentially rendered on the barrage display area.
In the embodiment of the invention, the bullet screen acquired from the bullet screen server is divided according to the attributes of the bullet screen and based on the payment principle and the principle of the association between the character content of the bullet screen and the live broadcast content of the current live broadcast room in a scoring mode, and then is displayed in sequence according to the priority order, so that the bullet screen is displayed in order.
Optionally, on the basis of the third optional embodiment corresponding to fig. 1, in a fourth optional embodiment of the live broadcast room barrage display method provided in the embodiment of the present invention, the barrage is scored, specifically:
s311: and judging the attribute of the bullet screen, if the sender of the bullet screen is a VIP user or the bullet screen is a paid bullet screen, scoring the bullet screen as a, otherwise, scoring the bullet screen as 0. The payment bullet screen is a special bullet screen sent after the payment prop is used, if the color of the bullet screen is white under normal conditions, the red bullet screen is the payment bullet screen, and the red bullet screen can be sent only after the related prop is used. Both the VIP user and the sender of the paid bullet screen belong to paying users of the live platform,
s312: and judging the text content of the bullet screen, if the text content is related to the live broadcast content of the current live broadcast room, scoring b, otherwise, scoring 0. For example, the anchor is performing live broadcast of a car show, if a certain barrage contains the content of a car, the text content of the barrage is related to the live broadcast content in the current live broadcast room, the principle that the text content of the barrage is related to the live broadcast content in the current live broadcast room is met, and scoring is performed on the barrage related to the live broadcast content so as to facilitate subsequent preferential display.
S323: the calculation formula of the bullet screen final score is as follows:
score＝a*p+b*(1-p)
wherein a and b are positive integers, p is an adjustable scale factor, and the value range is that p is more than or equal to 0 and less than or equal to 1, and is usually 0.5.
In the embodiment of the invention, the bullet screens are scored, the bullet screen with the score of 0 is a common bullet screen, the bullet screen with the score of not 0 is a special bullet screen, and the rendering rule of the common bullet screen on the bullet screen display area is as follows: and if and only if no special barrage is rendered on the barrage display area, rendering on the barrage display area in sequence according to the sending time sequence of the common barrage. And the special barrage is displayed according to the scores in sequence, and if the scores are the same, the special barrage is displayed according to the sending time of the special barrage in sequence. A plurality of bullet screen display areas can be divided in the image, so that all bullet screen display areas on the image are subjected to bullet screen rendering, and at least one bullet screen is rendered on each bullet screen display area.
According to the live broadcast room bullet screen display method, through a deep learning algorithm, image semantic segmentation is carried out on images of a video stream to form a plurality of different target scene areas, then bullet screen display areas are screened out from the target scene areas, bullet screen display is only carried out in the bullet screen display areas when bullet screen display is carried out, the bullet screen display areas are areas which do not influence live broadcast contents, the live broadcast and bullet screen watching requirements of users are met, and meanwhile the use experience of the users is effectively guaranteed.
An embodiment of a storage medium provided in an embodiment of the present invention includes: the storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring a video stream of the current live broadcast room according to the video stream address of the current live broadcast room, and acquiring a barrage of the current live broadcast room from a barrage server;
performing image semantic segmentation on the image of the video stream of the current live broadcast room based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and screening a bullet screen display area in the target scene area, and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
Optionally, on the basis of the above storage medium embodiment, in a first optional embodiment of the storage medium provided by the embodiment of the present invention, the storage medium may adopt any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-readable storage medium may be, for example but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Alternatively, on the basis of the above embodiment of the storage medium and the first alternative embodiment, in a second alternative embodiment of the storage medium provided by the embodiment of the present invention, the computer-readable signal medium may include a data signal propagating in a baseband or as a part of a carrier wave, in which a computer-readable program code is carried. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Optionally, on the basis of the above embodiment of the storage medium and the first and second alternative embodiments, in a third alternative embodiment of the storage medium provided by the embodiment of the present invention, the computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, the programming languages including an object oriented programming language, such as Java, Smalltalk, C + +, and further including a conventional procedural programming language, such as "C" programming language or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
Referring to fig. 2, an embodiment of an electronic device provided in an embodiment of the present invention includes:
the system comprises an acquisition unit, a live broadcast room processing unit and a barrage server, wherein the acquisition unit is used for acquiring a video stream of the current live broadcast room according to a video stream address of the current live broadcast room and acquiring a barrage of the current live broadcast room from the barrage server;
the segmentation unit is used for performing image semantic segmentation on the image of the current live broadcast video stream based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and the rendering unit is used for screening the bullet screen display area in the target scene area and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
An embodiment of a live broadcast room bullet screen display system provided in an embodiment of the present invention includes: the device comprises an acquisition module, a segmentation module and a rendering module.
The acquisition module is used for acquiring the video stream of the current live broadcast room according to the video stream address of the current live broadcast room and acquiring the barrage of the current live broadcast room from the barrage server; the segmentation module is used for performing image semantic segmentation on the image of the current live broadcast video stream based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas; the rendering module is used for screening a bullet screen display area in the target scene area and rendering the bullet screen of the current live broadcast room on the bullet screen display area. When screening is performed, many target scenes are of no particular significance in live scenes, such as target scenes including tables, sky, walls, roads, numbers and the like, and the area where the target scenes are located can be used as a bullet screen display area. The specific screening rule is manually flexibly specified according to the live broadcast scene. Through the bullet screen is rendered in the bullet screen display area, then the bullet screen is displayed, the bullet screen display area is generally an unimportant scene area, and the blocking of important contents in a live-action picture by the bullet screen is avoided through the mode.
Optionally, on the basis of the embodiment corresponding to the live broadcast room barrage display system, in a first optional embodiment of the live broadcast room barrage display system provided in the embodiment of the present invention, the barrage display area is screened out from the target scene area, and specifically:
s301: judging the live broadcast type of the current live broadcast room, if the live broadcast is outdoor live broadcast, turning to S302, and if the live broadcast is indoor live broadcast, turning to S303; the live broadcast scene types are divided from a large aspect and generally divided into outdoor live broadcast and indoor live broadcast.
S302: the target scene area except the target scene area where the anchor character and the outdoor related prop are located is a bullet screen display area; outdoor live broadcast is generally a live broadcast type of a main broadcast survival class or an activity display class, so main broadcast characters and outdoor related props are more important, and a target scene area except a target scene area where the main broadcast characters and the outdoor related props are located is a bullet screen display area.
S303: and the target scene areas except the target scene areas where the anchor character, the display, the keyboard and the mouse are positioned are bullet screen display areas. The indoor live broadcast is generally game live broadcast and talent live broadcast, so that the area where the anchor character, the display, the keyboard and the mouse are located is an area capable of perfectly showing the operation of the anchor character, and the target scene area except the target scene area where the anchor character, the display, the keyboard and the mouse are located is a bullet screen display area.
Optionally, on the basis of the embodiment corresponding to the live broadcast room barrage display system, in a second optional embodiment of the live broadcast room barrage display system provided in the embodiment of the present invention, when the rendering module renders the barrage on the barrage display area, based on a payment principle and a principle that the text content of the barrage is related to the live broadcast content of the current live broadcast room, the barrage is scored, and then, according to a principle that the score is from high to low, the barrage with a high score is preferentially rendered on the barrage display area.
According to the live broadcast room bullet screen display system, through a deep learning algorithm, image semantic segmentation is carried out on images of a video stream to form a plurality of different target scene areas, then bullet screen display areas are screened out from the target scene areas, bullet screen display is only carried out in the bullet screen display areas when bullet screen display is carried out, the bullet screen display areas are areas which do not influence live broadcast contents, the live broadcast and bullet screen watching requirements of users are met, and meanwhile the use experience of the users is effectively guaranteed.
The present invention is not limited to the above-described embodiments, and it will be apparent to those skilled in the art that various modifications and improvements can be made without departing from the principle of the present invention, and such modifications and improvements are also considered to be within the scope of the present invention. Those not described in detail in this specification are within the skill of the art.
Claims (10)
1. A live broadcast room barrage display method is characterized by comprising the following steps:
acquiring a video stream of the current live broadcast room according to the video stream address of the current live broadcast room, and acquiring a barrage of the current live broadcast room from a barrage server;
performing image semantic segmentation on the image of the video stream of the current live broadcast room based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and screening a bullet screen display area in the target scene area, and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
2. The live broadcast room barrage display method as claimed in claim 1, characterized in that: the deep learning algorithm based on the PascalContext data set performs image semantic segmentation on the image of the video stream to form a plurality of different target scene areas, and the specific steps are as follows:
s201: acquiring historical live broadcast image data stored in a live broadcast server, and marking target scenes contained in images in the historical live broadcast image data;
s202: training a deep learning algorithm based on a PascalContext data set by using marked historical live image data;
s203: and performing image semantic segmentation on the image of the current live broadcast room video stream by adopting a trained deep learning algorithm, and forming a plurality of regions by taking a target scene contained in the image as a unit.
3. The live broadcast room barrage display method as claimed in claim 2, characterized in that: the method comprises the following steps of screening a bullet screen display area in a target scene area, specifically:
s301: judging the live broadcast type of the current live broadcast room, if the live broadcast is outdoor live broadcast, turning to S302, and if the live broadcast is indoor live broadcast, turning to S303;
s302: the target scene area except the target scene area where the anchor character and the outdoor related prop are located is a bullet screen display area;
s303: and the target scene areas except the target scene areas where the anchor character, the display, the keyboard and the mouse are positioned are bullet screen display areas.
4. The live broadcast room barrage display method as claimed in claim 1, characterized in that: when the barrage is rendered on the barrage display area, based on the payment principle and the principle that the barrage text content is related to the live broadcast content of the current live broadcast room, the barrage is scored, and then according to the principle that scores are from high to low, the barrage with high scores is preferentially rendered on the barrage display area.
5. The live broadcast room barrage display method of claim 4, wherein: the bullet screen is scored, and the scoring specifically comprises the following steps:
s311: judging the attribute of the bullet screen, if the sender of the bullet screen is a VIP user or the bullet screen is a paid bullet screen, scoring the bullet screen as a, otherwise, scoring the bullet screen as 0;
s312: judging the text content of the bullet screen, if the text content is related to the live broadcast content of the current live broadcast room, scoring b, otherwise, scoring 0;
s323: the calculation formula of the bullet screen final score is as follows:
score＝a*p+b*(1-p)
wherein a and b are positive integers, p is an adjustable scale factor, and the value range is that p is more than or equal to 0 and less than or equal to 1.
6. The live broadcast room barrage display method as claimed in claim 5, characterized in that: scoring the bullet screen, the bullet screen that the score is 0 is ordinary bullet screen, and the bullet screen that the score is not 0 is special bullet screen, ordinary bullet screen is the rule of rendering on bullet screen display area does: and if and only if no special barrage is rendered on the barrage display area, rendering on the barrage display area in sequence according to the sending time sequence of the common barrage.
7. A storage medium having a computer program stored thereon, characterized in that: the computer program when executed by a processor implementing the steps of:
acquiring a video stream of the current live broadcast room according to the video stream address of the current live broadcast room, and acquiring a barrage of the current live broadcast room from a barrage server;
performing image semantic segmentation on the image of the video stream of the current live broadcast room based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and screening a bullet screen display area in the target scene area, and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
8. An electronic device, characterized in that the electronic device comprises:
the system comprises an acquisition unit, a live broadcast room processing unit and a barrage server, wherein the acquisition unit is used for acquiring a video stream of the current live broadcast room according to a video stream address of the current live broadcast room and acquiring a barrage of the current live broadcast room from the barrage server;
the segmentation unit is used for performing image semantic segmentation on the image of the current live broadcast video stream based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and the rendering unit is used for screening the bullet screen display area in the target scene area and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
9. A live broadcast room barrage display system, comprising:
the acquisition module is used for acquiring the video stream of the current live broadcast room according to the video stream address of the current live broadcast room and acquiring the barrage of the current live broadcast room from the barrage server;
the segmentation module is used for performing image semantic segmentation on the image of the current live broadcast video stream based on a deep learning algorithm of a PascalContext data set to form a plurality of different target scene areas;
and the rendering module is used for screening the bullet screen display area in the target scene area and rendering the bullet screen of the current live broadcast room on the bullet screen display area.
10. The live room bullet screen display system of claim 9, wherein: when the module of rendering renders the barrage on barrage display area, based on the principle of paying to and barrage literal content and the current live broadcast relevant principle of room live broadcast content, score the barrage, then according to the rule of score from high to low, the barrage that scores is high is preferentially rendered on barrage display area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811253120.7A CN111107381A (en) | 2018-10-25 | 2018-10-25 | Live broadcast room bullet screen display method, storage medium, equipment and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811253120.7A CN111107381A (en) | 2018-10-25 | 2018-10-25 | Live broadcast room bullet screen display method, storage medium, equipment and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111107381A true CN111107381A (en) | 2020-05-05 |
Family
ID=70418298
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811253120.7A Pending CN111107381A (en) | 2018-10-25 | 2018-10-25 | Live broadcast room bullet screen display method, storage medium, equipment and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111107381A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112752110A (en) * | 2020-08-14 | 2021-05-04 | 腾讯科技（深圳）有限公司 | Video presentation method and device, computing equipment and storage medium |
| CN114520923A (en) * | 2020-11-20 | 2022-05-20 | 武汉斗鱼网络科技有限公司 | Barrage information display method, first terminal, second terminal, medium and equipment |
| WO2023207516A1 (en) * | 2022-04-27 | 2023-11-02 | 北京字跳网络技术有限公司 | Live streaming video processing method and apparatus, electronic device, and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102222101A (en) * | 2011-06-22 | 2011-10-19 | 北方工业大学 | Method for video semantic mining |
| CN106028163A (en) * | 2016-05-19 | 2016-10-12 | 乐视控股（北京）有限公司 | Bullet curtain display method and device for client |
| CN106303731A (en) * | 2016-08-01 | 2017-01-04 | 北京奇虎科技有限公司 | The display packing of barrage and device |
| CN106792100A (en) * | 2016-12-30 | 2017-05-31 | 北京奇艺世纪科技有限公司 | A kind of video barrage display methods and device |
| CN107181976A (en) * | 2017-04-28 | 2017-09-19 | 华为技术有限公司 | A kind of barrage display methods and electronic equipment |
| CN108471557A (en) * | 2018-03-28 | 2018-08-31 | 武汉斗鱼网络科技有限公司 | A kind of barrage broadcasting method, device, server and medium |
| US20180267970A1 (en) * | 2016-06-03 | 2018-09-20 | Tencent Technology (Shenzhen) Company Limited | Bullet screen content processing method, application server, and user terminal |
| CN108632658A (en) * | 2018-03-14 | 2018-10-09 | 维沃移动通信有限公司 | A kind of barrage display methods, terminal |
-
2018
- 2018-10-25 CN CN201811253120.7A patent/CN111107381A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102222101A (en) * | 2011-06-22 | 2011-10-19 | 北方工业大学 | Method for video semantic mining |
| CN106028163A (en) * | 2016-05-19 | 2016-10-12 | 乐视控股（北京）有限公司 | Bullet curtain display method and device for client |
| US20180267970A1 (en) * | 2016-06-03 | 2018-09-20 | Tencent Technology (Shenzhen) Company Limited | Bullet screen content processing method, application server, and user terminal |
| CN106303731A (en) * | 2016-08-01 | 2017-01-04 | 北京奇虎科技有限公司 | The display packing of barrage and device |
| CN106792100A (en) * | 2016-12-30 | 2017-05-31 | 北京奇艺世纪科技有限公司 | A kind of video barrage display methods and device |
| CN107181976A (en) * | 2017-04-28 | 2017-09-19 | 华为技术有限公司 | A kind of barrage display methods and electronic equipment |
| CN108632658A (en) * | 2018-03-14 | 2018-10-09 | 维沃移动通信有限公司 | A kind of barrage display methods, terminal |
| CN108471557A (en) * | 2018-03-28 | 2018-08-31 | 武汉斗鱼网络科技有限公司 | A kind of barrage broadcasting method, device, server and medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112752110A (en) * | 2020-08-14 | 2021-05-04 | 腾讯科技（深圳）有限公司 | Video presentation method and device, computing equipment and storage medium |
| CN114520923A (en) * | 2020-11-20 | 2022-05-20 | 武汉斗鱼网络科技有限公司 | Barrage information display method, first terminal, second terminal, medium and equipment |
| CN114520923B (en) * | 2020-11-20 | 2023-09-01 | 武汉斗鱼网络科技有限公司 | Barrage information display method, first terminal, second terminal, medium and equipment |
| WO2023207516A1 (en) * | 2022-04-27 | 2023-11-02 | 北京字跳网络技术有限公司 | Live streaming video processing method and apparatus, electronic device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021004247A1 (en) | Method and apparatus for generating video cover and electronic device | |
| CN110189336B (en) | Image generation method, system, server and storage medium | |
| CN110602554A (en) | Cover image determining method, device and equipment | |
| US11871086B2 (en) | Method of displaying comment information, computing device, and readable storage medium | |
| US20220277481A1 (en) | Panoramic video processing method and apparatus, and storage medium | |
| CN111080799A (en) | Scene roaming method, system, device and storage medium based on three-dimensional modeling | |
| CN111654746B (en) | Video frame insertion method and device, electronic equipment and storage medium | |
| CN103971391A (en) | Animation method and device | |
| CN111107381A (en) | Live broadcast room bullet screen display method, storage medium, equipment and system | |
| CN110177295B (en) | Subtitle out-of-range processing method and device and electronic equipment | |
| CN109729429B (en) | Video playing method, device, equipment and medium | |
| CN110446093A (en) | A kind of video progress bar display methods, device and storage medium | |
| CN110930220A (en) | Display method, display device, terminal equipment and medium | |
| CN114205635A (en) | Live comment display method, device, equipment, program product and medium | |
| CN107404662B (en) | Method and device for content promotion in live broadcast room | |
| CN112827172A (en) | Shooting method, shooting device, electronic equipment and storage medium | |
| CN112995694B (en) | Video display method and device, electronic equipment and storage medium | |
| CN110728129A (en) | Method, device, medium and equipment for typesetting text content in picture | |
| CN113301382B (en) | Video processing method, device, medium, and program product | |
| US11335084B2 (en) | Image object anomaly detection | |
| CN109871465B (en) | Time axis calculation method and device, electronic equipment and storage medium | |
| CN116089757A (en) | Page display method and device, electronic equipment, storage medium and program product | |
| CN113641853A (en) | Dynamic cover generation method, device, electronic equipment, medium and program product | |
| CN111556372A (en) | Method and device for adding subtitles to video and audio programs in real time | |
| CN112465692A (en) | Image processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200505 |
|
| RJ01 | Rejection of invention patent application after publication |