CN107911708B

CN107911708B - Barrage display method, live broadcast method and related devices

Info

Publication number: CN107911708B
Application number: CN201711100419.4A
Authority: CN
Inventors: 向晨宇; 李论; 范保成; 张俊辉
Original assignee: Tencent Cyber Tianjin Co Ltd
Current assignee: Tencent Cyber Tianjin Co Ltd
Priority date: 2017-11-09
Filing date: 2017-11-09
Publication date: 2022-04-05
Anticipated expiration: 2037-11-09
Also published as: CN107911708A

Abstract

The embodiment of the invention discloses a bullet screen display method, a live broadcast method and a related device, wherein the bullet screen display method is applied to game application in a bullet screen display device for live game broadcast, and the bullet screen display method comprises the following steps: starting a game in a live broadcast application, and utilizing the live broadcast application to carry out live broadcast on a game picture of the game application; establishing a special communication channel with the live broadcast application; acquiring barrage data acquired by the live broadcast application through the special communication channel, wherein the barrage data is comment data issued by a user corresponding to each terminal aiming at a live broadcast game picture; and rendering the bullet screen data in a GPU (graphics processing Unit), and superposing the rendered bullet screen data in the current game picture for display. The embodiment of the invention can realize the display of the bullet screen during live game and enhance the interaction among users.

Description

Barrage display method, live broadcast method and related devices

Technical Field

The embodiment of the invention relates to the field of multimedia display, in particular to a barrage display method, a live broadcast method and a related device.

Background

With the hardware development of mobile devices such as smart phones and tablet computers and the development of communication networks, a new industry, namely 'live game', is promoted. In a video picture being played, a manner of displaying a comment in a caption fashion from the picture is called a bullet screen. If live game and barrage are combined, a new experience is provided for users, and the interaction between the users can be enhanced, but the prior art lacks a related implementation scheme.

Disclosure of Invention

In view of this, embodiments of the present invention provide a bullet screen display method, a live broadcast method, and a related device, which can display a bullet screen during live broadcast of a game, and enhance interaction between users.

The bullet screen display method provided by the embodiment of the invention is applied to game application in a bullet screen display device for live game, and comprises the following steps:

starting a game in a live broadcast application, and utilizing the live broadcast application to carry out live broadcast on a game picture of the game application;

establishing a special communication channel with the live broadcast application;

acquiring barrage data acquired by the live broadcast application through the special communication channel, wherein the barrage data is comment data issued by a user corresponding to each terminal aiming at a live broadcast game picture;

and rendering the bullet screen data in a GPU (graphics processing Unit), and superposing the rendered bullet screen data in the current game picture for display.

The live broadcasting method provided by the embodiment of the invention is applied to live broadcasting application in a live barrage display device for game live broadcasting, and comprises the following steps:

starting a live broadcast function, and carrying out live broadcast on a game picture of the game application;

establishing a dedicated communication channel with the game application;

acquiring bullet screen data, wherein the bullet screen data are comment data issued by a user corresponding to each terminal aiming at a live game picture;

and sending the bullet screen data to the game application through the special communication channel, so that the game application renders the bullet screen data and then displays the bullet screen data in a superposition mode in a current game picture.

The embodiment of the present invention further provides a bullet screen display device, including:

the system comprises a first starting unit, a second starting unit and a third starting unit, wherein the first starting unit is used for starting a game in a live broadcast application and utilizing the live broadcast application to carry out live broadcast on a game picture of the game application;

the first establishing unit is used for establishing a special communication channel with the live broadcast application;

a first obtaining unit, configured to obtain, through the dedicated communication channel, bullet screen data obtained by the live broadcast application, where the bullet screen data is comment data issued by a user corresponding to each terminal for a game picture being live broadcast;

and the rendering processing unit is used for rendering the bullet screen data in the GPU and superposing the rendered bullet screen data in the current game picture for display.

An embodiment of the present invention further provides a live broadcasting device, including:

the second starting unit is used for starting a live broadcast function and conducting live broadcast on game pictures of the game application;

a second establishing unit, configured to establish a dedicated communication channel with the game application;

the second acquisition unit is used for acquiring barrage data, and the barrage data is comment data issued by a user corresponding to each terminal aiming at a live game picture;

and the sending unit is used for sending the bullet screen data to the game application through the special communication channel so that the game application renders the bullet screen data and then displays the bullet screen data in a superposed manner in a current game picture.

The embodiment of the invention also provides a live game barrage display device, which comprises the live game barrage display device and the live game device.

In the embodiment of the invention, a game application starts a game in a live broadcast application, a game picture of the game application is live broadcast by using the live broadcast application, then a special communication channel is established between the game application and the live broadcast application, bullet screen data obtained by the live broadcast application is obtained through the special communication channel, and finally the bullet screen data is rendered in a GPU and is superposed in a current game picture for display. In the embodiment of the invention, the bullet screen is displayed during the live play of the game by adopting multi-process (multi-application) communication, so that brand new experience is brought to users, and the interaction among the users is enhanced.

In addition, the bullet screen data are transmitted by establishing a special communication channel, so that the bullet screen data can be prevented from being intercepted, and the data safety is ensured.

In addition, because the increment of the installation package of the game application is limited, part of implementation logic of the bullet screen display is placed at the live broadcast application end, so that the problem that the size of the installation package of the game application is overlarge due to the fact that all the implementation logic of the bullet screen display is placed at the game application end can be solved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic view of an application scenario of a bullet screen display method according to an embodiment of the present invention;

fig. 2a is a schematic flowchart of a bullet screen display method according to an embodiment of the present invention;

FIG. 2b is a rendering pipeline schematic of an embodiment of the present invention;

fig. 3a is a schematic flow chart of a live broadcasting method provided in an embodiment of the present invention;

FIG. 3b is a diagram illustrating a method for broadcasting service information according to an embodiment of the present invention;

FIG. 4 is a schematic flow chart of implementing the display of a bullet screen during live game play according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a bullet screen display device according to an embodiment of the present invention;

fig. 6 is another schematic structural diagram of a bullet screen display device according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a live broadcast apparatus provided in an embodiment of the present invention;

fig. 8 is another schematic structural diagram of a live broadcast apparatus provided in an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a live game barrage display device according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The embodiment of the invention provides a bullet screen display method, a live broadcast method and a related device, which can display a bullet screen during live broadcast of a game. The barrage display method and the live broadcast method of the embodiment of the invention are both used for a live broadcast end, namely electronic equipment of the live broadcast end, and the electronic equipment comprises the following components: the mobile phone, the tablet computer, the computer and the like at the anchor end enable the anchor to be the user who carries out the live broadcast, and enable the game player in the live broadcast to be the anchor. Specifically, in the embodiment of the present invention, a live application (e.g., a live assistant, a live companion) and a game application (e.g., a fighter and a royal glory) are installed in the electronic device of the anchor. A specific application scenario can be as shown in fig. 1, when a anchor needs to use a live application for live game playing, the live application may be started, a game is started through the live application, that is, the game application is started in the live application, after the start, a dedicated communication channel is established between the game application and the live application, the game application obtains bullet screen data obtained by the live application through the established dedicated communication channel (the live application may obtain bullet screen data from a background server, and the background server may be a bullet screen server or a background server of the game application), the bullet screen data may be comment data issued by a user (i.e., a viewer) at each terminal on a live game picture, and then renders the bullet screen data in a Graphics Processing Unit (GPU), and superimposes the rendered bullet screen data on a current game picture for display, finally, the anchor end will present a game picture with a bullet screen based on which the anchor can interact with the audience.

The following detailed description will be made separately, and the description sequence of each embodiment below does not limit the specific implementation sequence.

In this embodiment, a bullet screen display method will be described from the perspective of a bullet screen display device, in which a game application is installed, and which may be specifically integrated in an electronic device at a main broadcast end.

As shown in fig. 2a, the bullet screen display method of the present embodiment includes the following steps:

step S201, starting a game in a live broadcast application, and utilizing the live broadcast application to carry out live broadcast on a game picture of the game application.

And the live application refers to software for live broadcasting, such as a live broadcasting assistant, a live broadcasting partner and the like. The game application refers to software for playing a game, such as a fighter, a royal glory, and the like.

The method of the embodiment is mainly applied to a main broadcasting end, the main broadcasting end can simultaneously install the live broadcasting application and the game application, and the main broadcasting uses the game application to play the game and simultaneously utilizes the live broadcasting application to carry out the live broadcasting of the game.

Specifically, the anchor may start a live application, select a live mode in the live application, and then start a game application in the live application, that is, the game application is started in the live application, so as to use the live application to live a game screen of the game application.

In this embodiment, live broadcasting can be performed in a screen recording live broadcasting mode. For example, the live broadcast application may collect a screen image and an audio signal of a main broadcast terminal, encode the collected screen image and audio signal, and send the encoded screen image and audio signal to a live broadcast server, and the live broadcast server sends live broadcast data to each terminal that needs to watch the live broadcast, so as to realize screen recording and live broadcast. Of course, in practice, other modes can be adopted for live broadcast, such as: the window is live, and is not limited in particular here.

And S202, establishing a special communication channel with the live broadcast application.

During the running process of the game application, a special communication channel can be established between the game application and the live broadcast application so as to acquire bullet screen data. Specifically, the dedicated communication channel may be established by a specific IP address and port number.

That is, after the live application is started, the zero-configuration networking Bonjour service of the live application may be started, the service information of the live application may be broadcast, where the service information includes an IP address and identification information (e.g., a host name, a model number, etc.) of a host running the live application, and may also include other information such as a service type and a port number, which is not specifically limited herein.

Since Bonjour distributes service information in a broadcast manner, all electronic devices (including local devices) in the local area network can receive the service information. Similarly, when the live application on other electronic equipment publishes service information by using Bonjour, the live application can also receive the service information. The native machine mentioned in this embodiment refers to a host machine running the game application.

Therefore, in order to identify the live application running on the local computer, after receiving the service information broadcasted by the live application, the game application can determine whether the IP address and the identification information contained in the service information are corresponding and consistent with the IP address and the identification information of the local computer, and if so, it indicates that the game application and the live application run on the same host computer, and the game application can establish a socket link channel with the live application based on the consistent IP address and the identification information. If the game application and the live application are not consistent, the game application and the live application are not operated in the same host, and the service information of the live application is lost.

Because the established special communication channel is used for acquiring bullet screen data, the special communication channel is established by adopting the consistent IP address and the consistent port number, the data filtering of other players who open the live game under the same wireless local area network (WIFI) can be realized, and the data processing efficiency is improved.

And step S203, acquiring the bullet screen data acquired by the live broadcast application through the special communication channel.

And the bullet screen data is comment data which is published by the user corresponding to each terminal aiming at the live game picture.

When users corresponding to the terminals watch the live broadcasts, comment data can be published through the corresponding terminals, the comment data published by the corresponding users are sent to a background server by the terminals, and the background server can be a bullet screen server or a background server applied to games. In the live broadcast process, the live broadcast application can send a bullet screen acquisition request to the background server, the background server can feed back bullet screen data to the live broadcast application according to the bullet screen acquisition request, and the game application can acquire the bullet screen data through a special communication channel established between the game application and the live broadcast application.

When the game application acquires the bullet screen data through the dedicated communication channel, a preset Protocol may be adopted, and the preset Protocol includes, but is not limited to, a Transmission Control Protocol (TCP) Protocol and a User Datagram Protocol (UDP) Protocol. Preferably, a TCP protocol may be used, which is more secure and has a lower data loss rate than a UDP protocol.

And S204, rendering the bullet screen data in the GPU, and superposing the rendered bullet screen data in the current game picture for display.

Specifically, the game application may call an OpenGL ES in the GPU, and render the bullet screen data according to the bullet screen display mode set by the anchor user. The bullet screen display mode comprises the following steps: font style, font size, font color and display effect when bullet screen data is displayed.

The font style, the font size, the font color and the display effect can be set according to the actual requirement. Font styles are for example: regular script, song script, black body, etc. Font sizes are for example: the setting may be made in conjunction with the size of the screen of the electronic device. Font colors are for example: it may be set in conjunction with the color of the game screen, and when the screen is white, the font is set to black, red, or the like. The display effects are, for example: planar display effect, stereoscopic display effect, uniform speed display and the like.

In addition, the bullet screen display mode may further include other contents, such as a display position, and the like, which is not specifically limited herein.

OpenGL ES defines the concept of buffers (buffers) for data exchange between two memory areas in order to improve rendering performance. A buffer refers to a continuous Random Access Memory (RAM) that the GPU can control and manage. The program copies data from the memory of a Central Processing Unit (CPU) to the buffer of the OpenGL ES. By monopolizing the buffer, the GPU can read and write memory as efficiently as possible. The GPU applies its ability to process data asynchronously on the buffer, meaning that programs running in the CPU can continue to execute while the GPU works with the data in the buffer. The GPU needs to know which location in memory to store the rendered image pixel data, and the buffer that receives the rendering results is called a frame buffer. Rendering instructions will replace the contents of the frame buffer at the appropriate time, and OpenGL ES will set the data type and offset according to the specific platform hardware configuration and function. In general, the rendering results may be stored in any number of frame buffers.

Specifically, in this implementation, the rendering pipeline may be as shown in fig. 2b, and the rendering process may be as follows:

vertex Array (Vertex Array) or Buffer objects (Buffer objects): vertex data (i.e., bullet screen data to be rendered, in OpenGL ES, all objects and geometric elements are finally expressed in the form of vertices, which generally generate triangles, lines or points) is obtained through an Application Programming Interface (API), and the Vertex data is loaded into a Vertex Array (Vertex Array) or Buffer objects (Buffer objects).

Vertex Shader (Vertex Shader): the Vertex Shader (Vertex Shader) realizes the operation of Vertex data in a Vertex array or a buffer object in a programmable mode, such as coordinate space conversion, texture coordinate calculation and the like.

Primitive Assembly (primative Assembly): vertex data after shader processing is assembled into basic primitives in a Primitive Assembly (primative Assembly) phase, and OpenGL ES supports three basic primitives: points, lines and triangles, which are renderable by OpenGL ES. Optionally, clipping (clip) may be performed on the assembled primitive, the primitive completely in the view frustum is retained, the primitive completely not in the view frustum is discarded, and the primitive half not in the view frustum is clipped; the primitives in the view frustum are then culled (cut), which can be encoded to determine whether to cull the front, back, or all of the primitives.

Rasterization (Rasterization): in the Rasterization stage, the primitive is converted into two-dimensional fragments (fragments) representing pixels that can be rendered on the screen, which contain information on position, color, texture coordinates, etc., which are interpolated from the vertex information of the primitive.

Fragment Shader (Fragment Shader): the fragment shader implements operations on the fragments in a programmable manner. At this stage it accepts as input the fragments, colors, depth values, template values after the rasterization process.

Per Fragment Operation (Per-Fragment Operation): at this stage, a series of tests and processing are performed on each fragment output by the fragment shader to determine the pixels that are ultimately used for rendering.

Frame Buffer (Frame Buffer): the final result is written into a Frame Buffer.

It should be noted that the bullet screen display method provided in this embodiment may be implemented by embedding a Software Development Kit (SDK) in an installation package of the game application.

In this embodiment, a game application starts a game in a live application, the live application is used to live a game picture of the game application, then a dedicated communication channel is established between the game application and the live application, bullet screen data obtained by the live application is obtained through the dedicated communication channel, and finally the bullet screen data is rendered in a GPU, and the rendered bullet screen data is superimposed on a current game picture for display. In the embodiment of the invention, the bullet screen is displayed during the live play of the game by adopting multi-process (multi-application) communication, so that brand new experience is brought to users, and the interaction among the users is enhanced.

In addition, by establishing a dedicated communication channel between applications to transmit the bullet screen data, the bullet screen data can be prevented from being eavesdropped.

In this embodiment, a live broadcast method will be described from the perspective of a live broadcast device, in which a live broadcast application is installed, and which may be specifically integrated in an electronic device at a main broadcast end.

As shown in fig. 3a, the live broadcasting method of the present embodiment includes the following steps:

and S301, starting a live broadcast function, and carrying out live broadcast on a game picture of the game application.

In specific implementation, when the anchor needs to play the game live broadcast, the live broadcast application can start a live broadcast function according to the operation of the anchor, and the game picture of the game application is live broadcast. In this embodiment, the live application may be live in a screen recording and live broadcasting manner. For example, the live broadcast application can collect a screen image and an audio signal of a main broadcast end, codes the collected screen image and audio signal and then sends the coded screen image and audio signal to a live broadcast server, and the live broadcast server sends live broadcast data to terminals needing to watch the live broadcast, so that screen recording and live broadcast are achieved. Of course, in practice, other modes can be adopted for live broadcast, such as: the window is live, and is not limited in particular here.

Thereafter, the live broadcast application may start its own zero-configuration networking Bonjour service, and broadcast its own service information, where the service information includes an IP address and identification information (e.g., a host name, a model number, etc.) of a host running the live broadcast application, and may also include other information such as a service type and a port number, which is not specifically limited herein.

The zero configuration networking Bonjour is a Name from apple company for an open zero setup network standard based on multicast Domain Name service (multicast Domain Name System), and is a protocol capable of automatically querying devices or applications in an access network. Bonjour's device or application is used to automatically disseminate their own service information in the network and listen to the service information of other devices or applications as between devices or applications are on call, which makes devices or applications in a local area network easy to find even without a network administrator.

The work process of Bonjour can be summarized as: announcements and queries. Announcing: any device or application accessing the network will advertise its own information, such as IP address, port number, service type, etc. And (3) inquiring: whether the service which the user wants exists in the whole network or not is inquired, whether the service which the user can access exists is inquired, whether conflict exists in the network is inquired, and the IP address, the port number and the like of the target host are inquired.

After broadcasting the service information of the live application itself, the live application may perform process residence in the background in a preset manner to avoid losing network connection, where the preset manner includes, but is not limited to, the following:

(1) and audio control, namely broadcasting or recording continuously in the background.

(2) And receiving the position update, and continuously acquiring the geographical position of the local machine and updating.

(3) A task of limited length is performed.

(4) And magazines and magazine application special cases can be downloaded in the background.

(5) And providing network telephone service.

Preferably, audio control may be employed, i.e. the live application may play an audio file in an endless loop in the background, using a play component provided by the system, which audio file may be sound-free, e.g. an empty or fake audio file.

And step S302, establishing a special communication channel with the game application.

Since Bonjour distributes service information in a broadcast manner, all electronic devices (including local devices) in the local area network can receive the service information. For example, as shown in fig. 3b, for example, n mobile phones are in a local area network for live broadcasting, after the live broadcast application on the mobile phone 1 broadcasts service information, the game application on the mobile phone 1 receives the service information, and similarly, the game applications on the mobile phones 2 to n all receive the service information. After receiving the service information, each game application judges whether the live broadcast application is the live broadcast application on the own local machine or not based on the service information (namely, whether the IP address and the identification information carried in the service information are corresponding to the IP address and the identification information on the own local machine or not is judged), and if not, the corresponding service information is lost.

Specifically, in the above example, the game applications in the mobile phones 2 to n all discard the service information, the game application in the mobile phone 1 retains the service information, and the live application in the mobile phone 1 establishes a socket link channel with the game application in the mobile phone 1 based on the IP address and the identification information carried in the service information. After the socket link channel is established, the live application can close the Bonjour service of the live application.

Step S303, bullet screen data is obtained, and the bullet screen data is comment data issued by a user corresponding to each terminal aiming at a live game picture.

The live broadcast application carries out live broadcast on a game picture of the currently running game application, a user corresponding to each terminal watches the live broadcast, comment data can be published through the corresponding terminal when watching, each terminal sends the comment data published by the corresponding user to a background server, and the background server can be a bullet screen server or a background server of the game application.

In the live broadcast process, the live broadcast application can send a bullet screen acquisition request to the background server and then receive bullet screen data fed back by the background server according to the bullet screen acquisition request. Or, after receiving the barrage data of the game picture which is sent by each terminal and is directed to the live broadcast application, the background server may directly send the barrage data to the live broadcast application, and the live broadcast application may receive the barrage data sent by the background server.

Step S304, the bullet screen data is sent to the game application through the special communication channel, so that the game application renders the bullet screen data and then superposes the bullet screen data on a current game picture for display

And the live broadcast application sends the acquired bullet screen data to the game application through a special communication channel established between the live broadcast application and the game application. When sending, a preset Protocol may be adopted, and the preset Protocol includes, but is not limited to, a Transmission Control Protocol (TCP) Protocol and a User Datagram Protocol (UDP) Protocol. Preferably, a TCP protocol may be used, which is more secure and has a lower data loss rate than a UDP protocol.

After receiving the bullet screen data, the game application can render the bullet screen data in the GPU, and superimpose the rendered bullet screen data on the current game picture for display.

After the game ends, the live application may exit from the background.

In this embodiment, live broadcast application starts the live broadcast function, carries out the live broadcast to the recreation picture of recreation application, then with establish the private communication passageway between the recreation application, acquire the bullet screen data, will bullet screen data passes through private communication passageway sends for the recreation application, so that the recreation application will the bullet screen data is played back stack and is shown in current recreation picture, and this embodiment has realized the live bullet screen display of recreation through the mode that adopts the multi-process communication, has brought brand-new experience for the user, has strengthened the interdynamic between the user.

In addition, because the increment of the installation package of the game application is limited, the live broadcast application end shares part of realization logic of bullet screen display, and the problem of overlarge volume of the game application installation package caused by putting all realization logic of bullet screen display at the game application end can be avoided.

In this embodiment, a description will be given from the perspective of a live game barrage display device, in which a game application and a live application are installed, and the live game barrage display device may be specifically integrated in an electronic device at a main broadcast end.

As shown in fig. 4, the method of the present embodiment includes:

and step S401, the live application starts a live function.

The method of the embodiment is mainly applied to a main broadcasting end, the main broadcasting end can simultaneously install the live broadcasting application and the game application, and the main broadcasting uses the game application to play the game and simultaneously utilizes the live broadcasting application to carry out the live broadcasting of the game. When the anchor needs to play the game live broadcast, the live broadcast application can start the live broadcast function according to the operation of the anchor.

Step S402, the game application starts the game in the live application.

That is, the game application starts the game in the live application according to the operation of the anchor user.

Step S403, the live application starts Bonjour service and broadcasts the service information of the live application.

The service information includes an IP address and identification information (e.g., host name, model number, etc.) of a host running the live broadcast application, and may also include other information such as a service type and a port number, which is not specifically limited herein.

Thereafter, the live application can play an audio file in the background, which can be sound-free, such as an empty or fake audio file, in an endless loop using a system-provided play component, to enable the process to be resident in the background until the game is over.

Step S404, the game application judges whether the IP address and the identification information contained in the received service information are consistent with the IP address and the identification information of the game application, if so, the step S405 is executed, otherwise, the step S409 is executed, and the service information is discarded.

In order to identify a live application running on a local computer, after receiving service information broadcasted by the live application, a game application can judge whether an IP address and identification information contained in the service information are corresponding and consistent with an IP address and identification information of the local computer (a host computer where the game application runs), if so, the game application and the live application run on the same host computer, and the game application can establish a socket link channel with the live application based on the consistent IP address and identification information. If the game application and the live application are not consistent, the game application and the live application are not operated in the same host, and the service information of the live application is lost.

Step S405, the game application establishes a special communication channel with the live broadcast application by using the consistent IP address and the identification information.

The gaming application may establish a socket link channel with the live application based on the consistent IP address and identification information.

Step S406, the live broadcast application acquires bullet screen data.

In the live broadcast process, the live broadcast application can send a bullet screen acquisition request to the background server and then receive bullet screen data fed back by the background server according to the bullet screen acquisition request.

Step S407, the game application obtains the bullet screen data obtained by the live broadcast application through the special communication channel.

And step S408, rendering the bullet screen data in the GPU by the game application, and superposing the rendered bullet screen data in the current game picture for display.

Specifically, the game application may call an OpenGL ES in the GPU, and render the bullet screen data according to the bullet screen display mode set by the anchor user. The bullet screen display mode comprises the following steps: font style, font size, font color and display effect when bullet screen data is displayed. And after the barrage rendering is finished, the rendered barrage data is superposed in the current game picture for display.

When the game is over, the live application exits from the background.

In the embodiment, by adopting multi-process (multi-application) communication, the barrage is displayed during the live playing of the game, so that brand new experience is brought to users, and the interaction among the users is enhanced; the bullet screen data are transmitted by establishing a special communication channel, so that the bullet screen data can be prevented from being intercepted, and the data safety is ensured; furthermore, because the increment of the installation package of the game application is limited, part of implementation logic displayed by the bullet screen is placed at the live broadcast application end, so that the problem that the size of the installation package of the game application is overlarge due to the fact that all implementation logic displayed by the bullet screen is placed at the game application end can be solved.

An embodiment of the present invention further provides a bullet screen display device, as shown in fig. 5, the bullet screen display device of this embodiment includes: the first starting unit 501, the first establishing unit 502, the first obtaining unit 503 and the rendering processing unit 504 are as follows:

a first starting unit 501, configured to start a game in a live broadcast application, and perform live broadcast on a game screen of the game application by using the live broadcast application;

a first establishing unit 502, configured to establish a dedicated communication channel with the live application;

a first obtaining unit 503, configured to obtain, through the dedicated communication channel, barrage data obtained by the live application, where the barrage data is comment data issued by a user corresponding to each terminal for a game picture being live;

and the rendering processing unit 504 is configured to render the bullet screen data in the GPU, and superimpose the rendered bullet screen data on the current game screen for display.

In some embodiments, as shown in fig. 6, the bullet screen display device further includes:

a receiving unit 505, configured to receive service information broadcasted by the live application, where the service information includes an internet protocol address IP address and identification information of a host running the live application;

a judging unit 506, configured to judge whether the internet protocol address and the identification information included in the service information correspond to and are consistent with the internet protocol address and the identification information of the local computer;

the first establishing unit 502 is configured to execute a step of establishing a dedicated communication channel with the live application when the internet protocol address and the identification information included in the service information correspond to and are consistent with the internet protocol address and the identification information of the local device.

In some embodiments, the first establishing unit 502 is specifically configured to:

and establishing a socket link channel between the live broadcast application and the consistent Internet protocol address and identification information.

In some embodiments, the rendering processing unit 504 is specifically configured to:

and rendering the bullet screen data in the graphic processor according to a bullet screen display mode set by the anchor user.

In some embodiments, the bullet screen display mode includes: font style, font size, font color and display effect when bullet screen data is displayed.

As can be seen from the above, in this embodiment, in the bullet screen display device, the first starting unit 501 starts a game in a live application, the live application is used to live a game screen of the game application, then the first establishing unit 502 establishes a dedicated communication channel with the live application, the first obtaining unit 503 obtains bullet screen data obtained by the live application through the dedicated communication channel, and finally the rendering processing unit 504 renders the bullet screen data in the GPU, and superimposes the rendered bullet screen data on a current game screen for display. In the embodiment of the invention, the bullet screen is displayed during the live play of the game by adopting multi-process (multi-application) communication, so that brand new experience is brought to users, and the interaction among the users is enhanced.

In specific implementation, the above modules may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and specific implementation of the above modules may refer to the foregoing method embodiments, which are not described herein again.

An embodiment of the present invention further provides a live broadcasting device, as shown in fig. 7, the live broadcasting device of this embodiment includes: the second starting unit 601, the second establishing unit 602, the second obtaining unit 603, and the sending unit 604 are as follows:

a second starting unit 601, configured to start a screen recording live broadcast function, and live broadcast a game picture of a game application;

a second establishing unit 602, configured to establish a dedicated communication channel with the game application;

a second obtaining unit 603, configured to obtain barrage data, where the barrage data is comment data issued by a user corresponding to each terminal for a live game picture;

a sending unit 604, configured to send the bullet screen data to the game application through the dedicated communication channel, so that the game application renders the bullet screen data and then superimposes the bullet screen data on a current game screen for display.

In some embodiments, as shown in fig. 8, the live device further includes:

a broadcasting unit 605, configured to start a zero configuration networking Bonjour service, and broadcast service information of the live broadcast application, where the service information includes an internet protocol address IP address and identification information of a host running the live broadcast application.

In some embodiments, the second establishing unit 602 is specifically configured to:

and establishing a socket link channel between the IP address and the identification information of the Internet protocol address and the game application.

In some embodiments, as shown in fig. 8, the second obtaining unit 603 includes:

a sending subunit 6031, configured to send a bullet screen obtaining request to the backend server;

a receiving subunit 6032, configured to receive the bullet screen data fed back by the backend server according to the bullet screen obtaining request.

In some embodiments, as shown in fig. 8, the live device further includes:

and the background resident unit 606 is configured to perform process resident in the background in a preset manner until the game is ended.

Therefore, in the embodiment, in the live broadcasting device, the second starting unit 601 is used for starting the screen recording live broadcasting function, the game picture of the game application is live broadcasted, then the second establishing unit 602 establishes a special communication channel with the game application, the second obtaining unit 603 obtains the bullet screen data, and finally the sending unit 604 sends the bullet screen data to the game application through the special communication channel, so that the game application renders the bullet screen data and then displays the bullet screen data in the current game picture in a superposed manner.

Correspondingly, an embodiment of the present invention further provides a live game barrage display apparatus, as shown in fig. 9, the live game barrage display apparatus 700 includes a barrage display apparatus 701 and a live broadcast apparatus 702. The live-game barrage display device 700 may be an electronic device at a main player, the barrage display device 701 may be a game application client, the live-broadcast device 702 may be a live-broadcast application client, and both the game application client and the live-broadcast application client are installed in the electronic device.

The bullet screen display device 701 and the live broadcast device 702 can refer to the description of the previous embodiments, and are not described herein again.

An embodiment of the present invention further provides an electronic device, as shown in fig. 10, which shows a schematic structural diagram of the electronic device according to the embodiment of the present invention, specifically:

the electronic device may include Radio Frequency (RF) circuitry 801, memory 802 including one or more computer-readable storage media, input unit 803, display unit 804, sensor 805, audio circuitry 806, Wireless Fidelity (WiFi) module 807, processor 808 including one or more processing cores, and power supply 809. Those skilled in the art will appreciate that the configuration of the electronic device shown in fig. 10 does not constitute a limitation of the electronic device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:

the RF circuit 801 may be used for receiving and transmitting signals during information transmission and reception or during a call, and in particular, receive downlink information from a base station and then send the received downlink information to one or more processors 808 for processing; in addition, data relating to uplink is transmitted to the base station. In general, the RF circuitry 801 includes, but is not limited to, an antenna, at least one Amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuitry 801 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to Global System for Mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), and the like.

The memory 802 may be used to store software programs and modules, and the processor 808 may execute various functional applications and data processing by operating the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the electronic device, and the like. Further, the memory 802 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 808 and the input unit 803 access to the memory 802.

The input unit 803 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. In particular, in a particular embodiment, the input unit 803 may include a touch-sensitive surface as well as other input devices. The touch-sensitive surface, also referred to as a touch display screen or a touch pad, may collect touch operations by a user (e.g., operations by a user on or near the touch-sensitive surface using a finger, a stylus, or any other suitable object or attachment) thereon or nearby, and drive the corresponding connection device according to a predetermined program. Alternatively, the touch sensitive surface may comprise two parts, a touch detection means and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 808, and can receive and execute commands sent by the processor 808. In addition, touch sensitive surfaces may be implemented using various types of resistive, capacitive, infrared, and surface acoustic waves. The input unit 803 may include other input devices in addition to the touch-sensitive surface. In particular, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 804 may be used to display information input by or provided to a user and various graphical user interfaces of the electronic device, which may be made up of graphics, text, icons, video, and any combination thereof. The Display unit 804 may include a Display panel, and optionally, the Display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch-sensitive surface may overlay the display panel, and when a touch operation is detected on or near the touch-sensitive surface, the touch operation is transmitted to the processor 808 to determine the type of touch event, and the processor 808 then provides a corresponding visual output on the display panel according to the type of touch event. Although in FIG. 10 the touch sensitive surface and the display panel are two separate components to implement input and output functions, in some embodiments the touch sensitive surface may be integrated with the display panel to implement input and output functions.

The electronic device may also include at least one sensor 805, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel according to the brightness of ambient light, and a proximity sensor that may turn off the display panel and/or the backlight when the terminal is moved to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), detect the magnitude and direction of gravity when the motion sensor is stationary, and can be used for applications (such as horizontal and vertical screen switching, related games, magnetometer attitude calibration) for recognizing the attitude of an electronic device, vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which may be further configured to the electronic device, detailed descriptions thereof are omitted.

Audio circuitry 806, a speaker, and a microphone may provide an audio interface between the user and the terminal. The audio circuit 806 may transmit the electrical signal converted from the received audio data to a speaker, and convert the electrical signal into an audio signal for output; on the other hand, the microphone converts the collected sound signal into an electric signal, which is received by the audio circuit 806 and converted into audio data, which is then processed by the audio data output processor 808, and then sent to, for example, another electronic device via the RF circuit 801, or the audio data is output to the memory 802 for further processing. The audio circuit 806 may also include an earbud jack to provide communication of a peripheral headset with the electronic device.

WiFi belongs to short-distance wireless transmission technology, and the electronic device can help the user send and receive e-mail, browse web pages, access streaming media, etc. through the WiFi module 807, which provides wireless broadband internet access for the user. Although fig. 10 shows the WiFi module 807, it is understood that it does not belong to the essential constitution of the electronic device, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The processor 808 is a control center of the electronic device, connects various parts of the entire apparatus using various interfaces and lines, and performs various functions of the electronic device and processes data by operating or executing software programs and/or modules stored in the memory 802 and calling data stored in the memory 802. Optionally, processor 808 may include one or more processing cores; preferably, the processor 808 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 808.

The electronic device also includes a power supply 809 (e.g., a battery) for powering the various components, which may be logically coupled to the processor 808 via a power management system to manage charging, discharging, and power consumption via the power management system. The power supply 809 may also include one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, or any other component.

Although not shown, the electronic device may further include a camera, a bluetooth module, and the like, which are not described in detail herein.

Specifically, in this embodiment, the processor 808 in the electronic device loads an executable file corresponding to a process of one or more application programs into the memory 802 according to the following instructions, and the processor 808 runs the application programs stored in the memory 802, so as to implement the following functions:

In some embodiments, before establishing the dedicated communication channel with the live application, the processor 808 is further configured to:

receiving service information broadcasted by the live broadcast application, wherein the service information comprises an Internet Protocol (IP) address and identification information of a host running the live broadcast application;

judging whether the internet protocol address and the identification information contained in the service information correspond to the internet protocol address and the identification information of the local machine or not;

and if the correspondence is consistent, executing the step of establishing a special communication channel with the live broadcast application.

In some embodiments, when establishing the dedicated communication channel with the live application, the processor 808 is specifically configured to perform the following steps:

In some embodiments, when rendering the barrage data in a graphics processor, processor 808 is specifically configured to perform the following steps:

Specifically, in this embodiment, the processor 808 in the electronic device further loads an executable file corresponding to a process of one or more application programs into the memory 802 according to the following instructions, and the processor 808 runs the application programs stored in the memory 802, so as to implement the following functions:

establishing a dedicated communication channel with the game application;

In some embodiments, prior to establishing the dedicated communication channel with the gaming application, processor 808 is further configured to:

and starting a zero configuration networking Bonjour service, broadcasting service information of the live broadcast application, wherein the service information comprises an Internet protocol address (IP) address and identification information of a host running the live broadcast application.

In some embodiments, upon establishing the dedicated communication channel with the gaming application, processor 808 is specifically configured to perform the following steps:

In some embodiments, in acquiring the bullet screen data, the processor 808 is specifically configured to perform the following steps:

sending a bullet screen acquisition request to a background server;

and receiving the barrage data fed back by the background server according to the barrage acquisition request.

In some embodiments, after the video recording live broadcasting function is started and the game screen of the game application is live broadcasted, the processor 808 is further configured to perform the following steps:

and carrying out process resident on the background in a preset mode until the game is finished.

In the electronic equipment, the barrage is displayed during live game play by adopting multi-process (multi-application) communication, so that brand-new experience is brought to users, and the interaction among the users is enhanced; the bullet screen data are transmitted by establishing a special communication channel, so that the bullet screen data can be prevented from being intercepted, and the data safety is ensured; furthermore, because the increment of the installation package of the game application is limited, part of implementation logic displayed by the bullet screen is placed at the live broadcast application end, so that the problem that the size of the installation package of the game application is overlarge due to the fact that all implementation logic displayed by the bullet screen is placed at the game application end can be solved.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form. The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer (which may be a personal computer, an apparatus, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A bullet screen display method is applied to game application in a bullet screen display device for live game, and is characterized by comprising the following steps:

rendering the bullet screen data in a GPU (graphics processing Unit), and superposing the rendered bullet screen data in a current game picture for display;

before establishing a dedicated communication channel with the live application, the method further comprises the following steps:

if the correspondence is consistent, executing a step of establishing a special communication channel with the live broadcast application;

the establishing of the special communication channel with the live broadcast application comprises the following steps:

2. The bullet screen display method of claim 1 wherein said rendering said bullet screen data within a graphics processor comprises:

and rendering the bullet screen data in the graphics processor according to a bullet screen display mode set by a main broadcasting user.

3. The bullet screen display method according to claim 2, wherein the bullet screen display mode comprises: font style, font size, font color and display effect when bullet screen data is displayed.

4. A live broadcast method is applied to live broadcast application in a live-play screen display device of game live broadcast, and is characterized by comprising the following steps:

establishing a dedicated communication channel with the game application;

sending the bullet screen data to the game application through the special communication channel, so that the game application renders the bullet screen data and then displays the bullet screen data in a superposition mode in a current game picture;

before establishing a dedicated communication channel with the game application, further comprising:

starting a zero configuration networking Bonjour service, broadcasting service information of the live broadcast application, wherein the service information comprises an Internet Protocol (IP) address and identification information of a host running the live broadcast application;

the establishing of the special communication channel with the game application comprises the following steps:

5. The live broadcasting method of claim 4, wherein the obtaining of the barrage data comprises:

sending a bullet screen acquisition request to a background server;

6. The live broadcasting method according to any one of claims 4 or 5, wherein after the live broadcasting function is started and the game screen of the game application is live broadcasted, the live broadcasting method further comprises:

7. A bullet screen display device, comprising:

the rendering processing unit is used for rendering the barrage data in the GPU and superposing the rendered barrage data in the current game picture for display;

the receiving unit is used for receiving service information broadcasted by the live broadcast application, wherein the service information comprises an internet protocol address (IP) address and identification information of a host running the live broadcast application;

the judging unit is used for judging whether the Internet protocol address and the identification information contained in the service information correspond to the Internet protocol address and the identification information of the local machine or not;

the first establishing unit is used for executing the step of establishing a special communication channel with the live broadcast application when the internet protocol address and the identification information contained in the service information correspond and accord with the internet protocol address and the identification information of the local device.

8. A live broadcast apparatus, comprising:

a second establishing unit, configured to establish a dedicated communication channel with the game application, and before establishing the dedicated communication channel with the game application, further include: starting a zero configuration networking Bonjour service, broadcasting service information of the live broadcast application, wherein the service information comprises an Internet Protocol (IP) address and identification information of a host running the live broadcast application; the establishing of the special communication channel with the game application comprises the following steps: establishing a socket link channel between the game application and the Internet protocol address IP address and the identification information;

9. The live device according to claim 8, wherein the second acquiring unit comprises:

the sending subunit is used for sending a barrage obtaining request to the background server;

and the receiving subunit is used for receiving the barrage data fed back by the background server according to the barrage acquisition request.

10. A live game barrage display device, comprising the barrage display device according to claim 7 and a live game device according to claim 8 or 9.