CN110784757A - Game live broadcast method and device in live broadcast client, medium and electronic equipment - Google Patents

Game live broadcast method and device in live broadcast client, medium and electronic equipment Download PDF

Info

Publication number
CN110784757A
CN110784757A CN201911071870.7A CN201911071870A CN110784757A CN 110784757 A CN110784757 A CN 110784757A CN 201911071870 A CN201911071870 A CN 201911071870A CN 110784757 A CN110784757 A CN 110784757A
Authority
CN
China
Prior art keywords
live
game
live broadcast
client
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911071870.7A
Other languages
Chinese (zh)
Inventor
赵崇博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Netease Hangzhou Network Co Ltd
Original Assignee
Netease Hangzhou Network Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Netease Hangzhou Network Co Ltd filed Critical Netease Hangzhou Network Co Ltd
Priority to CN201911071870.7A priority Critical patent/CN110784757A/en
Publication of CN110784757A publication Critical patent/CN110784757A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/55—Push-based network services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4335—Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4781—Games
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8166—Monomedia components thereof involving executable data, e.g. software
    • H04N21/8173—End-user applications, e.g. Web browser, game

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Information Transfer Between Computers (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention discloses a game live broadcast method in a live broadcast client, a game live broadcast device in the live broadcast client, a computer readable storage medium and electronic equipment, and relates to the technical field of video live broadcast. In the game live broadcasting method in the live broadcasting client, the live broadcasting client runs on an electronic terminal, and the electronic terminal comprises a game client, wherein the game live broadcasting method in the live broadcasting client comprises the following steps: the method comprises the steps of obtaining live broadcast images of a game through an image rendering thread of a game client, obtaining live broadcast audio of the game, coding the live broadcast images and the live broadcast audio, and pushing the coded live broadcast images and the live broadcast audio to a live broadcast server. The method and the device can reduce the jamming problem of the game and improve the live broadcast processing efficiency in the live broadcast process.

Description

Game live broadcast method and device in live broadcast client, medium and electronic equipment
Technical Field
The present disclosure relates to the field of video live broadcast technologies, and in particular, to a game live broadcast method in a live broadcast client, a game live broadcast device in the live broadcast client, a computer-readable storage medium, and an electronic device.
Background
With the progress of internet technology, live webcasts are also rapidly developed and widely applied. For example, large game manufacturers combine gaming with live broadcasting to enhance the promotion and promotion of game products.
At present, when a game is played directly, two technical schemes are adopted, wherein the first scheme comprises the following steps: and recording game pictures of the game client by using third-party screen recording software, and pushing the recorded pictures to a server for image processing to obtain live video. The second scheme is as follows: the live broadcast device is embedded into the game processor, and the game processor processes the pictures of the game pictures and then pushes the processed game pictures to the live broadcast device. However, the two existing technical solutions can cause the problems of complex live broadcast operation process, game pause and the like.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present disclosure is directed to a game live broadcasting method in a live broadcasting client, a game live broadcasting device in a live broadcasting client, a computer-readable storage medium, and an electronic device, thereby overcoming, at least to some extent, a problem of game stutter due to limitations and disadvantages of related art.
According to a first aspect of the present disclosure, a game live broadcast method in a live broadcast client is provided, where the live broadcast client operates in an electronic terminal, the electronic terminal includes a game client, and the game live broadcast method in the live broadcast client includes: acquiring a live broadcast image of a game through an image rendering thread of a game client; acquiring live broadcast audio of a game; and coding and pushing the live images and the live audio to a live server.
According to a second aspect of the present disclosure, there is provided a game live broadcasting device in a live broadcasting client, including: the image acquisition module is used for acquiring a live image of the game through an image rendering thread of the game client; the audio acquisition module is used for acquiring live audio of the game; and the data coding module is used for coding the live image and the live audio and pushing the coded live audio to a live server.
Optionally, the data encoding module comprises: the image coding unit is used for coding the live broadcast image to generate a streaming media file; the streaming media file comprises identification information of the streaming media file and image data; the video data generating unit is used for extracting image data from the streaming media file and generating target video data corresponding to the image data if the identification information of the streaming media file is consistent with the identification information of the standard streaming media file; the audio data generating unit is used for coding the live broadcast audio to generate target audio data; and the data processing unit is used for processing the target video data and the target audio data according to the timestamp for generating the target video data and the timestamp for generating the target audio data, and pushing the target video data and the target audio data to the live broadcast server.
Alternatively, the image encoding unit may be configured to perform: and coding the live broadcast image based on the preset video resolution to generate a streaming media file.
Optionally, the image acquisition module may be configured to perform: and acquiring a live image of the game from an image rendering thread of the game client based on a preset video frame rate.
Alternatively, the video data generation unit may be configured to perform: and extracting image data from the streaming media file according to a preset video frame rate, and generating target video data corresponding to the image data.
Alternatively, the image encoding unit may be configured to perform: and zooming the live broadcast image, and encoding the zoomed live broadcast image.
Optionally, the game live broadcasting device in the live client further includes: and the information adding module is used for adding protection information to the coded live image.
Alternatively, the image encoding unit may be configured to perform: and carrying out zooming processing on the live image, and carrying out coding and protection information adding processing on the zoomed live image.
Optionally, the game live broadcasting device in the live client further includes: the storage space determining module is used for determining the available storage space of a file library for storing the streaming media file; and the file library emptying module is used for emptying the file library and recoding the direct-broadcast image to generate the streaming media file to be stored in the file library if the streaming media file is larger than the available storage space.
Optionally, the game live broadcasting device in the live client further includes: the encoding duration determining module is used for determining the encoding duration for encoding the live image; and the coding duration judging module is used for recoding the direct broadcast image if the coding duration is greater than a preset threshold.
According to a third aspect of the present disclosure, there is provided a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements a game live method in a live client as described above.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising: one or more processors; a storage device for storing one or more programs which, when executed by one or more processors, cause the one or more processors to implement a method of live gaming in a live client as described above.
Exemplary embodiments of the present disclosure have the following advantageous effects:
in some embodiments of the present disclosure, a live client operates in an electronic terminal, and the electronic terminal includes a game client. Specifically, a live broadcast client acquires a live broadcast image of a game through an image rendering thread of a game client; acquiring live broadcast audio of a game; and coding and pushing the live images and the live audio to a live server. On the one hand, the live broadcast client can acquire live broadcast images of games from the game client, so that the processing steps of pushing the live broadcast images for the live broadcast client by the game client are reduced, the live broadcast images are not acquired by using third-party recording software, the operation in the live broadcast process is simplified, and the live broadcast processing efficiency in the live broadcast process is improved. On the other hand, the live broadcast client encodes the live broadcast images and the live broadcast audio, so that the process of encoding and streaming the images through the game client in the condition that the live broadcast client is embedded into the game client is avoided, the consumption of game client resources is reduced, the pause of a game is reduced, and the smoothness of the game is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:
fig. 1 schematically illustrates a flow chart of a game live method in a live client according to an exemplary embodiment of the present disclosure;
FIG. 2 schematically illustrates an interaction flow diagram of a game live method in a live client according to an exemplary embodiment of the present disclosure;
FIG. 3 schematically illustrates a block diagram of a gaming live device in a live client according to an exemplary embodiment of the present disclosure;
FIG. 4 schematically illustrates a block diagram of a data encoding module according to an exemplary embodiment of the present disclosure;
FIG. 5 schematically illustrates a block diagram of a gaming live device in a live client according to another exemplary embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a gaming live device in a live client according to another exemplary embodiment of the present disclosure;
FIG. 7 schematically illustrates a block diagram of a gaming live device in a live client according to another exemplary embodiment of the present disclosure;
fig. 8 schematically shows a block diagram of an electronic device in an exemplary embodiment according to the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
In the present disclosure, the terms "comprises" and "comprising" are used in an open-ended fashion, and mean that there may be additional elements/components/etc. in addition to the listed elements/components/etc.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the steps. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
For ease of understanding, the scenario to which the present solution relates will be explained first.
With the progress of internet technology, live webcasts are rapidly developed and applied, for example, live games.
In the live game, the anchor firstly opens third-party screen recording software to collect screen pictures containing game content pictures, then preprocesses the recorded screen pictures to extract the game content pictures, then pushes the processed game content pictures to a live broadcast server, and then sends the processed game content pictures to a user for watching by the live broadcast server. However, in the above process, the game software and the third-party screen recording software are incompatible, complicated to operate and the like, and the game jamming problem is also caused. In response to these problems, the present disclosure provides a method for live game play in a live client.
It should be noted that, in the exemplary embodiment of the present disclosure, the game live method in the live client described below may be generally implemented by the live client, that is, each step of the game live method in the live client may be executed by the live client, in which case, the game live device in the live client may be configured in the live client.
In addition, the game live broadcasting method in the live client described below may be generally implemented by a terminal device (e.g., a mobile phone, a tablet, a personal computer, etc.), that is, the steps of the game live broadcasting method in the live client may be performed by the terminal device, in which case, the game live broadcasting device in the live client may be configured in the terminal device.
Hereinafter, the steps of the game live broadcast method in the live broadcast client in the present exemplary embodiment will be described in more detail with reference to the drawings and the examples.
Fig. 1 schematically shows a flowchart of a game live method in a live client according to an exemplary embodiment of the present disclosure. In the following description, a live client is taken as an execution subject for illustration, the live client runs on an electronic terminal, and the electronic terminal includes a game client. Referring to fig. 1, a game live broadcasting method in a live client may include the steps of:
and S102, acquiring a live broadcast image of the game through an image rendering thread of the game client.
In an exemplary embodiment of the present disclosure, a game image is acquired from a game in which a game client is running to be live-played, wherein the game image is a live-played image of the game in the present exemplary embodiment. The image rendering thread of the game client can render each frame of image in the game. The live image of the game may contain the picture content of the game. It should be noted that, in the exemplary embodiment of the present disclosure, the type of game executed in the game client is not limited, for example, a shooting game, a fighting game, and the like.
According to one embodiment of the present disclosure, the live client may set a preset video frame rate to acquire live images, where the preset video frame rate may be set to 20fps, may be set to 30fps, and so on. In addition, the live broadcast client can also set a plurality of preset video frame rates, and display the plurality of preset video frame rates to the main broadcast in a mode of acquiring the speed gear of the live broadcast image.
That is, in an exemplary embodiment of the present disclosure, the live client may acquire a live image of the game from an image rendering thread of the game client based on a preset video frame rate. For example, the anchor may obtain a game image from an image rendering thread of the game client based on a preset video frame rate of 20 fps.
After the live image is acquired, the live client may also perform scaling processing on the acquired live image. In addition, in order to protect the game from being copied, the live client may add protection information to the game image after acquiring the game image, where the protection information may show an identifier of the anchor in the form of a watermark, and the identifier of the anchor may contain a nickname of the anchor, an anchor number of the anchor, and so on. The live broadcast client can also perform processing of adding protection information after performing scaling processing on live broadcast images.
The live broadcast client side of the embodiment can acquire live broadcast images of games, processing of the game client side on the live broadcast images is reduced, the fact that third-party recording software is used for acquiring the live broadcast images is avoided, operation in a live broadcast process is simplified, and live broadcast cost is reduced. Meanwhile, the live client adds the protection information to visually display the ownership of the live video of the anchor broadcast, so that the user can conveniently inquire and understand the ownership.
And S104, acquiring live audio of the game.
The live audio of the game may comprise audio in the game and may also comprise audio for the main lecture game. It should be noted that the audio of the main broadcasting explanation game can be collected by a third-party recording device and transmitted to the live broadcasting client, or collected by a recording device in the electronic terminal. Live audio of the game may also be captured using microphones in exemplary embodiments of the present disclosure. The microphone may be a microphone device or a set of microphone arrays.
In addition, this example embodiment may also perform preprocessing on the captured live audio of the game, where the preprocessing may include: the sound of the audio of the main broadcasting explaining game is filtered, for example, the sound of the road, the noise contained in the audio of the game in the case of unstable current, and the like.
And S106, coding the live image and the live audio and pushing the coded live image and the live audio to a live server.
In an exemplary embodiment of the present disclosure, a live client may encode live images, encode live audio, and then synchronize the live images and the live audio.
Specifically, according to an embodiment of the present disclosure, a live client may encode a live image to generate a streaming media file; the streaming media file may include identification information of the streaming media file and image data, and the identification information of the streaming media file may include coding information, index information, and the like of the streaming media file; if the identification information of the streaming media file is consistent with the identification information of the standard streaming media file, extracting image data from the streaming media file and generating target video data corresponding to the image data; the identification information of the standard streaming media file may include corresponding standard encoding information and a corresponding data format standard; encoding live audio to generate target audio data; and processing the target video data and the target audio data according to the timestamp for generating the target video data and the timestamp for generating the target audio data, and pushing the target video data and the target audio data to a live broadcast server.
In an exemplary embodiment of the present disclosure, the streaming media file may be a file in MP4 format, a file in MOV format, a file in M4V format, and the like, and the live client may extract image data from the streaming media file according to a structural composition of the streaming media file. The time stamp may be a current system time stamp from which the image data is extracted as a standard.
Before generating the target video data, the live broadcast client may compare the identification information of the streaming media file with the identification information of the standard streaming media file. If the identification information of the streaming media file is consistent with the identification information of the standard streaming media file, image data can be extracted from the coded streaming media file and packaged into target image data. In addition, the encoding of the live video can be performed according to the existing video encoding technology, or the live video of the game can be written and encoded by the system interface to obtain a streaming media file.
Taking the MP4 streaming media format as an example, the data format of the MP4 file includes moov element and mdat element, where the moov element includes file encoding information, and the mdat element includes image data.
The live broadcast client acquires live broadcast images of the game and writes the live broadcast images into an MP4 streaming media format. Before extracting image data from the written MP4 streaming media format, the live broadcast client firstly extracts moov elements of MP4 streaming media written with live broadcast images, and then judges whether file encoding information contained in the moov elements meets the encoding information of the standard MP4 and whether the data structure meets the standard MP4 streaming media format. If the format of the stream media is consistent with the standard MP4 stream media format, the MP4 stream media written with live images is correct, and the image data can be extracted from the mdat element. It should be noted that, in the case where the streaming media of the MP4 that writes live images has a plurality of mdat elements, the extraction may be performed in mdat units and packaged into VIDEO FLV data (FLASH VIDEO, FLV), that is, target VIDEO data.
In an exemplary embodiment of the present disclosure, a live client encodes live Audio, may encode the live Audio according to an existing Advanced Audio Coding (AAC), and encapsulates the encoded live Audio into Audio FLV data, that is, target Audio data. It should be noted that the method adopted by the encapsulation process of the video FLV data and the audio FLV data may be an encapsulation method in the prior art.
In addition, the target video data and the target audio data are the time stamps and the targets of the target video data analyzed each time according to the system time stamps, and the live broadcast client compares the time stamps of the target video data with the time stamps of the target audio data to perform synchronous processing. That is to say, the live broadcast client may also temporarily store the target audio data and the target video data, and then, based on the timestamp of the target video data and the timestamp of the target audio data, push the target audio data and the target video data to the live broadcast server according to the sequence of the timestamps. The live broadcast client may push target video data and target audio data to a live broadcast server according to a Real Time Message Protocol (RTMP).
According to one embodiment of the present disclosure, a live client may encode a live image based on a preset video resolution to generate a streaming media file. Wherein the preset video resolution may be set to 640 × 320, 720 × 480, etc., and presented to the main broadcast in the form of buttons. In case a live image needs to be encoded, the selection can be made by the anchor.
In addition, the live broadcast client can also extract image data from the streaming media file according to a preset video frame rate and generate target video data corresponding to the image data. For example, in the case where the anchor acquires a live image of a game at a preset video frame rate of 20fps, image data may be extracted according to the same preset video frame rate.
According to the exemplary embodiment of the disclosure, after the live broadcast client performs scaling processing on the live broadcast image, the scaled live broadcast image may be encoded to generate a streaming media file. In addition, the live broadcast client can also add protection information to the live broadcast image, and then encode the live broadcast image added with the protection information to generate a streaming media file. That is, after the live broadcast client performs scaling processing and protection information adding processing on the live broadcast image, the live broadcast image after processing may also be encoded to generate a streaming media file. The live broadcast client carries out zooming processing and protection information adding processing on the live broadcast image, avoids the processes of image zooming and protection information adding executed by the game client in the prior art, reduces the resource waste of the game client, provides more resources for the game and improves the smoothness of the game.
It is noted that in an exemplary embodiment of the present disclosure, the available storage space of a file library storing streaming media files is determined; and if the streaming media file is larger than the available storage space, clearing the file library, and recoding the live image to generate the streaming media file to be stored in the file library. The file library can be arranged in a live client and also can be arranged in an electronic terminal for operating the live client. Due to the limited disk space of the electronic terminal, the live client of the terminal can set the available storage space to 50 MB.
That is, in the case of storing the generated streaming media file, if the size of the streaming media file exceeds the available storage space after the live broadcast client determines that the streaming media file is stored, the current storage step needs to be terminated, the file library needs to be emptied, the live broadcast image needs to be encoded again, and the live broadcast image needs to be stored again.
According to another embodiment of the present disclosure, a live client may determine an encoding duration for encoding a live image; and if the encoding duration is greater than a preset threshold value, re-encoding the live broadcast image. Wherein the preset threshold is calculated according to the existing coding technology.
As shown in fig. 2, an interactive flowchart of a game live method in a live client according to an exemplary embodiment of the present disclosure may be executed in an electronic terminal, where the electronic terminal includes a game client. Specifically, the interaction flow of the game live broadcasting method in the live broadcasting client may include: in step S201, the live broadcast client may obtain a live broadcast image of the game through an image rendering thread of the game client; in step S203, the live client may obtain the audio of the game, and may also obtain the audio of the main broadcasting explaining the game; in step S205, the live client may encode the live image and generate target video data; in step S207, the live client may encode live audio and generate target audio data; in step S209, the live client may push the processed target video data and target audio data to the server.
In an exemplary embodiment of the present disclosure, the live image obtained from the image rendering thread of the game may be in the form of a graphics library Texture unit, e.g., OpenGL Texture.
For example, in the case where the anchor needs to play a live game, the streaming media format of MP4 is adopted.
Firstly, a live broadcast client acquires a live broadcast image of a game at a preset video frame rate of 20fps, and writes the live broadcast image into an MP4 streaming media format; then, storing the streaming media format into a file library with available storage space of 50MB, and if the MP4 streaming media file exceeds 50MB, re-encoding the live image; secondly, before extracting image data from the written MP4 streaming media format, the live client extracts moov elements of MP4 streaming media written with live images; thirdly, whether the file encoding information contained in the moov element conforms to the encoding information of the standard MP4 or not and whether the data structure conforms to the standard MP4 streaming media format or not are judged. If the video data is consistent with the standard MP4 streaming media format, then, the image data is extracted in units of mdat elements, the system time stamp of the extracted image data is recorded, and the image data is packaged into video FLV data.
The live broadcast client collects the audio frequency of the game and the audio frequency of the main broadcast explanation game through a microphone, outputs audio data according to an advanced audio coding algorithm, records a system time stamp corresponding to the output audio data, and encapsulates the audio data into audio FLV data.
And then, the live broadcast client pushes the video FLV data and the audio FLV data to a live broadcast server based on the time stamp of the video FLV data and the time stamp of the audio FLV data so as to send the video FLV data and the audio FLV data to the watching client.
According to the game live broadcasting method in the live broadcasting client, the live broadcasting client is used for acquiring the game live broadcasting image and the live broadcasting audio and encoding the live broadcasting image and the live broadcasting audio, so that encoding processing and packaging processing of the game on the live broadcasting image are avoided, operation steps of the game are reduced, and the game pause is further reduced.
It should be noted that although the various steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.
Further, in an exemplary embodiment of the present disclosure, a game live device in a live client is also provided.
Fig. 3 schematically illustrates a block diagram of a game live device in a live client according to an exemplary embodiment of the present disclosure. Referring to fig. 3, a game live device 300 in a live client according to an exemplary embodiment of the present disclosure may include: an image acquisition module 302, an audio acquisition module 304, and a data encoding module 306.
The image obtaining module 302 is configured to obtain a live image of a game through an image rendering thread of a game client; an audio acquisition module 304, configured to acquire live audio of a game; and the data encoding module 306 is configured to encode the live image and the live audio and push the encoded live image and the live audio to a live server.
According to an exemplary embodiment of the present disclosure, referring to fig. 4, the data encoding module 306 may include: an image encoding unit 401, a video data generation unit 403, an audio data generation unit 405, and a data processing unit 407.
The image encoding unit 401 is configured to encode a live image and generate a streaming media file; the streaming media file comprises identification information of the streaming media file and image data; a video data generating unit 403, configured to extract image data from the streaming media file and generate target video data corresponding to the image data if the identification information of the streaming media file is consistent with the identification information of the standard streaming media file; an audio data generating unit 405, configured to encode live audio to generate target audio data; and the data processing unit 407 is configured to process the target video data and the target audio data according to the timestamp for generating the target video data and the timestamp for generating the target audio data, and push the target video data and the target audio data to the live broadcast server.
According to an exemplary embodiment of the present disclosure, the image encoding unit 401 may be configured to perform: and coding the live broadcast image based on the preset video resolution to generate a streaming media file.
According to an example embodiment of the present disclosure, the image acquisition module 302 may be configured to perform: and acquiring a live image of the game from an image rendering thread of the game client based on a preset video frame rate.
According to an exemplary embodiment of the present disclosure, the video data generation unit 403 may be configured to perform: and extracting image data from the streaming media file according to a preset video frame rate, and generating target video data corresponding to the image data.
According to another exemplary embodiment of the present disclosure, the image encoding unit 401 may be configured to perform: and zooming the live broadcast image, and encoding the zoomed live broadcast image.
According to another exemplary embodiment of the present disclosure, referring to fig. 5, compared to the game live device 300 in the live client, the game live device 500 in the live client may further include: an information adding module 502, configured to perform information adding protection processing on the encoded live video.
According to another exemplary embodiment of the present disclosure, the image encoding unit 401 may be configured to perform: and carrying out zooming processing on the live image, and carrying out coding and protection information adding processing on the zoomed live image.
According to another exemplary embodiment of the present disclosure, referring to fig. 6, compared to the game live device 300 in the live client, the game live device 600 in the live client may further include: a storage space determination module 601 and a file library clearing module 603.
The storage space determining module 601 is configured to determine an available storage space of a file library for storing the streaming media file; the file library emptying module 603 is configured to empty the file library if the streaming media file is larger than the available storage space, and re-encode the live image to generate the streaming media file for storage in the file library.
According to another exemplary embodiment of the present disclosure, referring to fig. 7, compared to the game live device 300 in the live client, the game live device 700 in the live client may further include: a code duration determination module 702 and a code duration determination module 704.
The encoding duration determining module 702 is configured to determine an encoding duration for encoding a live image; the encoding duration determining module 704 is configured to re-encode the live image if the encoding duration is greater than a preset threshold.
The details of each module/unit in the above-mentioned apparatus have been described in detail in the embodiments of the method section, and thus are not described again.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above-mentioned "exemplary methods" section of the present description, when the program product is run on the terminal device.
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 800 according to this embodiment of the invention is described below with reference to fig. 8. The electronic device 800 shown in fig. 8 is only an example and should not bring any limitations to the function and scope of use of the embodiments of the present invention.
As shown in fig. 8, electronic device 800 is in the form of a general purpose computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one memory unit 820, a bus 830 connecting different system components (including the memory unit 820 and the processing unit 810), and a display unit 840.
Wherein the storage unit stores program code that is executable by the processing unit 810 to cause the processing unit 810 to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary methods" of the present specification. For example, the processing unit 810 may perform steps S102 to S106 shown in fig. 1.
The storage unit 820 may include readable media in the form of volatile memory units such as a random access memory unit (RAM)8201 and/or a cache memory unit 8202, and may further include a read only memory unit (ROM) 8203.
The storage unit 820 may also include a program/utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
Bus 830 may be any of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to communicate with the electronic device 800, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 800 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 850. Also, the electronic device 800 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 860. As shown, the network adapter 860 communicates with the other modules of the electronic device 800 via the bus 830. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.

Claims (13)

1. A game live broadcast method in a live broadcast client is characterized in that the live broadcast client runs in an electronic terminal, the electronic terminal comprises a game client, and the game live broadcast method in the live broadcast client comprises the following steps:
acquiring a live broadcast image of the game through an image rendering thread of the game client;
acquiring live broadcast audio of the game;
and coding the live broadcast image and the live broadcast audio and pushing the coded live broadcast image and the live broadcast audio to a live broadcast server.
2. The live game playing method in the live client side according to claim 1, wherein the step of encoding and pushing the live image and the live audio to a live server comprises:
encoding the live broadcast image to generate a streaming media file; the streaming media file comprises identification information of the streaming media file and image data;
if the identification information of the streaming media file is consistent with the identification information of a standard streaming media file, extracting the image data from the streaming media file and generating target video data corresponding to the image data;
encoding the live broadcast audio to generate target audio data;
and processing the target video data and the target audio data according to the timestamp for generating the target video data and the timestamp for generating the target audio data, and pushing the target video data and the target audio data to a live broadcast server.
3. The live game playing method in the live client according to claim 2, wherein encoding the live image and generating a streaming media file comprises:
and coding the live broadcast image based on a preset video resolution ratio to generate a streaming media file.
4. The live game playing method in the live game client according to claim 2 or 3, wherein the step of acquiring the live game image through the image rendering thread of the game client comprises:
and acquiring the live broadcast image of the game from the image rendering thread of the game client based on a preset video frame rate.
5. The game live method in the live client according to claim 4, wherein extracting the image data from the streaming media file and generating target video data corresponding to the image data comprises:
and extracting image data from the streaming media file according to the preset video frame rate, and generating target video data corresponding to the image data.
6. A game live method in a live client according to any one of claims 1 to 3, wherein encoding the live image comprises:
and zooming the live broadcast image, and encoding the zoomed live broadcast image.
7. A game live method in a live client according to claim 2 or 3, wherein before generating the streaming media file, the game live method in the live client further comprises:
and adding protection information to the encoded live image.
8. The live game playing method in the live client according to claim 2, wherein encoding the live image comprises:
and zooming the live broadcast image, and encoding and adding protection information to the zoomed live broadcast image.
9. A game live method in a live client according to claim 2 or 3, wherein the game live method in the live client further comprises:
determining available storage space of a file library for storing the streaming media file;
and if the streaming media file is larger than the available storage space, clearing the file library, recoding the live broadcast image, and generating the streaming media file to be stored in the file library.
10. A game live method in a live client according to any one of claims 1 to 3, wherein the game live method in the live client further comprises:
determining the encoding time length for encoding the live broadcast image;
and if the encoding duration is greater than a preset threshold value, re-encoding the live broadcast image.
11. A game live broadcast device in a live broadcast client, wherein the live broadcast client runs on an electronic terminal, and the electronic terminal comprises a game client, and the game live broadcast device is characterized by comprising:
the image acquisition module is used for acquiring a live image of the game through an image rendering thread of the game client;
the audio acquisition module is used for acquiring the live audio of the game;
and the data coding module is used for coding the live broadcast image and the live broadcast audio and pushing the coded live broadcast image and the live broadcast audio to a live broadcast server.
12. A computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements a method of live gaming in a live client as claimed in any one of claims 1 to 10.
13. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement a method of live gaming in a live client as claimed in any one of claims 1 to 10.
CN201911071870.7A 2019-11-05 2019-11-05 Game live broadcast method and device in live broadcast client, medium and electronic equipment Pending CN110784757A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911071870.7A CN110784757A (en) 2019-11-05 2019-11-05 Game live broadcast method and device in live broadcast client, medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911071870.7A CN110784757A (en) 2019-11-05 2019-11-05 Game live broadcast method and device in live broadcast client, medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN110784757A true CN110784757A (en) 2020-02-11

Family

ID=69389136

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911071870.7A Pending CN110784757A (en) 2019-11-05 2019-11-05 Game live broadcast method and device in live broadcast client, medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN110784757A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111330281A (en) * 2020-03-05 2020-06-26 网易（杭州）网络有限公司 Game data processing method, game data processing device, medium, and electronic device
CN113056030A (en) * 2021-02-01 2021-06-29 广州朗国电子科技有限公司 Wireless screen projection module and screen projection method of all-in-one machine
CN113596502A (en) * 2021-08-03 2021-11-02 广州繁星互娱信息科技有限公司 Live broadcast room sound effect adjusting method and device, electronic equipment and medium
CN114390300A (en) * 2020-10-20 2022-04-22 腾讯科技（深圳）有限公司 Live broadcast control method and device, electronic equipment and computer storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8381259B1 (en) * 2012-01-05 2013-02-19 Vinod Khosla Authentication and synchronous interaction between a secondary device and a multi-perspective audiovisual data stream broadcast on a primary device
US20140259084A1 (en) * 2009-09-22 2014-09-11 Caption Colorado Llc Caption and/or Metadata Synchronization for Replay of Previously or Simultaneously Recorded Live Programs
CN105407361A (en) * 2015-11-09 2016-03-16 广州华多网络科技有限公司 Audio and video live broadcast data processing method and device
CN106210753A (en) * 2016-05-24 2016-12-07 北京畅游天下网络技术有限公司 A kind of net cast method and device
CN106303329A (en) * 2016-08-11 2017-01-04 广州爱九游信息技术有限公司 Record screen live broadcasting method and device, mobile device and live broadcast system
CN109168014A (en) * 2018-09-26 2019-01-08 广州虎牙信息科技有限公司 A kind of live broadcasting method, device, equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140259084A1 (en) * 2009-09-22 2014-09-11 Caption Colorado Llc Caption and/or Metadata Synchronization for Replay of Previously or Simultaneously Recorded Live Programs
US8381259B1 (en) * 2012-01-05 2013-02-19 Vinod Khosla Authentication and synchronous interaction between a secondary device and a multi-perspective audiovisual data stream broadcast on a primary device
CN105407361A (en) * 2015-11-09 2016-03-16 广州华多网络科技有限公司 Audio and video live broadcast data processing method and device
CN106210753A (en) * 2016-05-24 2016-12-07 北京畅游天下网络技术有限公司 A kind of net cast method and device
CN106303329A (en) * 2016-08-11 2017-01-04 广州爱九游信息技术有限公司 Record screen live broadcasting method and device, mobile device and live broadcast system
CN109168014A (en) * 2018-09-26 2019-01-08 广州虎牙信息科技有限公司 A kind of live broadcasting method, device, equipment and storage medium

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111330281A (en) * 2020-03-05 2020-06-26 网易（杭州）网络有限公司 Game data processing method, game data processing device, medium, and electronic device
CN111330281B (en) * 2020-03-05 2023-03-28 网易（杭州）网络有限公司 Game data processing method, game data processing device, medium, and electronic device
CN114390300A (en) * 2020-10-20 2022-04-22 腾讯科技（深圳）有限公司 Live broadcast control method and device, electronic equipment and computer storage medium
CN114390300B (en) * 2020-10-20 2024-04-26 腾讯科技（深圳）有限公司 Live broadcast control method and device, electronic equipment and computer storage medium
CN113056030A (en) * 2021-02-01 2021-06-29 广州朗国电子科技有限公司 Wireless screen projection module and screen projection method of all-in-one machine
CN113596502A (en) * 2021-08-03 2021-11-02 广州繁星互娱信息科技有限公司 Live broadcast room sound effect adjusting method and device, electronic equipment and medium

Similar Documents

Publication Publication Date Title
CN110784757A (en) Game live broadcast method and device in live broadcast client, medium and electronic equipment
CN112291627B (en) Video editing method and device, mobile terminal and storage medium
US10229651B2 (en) Variable refresh rate video capture and playback
WO2016150317A1 (en) Method, apparatus and system for synthesizing live video
US9363542B2 (en) Techniques to provide an enhanced video replay
CN109168014B (en) Live broadcast method, device, equipment and storage medium
KR101387212B1 (en) System and method for generating and playing three dimensional image files based on two dimensional image media standards
CN110324643B (en) Video recording method and system
JP6499324B2 (en) Method, client and computer storage medium for playing video
CN112272327B (en) Data processing method, device, storage medium and equipment
CN114697668B (en) Encoding and decoding method of point cloud media and related products
CN113535063A (en) Live broadcast page switching method, video page switching method, electronic device and storage medium
EP2651132A1 (en) Data management apparatus and data management method
CN112235600B (en) Method, device and system for processing video data and video service request
CN109862385B (en) Live broadcast method and device, computer readable storage medium and terminal equipment
CN111918074A (en) Live video fault early warning method and related equipment
CN111918082A (en) Cloud game live broadcasting method and device, storage medium and equipment
KR101085718B1 (en) System and method for offering augmented reality using server-side distributed image processing
CN111093091A (en) Video processing method, server and system
CN112954374B (en) Video data processing method and device, electronic equipment and storage medium
CN110798700B (en) Video processing method, video processing device, storage medium and electronic equipment
JP2024506623A (en) Point cloud encoding/decoding method, device, computer readable medium, and electronic device
CN114339284A (en) Method, device, storage medium and program product for monitoring live broadcast delay
JP7376235B2 (en) Video distribution system
CN113395531B (en) Play switching method and device, electronic equipment and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200211