CN111405312A - Live broadcast stream pushing method, device, terminal, server and storage medium - Google Patents

Live broadcast stream pushing method, device, terminal, server and storage medium Download PDF

Info

Publication number
CN111405312A
CN111405312A CN202010339447.7A CN202010339447A CN111405312A CN 111405312 A CN111405312 A CN 111405312A CN 202010339447 A CN202010339447 A CN 202010339447A CN 111405312 A CN111405312 A CN 111405312A
Authority
CN
China
Prior art keywords
stream
stream pushing
pushing
server
live
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010339447.7A
Other languages
Chinese (zh)
Inventor
何思远
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Kugou Computer Technology Co Ltd
Original Assignee
Guangzhou Kugou Computer Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Kugou Computer Technology Co Ltd filed Critical Guangzhou Kugou Computer Technology Co Ltd
Priority to CN202010339447.7A priority Critical patent/CN111405312A/en
Publication of CN111405312A publication Critical patent/CN111405312A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2408—Monitoring of the upstream path of the transmission network, e.g. client requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The application discloses a live broadcast stream pushing method, a live broadcast stream pushing device, a terminal, a server and a storage medium, and belongs to the technical field of networks. The method is used for a plug flow end, and comprises the following steps: when a live broadcast instruction is received, a first live broadcast video stream is pushed to a stream pushing server, and the first live broadcast video stream is obtained by coding in a first coding mode; sending plug flow information to a plug flow server, wherein the plug flow information is used for representing the coding capability and the plug flow capability of a plug flow end; receiving a stream pushing instruction sent by a stream pushing server, wherein the stream pushing instruction is sent by the stream pushing server when stream pushing information meets a preset stream pushing condition, the stream pushing instruction is used for indicating a stream pushing end to push a second live video stream, and the second live video stream is obtained by coding in a second coding mode; and pushing the second live video stream to a stream pushing server according to the stream pushing instruction.

Description

Live broadcast stream pushing method, device, terminal, server and storage medium
Technical Field
The embodiment of the application relates to the technical field of networks, in particular to a live streaming method, a live streaming device, a live streaming terminal, a live streaming server and a storage medium.
Background
With the development of network technology, the application of the live broadcast platform is more and more extensive. The live broadcast platform can comprise a main broadcast and a watching user, wherein the main broadcast corresponds to a stream pushing end in the live network broadcast process, and the watching user corresponds to a stream pulling end in the live network broadcast process. The plug flow end mainly comprises four links in the plug flow process: collecting an image; preprocessing the acquired information; coding and compressing the collected information to obtain a video stream; and pushing the video stream to the server so as to facilitate the video stream to be distributed by the pushing server.
For the coding compression link, the related art includes providing H264 video coding technology and H265 video coding technology. The H265 video coding technology has higher picture quality under the same uplink bandwidth condition by increasing the computational complexity, but requires the playback device to support decoding playback of the H265 video.
When the playing device does not have the hardware capability of decoding and playing the H265 video, the stream pushing server needs to perform transcoding operation, which results in significant increase of server cost.
Disclosure of Invention
The embodiment of the application provides a live streaming method, a live streaming device, a terminal, a server and a storage medium, and the technical scheme is as follows:
in one aspect, a live streaming method is provided, where the method is used at a streaming end, and the method includes:
when a live broadcast instruction is received, pushing a first live broadcast video stream to a stream pushing server, wherein the first live broadcast video stream is obtained by coding in a first coding mode;
sending plug flow information to the plug flow server, wherein the plug flow information is used for representing the coding capability and the plug flow capability of the plug flow end;
receiving a stream pushing instruction sent by the stream pushing server, wherein the stream pushing instruction is sent by the stream pushing server when the stream pushing information meets a preset stream pushing condition, the stream pushing instruction is used for indicating the stream pushing end to push a second live video stream, and the second live video stream is obtained by coding in a second coding mode;
and pushing the second live video stream to the stream pushing server according to the stream pushing instruction.
In another aspect, a live streaming method is provided, where the method is used for a streaming server, and the method includes:
receiving a first direct-playing video stream pushed by a stream pushing end, wherein the first direct-playing video stream is obtained by coding in a first coding mode;
acquiring the push flow information corresponding to the push flow end, wherein the push flow information is used for representing the coding capability and the push flow capability of the push flow end;
responding to the stream pushing information to indicate that the stream pushing end meets a preset stream pushing condition, and sending a stream pushing instruction to the stream pushing end, wherein the stream pushing instruction is used for indicating the stream pushing end to push a second live video stream, and the second live video stream is obtained by coding in a second coding mode;
and receiving the second live video stream pushed by the stream pushing end.
In another aspect, a live streaming apparatus is provided, where the apparatus is used for a streaming end, and the apparatus includes:
the first pushing module is used for pushing a first direct-broadcasting video stream to a stream pushing server when receiving a live-broadcasting instruction, wherein the first direct-broadcasting video stream is obtained by coding in a first coding mode;
the information sending module is used for sending the stream pushing information to the stream pushing server, wherein the stream pushing information is used for representing the coding capability and the stream pushing capability of the stream pushing end;
the instruction receiving module is used for receiving a stream pushing instruction sent by the stream pushing server, the stream pushing instruction is sent by the stream pushing server when the stream pushing information meets a preset stream pushing condition, the stream pushing instruction is used for indicating the stream pushing end to push a second live video stream, and the second live video stream is obtained by coding in a second coding mode;
and the second pushing module is used for pushing the second live video stream to the stream pushing server according to the stream pushing instruction.
In another aspect, a live streaming apparatus is provided, where the apparatus is used for a streaming server, and the apparatus includes:
the first receiving module is used for receiving a first direct-playing video stream pushed by a stream pushing end, and the first direct-playing video stream is obtained by coding in a first coding mode;
the information receiving module is used for acquiring the push flow information corresponding to the push flow end, and the push flow information is used for representing the coding capability and the push flow capability of the push flow end;
the instruction sending module is used for responding to the stream pushing information to indicate that the stream pushing end meets a preset stream pushing condition, and sending a stream pushing instruction to the stream pushing end, wherein the stream pushing instruction is used for indicating the stream pushing end to push a second live video stream, and the second live video stream is obtained by coding in a second coding mode;
and the second receiving module is used for receiving the second live video stream pushed by the stream pushing end.
In another aspect, a terminal is provided that includes a processor and a memory; the memory stores at least one instruction for execution by the processor to implement a live streaming method as described in the above aspect.
In another aspect, a server is provided, the server comprising a processor and a memory; the memory stores at least one instruction for execution by the processor to implement a live streaming method as described in the above aspect.
In another aspect, a computer-readable storage medium is provided that stores at least one instruction for execution by a processor to implement a live streaming method as described in the above aspect.
In another aspect, a computer program product is provided, where at least one instruction is stored, and the at least one instruction is loaded and executed by the processor to implement the live streaming method in the above aspect.
In the embodiment of the application, for a live broadcast system, after a live broadcast, a live broadcast video stream is compressed and encoded in a default first encoding mode and pushed to a live broadcast server by a live broadcast end; however, for the push of the live video stream, there is a live video stream in a more optimal coding mode, that is, a second live video stream, but the push of the second live video stream requires that the push end has certain coding capability and push capability, so that the push end sends push information to the push server, so that the push server judges whether the push end meets the preset push condition according to the push information, and when the preset push condition is met, the push server sends a push instruction to the push end; furthermore, the stream pushing end simultaneously pushes the first live video stream and the second live video stream according to the stream pushing instruction. Compared with the method for performing compression coding on the live video stream and performing transcoding operation by the stream pushing server in the related art by adopting a coding mode, the method provided by the embodiment of the application can realize that the stream pushing end simultaneously pushes the live video stream in different coding modes so as to improve the pushing efficiency of the live video stream; and the compression coding process is realized by the plug flow end, so that the coding operation pressure of the plug flow server can be reduced, and the cost of the plug flow server can be reduced.
Drawings
Fig. 1 is a schematic diagram illustrating a live system provided by an exemplary embodiment of the present application;
fig. 2 is a flowchart illustrating a live streaming method according to an exemplary embodiment of the present application;
fig. 3 is a flowchart illustrating a live streaming method according to another exemplary embodiment of the present application;
fig. 4 shows a flowchart of a live streaming method provided by another exemplary embodiment of the present application;
FIG. 5 is a flow chart illustrating a push terminal implementing simultaneous push provided by an exemplary embodiment of the present application;
fig. 6 shows a block diagram of a live streaming apparatus according to an exemplary embodiment of the present application;
fig. 7 shows a block diagram of a live streaming apparatus according to another exemplary embodiment of the present application;
fig. 8 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application;
fig. 9 is a block diagram illustrating a server according to an exemplary embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
Fig. 1 is a schematic diagram of a live broadcast system according to an exemplary embodiment of the present application. Referring to fig. 1, the live system may include: a push streaming end 100, a push streaming server 200, and a pull streaming end 300.
Optionally, an anchor client used by a network anchor is installed and operated in the stream pushing end 100. The network anchor can register a live broadcast room in the anchor client, and can perform interaction such as audio, video, desktop sharing, document sharing and the like with other users watching the live broadcast through the live broadcast room. In the embodiment of the present application, the network anchor triggers the anchor client to start playing, the streaming client 100 collects real-time images, performs compression coding on the collected images to obtain a live video stream, and then sends the live video stream to the streaming server 200.
The push streaming terminal 100 is connected to the push streaming server 200 through a wireless network or a wired network.
The stream pushing server 200 is a transfer station for exchanging information between live broadcast rooms in a live broadcast system, and is configured to receive a live broadcast video stream from the stream pushing terminal 100 and push the live broadcast video stream to the stream pulling terminal 300, and then the stream pulling terminal 300 watches a user client corresponding to the live broadcast to receive the live broadcast video stream, and performs decoding and playing of the live broadcast video stream in a hardware decoding manner or an online decoding manner through a terminal device. Optionally, the plug flow server 200 may be an independent server, or may be integrated into any one of a server cluster, a virtual cloud storage, or a cloud computing center, taking the plug flow server 200 as the server cluster as an example, in this embodiment of the present application, the plug flow server 200 may include a plug flow control server 210 and a video flow management server 220, where the plug flow control server 210 and the video flow management server 220 are connected through a wireless network or a wired network. On one side of the stream pushing end, the stream pushing end 100 is connected with a stream pushing control server 210 and a video stream management server 220 through a wireless network or a wired network respectively; on the side of the stream pulling end, the stream pulling end 300 is connected to the stream pushing control server 210 and the video stream management server 220 through a wireless network or a wired network, respectively.
In combination with the composition of the stream pushing server 200, in the embodiment of the present application, the stream pushing terminal 100 pushes the compressed and encoded live video stream to the video stream managing server 220, the video stream managing server 220 distributes the live video stream according to the viewing requirement of the user of the stream pulling terminal 300, and then the received live video stream is decoded and played by the stream pulling terminal 300; in addition, in this embodiment of the application, the stream pushing end 100 may also push a live video stream obtained based on different encoding modes. As shown in fig. 1, the stream pushing end 100 first pushes a first live video stream to the video stream management server 220, where the first live video stream is a live video stream obtained in a first encoding manner, and when the stream pushing end 100 pushes a live video stream obtained in another encoding manner (such as a second encoding manner), the live video stream needs to be audited by the stream pushing control server 210, the process is that the stream pushing end 100 sends stream pushing information to the stream pushing control server 210, and the stream pushing control server 210 audits the stream pushing information, and determines whether the stream pushing end 100 currently has a condition for pushing live video streams in different encoding manners, and when the condition is met, the stream pushing control server 210 sends a stream pushing instruction indicating that push is granted to the stream pushing end 100, and then the stream pushing end 100 pushes the first live video stream and the second live video stream.
Optionally, the stream pushing server 200 may further include a anchor identifier management server, where the anchor identifier management server is configured to manage events such as anchor identifier creation and anchor identifier supervision in a network live broadcast process; optionally, the stream pushing server 200 may further include a live broadcast room identifier management server, where the live broadcast room identifier management server is configured to manage events such as creation of a live broadcast room identifier and monitoring of a live broadcast room identifier in a network live broadcast process. The server that the aforementioned plug flow server 200 may include is only an illustrative example, and may also be supplemented according to the content of the embodiment of the present application, which is not limited in this embodiment of the present application.
In addition, when the plug flow server 200 is an independent server, the plug flow server 200 has the background service function of each server.
Referring to fig. 2, a flowchart of a live streaming method provided by an exemplary embodiment of the present application is shown. The method is suitable for the live broadcast system shown in fig. 1, and comprises the following steps:
step 201, when the stream pushing end receives a live broadcasting instruction, pushing a first live broadcasting video stream to a stream pushing server.
The video stream refers to transmission of video data. After the push stream end collects the image through the camera, the collected image needs to be compressed and encoded, and the compressed and encoded video data is pushed to the push stream server.
In an embodiment of the present application, the video stream includes a first live video stream and a second live video stream. The first direct-playing video stream refers to a video stream obtained by coding in a first coding mode; correspondingly, the second live video stream refers to a video stream encoded in the second encoding mode.
In the embodiment of the present application, an implementation scenario in which the bandwidth occupancy rate of the second live video stream is lower than the bandwidth occupancy rate of the first live video stream under the same picture quality is exemplarily described, that is, the playback image quality of the second live video stream is higher than the playback image quality of the first live video stream under the same bandwidth condition, but an implementation scenario in which the bandwidth occupancy rate of the second live video stream is higher than the bandwidth occupancy rate of the first live video stream under the same picture quality is not excluded. The first encoding method may be a default encoding method.
In the embodiment of the application, the live broadcasting is carried out when the stream pushing end receives a live broadcasting instruction, and the first live broadcasting video stream is pushed to the stream pushing server. The live broadcasting instruction is obtained according to triggering operation of the anchor on the playing control, if the playing control for indicating playing is displayed on the live broadcasting client interface, when the anchor touches and clicks the playing control, the live broadcasting client generates the live broadcasting instruction according to the triggering operation.
Step 202, the stream pushing server receives a first live video stream pushed by a stream pushing end.
Correspondingly, the stream pushing server receives the first direct-playing video stream pushed by the stream pushing end. Optionally, the streaming server stores the first live video stream; when the pull stream end has the watching user entering the live broadcast room, the push stream server distributes the first live broadcast video stream to each user client corresponding to the pull stream end.
And step 203, the stream pushing end sends stream pushing information to the stream pushing server.
Optionally, in the embodiment of the present application, an example in which the first encoding method is an H264 encoding method and the second encoding method is an H265 encoding method is schematically described.
In the related art, both the H264 coding scheme and the H265 coding scheme are video compression coding standards. Among them, the H264 coding scheme is a video compression coding technique that is common in the related art, and the H265 coding scheme is an improvement of the H264 coding scheme. Compared with the H264 coding method, except for the above-mentioned same picture quality, the bandwidth occupancy rate of the second live video stream in the H265 coding method is lower than that of the first live video stream in the H264 coding method, and in addition, the storage space required by the second live video stream in the H265 coding method is lower than that of the first live video stream in the H264 coding method under the same picture quality and code rate.
As can be seen from this, in the push streaming process, in addition to the compression encoding of the video stream by using the H264 encoding method, which is relatively common, it is necessary to generalize the compression encoding of the video stream by using the H265 encoding method.
However, whether to use the H265 encoding scheme for compression encoding of the video stream needs to be determined in combination with the actual network environment and the device support information. When the user equipment or the live system at the stream pushing end does not support the compression coding of the live video stream in the H265 coding mode, the stream pushing of the second live video stream cannot be performed.
Therefore, in a possible implementation, the stream pushing end sends stream pushing information to the stream pushing server, wherein the stream pushing information is used for characterizing the encoding capability of the stream pushing end. When the stream pushing information represents that the stream pushing end has the coding capability in the second coding mode, the stream pushing end can push the second live video stream; and when the stream pushing information indicates that the stream pushing end does not have the coding capability in the second coding mode, the stream pushing end cannot push the second live video stream.
In addition, when the stream pushing end simultaneously pushes the first live video stream and the second live video stream, it indicates that the currently occupied uplink bandwidth is larger. Therefore, when the stream pushing end has the encoding capability in the second encoding method, if the current uplink bandwidth is not enough, the stream pushing end cannot push the second live video stream. Therefore, the stream pushing information sent by the stream pushing end to the stream pushing server is also used to characterize the stream pushing capability of the stream pushing end, that is, the stream pushing information is used to characterize the encoding capability and the stream pushing capability of the stream pushing end.
And step 204, the plug flow server acquires the plug flow information corresponding to the plug flow end.
Correspondingly, after the stream pushing end sends the stream pushing information to the stream pushing server, the stream pushing server obtains the stream pushing information corresponding to the stream pushing end.
Step 205, in response to the indication that the stream pushing end meets the preset stream pushing condition, the stream pushing server sends a stream pushing instruction to the stream pushing end.
And the stream pushing instruction is sent by the stream pushing server when the stream pushing information meets the preset stream pushing condition and is used for indicating the stream pushing end to push the second live video stream.
In a possible implementation manner, a preset stream pushing condition is preset in the stream pushing server, and is used for judging whether the stream pushing end meets a condition for pushing the second live video stream. Optionally, the preset stream pushing condition includes a preset stream pushing condition related to the encoding capability and a preset encoding condition related to the stream pushing capability.
And when the plug flow information indicates that the plug flow end meets the preset plug flow condition, the plug flow server sends a plug flow instruction to the plug flow end.
In an example, the stream pushing information represents that the stream pushing end has the coding capability in the second coding mode, and the current network environment meets the network requirement for pushing the first live video stream and the second live video stream simultaneously, the stream pushing server judges that the stream pushing information indicates that the stream pushing end meets the preset stream pushing condition, and then sends a stream pushing instruction to the stream pushing end.
And step 206, the stream pushing end receives a stream pushing instruction sent by the stream pushing server.
Correspondingly, after the stream pushing server sends the stream pushing instruction to the stream pushing end, the stream pushing end receives the stream pushing instruction sent by the stream pushing server.
And step 207, pushing the second live video stream to the stream pushing server by the stream pushing end according to the stream pushing instruction.
Because the stream pushing instruction is used for instructing the stream pushing end to push the second live video stream, the stream pushing end simultaneously pushes the second live video stream on the basis of continuously pushing the first live video stream, that is, the stream pushing end simultaneously pushes the first live video stream and the second live video stream to the stream pushing server according to the stream pushing instruction.
In addition, the stream pushing end may also push the second live video stream to the stream pushing server according to the stream pushing instruction, and stop pushing the first live video stream to the stream pushing server, which is not limited in this embodiment of the application.
And step 208, the stream pushing server receives the second live video stream pushed by the stream pushing end.
Correspondingly, the stream pushing server receives the second live video stream while receiving the first live video stream pushed by the stream pushing end. Optionally, the stream pushing server stores the first live video stream and the second live video stream; and when the pull stream end has the watching user entering the live broadcast room, the push stream server distributes the first live broadcast video stream and the second live broadcast video stream to each user client corresponding to the pull stream end.
And when the stream pushing server only receives the second live video stream pushed by the stream pushing end, the stream pushing server stores the second live video stream.
Optionally, in this embodiment of the application, after step 206, the following is also included.
And the content I, the stream pushing end sends a stream pushing address obtaining request to the stream pushing server, wherein the stream pushing address obtaining request is used for requesting to obtain a stream pushing address of the second live video stream.
And secondly, the stream pushing server sends a stream pushing address to the stream pushing end.
And thirdly, the stream pushing end pushes a second live video stream to the stream pushing server according to the stream pushing address while continuing to push the first live video stream.
To sum up, in the embodiment of the present application, for the live broadcast system, after the live broadcast, the stream pushing end performs compression coding on the live broadcast video stream in the default first coding mode, and pushes the live broadcast video stream to the stream pushing server; however, for the push of the live video stream, there is a live video stream in a more optimal coding mode, that is, a second live video stream, but the push of the second live video stream requires that the push end has certain coding capability and push capability, so that the push end sends push information to the push server, so that the push server judges whether the push end meets the preset push condition according to the push information, and when the preset push condition is met, the push server sends a push instruction to the push end; furthermore, the stream pushing end simultaneously pushes the first live video stream and the second live video stream according to the stream pushing instruction. Compared with the method for performing compression coding on the live video stream and performing transcoding operation by the stream pushing server in the related art by adopting a coding mode, the method provided by the embodiment of the application can realize that the stream pushing end simultaneously pushes the live video stream in different coding modes so as to improve the pushing efficiency of the live video stream; and the compression coding process is realized by the plug flow end, so that the coding operation pressure of the plug flow server can be reduced, and the cost of the plug flow server can be reduced.
Referring to fig. 3, a flowchart of a live streaming method according to another exemplary embodiment of the present application is shown. The method is suitable for the live broadcast system shown in fig. 1, and comprises the following steps:
step 301, the stream pushing end uploads a test data packet to the stream pushing server.
For the stream side, in order to test whether the current network environment can be played, in one possible implementation, the stream side uploads a test packet to the stream server.
When the plug flow server successfully receives the test data packet, the current network environment is available. Optionally, the stream pushing server may send feedback information to the stream pushing end to notify the stream pushing end that the current network is available; in addition, when the stream pushing end does not receive the feedback information sent by the stream pushing server after a certain time, the current network is indicated to be unavailable, and further, the stream pushing end generates network connection prompt information for prompting the anchor to open the network connection.
Step 302, the stream pushing end determines the available uplink bandwidth according to the data volume and the uploading duration of the test data packet.
Wherein, the data volume of the test data packet is known by the stream pushing end. Therefore, the stream pushing end can judge whether the current network is available or not through the feedback information of the stream pushing server to the test data packet, and can determine the available uplink bandwidth according to the data volume and the uploading duration of the test data packet.
In a possible implementation manner, when the stream pushing server receives the test data packet, the stream pushing server obtains arrival time information of the test data packet and returns feedback information containing the arrival time information to the stream pushing end, the stream pushing end obtains an upload time length of the test data packet according to the arrival time information and sending time information of the test data packet, and then determines an available uplink bandwidth according to a data volume and the upload time length of the test data packet.
And 303, when the stream pushing end receives the live broadcasting instruction, pushing the first live broadcasting video stream to the stream pushing server.
Please refer to step 201, and the embodiments of the present application are not described herein again.
Step 304, the stream pushing server receives the first live video stream pushed by the stream pushing end.
Please refer to step 202, which is not described herein again in this embodiment.
Before sending the push flow information to the push flow server, the push flow end needs to acquire the push flow information. Optionally, in this embodiment of the present application, the stream pushing information includes a hardware parameter, an available uplink bandwidth, and a stream pushing parameter, where the available uplink bandwidth is determined by the stream pushing end through uploading a test data packet, and the stream pushing parameter is used to indicate a stream pushing condition of the first direct-playing video stream within a preset stream pushing duration. Therefore, the process of acquiring the push flow information by the push flow terminal includes step 305 and step 306.
Step 305, the stream pushing end obtains hardware parameters and available uplink bandwidth.
In the hardware parameters, the available uplink bandwidth and the push flow parameters included in the push flow information, the hardware parameters and the available uplink bandwidth can be directly obtained. The hardware parameters include encoder type information supported by the anchor user side corresponding to the stream pushing side, such as H264 encoder information, H265 encoder information and the like; and the available uplink bandwidth is determined by uploading the test data packet by the stream pushing end.
Step 306, in response to that the stream pushing duration of the first direct-playing video stream reaches the preset stream pushing duration, the stream pushing end obtains stream pushing parameters within the preset stream pushing duration.
In the hardware parameters, the available uplink bandwidth and the push flow parameters included in the push flow information, the push flow parameters cannot be directly acquired.
The stream pushing parameter is used for indicating the stream pushing condition of the first direct-playing video stream within a preset stream pushing duration. In a possible implementation manner, when the stream pushing duration of the first direct-playing video stream reaches the preset stream pushing duration, the stream pushing end obtains the stream pushing parameters within the preset stream pushing duration.
Optionally, the push stream parameter may include packet loss rate, code rate, resolution, and other push stream parameters for determining the push stream condition. In the embodiment of the present application, a flow pushing parameter is taken as an example to schematically illustrate the packet loss rate, and the packet loss rate is obtained by frame rate calculation.
When the uplink bandwidth is not enough, in order to realize the push of the live video stream, the frame rate of the push stream end is continuously reduced. And the stream pushing end acquires the frame number in the preset stream pushing duration, and compares the frame number with the scheduled sending frame number to obtain the packet loss rate. In one example, the predetermined push flow duration is one minute, if the push is performed according to the initial frame rate, the scheduled sending frame number in one minute is 2000 frames, and after the frame rate is continuously decreased, the actual sending frame number in one minute is 1200 frames, so that the obtained packet loss rate is a ratio of 800 to 2000, that is, 0.4.
And 307, the stream pushing end sends the hardware parameters, the available uplink bandwidth and the stream pushing parameters as stream pushing information to the stream pushing server.
To this end, the plug flow end can obtain all the contents included in the plug flow information through the step 305 and the step 306; further, the stream pushing end sends the hardware parameters, the available uplink bandwidth and the stream pushing parameters as stream pushing information to the stream pushing server.
And 308, the plug flow server acquires the plug flow information corresponding to the plug flow end.
Please refer to step 204, which is not described herein again in this embodiment.
After the stream pushing server obtains the stream pushing information corresponding to the stream pushing end, whether the stream pushing end meets the condition of pushing the second live video stream needs to be judged according to the stream pushing information. Optionally, the process of determining the push flow information by the push flow server includes step 309 and step 310.
Step 309, in response to that the hardware parameter belongs to the preset hardware parameter, the stream pushing server determines that the stream pushing end meets the preset encoding capability.
The preset hardware parameter is a parameter of hardware supporting the second encoding mode. In the above-mentioned content, in the embodiment of the present application, taking the second encoding method as the H265 encoding method as an example, the preset hardware parameter is a parameter of hardware supporting the H265 encoding method.
In an example, if the hardware parameter in the stream pushing information includes H265 encoder information, the stream pushing server determines that the hardware parameter belongs to a preset hardware parameter, that is, the stream pushing server determines that the stream pushing end meets the preset encoding capability.
Step 310, in response to that the available uplink bandwidth is greater than the uplink bandwidth threshold and the stream pushing parameter indicates that the packet loss rate is less than the packet loss rate threshold, the stream pushing server determines that the stream pushing end meets the preset stream pushing capability.
And the uplink bandwidth threshold is determined according to the resolution corresponding to the first direct-playing video stream.
Watch 1
Resolution ratio H264 upstream bandwidth requirement H265 upstream bandwidth requirement Uplink bandwidth threshold
480P 150KB 120KB 270KB
720P 250KB 200KB 450KB
1080P 500KB 400KB 900KB
As shown in table i, when the resolution is different and the coding scheme is different, the corresponding uplink bandwidth threshold is also different. If the resolution corresponding to the first live video stream is 480P, the required uplink bandwidth threshold is 170KB in order to realize simultaneous pushing of the first live video stream and the second live video stream by the stream pushing end.
In addition, a packet loss rate threshold is preset in the push streaming server, and in a possible implementation manner, in response to that the available uplink bandwidth is greater than the uplink bandwidth threshold and the push streaming parameter indicates that the packet loss rate is less than the packet loss rate threshold, the push streaming server determines that the push streaming end meets the preset push streaming capability.
In an example, the uplink bandwidth threshold is 450KB, the packet loss rate threshold is 0.3, and the push streaming server determines that the push streaming end does not meet the preset push streaming capability if the push streaming information indicates that the available uplink bandwidth is 600KB and the packet loss rate is 0.4; in another example, the uplink bandwidth threshold is 450KB, the packet loss rate threshold is 0.3, the push streaming server determines that the push streaming end satisfies the predetermined push streaming capability, where the push streaming information indicates that the available uplink bandwidth is 600KB and the packet loss rate is 0.2.
Optionally, the execution sequence of step 309 and step 310 is not sequential, or the plug flow server executes step 309 and step 310 simultaneously. In step 309, if the plug flow server determines that the hardware parameter does not belong to the preset hardware parameter, the process after step 309 is not executed.
Step 311, in response to the hardware parameter indicating that the stream pushing end satisfies the preset encoding capability and the available uplink bandwidth and the stream pushing parameter indicating that the stream pushing end satisfies the preset stream pushing capability, the stream pushing server determines that the stream pushing end satisfies the preset stream pushing condition and sends a stream pushing instruction to the stream pushing end.
For the preset coding capacity and the preset stream pushing capacity, the stream pushing server requires that the stream pushing information simultaneously meet the preset coding capacity and the preset stream pushing capacity, so that the stream pushing end can be determined to meet the preset stream pushing condition, and a stream pushing instruction is sent to the stream pushing end.
In step 312, the stream pushing end receives the stream pushing command sent by the stream pushing server.
Please refer to step 206, which is not described herein again in this embodiment.
And 313, the stream pushing end pushes the second live video stream to the stream pushing server according to the stream pushing instruction.
Please refer to step 207, which is not described herein again in this embodiment.
Optionally, in this embodiment of the application, the stream pushing server determines whether to allow the stream pushing end to push the second live video stream according to the stream pushing information, and there are other determination methods: and the anchor client carries out unique identification through an anchor user account, and if the anchor user account corresponding to the stream pushing end is a preset account, the stream pushing server sends a transcoding instruction to the stream pushing end to prompt the stream pushing end to only push a second live video stream.
The preset account can be a popular anchor user account, and in order to improve the live broadcast fluency of the popular anchor, the stream pushing end can encode the live broadcast video stream only by using the second encoding mode and push the live broadcast video stream to the stream pushing server.
And step 314, the stream pushing server receives the second live video stream pushed by the stream pushing end.
Please refer to step 208, which is not described herein again in this embodiment.
On the basis of the above embodiment, the embodiment of the present application further describes a process of uploading test data by the plug flow end, a process of generating plug flow information by the plug flow end, and a process of determining the plug flow information by the plug flow server. In the process of uploading the test data by the plug flow end, the plug flow end can simultaneously determine whether the network is available and the available uplink bandwidth through the test data packet; in the process of generating the stream pushing information by the stream pushing end, the stream pushing end not only acquires hardware parameters and available uplink bandwidth, but also acquires stream pushing parameters within a preset stream pushing duration, so that the phenomenon that the pushing of a first live video stream is influenced when a second live video stream cannot be successfully pushed due to poor network environment is avoided, and the live viewing effect of the stream pulling end is reduced; in the process that the stream pushing server judges the stream pushing information, the stream pushing server judges whether the stream pushing end meets the preset stream pushing condition or not according to the preset condition corresponding to the stream pushing information, the pushing success rate of the second live video stream can be improved, the pushing efficiency of the whole live video stream is also improved when the pushing of the first live video stream is not influenced, and then the live picture effect is improved.
Referring to fig. 4, a flowchart of a live streaming method according to another exemplary embodiment of the present application is shown. The method is suitable for the live broadcast system shown in fig. 1, and comprises the following steps:
step 401, the stream pushing end uploads a test data packet to the stream pushing server.
Please refer to step 301, which is not described herein again in this embodiment.
And step 402, the stream pushing end determines the available uplink bandwidth according to the data volume and the uploading duration of the test data packet.
Please refer to step 302, which is not described herein again in this embodiment.
And step 403, when the stream pushing end receives the live broadcasting instruction, pushing the first live broadcasting video stream to the stream pushing server.
Please refer to step 201, and the embodiments of the present application are not described herein again.
In step 404, the streaming server receives a first live video stream pushed by the streaming end.
Please refer to step 202, which is not described herein again in this embodiment.
Step 405, the stream pushing end obtains the hardware parameters and the available uplink bandwidth.
Please refer to step 305, which is not described herein again in this embodiment.
Step 406, in response to that the stream pushing duration of the first direct-playing video stream reaches the preset stream pushing duration, the stream pushing end obtains a stream pushing parameter within the preset stream pushing duration.
Please refer to step 306, which is not described herein again.
Step 407, the stream pushing end sends the hardware parameter, the available uplink bandwidth and the stream pushing parameter as stream pushing information to the stream pushing server.
Please refer to step 307, which is not described herein again in this embodiment.
And step 408, the plug flow server acquires the plug flow information corresponding to the plug flow end.
Please refer to step 204, which is not described herein again in this embodiment.
Step 409, in response to the fact that the hardware parameter belongs to the preset hardware parameter, the stream pushing server determines that the stream pushing end meets the preset coding capacity.
Please refer to step 309, and the embodiments of the present application are not described herein again.
Step 410, in response to that the available uplink bandwidth is greater than the uplink bandwidth threshold and the stream pushing parameter indicates that the packet loss rate is less than the packet loss rate threshold, the stream pushing server determines that the stream pushing end meets the preset stream pushing capability.
Please refer to step 310, which is not described herein again.
Step 411, in response to the hardware parameter indicating that the stream pushing end satisfies the preset encoding capability and the available uplink bandwidth and the stream pushing parameter indicating that the stream pushing end satisfies the preset stream pushing capability, the stream pushing server determines that the stream pushing end satisfies the preset stream pushing condition and sends a stream pushing instruction to the stream pushing end.
Please refer to step 311, which is not described herein again in this embodiment.
In step 412, the stream pushing end receives a stream pushing command sent by the stream pushing server.
Please refer to step 206, which is not described herein again in this embodiment.
And 413, the stream pushing end pushes the second live video stream to the stream pushing server according to the stream pushing instruction.
Please refer to step 207, which is not described herein again in this embodiment.
In step 414, the stream pushing server receives the second live video stream pushed by the stream pushing end.
Please refer to step 208, which is not described herein again in this embodiment.
And 415, responding to the existence of the plug flow abnormity, and displaying prompt information by the plug flow end.
In the process of simultaneously pushing the first live video stream and the second live video stream, the stream pushing end may have stream pushing abnormalities, such as network fluctuation, live picture machine jamming, and the like. The above-mentioned plug flow abnormal situation may be related to the network environment of the operator, or may be caused by that other application software at the plug flow end preempts the device resource, for example, a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU) are occupied by a background application program too high.
In one possible implementation mode, when the plug flow abnormity exists, the plug flow end displays the prompt message. Wherein, the prompt message includes the reason for the abnormal plug flow.
Optionally, step 415 includes the following.
The method comprises the steps that firstly, a stream pushing end obtains hardware occupancy rate, and the hardware occupancy rate comprises at least one of CPU occupancy rate and GPU occupancy rate.
And secondly, responding to the situation that the hardware occupancy rate is larger than the occupancy rate threshold value, displaying first prompt information by the plug flow end, wherein the first prompt information is used for prompting that the plug flow is abnormal due to the fact that the hardware occupancy rate is too high.
And thirdly, the stream pushing end acquires network parameters, wherein the network parameters at least comprise the current uplink bandwidth.
And fourthly, responding to the situation that the current uplink bandwidth is smaller than the uplink bandwidth threshold value, displaying second prompt information by the stream pushing end, wherein the second prompt information is used for prompting that the network is blocked to cause stream pushing abnormity.
And step 416, in response to that the duration of the stream pushing exception reaches the duration threshold, the stream pushing end stops pushing the second live video stream to the stream pushing server.
When the stream pushing end displays the first prompt message and/or the second prompt message, the anchor user end does not perform an operation for improving the current stream pushing abnormity, namely the duration of the stream pushing abnormity reaches a duration threshold, the stream pushing end stops pushing the second live video stream to the stream pushing server. If the background application programs are excessively occupied, the user does not close at least one background application program within the time length threshold value; if the current network is blocked and the user does not improve the network condition within the time length threshold value, the stream pushing end stops pushing the second live video stream to the stream pushing server.
On the basis of the above embodiment, the embodiment of the present application further discloses how the plug flow end solves the content of the plug flow abnormality, when the plug flow abnormality exists, the plug flow end displays the prompt information, and displays different prompt information according to different plug flow abnormality conditions, so as to realize targeted information prompt; in addition, when the duration of the stream pushing abnormity reaches the duration threshold, the stream pushing end stops pushing the second live video stream to the stream pushing server so as to maintain a normal network live environment.
Please refer to fig. 5, which illustrates a flowchart of implementing simultaneous stream pushing by the stream pushing end according to an exemplary embodiment of the present application.
For the push flow end, in the embodiments of the present application, the push flow end mainly includes four links in the simultaneous push flow process: collecting an image; preprocessing the collected image; encoding and compressing the collected image through an H264 encoder and an H265 encoder; and pushing the first live video stream and the second live video stream which are subjected to compression coding to a push streaming server.
When the anchor starts a beautifying function, a filter function and the like on a current live broadcast picture, the stream pushing end needs to perform corresponding preprocessing on the acquired image; or, the current live broadcast system has an image processing function, such as performing image brightening adjustment on a live broadcast image with low brightness, and the like, and the stream push end also needs to perform corresponding preprocessing on the acquired image.
Referring to fig. 6, a block diagram of a live streaming apparatus provided in an exemplary embodiment of the present application is shown, where the apparatus is used in a streaming end, and the apparatus includes:
the first pushing module 601 is configured to push a first direct-broadcast video stream to a stream pushing server when receiving a live-broadcast instruction, where the first direct-broadcast video stream is obtained by encoding in a first encoding manner;
an information sending module 602, configured to send, to the stream pushing server, stream pushing information, where the stream pushing information is used to represent an encoding capability and a stream pushing capability of the stream pushing end;
the instruction receiving module 603 is configured to receive a stream pushing instruction sent by the stream pushing server, where the stream pushing instruction is sent by the stream pushing server when the stream pushing information meets a preset stream pushing condition, and the stream pushing instruction is used to instruct the stream pushing end to push a second live video stream, where the second live video stream is obtained by encoding in a second encoding manner;
a second pushing module 604, configured to push the second live video stream to the stream pushing server according to the stream pushing instruction.
Optionally, the information sending module 602 includes:
a first information sending unit, configured to obtain a hardware parameter and an available uplink bandwidth;
a second information sending unit, configured to obtain a stream pushing parameter within a preset stream pushing duration in response to that the stream pushing duration of the first direct-playing video stream reaches the preset stream pushing duration;
a third information sending unit, configured to send the hardware parameter, the available uplink bandwidth, and the push flow parameter as the push flow information to the push flow server.
Optionally, the apparatus further comprises:
the data packet sending module is used for uploading a test data packet to the plug flow server;
and the bandwidth determining module is used for determining the available uplink bandwidth according to the data volume and the uploading duration of the test data packet.
Optionally, the apparatus further comprises:
the first exception handling module is used for responding to the existence of the plug flow exception and displaying prompt information, wherein the prompt information comprises the reason of the plug flow exception;
and the second exception handling module is used for responding to the situation that the duration of the stream pushing exception reaches a duration threshold value and stopping pushing the second live video stream to the stream pushing server.
Optionally, the first exception handling module includes:
the first exception handling unit is used for acquiring hardware occupancy rate, wherein the hardware occupancy rate comprises at least one of CPU occupancy rate and GPU occupancy rate; responding to the fact that the hardware occupancy rate is larger than the occupancy rate threshold value, and displaying first prompt information, wherein the first prompt information is used for prompting that the pushing flow is abnormal due to the fact that the hardware occupancy rate is too high;
the second exception handling unit is used for acquiring network parameters, and the network parameters at least comprise the current uplink bandwidth; and responding to the fact that the current uplink bandwidth is smaller than an uplink bandwidth threshold value, and displaying second prompt information, wherein the second prompt information is used for prompting that the network is blocked to cause the flow pushing abnormity.
Referring to fig. 7, a block diagram of a live streaming apparatus according to another exemplary embodiment of the present application is shown, where the apparatus is used in a streaming server, and the apparatus includes:
a first receiving module 701, configured to receive a first direct-play video stream pushed by a stream pushing end, where the first direct-play video stream is obtained by coding in a first coding mode;
an information receiving module 702, configured to obtain stream pushing information corresponding to the stream pushing end, where the stream pushing information is used to represent a coding capability and a stream pushing capability of the stream pushing end;
the instruction sending module 703 is configured to send a stream pushing instruction to the stream pushing end in response to the stream pushing information indicating that the stream pushing end meets a preset stream pushing condition, where the stream pushing instruction is used to indicate the stream pushing end to push a second live video stream, and the second live video stream is obtained by encoding in a second encoding manner;
a second receiving module 704, configured to receive the second live video stream pushed by the stream pushing end.
Optionally, the stream pushing information includes a hardware parameter, an available uplink bandwidth, and a stream pushing parameter, where the available uplink bandwidth is determined by the stream pushing end through uploading a test data packet, and the stream pushing parameter is used to indicate a stream pushing condition of the first direct-playing video stream within a preset stream pushing duration;
optionally, the instruction sending module 703 includes:
and the instruction sending unit is used for responding to the situation that the hardware parameter indicates that the stream pushing end meets the preset coding capacity, and the available uplink bandwidth and the stream pushing parameter indicate that the stream pushing end meets the preset stream pushing capacity, determining that the stream pushing end meets the preset stream pushing condition, and sending the stream pushing instruction to the stream pushing end.
Optionally, the apparatus further comprises:
the first plug flow judging module is used for responding to the fact that the hardware parameter belongs to a preset hardware parameter, and determining that the plug flow end meets the preset coding capacity, wherein the preset hardware parameter is a parameter of hardware supporting the second coding mode;
and the second stream pushing judgment module is used for determining that the stream pushing end meets the preset stream pushing capability in response to the fact that the available uplink bandwidth is larger than the uplink bandwidth threshold and the stream pushing parameter indicates that the packet loss rate is smaller than the packet loss rate threshold, and the uplink bandwidth threshold is determined according to the resolution corresponding to the first direct-playing video stream.
Referring to fig. 8, a block diagram of a terminal 800 according to an exemplary embodiment of the present disclosure is shown, where the terminal 800 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving picture Experts Group Audio L layer III, mpeg Audio layer 3), an MP4 player (Moving picture Experts Group Audio L layer IV, mpeg Audio layer 4).
In general, the terminal 800 includes: a processor 801 and a memory 802.
The processor 801 may also include a main processor, which is a processor for Processing data in a wake-up state, also referred to as a Central Processing Unit (CPU), and a coprocessor, which is a low power processor for Processing data in a standby state, in some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit) for rendering and rendering content for display, in some embodiments, the processor 801 may also include an intelligent processor (AI) for learning operations related to AI (Artificial Intelligence processor) for computing operations related to display screens.
Memory 802 may include one or more computer-readable storage media, which may be tangible and non-transitory. Memory 802 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 802 is used to store at least one instruction for execution by the processor 801 to implement the methods provided by embodiments of the present application.
In some embodiments, the terminal 800 may further include: a peripheral interface 803 and at least one peripheral. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a touch screen display 805, a camera 806, an audio circuit 807, a positioning component 808, and a power supply 809.
The peripheral interface 803 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, memory 802, and peripheral interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral interface 803 may be implemented on separate chips or circuit boards, which are not limited by this embodiment.
The Radio Frequency circuit 804 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 804 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 804 converts an electrical signal into an electromagnetic signal to be transmitted, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuit 804 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: the world wide web, metropolitan area networks, intranets, generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 804 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.
Touch Display 805 is used to Display a UI (User Interface) that may include graphics, text, icons, video, and any combination thereof, touch Display 805 also has the ability to capture touch signals on or over the surface of touch Display 805, which may be input as control signals to processor 801 for processing, touch Display 805 is used to provide virtual buttons and/or a virtual keyboard, also known as soft buttons and/or a soft keyboard.
The camera assembly 806 is used to capture images or video. Optionally, camera assembly 806 includes a front camera and a rear camera. Generally, a front camera is used for realizing video call or self-shooting, and a rear camera is used for realizing shooting of pictures or videos. In some embodiments, the number of the rear cameras is at least two, and each of the rear cameras is any one of a main camera, a depth-of-field camera and a wide-angle camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting function and a VR (Virtual Reality) shooting function. In some embodiments, camera assembly 806 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.
The audio circuit 807 is used to provide an audio interface between the user and the terminal 800. The audio circuit 807 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 801 for processing or inputting the electric signals to the radio frequency circuit 804 to realize voice communication. For the purpose of stereo sound collection or noise reduction, a plurality of microphones may be provided at different portions of the terminal 800. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 801 or the radio frequency circuit 804 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, the audio circuitry 807 may also include a headphone jack.
The positioning component 808 is used to locate the current geographic location of the terminal 800 to implement navigation or L BS (L geographic based Service). the positioning component 808 can be a positioning component based on the united states GPS (global positioning System), the chinese beidou System, or the russian galileo System.
Power supply 809 is used to provide power to various components in terminal 800. The power supply 809 can be ac, dc, disposable or rechargeable. When the power supply 809 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.
In some embodiments, terminal 800 also includes one or more sensors 810. The one or more sensors 810 include, but are not limited to: acceleration sensor 811, gyro sensor 812, pressure sensor 813, fingerprint sensor 814, optical sensor 815 and proximity sensor 816.
The acceleration sensor 811 may detect the magnitude of acceleration in three coordinate axes of the coordinate system established with the terminal 800. For example, the acceleration sensor 811 may be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 801 may control the touch screen 805 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 811. The acceleration sensor 811 may also be used for acquisition of motion data of a game or a user.
The gyro sensor 812 may detect a body direction and a rotation angle of the terminal 800, and the gyro sensor 812 may cooperate with the acceleration sensor 811 to acquire a 3D motion of the user with respect to the terminal 800. From the data collected by the gyro sensor 812, the processor 801 may implement the following functions: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.
Pressure sensors 813 may be disposed on the side bezel of terminal 800 and/or underneath touch display 805. When the pressure sensor 813 is disposed on the side frame of the terminal 800, a user's grip signal on the terminal 800 can be detected, and left-right hand recognition or shortcut operation can be performed based on the grip signal. When the pressure sensor 813 is disposed at the lower layer of the touch display screen 805, it is possible to control an operability control on the UI interface according to a pressure operation of the user on the touch display screen 805. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.
The fingerprint sensor 814 is used for collecting a fingerprint of a user to identify the identity of the user according to the collected fingerprint, when the identity of the user is identified as a trusted identity, the user is authorized to perform related sensitive operations by the processor 801, the sensitive operations include unlocking a screen, viewing encrypted information, downloading software, paying and changing settings, and the like.
The optical sensor 815 is used to collect the ambient light intensity. In one embodiment, the processor 801 may control the display brightness of the touch screen 805 based on the ambient light intensity collected by the optical sensor 815. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 805 is increased; when the ambient light intensity is low, the display brightness of the touch display 805 is turned down. In another embodiment, the processor 801 may also dynamically adjust the shooting parameters of the camera assembly 806 based on the ambient light intensity collected by the optical sensor 815.
A proximity sensor 816, also known as a distance sensor, is typically provided on the front side of the terminal 800. The proximity sensor 816 is used to collect the distance between the user and the front surface of the terminal 800. In one embodiment, when the proximity sensor 816 detects that the distance between the user and the front surface of the terminal 800 gradually decreases, the processor 801 controls the touch display 805 to switch from the bright screen state to the dark screen state; when the proximity sensor 816 detects that the distance between the user and the front surface of the terminal 800 becomes gradually larger, the processor 801 controls the touch display 805 to switch from the screen-on state to the screen-on state.
Those skilled in the art will appreciate that the configuration shown in fig. 8 is not intended to be limiting of terminal 800 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.
Referring to fig. 9, a schematic structural diagram of a server 900 according to an embodiment of the present application is shown. The server 900 may be used to implement the live streaming method provided in the above embodiments. The server 900 may be the push streaming server 200 described in the embodiment of fig. 1. Specifically, the method comprises the following steps:
the server 900 includes a Central Processing Unit (CPU)901, a system memory 904 including a Random Access Memory (RAM)902 and a Read Only Memory (ROM)903, and a system bus 905 connecting the system memory 904 and the central processing unit 901. The server 900 also includes a basic input/output system (I/O system) 906 for facilitating the transfer of information between devices within the computer, and a mass storage device 907 for storing an operating system 913, application programs 914, and other program modules 915.
The basic input/output system 906 includes a display 908 for displaying information and an input device 909 such as a mouse, keyboard, etc. for user input of information. Wherein the display 908 and the input device 909 are connected to the central processing unit 901 through an input output controller 910 connected to the system bus 905. The basic input/output system 906 may also include an input/output controller 910 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 910 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 907 is connected to the central processing unit 901 through a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable media provide non-volatile storage for the server 900. That is, the mass storage device 907 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM drive.
Without loss of generality, the computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the foregoing. The system memory 904 and mass storage device 907 described above may be collectively referred to as memory.
The server 900 may also operate as a remote computer connected to a network via a network, such as the internet, according to various embodiments of the present application. That is, the server 900 may be connected to the network 912 through the network interface unit 911 coupled to the system bus 905, or the network interface unit 911 may be used to connect to other types of networks or remote computer systems (not shown).
The memory also includes one or more programs stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for implementing the server-side live streaming method.
The embodiment of the present application further provides a computer-readable storage medium, where at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the live streaming method provided in the above embodiments.
Optionally, the computer-readable storage medium may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a Solid State Drive (SSD), or an optical disc. The Random Access Memory may include a resistive Random Access Memory (ReRAM) and a Dynamic Random Access Memory (DRAM).
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (13)

1. A live streaming method is used for a streaming end, and comprises the following steps:
when a live broadcast instruction is received, pushing a first live broadcast video stream to a stream pushing server, wherein the first live broadcast video stream is obtained by coding in a first coding mode;
sending plug flow information to the plug flow server, wherein the plug flow information is used for representing the coding capability and the plug flow capability of the plug flow end;
receiving a stream pushing instruction sent by the stream pushing server, wherein the stream pushing instruction is sent by the stream pushing server when the stream pushing information meets a preset stream pushing condition, the stream pushing instruction is used for indicating the stream pushing end to push a second live video stream, and the second live video stream is obtained by coding in a second coding mode;
and pushing the second live video stream to the stream pushing server according to the stream pushing instruction.
2. The method of claim 1, wherein sending the push flow information to the push flow server comprises:
acquiring hardware parameters and available uplink bandwidth;
responding to the fact that the stream pushing duration of the first direct-playing video stream reaches a preset stream pushing duration, and obtaining stream pushing parameters in the preset stream pushing duration;
and sending the hardware parameters, the available uplink bandwidth and the stream pushing parameters as the stream pushing information to the stream pushing server.
3. The method of claim 2, wherein prior to pushing the first live video stream to the push streaming server, the method further comprises:
uploading a test data packet to the plug flow server;
and determining the available uplink bandwidth according to the data volume and the uploading duration of the test data packet.
4. The method of any of claims 1 to 3, wherein after the pushing the second live video stream to the streaming server, the method further comprises:
responding to the existence of the plug flow abnormity, and displaying prompt information, wherein the prompt information comprises the reason of the plug flow abnormity;
and in response to the fact that the duration of the stream pushing abnormity reaches a duration threshold value, stopping pushing the second live video stream to the stream pushing server.
5. The method of claim 4, wherein the displaying a prompt in response to the presence of a plug flow anomaly comprises:
acquiring hardware occupancy rate, wherein the hardware occupancy rate comprises at least one of Central Processing Unit (CPU) occupancy rate and Graphics Processing Unit (GPU) occupancy rate; responding to the fact that the hardware occupancy rate is larger than the occupancy rate threshold value, and displaying first prompt information, wherein the first prompt information is used for prompting that the pushing flow is abnormal due to the fact that the hardware occupancy rate is too high;
acquiring network parameters, wherein the network parameters at least comprise current uplink bandwidth; and responding to the fact that the current uplink bandwidth is smaller than an uplink bandwidth threshold value, and displaying second prompt information, wherein the second prompt information is used for prompting that the network is blocked to cause the flow pushing abnormity.
6. A live streaming method, which is used for a streaming server, and comprises the following steps:
receiving a first direct-playing video stream pushed by a stream pushing end, wherein the first direct-playing video stream is obtained by coding in a first coding mode;
acquiring the push flow information corresponding to the push flow end, wherein the push flow information is used for representing the coding capability and the push flow capability of the push flow end;
responding to the stream pushing information to indicate that the stream pushing end meets a preset stream pushing condition, and sending a stream pushing instruction to the stream pushing end, wherein the stream pushing instruction is used for indicating the stream pushing end to push a second live video stream, and the second live video stream is obtained by coding in a second coding mode;
and receiving the second live video stream pushed by the stream pushing end.
7. The method according to claim 6, wherein the stream pushing information includes hardware parameters, an available uplink bandwidth and a stream pushing parameter, the available uplink bandwidth is determined by the stream pushing end through uploading a test packet, and the stream pushing parameter is used to indicate a stream pushing condition of the first direct-view video stream within a preset stream pushing duration;
the sending a stream pushing instruction to the stream pushing end in response to the stream pushing information indicating that the stream pushing end meets a preset stream pushing condition comprises:
and in response to the hardware parameter indicating that the stream pushing end meets the preset coding capability and the available uplink bandwidth and the stream pushing parameter indicating that the stream pushing end meets the preset stream pushing capability, determining that the stream pushing end meets the preset stream pushing condition, and sending the stream pushing instruction to the stream pushing end.
8. The method according to claim 7, wherein after obtaining the push flow information corresponding to the push flow end, the method comprises:
determining that the stream pushing end meets the preset coding capacity in response to the fact that the hardware parameter belongs to a preset hardware parameter, wherein the preset hardware parameter is a parameter of hardware supporting the second coding mode;
and in response to that the available uplink bandwidth is larger than an uplink bandwidth threshold and that the stream pushing parameter indicates that the packet loss rate is smaller than a packet loss rate threshold, determining that the stream pushing end meets a preset stream pushing capability, wherein the uplink bandwidth threshold is determined according to the resolution corresponding to the first direct-playing video stream.
9. A live streaming device, the device being used at a streaming end, the device comprising:
the first pushing module is used for pushing a first direct-broadcasting video stream to a stream pushing server when receiving a live-broadcasting instruction, wherein the first direct-broadcasting video stream is obtained by coding in a first coding mode;
the information sending module is used for sending the stream pushing information to the stream pushing server, wherein the stream pushing information is used for representing the coding capability and the stream pushing capability of the stream pushing end;
the instruction receiving module is used for receiving a stream pushing instruction sent by the stream pushing server, the stream pushing instruction is sent by the stream pushing server when the stream pushing information meets a preset stream pushing condition, the stream pushing instruction is used for indicating the stream pushing end to push a second live video stream, and the second live video stream is obtained by coding in a second coding mode;
and the second pushing module is used for pushing the second live video stream to the stream pushing server according to the stream pushing instruction.
10. A live streaming apparatus, wherein the apparatus is used for a streaming server, the apparatus comprising:
the first receiving module is used for receiving a first direct-playing video stream pushed by a stream pushing end, and the first direct-playing video stream is obtained by coding in a first coding mode;
the information receiving module is used for acquiring the push flow information corresponding to the push flow end, and the push flow information is used for representing the coding capability and the push flow capability of the push flow end;
the instruction sending module is used for responding to the stream pushing information to indicate that the stream pushing end meets a preset stream pushing condition, and sending a stream pushing instruction to the stream pushing end, wherein the stream pushing instruction is used for indicating the stream pushing end to push a second live video stream, and the second live video stream is obtained by coding in a second coding mode;
and the second receiving module is used for receiving the second live video stream pushed by the stream pushing end.
11. A terminal, characterized in that the terminal comprises a processor and a memory; the memory stores at least one instruction for execution by the processor to implement a live streaming method of any of claims 1-5.
12. A server, comprising a processor and a memory; the memory stores at least one instruction for execution by the processor to implement a live streaming method of any of claims 6-8.
13. A computer-readable storage medium having stored thereon at least one instruction for execution by a processor to implement a live streaming method of any of claims 1-5 or to implement a live streaming method of any of claims 6-8.
CN202010339447.7A 2020-04-26 2020-04-26 Live broadcast stream pushing method, device, terminal, server and storage medium Pending CN111405312A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010339447.7A CN111405312A (en) 2020-04-26 2020-04-26 Live broadcast stream pushing method, device, terminal, server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010339447.7A CN111405312A (en) 2020-04-26 2020-04-26 Live broadcast stream pushing method, device, terminal, server and storage medium

Publications (1)

Publication Number Publication Date
CN111405312A true CN111405312A (en) 2020-07-10

Family

ID=71429835

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010339447.7A Pending CN111405312A (en) 2020-04-26 2020-04-26 Live broadcast stream pushing method, device, terminal, server and storage medium

Country Status (1)

Country Link
CN (1) CN111405312A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112738445A (en) * 2020-12-23 2021-04-30 北京大米科技有限公司 Multi-line plug flow method and device, storage medium and electronic equipment
CN113905249A (en) * 2021-09-30 2022-01-07 上海哔哩哔哩科技有限公司 Plug flow abnormity detection method and device
CN114390363A (en) * 2021-12-22 2022-04-22 广州方硅信息技术有限公司 Adaptation method, device, system and storage medium of encoder
CN114760528A (en) * 2022-04-19 2022-07-15 广州方硅信息技术有限公司 Method, system, device, computer equipment and medium for video data transmission
CN115499673A (en) * 2022-08-30 2022-12-20 深圳市思为软件技术有限公司 Live broadcast method and device

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101047853A (en) * 2006-03-30 2007-10-03 株式会社东芝 Server apparatus and video transmission method
CN106162214A (en) * 2016-09-05 2016-11-23 腾讯科技（深圳）有限公司 Method for video coding and net cast client
CN107027045A (en) * 2017-04-11 2017-08-08 广州华多网络科技有限公司 Pushing video streaming control method, device and video flowing instructor in broadcasting end
CN108848390A (en) * 2018-05-28 2018-11-20 北京文香信息技术有限公司 A kind of multithread live streaming copic viewing system
CN108966008A (en) * 2018-08-02 2018-12-07 腾讯科技（深圳）有限公司 Live video back method and device
CN109068157A (en) * 2018-08-21 2018-12-21 北京潘达互娱科技有限公司 Method of adjustment, device and the server of plug-flow parameter in a kind of live streaming
CN109361893A (en) * 2018-11-21 2019-02-19 华平信息技术股份有限公司 Video conference is gone the same way multi-source live broadcasting method and system
CN109462773A (en) * 2018-08-31 2019-03-12 北京潘达互娱科技有限公司 A kind of plug-flow method, apparatus, electronic equipment and storage medium
CN109688425A (en) * 2019-01-11 2019-04-26 北京三体云联科技有限公司 Live data plug-flow method
CN109788303A (en) * 2019-01-28 2019-05-21 广州酷狗计算机科技有限公司 Method for pushing, device, electronic equipment and the storage medium of live video stream
US20190158889A1 (en) * 2016-09-18 2019-05-23 Tencent Technology (Shenzhen) Company Limited Live streaming method and system, server, and storage medium
CN110636327A (en) * 2019-10-28 2019-12-31 成都超有爱科技有限公司 Video caching method and device, electronic equipment and storage medium
CN111064973A (en) * 2019-11-28 2020-04-24 湖北工业大学 Live broadcast system based on IPV9

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101047853A (en) * 2006-03-30 2007-10-03 株式会社东芝 Server apparatus and video transmission method
CN106162214A (en) * 2016-09-05 2016-11-23 腾讯科技（深圳）有限公司 Method for video coding and net cast client
US20190158889A1 (en) * 2016-09-18 2019-05-23 Tencent Technology (Shenzhen) Company Limited Live streaming method and system, server, and storage medium
CN107027045A (en) * 2017-04-11 2017-08-08 广州华多网络科技有限公司 Pushing video streaming control method, device and video flowing instructor in broadcasting end
CN108848390A (en) * 2018-05-28 2018-11-20 北京文香信息技术有限公司 A kind of multithread live streaming copic viewing system
CN108966008A (en) * 2018-08-02 2018-12-07 腾讯科技（深圳）有限公司 Live video back method and device
CN109068157A (en) * 2018-08-21 2018-12-21 北京潘达互娱科技有限公司 Method of adjustment, device and the server of plug-flow parameter in a kind of live streaming
CN109462773A (en) * 2018-08-31 2019-03-12 北京潘达互娱科技有限公司 A kind of plug-flow method, apparatus, electronic equipment and storage medium
CN109361893A (en) * 2018-11-21 2019-02-19 华平信息技术股份有限公司 Video conference is gone the same way multi-source live broadcasting method and system
CN109688425A (en) * 2019-01-11 2019-04-26 北京三体云联科技有限公司 Live data plug-flow method
CN109788303A (en) * 2019-01-28 2019-05-21 广州酷狗计算机科技有限公司 Method for pushing, device, electronic equipment and the storage medium of live video stream
CN110636327A (en) * 2019-10-28 2019-12-31 成都超有爱科技有限公司 Video caching method and device, electronic equipment and storage medium
CN111064973A (en) * 2019-11-28 2020-04-24 湖北工业大学 Live broadcast system based on IPV9

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LIVEVIDEOSTACK: "《H.265在花椒直播中的应用与优化》", 《云+社区-腾讯云》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112738445A (en) * 2020-12-23 2021-04-30 北京大米科技有限公司 Multi-line plug flow method and device, storage medium and electronic equipment
CN112738445B (en) * 2020-12-23 2023-06-02 北京大米科技有限公司 Multi-line plug flow method and device, storage medium and electronic equipment
CN113905249A (en) * 2021-09-30 2022-01-07 上海哔哩哔哩科技有限公司 Plug flow abnormity detection method and device
CN114390363A (en) * 2021-12-22 2022-04-22 广州方硅信息技术有限公司 Adaptation method, device, system and storage medium of encoder
CN114760528A (en) * 2022-04-19 2022-07-15 广州方硅信息技术有限公司 Method, system, device, computer equipment and medium for video data transmission
CN115499673A (en) * 2022-08-30 2022-12-20 深圳市思为软件技术有限公司 Live broadcast method and device
CN115499673B (en) * 2022-08-30 2023-10-20 深圳市思为软件技术有限公司 Live broadcast method and device

Similar Documents

Publication Publication Date Title
CN111405312A (en) Live broadcast stream pushing method, device, terminal, server and storage medium
CN110881136B (en) Video frame rate control method and device, computer equipment and storage medium
CN110213616B (en) Video providing method, video obtaining method, video providing device, video obtaining device and video providing equipment
CN108495142B (en) Video coding method and device
CN108966008B (en) Live video playback method and device
CN109788303B (en) Live video stream pushing method and device, electronic equipment and storage medium
CN110022373B (en) Service distribution method, device, server and storage medium
CN111417006A (en) Video screen projection method, device, terminal and storage medium
CN111586431B (en) Method, device and equipment for live broadcast processing and storage medium
CN108769726B (en) Multimedia data pushing method and device, storage medium and equipment
CN111107389B (en) Method, device and system for determining live broadcast watching time length
CN110996117B (en) Video transcoding method and device, electronic equipment and storage medium
CN110856019A (en) Code rate allocation method, device, terminal and storage medium
CN112714327A (en) Interaction method, device and equipment based on live application program and storage medium
KR20140092517A (en) Compressing Method of image data for camera and Electronic Device supporting the same
CN111669640B (en) Virtual article transfer special effect display method, device, terminal and storage medium
CN111010588B (en) Live broadcast processing method and device, storage medium and equipment
KR101311463B1 (en) remote video transmission system
CN113141541B (en) Code rate switching method, device, equipment and storage medium
CN111586433B (en) Code rate adjusting method, device, equipment and storage medium
CN111050211B (en) Video processing method, device and storage medium
CN110113669B (en) Method and device for acquiring video data, electronic equipment and storage medium
CN109714628B (en) Method, device, equipment, storage medium and system for playing audio and video
CN111478915A (en) Live broadcast data stream pushing method and device, terminal and storage medium
CN111698262B (en) Bandwidth determination method, device, terminal and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200710

RJ01 Rejection of invention patent application after publication