CN109587551B

CN109587551B - Method, device, equipment and storage medium for judging live broadcast blockage of streaming media

Info

Publication number: CN109587551B
Application number: CN201710909775.4A
Authority: CN
Inventors: 武爱敏; 王文博
Original assignee: Beijing Kingsoft Cloud Network Technology Co Ltd; Beijing Kingsoft Cloud Technology Co Ltd
Current assignee: Beijing Kingsoft Cloud Network Technology Co Ltd; Beijing Kingsoft Cloud Technology Co Ltd
Priority date: 2017-09-29
Filing date: 2017-09-29
Publication date: 2020-12-25
Anticipated expiration: 2037-09-29
Also published as: CN109587551A

Abstract

The embodiment of the invention provides a method, a device, equipment and a storage medium for judging whether live streaming media are jammed, relates to the technical field of communication, and solves the problem that in the prior art, whether the live streaming media are jammed or not is not accurately judged. The method for judging the live video pause of the streaming media is applied to equipment for receiving the streaming media in the live video process, and the method comprises the following steps: acquiring streaming media audio data in streaming media data with a preset time length, and calculating the playing time of the acquired streaming media audio data; comparing the calculated playing time with the preset time length to obtain a comparison result; and judging whether the live streaming media is blocked according to the comparison result so as to accurately judge whether the live streaming media is blocked.

Description

Method, device, equipment and storage medium for judging live broadcast blockage of streaming media

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a method, an apparatus, a device, and a storage medium for determining live broadcast stuck of a streaming media.

Background

The current streaming media refers to a media format played on the Internet by a streaming transmission mode. Streaming media is also called streaming media, which means that a merchant sends programs as data packets to a network by using a video transmission server. After the user decompresses the data through the decompression device, the program is displayed as before transmission.

Generally, live streaming media passes through many links from a signal acquisition end to a decoding receiving end, and since the characteristics of real-time transmission of the streaming media have very high requirements on network quality, the problems of watching experience influenced by katton and the like are frequently encountered, the live-streaming katton condition is necessarily checked and analyzed by some effective means or methods.

The current widely applied katon judgment methods mainly comprise the following two methods:

1. analyzing the change situation of the video compression code rate in the live data;

acquiring video live broadcast data of streaming media;

compressing the video live broadcast data by a CBR (Constant bit rate) compression mode;

transmitting the compressed video live broadcast data, and comparing the difference value between the video bit rate of the current time point and the video bit rate corresponding to the preset time point; if the difference value is larger than a first preset threshold value set according to the requirements of the user, the transmission speed is insufficient, and the video live broadcast data is judged to be blocked; if the difference value is smaller than a first preset threshold value set according to the user requirement, the transmission speed is enough, and the video live broadcast data is judged not to be blocked. The preset time point is before the current time.

2. By analyzing the video frame rate change situation in the live data:

based on the technical basis of video coding, the encoder writes information such as set theoretical code rate, theoretical frame rate and the like into a media header, and the information is used for helping a decoder to correctly identify compressed data;

the decoder obtains a theoretical code rate and a theoretical frame rate set by the streaming media when the live video data is compressed and coded by reading the information in the media header, wherein the theoretical frame rate represents how many compressed video images are per second;

the decoder acquires the actual frame rate of receiving and decoding every second, compares the difference value between the value of the theoretical frame rate and the actual frame rate of receiving and decoding, and judges that the live video data is blocked if the difference value is larger than a second preset threshold value set according to the requirements of a user and indicates that the data is lost; and when the difference value is smaller than a second preset threshold value set according to the user requirement, if the data is enough transmitted, judging that the video live broadcast data is not blocked, wherein the actual receiving and decoding frame rate is the number of the video images actually received and decoded per second.

In the two types of stuck judgment methods in the prior art, whether a live video is stuck or not is judged according to video information of a video in a streaming media, such as a video bit rate or a frame rate. However, in the actual network transmission process of the live video data, the network may have a unstable state, and in order to ensure that the live video data can be transmitted, the network actively discards part of the video frames in the actual processing, so that the video actively discarded by the network is not actually jammed, but by using the two jam determination methods, it may be determined that the live video data is jammed, and therefore, the prior art has a problem that it is inaccurate to determine whether the live video data is jammed.

Disclosure of Invention

The embodiment of the invention aims to provide a method, a device, equipment and a storage medium for judging the pause of live streaming media so as to accurately judge whether the live streaming media is paused. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a method for determining a live video pause of a streaming media, where the method is applied to a device that receives the streaming media in a live video process, and the method includes:

acquiring streaming media audio data in streaming media data with a preset time length, and calculating the playing time of the acquired streaming media audio data;

comparing the calculated playing time with the preset time length to obtain a comparison result;

and judging whether the live streaming media is blocked or not according to the comparison result.

Optionally, the step of obtaining streaming media audio data in the streaming media data with the preset time length and calculating the playing time of the obtained streaming media audio data includes:

acquiring streaming media audio data in the streaming media data with the preset time length;

recording a first time stamp in a first audio data packet of the preset time length;

recording a second time stamp in the last audio data packet of the preset time length;

and calculating the absolute value of the difference between the second time stamp in the last audio data packet and the first time stamp in the first audio data packet to obtain the playing time length of the acquired streaming media audio data.

Optionally, the step of determining whether the live streaming media is stuck according to the comparison result includes:

when the comparison result shows that the calculated playing time length is equal to the preset time length,

and determining that the live streaming media is not blocked.

when the comparison result shows that the calculated playing time length is less than or greater than the preset time length,

and judging whether the data volume of the streaming media audio data meets a preset condition and judging whether the live streaming media is blocked, wherein the preset condition shows the comparison relationship between the data volume of the streaming media audio data and the data volume of the data buffer area.

Optionally, when the comparison result shows that the calculated playing time length is greater than the preset time length,

the step of judging whether the data volume of the streaming media audio data meets the preset condition and whether the live streaming media is blocked comprises the following steps: judging whether the data volume of the acquired streaming media audio data of the preset time length is larger than the data volume which can be accommodated by the data buffer area,

if the data volume of the acquired streaming media audio data of the preset time length is larger than the data volume which can be accommodated in the data buffer area, judging that the live streaming media is blocked;

and if the data volume of the acquired streaming media audio data of the preset time length is less than or equal to the data volume capable of being accommodated in the data buffer area, judging that the live streaming media is not blocked.

Optionally, if it is determined that the live streaming media is blocked, the method further includes:

and determining the data volume overflowing the data buffer, and deleting the overflowed data volume.

Optionally, when the comparison result shows that the calculated playing time length is less than the preset time length,

the step of judging whether the data volume of the streaming media audio data meets the preset condition and whether the live streaming media is blocked comprises the following steps: judging whether the data amount buffered in the data buffer area is zero or not and whether the data amount of the acquired streaming media audio data is zero or not,

if the data volume cached in the data buffer area is zero and the data volume of the acquired streaming media audio data is zero, judging that the live streaming media is blocked;

and if the data volume cached in the data buffer area is larger than zero and the data volume of the acquired streaming media audio data is larger than zero, judging that the live streaming media is not blocked.

In a second aspect, an embodiment of the present invention further provides a device for determining whether a live streaming media is blocked, where the device is applied to an apparatus that receives a streaming media in a live streaming media process, and the device includes:

the acquisition module is used for acquiring streaming media audio data in streaming media data with a preset time length and calculating the playing time of the acquired streaming media audio data;

the comparison obtaining module is used for comparing the calculated playing time length with the preset time length to obtain a comparison result;

and the judging module is used for judging whether the live streaming media is blocked or not according to the comparison result.

Optionally, the first obtaining module is configured to obtain streaming media audio data in the streaming media data with the preset time length;

the first recording module is used for recording a first time stamp in the first audio data packet with the preset time length;

the second recording module is used for recording a second time stamp in the last audio data packet with the preset time length;

and the calculation obtaining module is used for calculating the absolute value of the difference between the second time stamp in the last audio data packet and the first time stamp in the first audio data packet to obtain the playing time length of the acquired streaming media audio data.

Optionally, the first determining sub-module is configured to determine that the live streaming media is not paused when the comparison result indicates that the calculated playing time length is equal to the preset time length.

Optionally, the second determining submodule is configured to determine whether the data size of the streaming media audio data meets a preset condition and determine whether the streaming media live broadcast is paused when the comparison result indicates that the calculated playing time is shorter than or longer than the preset time length, where the preset condition indicates a comparison relationship between the data size of the streaming media audio data and the data size of the data buffer.

Optionally, the second judgment sub-module is specifically configured to: when the comparison result shows that the calculated playing time length is longer than the preset time length, judging whether the data volume of the acquired streaming media audio data of the preset time length is larger than the data volume which can be accommodated by the data buffer area or not,

Optionally, the deleting module is configured to determine an amount of data overflowing the data buffer, and delete the overflowing amount of data.

Optionally, the second judgment sub-module is specifically configured to: when the comparison result shows that the calculated playing time length is less than the preset time length, judging whether the data amount cached in the data buffer area is zero or not and whether the data amount of the acquired streaming media audio data is zero or not,

In a third aspect, an embodiment of the present invention provides an apparatus for receiving streaming media in a live streaming media process, including a processor, a memory, and a storage, where the memory is used to store a computer program;

a processor, configured to implement the steps of the first aspect when executing the program stored in the memory.

In a fourth aspect, an embodiment of the present invention further provides an apparatus for receiving streaming media in a live streaming media process, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the bus,

a memory for storing a computer program;

In a fifth aspect, the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program, when executed by a processor, implements the steps of the first aspect.

In a sixth aspect, embodiments of the present invention provide a computer program product containing instructions, which when executed on a computer, cause the computer to perform the steps described above in the first aspect.

In a seventh aspect, an embodiment of the present invention provides a computer program, which, when running on a computer, causes the computer to perform the above steps of the first aspect.

According to the method, the device, the equipment and the storage medium for judging the pause of the live streaming media, which are provided by the embodiment of the invention, the live streaming media is judged whether to be paused or not by acquiring the streaming media audio data in the streaming media data with the preset time length, calculating the playing time length of the acquired streaming media audio data, and comparing the playing time length of the streaming media audio data with the preset time length to obtain a comparison result. Therefore, the misjudgment of the video data in the streaming media for detecting the pause is eliminated, and whether the streaming media is paused or not can be accurately judged. The method is based on subsequent adjustment of the video fluency, and improves the user experience effect after the video fluency is effectively adjusted in the later period. Of course, it is not necessary for any product or method that implements an embodiment of the invention to achieve all of the advantages described above at the same time.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a first flowchart of a method for determining a live media pause according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart illustrating the step 101 in FIG. 1 according to an embodiment of the present invention;

fig. 3 is a second flowchart of a method for determining live media pause according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a device for determining live media pause according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a device for receiving streaming media in a live streaming media process according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without any creative effort belong to the protection scope of the embodiments of the present invention.

In the prior art, whether the video live broadcast is blocked or not is judged by the video information of the video in the streaming media data, such as the video bit rate or the frame rate. In the actual process of transmitting the live video data, probably because the network has a less stable state, the network actually and actively discards part of the video so as to ensure that the live video data can be transmitted.

In the prior art, video information of video in streaming media data is only processed, and whether live video is blocked or not is judged. In the prior art, the streaming media audio data in the streaming media data is not processed or compressed, and whether the live video is blocked or not is judged, so that the inventor also considers that the blocking of the video data does not affect the streaming media audio data in the transmission process of the streaming media data on the basis of not performing any processing on the streaming media audio data. That is, in the actual transmission process of the streaming media data, the video data is played according to the playing requirement of the streaming media data, the streaming media audio data can be played according to the playing requirement of the streaming media data, and the transmission of the streaming media can be completed as long as the synchronous transmission of the video data and the streaming media audio data in the streaming media data is realized. The playing requirement of the player is determined by the actual transmission requirement of the network and the playing capability of the streaming media playing end. For example, the actual transmission requirements include: video code rate; the playing capability includes: network bandwidth, video definition, and network throughput.

In the actual playing process of the streaming media audio data, once interruption or error occurs, the fact that the live streaming media is blocked can be accurately judged. Based on the consideration of the inventor, the embodiments of the present invention provide a method, an apparatus, a device and a storage medium for determining the live streaming pause in order to solve the problem of the prior art that the live streaming pause is erroneously determined, and the live streaming pause is determined by using the streaming media audio data in the streaming media data, so that the live streaming pause is accurately determined, a good foundation is made for improving the pause in the later period, the smoothness of live streaming is improved, and the user experience effect is further improved.

First, a method for determining live media pause in a streaming media provided in an embodiment of the present invention is described below.

It should be noted that the method for determining live media pause provided by the embodiment of the present invention may also be applied to a device that receives a streaming media in a live media live broadcast process. The equipment may specifically be: desktop computers, laptop computers, intelligent mobile terminals, servers, and the like. The present invention is not limited to this, and any device that can implement the embodiment of the present invention belongs to the protection scope of the embodiment of the present invention.

As shown in fig. 1, a method for determining whether a live streaming media is blocked according to an embodiment of the present invention is applied to a device that receives a streaming media in a live streaming media process, and may include the following steps:

the device for sending the streaming media in the live streaming media process is simply referred to as a device in the subsequent introduction of the embodiment of the present invention.

Step 101, acquiring streaming media audio data in streaming media data with a preset time length, and calculating a playing time of the acquired streaming media audio data.

Streaming media data refers to streaming media data that is compressed after a series of media data is compressed and sent through network segments. The streaming media data is transmitted to the video and audio on the network in real time for users to watch. The streaming audio data in the streaming media data refers to sound in a frequency range that can be heard by human ears.

The preset time length can be set according to the requirements of users. The preset time length is the operating system time of the equipment. Generally, the shortest preset time length is 1 second, and the duration can be increased on the basis of the 1 second, in the embodiment of the present invention, the upper limit of the preset time length is not limited. Any preset time length is set under the condition of meeting the bandwidth requirement, and the method and the device belong to the protection scope of the embodiment of the invention. The bandwidth requirement may refer to a transmission bandwidth less than or equal to the propagated streaming media data. This bandwidth requirement is used to guarantee the proper transmission of streaming media data. For example, when the preset time length is set to 1 second, the 1 second may refer to 1 second between the system time stamp of 14 o 'clock 01 min 05 s on 26 days monday 6/7 in 2017 and the system time stamp of 14 o' clock 01 min 06 s on 26 days monday 26/6 in 2017. According to the embodiment of the invention, the smaller the preset time length is, the more accurate whether the live streaming media is blocked or not is judged.

The system timestamp refers to adding time information, such as the time of year, month, day, and time, to a series of data, which may be streaming media data.

Optionally, in a specific implementation, here, the step 101 includes: and according to a preset period, acquiring streaming media audio data in the streaming media data with a preset time length, and calculating the playing time of the acquired streaming media audio data. The preset period is greater than or equal to the preset time length, so that the equipment is ensured to acquire the streaming media audio data in all the streaming media data in the preset period, and the calculation of the playing time length of the streaming media audio data is completed.

The streaming audio data may include time information of each streaming audio data packet, a data amount of each streaming audio data packet, and a total data amount of the streaming audio data packets. The time information may refer to a timestamp, or may refer to a time length corresponding to the normal transmission time of the data packet, and the time length corresponding to the normal transmission time of the data packet may be obtained through experiments, or may be set according to user needs. Any time information capable of obtaining the playing time length of the streaming media audio data belongs to the protection scope of the embodiments of the present invention, which is not exemplified herein.

Step 102, comparing the calculated playing time with the preset time length to obtain a comparison result.

And 103, judging whether the live streaming media is blocked or not according to the comparison result.

The streaming media live broadcast refers to a transmission mode in which program contents broadcasted through media such as a broadcast network, the internet and the like are synchronously transmitted with a live event. The mortgage is that when data is transmitted according to a fixed frequency or speed, the fixed frequency or speed is reduced, and the unsmooth viewing experience is caused.

The method for judging the pause of the live streaming media provided by the embodiment of the invention comprises the steps of obtaining the streaming media audio data in the streaming media data with the preset time length, calculating the playing time length of the obtained streaming media audio data, and comparing the playing time length of the streaming media audio data with the preset time length to obtain a comparison result, thereby judging whether the live streaming media is paused. Therefore, the misjudgment of the video data in the streaming media for detecting the pause is eliminated, and whether the streaming media is paused or not can be accurately judged. The method is based on subsequent adjustment of the video fluency, and improves the user experience effect after the video fluency is effectively adjusted in the later period.

In order to determine the playing time length of the streaming media audio data more quickly, as shown in fig. 2, optionally, in a specific implementation, the step 101 includes the following steps:

step 201, acquiring streaming media audio data in the streaming media data with the preset time length.

The data transmission is carried out through data packets in the transmission process of the streaming media data, and the steps for acquiring the streaming media audio data are as follows:

the method comprises the steps of applying a device for sending the streaming media in the live streaming media process, obtaining streaming media data through network or local storage, and extracting streaming media audio data and streaming media video data in the streaming media data.

According to the preset sampling rate set by a user, the streaming media audio data is sampled and encoded by an encoder according to the preset sampling frequency to obtain the audio frame rate of the streaming media audio data per second, the time length required for decoding by a decoder is calculated by using the audio frame rate, the encoder periodically packs a streaming media audio data packet by taking the time length as a packing period, and writes the time length in the streaming media audio data packet to ensure that each streaming media audio data packet is encoded according to the audio frame rate when the encoder encodes the data.

After the decoder receives the coded streaming media data, the decoder will decode the streaming media video data packet by referring to the time length. Under normal conditions, the time difference of every two consecutive streaming media data packets should be within the duration, and no large-amplitude jump occurs.

The device for receiving the streaming media in the live streaming media process is applied to read the streaming media data packet, and the streaming media audio data packet, the preset sampling rate and the duration in the streaming media data can be obtained. This ensures that the device receiving the streaming media data packet can decode the received streaming media data.

For example, the preset sampling rate may be encoded and compressed by using AAC (Advanced Audio Coding), and the preset sampling rate is set to 44.1 kHZ.

The AAC format packs the samples during compression. The packing mode is to pack one packet for every 1024 sampling points. Each packet is referred to as an AAC audio frame. Thus, packing 44100 samples in an up-packing manner produces 43 audio frames (i.e., 44100 divided by 1024 equals about 43), i.e., an audio frame rate of 43 fps. Using this audio frame rate, the decoding duration required for each audio frame can be calculated to be approximately 23 milliseconds (i.e., 1000 divided by 43 approximately 23). Of course, only the audio coding and decoding technique of AAC is taken as an example for description here, other MP3 audio coding and decoding techniques are also applicable, and any audio coding and decoding technique capable of implementing the embodiment of the present invention belongs to the protection scope of the embodiment of the present invention, which is not exemplified here.

The frame rate of the audio is the number of audio packets packed per second, which is a fixed number, for example, 1024, in the process of collecting and converting sound from analog to digital.

The format of AAC can provide better sound quality and higher compression rate. One of the most important references to measure audio quality is the sampling rate, i.e., the number of samples of digitized information acquisition per second for the amplitude of the sound wave during the sound from analog waveform to digital. The highest sampling rate that can be heard by the human ear is 20kHz, and in order to avoid sound distortion caused by compression, transmission and other processes, the audio sampling rate is 44.1kHz, that is, 44100 samples per second.

Step 202, record the first time stamp in the first audio data packet of the preset time length.

The first audio data packet is mainly composed of a destination IP (Internet Protocol) address, a source IP address, payload data, and an identification portion. The first audio data packet comprises a packet header and a packet body, wherein the packet header is of a fixed length, the length of the packet body is not fixed, the lengths of all the field sections are fixed, and the packet header structures of the request data packet and the response data packet of the two parties are consistent.

The source IP address refers to a sending device address of the first audio data packet, and the sending address of the first audio data packet in the embodiment of the present invention may be a signal acquisition encoding end. The destination IP address mentioned above refers to the receiving device address of the first audio packet. The payload data may be streaming audio data, and the identification part may be an identification of a total data amount of the data, or an identification of a data amount of each data packet.

The first audio data packet and the timestamp corresponding to the first audio data packet may be recorded in a local hard disk, or may be recorded in a network virtual disk in which a network connection is established in advance, and any position where recording of the audio data packet and the timestamp corresponding to the audio data packet can be achieved belongs to the protection scope of the embodiment of the present invention. Through recording the time stamp, the later-stage user can directly use the time stamp or the equipment reads the time stamp at the recorded position for direct use, and the running memory is occupied without acquiring the time stamp in real time, so that the burden of running memory and bandwidth is reduced, and the efficiency of data processing is improved.

The first audio data packet and the timestamp corresponding to the first audio data packet can be recorded in the temporary cache, so that the used timestamp can be deleted after the preset time length is reached, and the pressure of a storage space is reduced. Any way of recording the timestamp is within the scope of the embodiments of the present invention, and is not illustrated here.

Recording a time stamp in a first audio data packet as a first time stamp DS in the first audio data packet, wherein the first time stamp DS in the first audio data packet comprises: and the encoder performs sampling coding on the streaming media audio data according to a preset sampling rate, and writes the starting time stamp of the duration into the streaming media audio data packet.

And step 203, recording a second time stamp in the last audio data packet of the preset time length.

The last audio data packet is mainly composed of a destination IP (Internet Protocol) address, a source IP address, payload data, and an identification portion. The last audio data packet comprises a packet header and a packet body, wherein the packet header is of a fixed length, the length of the packet body is not fixed, the lengths of all the field sections are fixed, and the packet header structures of the request data packet and the response data packet of the two parties are consistent.

The source IP address refers to a sending device address of the last audio data packet, and the sending address of the last audio data packet in the embodiment of the present invention may be a signal acquisition encoding end. The destination IP address mentioned above refers to the receiving device address of the last audio data packet. The payload data may be streaming audio data, and the identification part may be an identification of a total data amount of the data, or an identification of a data amount of each data packet.

The last audio data packet and the timestamp corresponding to the last audio data packet may also be recorded in a local hard disk, may also be recorded in a network virtual disk in which a network connection is established in advance, and may also be recorded in a temporary cache. Any position capable of recording the audio data packet and the timestamp corresponding to the audio packet is within the protection scope of the embodiment of the present invention.

Recording the time stamp in the last audio data packet as the second time stamp DE in the last audio data packet, the second time stamp DE in the last audio data packet comprising: and the encoder performs sampling coding on the streaming media audio data according to a preset sampling rate, and writes the end timestamp of the duration into the streaming media audio data packet.

Step 204, calculating an absolute value of a difference between a second time stamp in the last audio data packet and a first time stamp in the first audio data packet, and obtaining a playing time length of the obtained streaming media audio data.

And obtaining the playing time length of the streaming media data by using the obtained playing time length of the streaming media audio data.

The absolute value calculated here is used as the playing time length of the acquired streaming media audio data, so as to obtain the playing time length of the streaming media data. When the playing of the streaming media data is stable, that is, under the normal playing condition, the difference between the first timestamp DS and the second timestamp DE is equal to the time length written in the streaming media audio data packet when the encoder performs sampling encoding on the streaming media audio data according to the preset sampling rate, that is, the playing time length of the streaming media data. Compared with the video frame rates of 30fps, 25fps, 20fps, 15fps and the like of the current common streaming media data, the lower the video frame rate is, the poorer the smoothness degree of continuous pictures is, but generally not lower than 15 frames per second, human eyes can consider smooth pictures, and in some application scenes, a coding mode or a transmission mode for actively discarding video frames is also available, and the lower video frame rate only brings about a picture effect which is not smooth for viewers. However, as long as the sound is continuous, the pause phenomenon and the feeling of the short interruption of the live broadcast do not exist, and the audio frame is different from the video frame, and as long as the audio frame is lost or abnormal, the viewer can easily perceive the pause, slow playing and discontinuity of the character conversation or music.

By applying the embodiment of the invention, when the first audio data packet and the last audio data packet are received, the corresponding first time stamp and the second time stamp are respectively recorded, and the difference value between the first time stamp and the second time stamp can be quickly utilized to determine the playing time length of the acquired streaming media audio data. The most basic audio data in the streaming media data are compared before and after an audio coding time stamp, the audio frame rate of audio coding and the difference between the audio frame rate and the audio coding time stamp are used as important judgment reference indexes for judging whether the video is blocked or not in the live streaming media process, the loss of the audio frame can be accurately known, audiences can more easily perceive the pause, slow playing and discontinuity of character conversation or music, and equipment can also conveniently detect the blocking.

In order to more accurately determine whether the playing time length is correct, the method for determining live broadcast pause of streaming media may further include, after step 202: when a first audio data packet is received, an operating System timestamp SS (System _ Time _ Start) corresponding to the first audio data packet is recorded, and

after the step 203, the method may further include: when the last audio data packet is received, recording an operating System timestamp SE (System _ Time _ End, System End Time) corresponding to the last audio data packet;

and when the difference value of the operating system timestamp is the same as the absolute value of the DE-DS difference value, the streaming media audio data packet is stably transmitted and the playing time length is judged to be correct. The streaming media frequency data is transmitted without interruption or delay in the transmission process.

In the process of playing the streaming media, the live streaming media may be jammed, and the live streaming media may not be jammed, so as to accurately judge whether the live streaming media is jammed. Optionally, in a specific implementation manner, step 103 may include: and when the comparison result shows that the calculated playing time length is equal to the preset time length, judging that the live streaming media is not blocked.

The comparison result shows that the calculated playing time length is equal to the preset time length, and it can be known that the preset time length allows the device to correctly receive the streaming media audio data with the equal time length, that is, the problem of streaming media audio data loss does not occur in the streaming media live broadcast process, and at this time, the data amount cached in the data buffer area is the data amount corresponding to the streaming media audio data with the preset time length, and it is determined that the streaming media live broadcast is not blocked.

For example, it is assumed that a first time stamp DS in a first audio data packet in the acquired streaming media audio data is 14 o 'clock 01/07 s at 26 days monday in 2017 and month 5, a second time stamp DE in a last audio data packet is 09 o' clock 01/14 s at 26 days monday in 2017 and month 26, an absolute value of a difference obtained by subtracting the first time stamp DS in the first audio data packet from the second time stamp DE in the last audio data packet is 2 s, and the 2 s are taken as the play duration.

Assuming that the preset time length is 2 seconds, the system timestamp received by the device in the first audio data packet is 14 monday 14 point 01 minute 08 seconds at day 5 month 26 year 2017, and the system timestamp exactly at 2 seconds is 14 monday 14 point 01 minute 10 seconds at 26 monday 26 month 2017, the device receives the last audio data packet SE, that is, the system timestamp in the last audio data packet is used for subtracting the system timestamp in the first audio data packet to obtain an absolute value of a difference value of 2 seconds, and the 2 seconds are taken as the preset time length, so that the preset time length is equal to the playing time length, that the live broadcast of the streaming media is not blocked is judged, and the audio and video data of the streaming media are not interrupted or transmitted in a delayed manner in the transmission process.

Compared with the conventional video coding technology, the video coding technology is changed from a CBR (constant Bit Rate) technology with poor video quality of coded, compressed and output to a VBR (Variable Bit Rate) technology with better video quality of coded, compressed and output, that is, the video Bit Rate in unit time changes in an up-and-down floating manner along with the change frequency of video content and the complexity of information such as color. Although the overall average bit rate is substantially constant, the immediate problem is that the value of the video bit rate per unit time fluctuates greatly, and a range of variation of 2 to 3 times or more is possible. At this moment, a second preset threshold value set according to user requirements is used as a reference value, and the second preset threshold value is compared with an actual video bit rate, so that large deviation may exist, and due to the fact that the fluctuation of video bit rate values in unit time is large, the second preset threshold value is difficult to set accurately and is used as the reference value, the video bit rate of the whole streaming media live broadcast process is compared and analyzed, and due to the fact that the network actively discards videos, the videos are not actually stuck, the situation can not be identified by the dynamic bit rate algorithm, the videos actively discarded by the network can be possibly determined, and the fact that the streaming media live broadcast is stuck is judged.

By applying the embodiment, when the playing time length of the streaming media audio data is equal to the preset time length, the fact that the live streaming media is not blocked is judged, video frames do not need to be identified, and a second preset value does not need to be set as a reference value, so that the video bit rate in the whole live broadcasting process is compared and analyzed, the situation that videos actively discarded by a network are not generated, the live streaming media is erroneously judged to be blocked, and the accuracy of judging that the live streaming media is not blocked is improved.

In order to accurately determine that the live streaming media is not blocked, in a specific implementation manner, the step 103 may include the following steps:

and when the comparison result shows that the calculated playing time length is less than or greater than the preset time length, judging whether the data volume of the streaming media audio data meets a preset condition and judging whether the live streaming media is blocked, wherein the preset condition shows the comparison relationship between the data volume of the streaming media audio data and the data volume of the data buffer area.

And when the comparison result shows that the calculated playing time length is less than the preset time length, the device is known to have received the streaming media audio data when the preset time length is not reached. Indicating that theoretically and practically the same preset length of time is used, the device receives less audio frame data than the theoretical value.

For example, it is assumed that a first time stamp DS in a first audio data packet in the acquired streaming media audio data is 02 minutes 07 seconds at 14 o 'clock at 26 days monday at 2017 and month 5, a second time stamp DE in a last audio data packet is 02 minutes 08 seconds at 14 o' clock at 26 days monday at 2017 and month 26, an absolute value of a difference obtained by subtracting the first time stamp DS in the first audio data packet from the second time stamp DE in the last audio data packet is 1 second, and the 1 second is taken as the play duration.

Assuming that the preset time length is 2 seconds, the system timestamp of the first audio data packet received by the device is 02 minutes 08 seconds at 14 o ' clock of 26 days monday at 2017 and 5 months, 02 minutes 09 seconds at 14 o ' clock of 26 days monday at 2017, that is, the last audio data packet SE is received by the device in 1 second, while the preset time length is 2 seconds, the audio data packet has been received in 1 second, the audio data packet has not been received in the remaining 1 second, that is, the audio data packet received by the device is zero at 02 minutes 10 seconds at 14 o ' clock of 26 days monday at 2017 and 26 months.

A device will typically set a data buffer, and the amount of data in the data buffer typically includes: the data amount that can be accommodated by the data buffer area can be defined as CL (Cache _ Length, the data amount that can be accommodated by the data buffer area) and is used for accommodating quantitative data. For example, the data buffer can hold data with a duration of 2 seconds, and CL is TI × 2, where TI is a preset duration.

The data amount that can be accommodated by the data buffer here includes a lower limit value of the data amount, an upper limit value of the data amount, and a data legal range in which the data can be cached between the upper limit value and the lower limit value. Optionally, in a specific implementation manner, when the amount of data cached in the data buffer is less than the lower limit of the amount of data, the live streaming media needs to read data for playing, and at this time, because there is no cached data or the amount of data that does not satisfy the played data in the data buffer, and the audio data of the streaming media is not received, it is determined that the live streaming media is stuck.

Optionally, in another specific implementation manner, when the cache data in the data cache region is greater than the upper limit of the data volume, since the data volume in the data cache region cannot be played in time, the data exceeding the upper limit of the data volume may be lost, and at this time, a jam may occur. Therefore, whether the live streaming media is blocked or not can be accurately judged by using the data amount of the data buffer area.

Specifically, network congestion may cause a pause in the live streaming media, as an example. The upper limit value of the data volume that can be held in the data buffer area is the data volume that corresponds to 3 seconds time length, because of network congestion, then probably the equipment received the data volume that streaming media audio data is less than 1 seconds time length and corresponds this moment, originally read the data volume that the 2 seconds time length that plays corresponds each time, and the data volume of buffering in the data buffer area at this moment is just not enough, can only read the data volume that 1 seconds time length corresponds, after the equipment broadcast finishes the data volume that 1 seconds time length corresponds this moment, streaming media broadcast will appear blank, that is to say, the live broadcast of streaming media has appeared stuck.

If the data volume of the streaming media audio data suddenly increases after the network is congested. For example, the data volume of the streaming media audio data is a data volume corresponding to a 5s time length, because the data volume corresponding to the 5s time length is greater than the data volume corresponding to a 3 s time length, the network will select compression by itself, and discard the redundant data volume corresponding to a 2 s time length, so that the data corresponding to the 2 s time length is not cached, the device cannot receive the data volume of the streaming media, and thus a breakpoint occurs in the streaming media playing, that is, a pause occurs in the streaming media live broadcasting.

After the network is recovered, the data volume of the streaming media audio data is played according to the upper limit value of the data volume which is exactly equal to or less than the data buffer zone of the original data buffer zone each time, and the live streaming media broadcast can not be blocked.

Specifically, for example, when the code is written or the encoder encodes, the encoder writes the timestamp of the data packet with an error. It is assumed that the normal time stamp should be set according to the decoding frequency, one data per second. For example, the 1 st second corresponds to a data packet 1, the 2 nd second corresponds to a data packet 2, the 3 rd second corresponds to a data packet 3, the 4 th second corresponds to a data packet 4, and the 5 th second corresponds to a data packet 5;

assume that the preset time duration is set to 5 seconds, i.e. the device can receive the 5 packets within exactly 5 seconds. However, when an error occurs during transmission, the 1 st second corresponds to a data packet 1, the 3 rd second corresponds to a data packet 2, the 5 th second corresponds to a data packet 3, the 7 th second corresponds to a data packet 4, and the 9 th second corresponds to a data packet 5. Thus, when actually transmitted at a later stage, the time stamp of the transmission data becomes 7 seconds. Therefore, the time length of the actual transmission of the network is longer than the time length of the normal timestamp transmission of the data packets, and then the data packets 1, 2 and 3 are sent by the 5 th second, but the data packets 4 and 5 are not sent in time, and the data packets 4 and 5 which are not sent in time within the time length of the normal timestamp transmission are retransmitted after the network is recovered, so that the decoding end cannot decode the data packets 4 and 5 which are not sent out in time within 5 seconds, and the live streaming media is blocked.

The preset condition shows that the comparison relationship between the data volume of the streaming media audio data and the data volume which can be accommodated by the data buffer area is to judge whether the data volume of the streaming media audio data exceeds the upper limit value of the data volume which can be accommodated by the data buffer area or not on the basis of the data volume which can be accommodated by the data buffer area, namely, whether the data overflow relationship of the data volume which can be accommodated by the data buffer area exists or not; by using the relationship, when the comparison result shows that the calculated playing time length is less than or greater than the preset time length, the data overflow of the data amount which can be accommodated in the data buffer area can be judged, and the live broadcast of the streaming media is judged to be blocked. Alternatively, the first and second electrodes may be,

whether the data volume of the streaming media audio data reaches the lower limit value of the data volume which can be accommodated by the data buffer area or not and whether the relation that no data exists in the data buffer area or not is judged. By using the relationship, when the comparison result shows that the calculated playing time length is less than or greater than the preset time length and the lower limit value is zero, and the data amount cached in the data buffer zone is zero, the live broadcast of the streaming media is judged to be blocked.

By applying the embodiment of the invention, whether live broadcast of the streaming media is blocked can be accurately judged through the comparison relation between the data amount which can be accommodated by the data buffer area and the data amount of the streaming media audio data or through the comparison relation between the data amount which is buffered by the data buffer area and the data amount of the streaming media audio data or not.

Optionally, in a specific implementation manner, when the comparison result indicates that the calculated playing duration is greater than the preset time length, the step of determining whether the data size of the streaming media audio data meets a preset condition includes:

and judging whether the data volume of the acquired streaming media audio data with the preset time length is larger than the data volume which can be accommodated in the data buffer area.

When the playing time length is longer than the preset time length, it is known that the device still does not receive the completed streaming media audio data after the preset time length is reached, and the problem of streaming media audio data catching up or streaming media data retransmission may occur in the live broadcasting process of the streaming media.

For example, it is assumed that a first time stamp DS in a first audio data packet in the acquired streaming audio data is 14 o ' clock 02/09 seconds at 26 days monday in 2017 and 5 months, a second time stamp DE in a last audio data packet is 12 o ' clock 02/12 seconds at 14 o ' clock 02/26 days monday in 2017 and 26 months, an absolute value of a difference obtained by subtracting the first time stamp DS in the first audio data packet from the second time stamp DE in the last audio data packet is 3 seconds, and the 3 seconds are taken as the playing time length.

Assuming that the preset time length is 2 seconds, the system timestamp received by the device in the first audio data packet is 14 monday at 26 days and 26 months in 2017, 02 minutes and 08 seconds, the system timestamp is 14 monday at 26 days and 26 months in 2017, 02 minutes and 10 seconds, the device still receives the audio data packet between the last audio data packet and the first audio data packet, the device does not receive the last audio data packet, possibly, the streaming media data of the network is excessive, the device does not receive the last audio data packet in time, or the network can select compression by itself, discards the excessive last audio data packet with the time length of 1 second, and thus the streaming media data can retransmit the last audio data packet.

When the comparison result shows that the calculated playing time length is longer than the preset time length, the device can accurately judge whether the live streaming media is blocked or not by knowing the comparison relation between the data volume of the audio data of the streaming media indicated by the preset condition and the data volume capable of being accommodated in the data buffer zone.

The comparison result here shows that the calculated playing time length is longer than the preset time length, the device receives more streaming media audio data than the theoretical value, and the jamming may occur before the device receives more streaming media audio data than the theoretical value, the streaming media audio data more than the theoretical value are data which are not sent out in time before the network congestion is recovered, and the streaming media audio data more than the theoretical value are subjected to catch up and are retransmitted after the network congestion is recovered to be normal, so that the playing time length is longer than the preset time length.

Whether the preset condition is met or not means that when the streaming media audio data which is more than the theoretical value is chased and retransmitted, whether the chased and retransmitted streaming media audio data is within the data volume which can be accommodated in the data buffer area or not needs to be judged, so that whether the live streaming media is blocked or not can be accurately judged by utilizing the preset condition.

And if the data volume of the acquired streaming media audio data with the preset time length is larger than the data volume which can be accommodated in the data buffer area, judging that the live streaming media is blocked.

The data volume of the acquired streaming media audio data is larger than the data volume which can be accommodated in the data buffer area, and it can be known that the streaming media audio data exceeds the data volume which can be accommodated in the data buffer area, the situation that the media audio data overflows the data buffer area occurs, and the problem that the overflowed data is discarded by the network occurs, and at this time, it is determined that the streaming media live broadcast is blocked.

And if the data volume of the acquired streaming media audio data with the preset time length is less than or equal to the data volume which can be accommodated in the data buffer area, judging that the live streaming media is not blocked.

The data volume of the acquired streaming media audio data is less than or equal to the data volume which can be accommodated by the data buffer area, and it is known that the data volume of the streaming media audio data is within a legal range of the data volume which can be accommodated by the data buffer area, and the data volume is not lost in the transmission process or is discarded without exceeding the data buffer area, so that the live broadcast of the streaming media is judged to be not blocked.

The data amount that can be accommodated by the data buffer herein includes an upper limit value of the data amount of the data buffer, or the data amount that can be accommodated by the data buffer set according to a user demand. The data volume that this data buffer can hold can be according to user's long-term test, obtains a preset numerical value, regards this preset numerical value as the data volume that data buffer can hold, will judge like this when the data volume of streaming media audio data is greater than this preset numerical value that the live broadcast of streaming media blocks, judges the accuracy that the live broadcast of streaming media blocks more meticulously.

In order to reduce the pressure on the network caused by data transmission, optionally, in a specific implementation manner, if it is determined that the streaming media live broadcast is paused, the method for determining that the streaming media live broadcast is paused further includes: and determining the data amount overflowing the data buffer, and deleting the overflowed data amount.

Compared with the current video coding Frame Rate selection according to dynamic pictures or static pictures, that is, the video coding Frame Rate is also constant and dynamic, and the video coding Frame Rate may be a VFR (Variable Frame Rate) coding algorithm, similar to VBR. With different degrees of dynamic changes of pictures, the encoder encodes and generates different frame rates, for example, a higher video frame rate is adopted in a dynamic picture, and a lower video frame rate is adopted in a static picture, so that great help is provided for saving the file size of video encoding, but if a second preset threshold is set according to the requirements of a user and is compared with the video frame rate, a video frame rate smaller than the second preset threshold may exist. The video frame rate smaller than the second preset threshold is determined as the live streaming pause. However, in practical cases, the video frame rate smaller than the second preset threshold is only to indicate that more still pictures appear, and it is not necessarily that live streaming media is paused. The video coding frame rate selected in this way cannot be recognized, and it may be determined that the live streaming media is jammed.

By applying the embodiment, when the comparison result shows that the calculated playing time length is longer than the preset time length, whether live streaming media is blocked or not is judged by using the comparison relation between the data volume of the audio data of the streaming media and the data volume which can be accommodated in the data buffer area, the accuracy of judging whether live streaming media is blocked or not is improved, comparison parameters do not need to be set through dynamic change of pictures, and the difficulty of setting proper comparison parameters is reduced.

In order to accurately detect whether the live streaming media is stuck, in a specific implementation manner, when a comparison result shows that the calculated playing time length is less than a preset time length, whether the data volume of the audio data of the streaming media meets a preset condition is determined, and the step of determining whether the live streaming media is stuck comprises the following steps:

and judging whether the data volume cached in the data buffer area is zero or not and whether the data volume of the acquired streaming media audio data is zero or not.

When the comparison result shows that the calculated playing time length is less than the preset time length, the device can accurately judge whether the live streaming media is blocked or not by knowing the comparison relation between the data volume of the streaming media audio data indicated by the preset condition and the data volume cached in the data buffer area.

And if the data volume cached in the data buffer area is judged to be zero and the data volume of the acquired streaming media audio data is judged to be zero, the live streaming of the streaming media is judged to be blocked.

And if the data volume cached in the data buffer area is judged to be larger than zero and the data volume of the acquired streaming media audio data is judged to be larger than zero, judging that the live streaming media is not blocked.

The comparison result shows that the calculated playing time length is less than the preset time length, and it can be known that the device receives less streaming media audio data than the theoretical value, and whether the data amount cached in the data buffer area is zero needs to be judged again, and the data amount cached in the data buffer area is not null (that is, the data amount cached in the data buffer area is not zero), which indicates that the streaming media audio data exists in the data buffer area and does not influence the decoding end to decode the streaming media audio data, so that it is determined that the streaming media live broadcast is not blocked;

the amount of buffered data in the data buffer is empty, and no streaming media audio data is received (i.e., the amount of data of the acquired streaming media audio data is zero), so that it is determined that the streaming media live broadcast is stuck.

Compared with the prior art that whether the live streaming media is blocked is judged by utilizing video data in the streaming media data, the live streaming media is judged by utilizing the embodiment of the invention according to the streaming media audio data, so that the problem of direct interference caused by the existing judgment is well avoided, and the blocking of the live streaming media can be accurately judged.

As shown in fig. 3, the device applied to receive the streaming media in the live streaming media process may be a streaming media data receiving end. The flow of the stream media live broadcast process of the embodiment of the invention is as follows:

step 301, starting live streaming;

step 302, a signal acquisition coding end acquires streaming media audio data in streaming media data;

step 303, the signal acquisition encoding end encodes the streaming media audio data in the compressed streaming media. The streaming media data with the streaming media audio data has a time length, wherein the time length includes a start time stamp and an end time stamp, that is, the first time stamp DS of the first audio data packet is used as the start time stamp, and the second time stamp DE of the last audio data packet is used as the end time stamp;

step 304, the signal acquisition coding end sends the coded and compressed streaming media data;

step 305, a streaming media data receiving end receives the encoded and compressed streaming media data, and sets a preset time length TI and a data amount CL which can be accommodated by a data buffer area;

step 306, the receiving end of the streaming media data records a first time stamp DS in the first audio data packet;

the streaming media audio data in the streaming media data is transmitted through the audio data packet.

Step 307, the streaming media data receiving end records the second timestamp DE in the last audio data packet and records the total data volume DL of the audio data packet;

the total data amount DL of the audio data packets may be obtained by recording the data amount of each audio data packet and adding the data amounts of all the audio data packets, where all the data packets include: the audio data packet comprises a first audio data packet, a last audio data packet and all intermediate audio packets corresponding to the first audio data packet and the last audio data packet.

In step 308, the streaming media data receiving end calculates | DE-DS | ═ DD, where | DE-DS | may refer to an absolute value of a difference between the second timestamp DE and the first timestamp DS, or an absolute value of a difference between the first timestamp DS and the second timestamp DE, and determines a relationship between the absolute value DD of the difference, a preset time length TI, a data amount CL that can be accommodated by the data buffer, and a total data amount DL of the audio data packet.

The absolute value DD of the difference is used as the playing time length of the acquired streaming media audio data.

Step 309, the receiving end of streaming media data judges whether the absolute value DD of the difference is equal to the preset time length TI; if yes, if the DD is greater than TI, then step 310 is executed; if no, and DD < TI, then step 311 is performed, or if no, and DD > TI, then step 313 is performed.

Step 310, determining that the live streaming media is not blocked until step 314 is executed.

Step 311, determining whether the total data amount DL of the audio data packet is greater than zero; if yes, indicating that DL is 0, go to step 310; if not, the DL is <0, and step 312 is executed.

Step 312, determining that the live streaming media is blocked, and executing step 314.

Step 313, determining whether the total data amount DL of the audio data packets is less than or equal to the data amount CL that can be accommodated by the data buffer, if so, indicating that DL is < CL, then executing step 310; if not, indicating DL > CL, there is an overflow of the data buffer, then step 312 is performed.

And step 314, finishing the live broadcast of the streaming media.

Streaming media data can be transmitted to a streaming media data receiving end from a signal acquisition coding end through a plurality of links, and due to the fact that the streaming media data have the characteristic of real-time transmission, the requirements on network quality are very high, and the problems that the viewing experience is affected by the fact that the streaming media are directly broadcast, blocked and the like can be frequently encountered. The prior video coding technology cannot accurately judge whether live streaming media is blocked, but the audio coding technology based on the embodiment of the invention can accurately judge whether live streaming media is blocked, and just avoids the problems caused by the video coding technology. Therefore, by using the method for judging the live broadcast stuck condition of the streaming media, the live broadcast stuck condition of the streaming media is very necessary to be checked and analyzed through the streaming media audio data in the streaming media data.

As shown in fig. 4, an embodiment of the present invention provides a device for determining whether a streaming media live broadcast is stuck, where the device is applied to an apparatus that receives a streaming media in a streaming media live broadcast process, and the device includes:

an obtaining module 401, configured to obtain streaming media audio data in streaming media data with a preset time length, and calculate a playing time of the obtained streaming media audio data;

a comparison obtaining module 402, configured to compare the calculated playing time with the preset time duration to obtain a comparison result;

the judging module 403 is configured to judge whether the live streaming media is blocked according to the comparison result.

The device for judging whether live streaming is stuck according to the embodiment of the present invention calculates the playing time of the obtained streaming audio data by obtaining the streaming audio data in the streaming data with the preset time length, and compares the playing time of the streaming audio data with the preset time length to obtain a comparison result, thereby judging whether live streaming is stuck. Therefore, the misjudgment of the video data in the streaming media for detecting the pause is eliminated, and whether the streaming media is paused or not can be accurately judged. The method is based on subsequent adjustment of the video fluency, and improves the user experience effect after the video fluency is effectively adjusted in the later period.

The embodiment of the invention also provides equipment for receiving the streaming media in the live broadcasting process of the streaming media, which comprises a processor, a memory and a storage, wherein the memory is used for storing the computer program;

the processor is used for realizing the following steps when executing the program stored in the memory:

As shown in fig. 5, an embodiment of the present invention further provides an apparatus for receiving streaming media in a live streaming media process, which includes a processor 501, a communication interface 502, a memory 503, and a communication bus 504, where the processor, the communication interface, and the memory complete mutual communication through the bus,

a memory 503 for storing a computer program;

the processor 501 is configured to implement the following steps when executing the program stored in the memory:

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

The device for receiving the streaming media in the streaming media live broadcast process, provided by the embodiment of the invention, is used for calculating the playing time of the acquired streaming media audio data by acquiring the streaming media audio data in the streaming media data with the preset time length, and comparing the playing time of the streaming media audio data with the preset time length to obtain a comparison result, so as to judge whether the streaming media live broadcast is blocked or not. Therefore, the misjudgment of the video data in the streaming media for detecting the pause is eliminated, and whether the streaming media is paused or not can be accurately judged. The method is based on subsequent adjustment of the video fluency, and improves the user experience effect after the video fluency is effectively adjusted in the later period.

An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps:

The most frequently occurring problem affecting the viewing experience of audiences in the live streaming media process is the jamming, but the jamming is a problem which can be perceived by human beings and occurs in a service layer, and for a network, a server and an application program which provide basic support for streaming media data transmission, only indexes such as speed, integrity and the like of streaming media data in the transmission process can be perceived, if the quality index of a transmission layer and the quality index of the service layer can be accurately correlated, namely the application program can judge whether the jamming is likely to occur by analyzing the quality change of the streaming media data transmission, the live streaming media service quality can be greatly improved.

Therefore, the embodiment of the present invention provides a method for determining live streaming blockage, which is applied to a current complex internet live streaming framework, and provides an important and feasible determination method for multi-stage cascaded live full-link blockage troubleshooting analysis, so as to prepare for further improving the live streaming service quality in the later stage.

The multi-stage cascade is used for the streaming media data to pass through a plurality of interconnected nodes from the signal acquisition coding end to the streaming media data receiving end. And the streaming media data receiving end decodes the streaming media live broadcast data. For example, one of every two adjacent forwarding nodes, i.e., four forwarding nodes, i.e., O, P, Q, and R, may serve as a sending end, and the other may serve as a receiving end. Specifically, when the forwarding node O serves as a sending end, the forwarding node P serves as a receiving end; when the forwarding node P is used as a sending end, the forwarding node Q is used as a receiving end; when the forwarding node Q serves as a transmitting end, the forwarding node R serves as a receiving end, and thus each transmitting end to an adjacent receiving end is called a link or a level, so that there are multiple links and multiple levels, and the multiple levels are called multi-level concatenation.

Illustratively, the equipment in the beijing area sends the streaming media data to the equipment in the west ampere area, the equipment in the beijing area and the equipment in the west ampere area are not directly connected, but send the streaming media data to a transfer node equipment which is established with both the equipment in the shanxi area and the equipment in the beijing area through a distribution network, and then send the streaming media data forwarded by the transfer node equipment to the equipment in the west ampere area, so that the problem that a link formed by directly connecting the equipment in the beijing area and the equipment in the west ampere area is unstable is solved, the transmission reliability can be improved, and the hiton of the streaming media data in the multi-stage cascade transmission process can be checked by using the method for judging the live hiton of the streaming media in the embodiment of the invention.

The embodiment of the invention provides a computer program product containing instructions, which when running on a computer, enables the computer to execute the steps of the method for judging whether the streaming media live broadcast is blocked.

The embodiment of the invention provides a computer program, which when running on a computer, enables the computer to execute the steps of the method for judging whether the streaming media live broadcast is blocked.

For the apparatus/device/storage medium embodiment/computer program product/computer program containing instructions, the description is relatively simple as it is substantially similar to the method embodiment, and reference may be made to some descriptions of the method embodiment for relevant points.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus/device/storage medium embodiment/computer program product/computer program embodiment containing instructions, the description is relatively simple as it is substantially similar to the method embodiment, and reference may be made to some descriptions of the method embodiment for relevant points.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for judging the live pause of streaming media is characterized in that the method is applied to equipment for receiving the streaming media in the live broadcasting process of the streaming media, and comprises the following steps:

judging whether the live streaming media is blocked or not according to the comparison result;

wherein, the step of judging whether the live streaming media is blocked or not according to the comparison result comprises the following steps:

and judging whether the data volume of the streaming media audio data meets a preset condition, and judging whether the live streaming media is blocked, wherein the preset condition shows the comparison relationship between the data volume of the streaming media audio data and the data volume of a data buffer area.

2. The method according to claim 1, wherein the step of obtaining streaming media audio data in streaming media data of a preset time length and calculating the playing time length of the obtained streaming media audio data comprises:

and calculating the absolute value of the difference value between the second time stamp in the last audio data packet and the first time stamp in the first audio data packet to obtain the playing time length of the acquired streaming media audio data.

3. The method of claim 1, wherein the step of determining whether the live streaming media is stuck according to the comparison result comprises:

and determining that the live streaming media is not blocked.

4. The method of claim 1,

when the comparison result shows that the calculated playing time length is greater than the preset time length,

the step of judging whether the data volume of the streaming media audio data meets a preset condition or not and judging whether the streaming media live broadcast is blocked or not comprises the following steps: judging whether the data volume of the acquired streaming media audio data of the preset time length is larger than the data volume which can be accommodated by the data buffer area,

if the data volume of the acquired streaming media audio data in the preset time length is larger than the data volume which can be accommodated in the data buffer area, judging that the live streaming media is blocked;

5. The method of claim 4, wherein if it is determined that the streaming media live broadcast is stuck, the method further comprises:

and determining the data amount overflowing the data buffer, and deleting the overflowed data amount.

6. The method of claim 1,

when the comparison result shows that the calculated playing time length is less than the preset time length,

the step of judging whether the data volume of the streaming media audio data meets a preset condition or not and judging whether the streaming media live broadcast is blocked or not comprises the following steps: judging whether the data amount buffered in the data buffer area is zero or not and whether the data amount of the acquired streaming media audio data is zero or not,

7. A judging device for live video pause of streaming media is characterized in that the device is applied to equipment for receiving the streaming media in the live video process of the streaming media, and the device comprises:

the judging module is used for judging whether the live streaming media is blocked or not according to the comparison result;

the judging module comprises:

and the second judging submodule is used for judging whether the data volume of the streaming media audio data meets a preset condition and judging whether the live streaming media is paused or not when the comparison result shows that the calculated playing time length is smaller than or larger than the preset time length, wherein the preset condition shows the comparison relationship between the data volume of the streaming media audio data and the data volume of the data buffer area.

8. The apparatus of claim 7, wherein the obtaining module comprises:

the first obtaining module is used for obtaining streaming media audio data in the streaming media data with the preset time length;

the second recording module is used for recording a second timestamp in the last audio data packet with the preset time length;

9. The apparatus of claim 7, wherein the determining module comprises:

and the first judgment submodule is used for judging that the live streaming media is not blocked when the comparison result shows that the calculated playing time length is equal to the preset time length.

10. The apparatus of claim 7,

the second judgment sub-module is specifically configured to: when the comparison result shows that the calculated playing time length is longer than the preset time length, judging whether the data volume of the acquired streaming media audio data of the preset time length is larger than the data volume which can be accommodated by the data buffer area or not,

11. The apparatus of claim 10, further comprising:

and the deleting module is used for determining the data volume overflowing the data buffer and deleting the overflowing data volume.

12. The apparatus of claim 7,

the second judgment sub-module is specifically configured to: when the comparison result shows that the calculated playing time length is less than the preset time length, whether the data volume cached in the data buffer area is zero or not and whether the data volume of the acquired streaming media audio data is zero or not are judged,

13. An apparatus for receiving streaming media during live streaming of the streaming media, comprising a processor and a memory, wherein the memory is used for storing a computer program;

the processor, when executing the program stored in the memory, implementing the method steps of any of claims 1-6.

14. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 6.