CN105142002A - Audio/video live broadcasting method and device as well as control method and device - Google Patents

Audio/video live broadcasting method and device as well as control method and device Download PDF

Info

Publication number
CN105142002A
CN105142002A CN201510483445.4A CN201510483445A CN105142002A CN 105142002 A CN105142002 A CN 105142002A CN 201510483445 A CN201510483445 A CN 201510483445A CN 105142002 A CN105142002 A CN 105142002A
Authority
CN
China
Prior art keywords
duration
buffer
value
data duration
delay variation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201510483445.4A
Other languages
Chinese (zh)
Other versions
CN105142002B (en
Inventor
陈健生
邹向群
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Boguan Information Technology Co Ltd
Original Assignee
Guangzhou Boguan Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Boguan Information Technology Co Ltd filed Critical Guangzhou Boguan Information Technology Co Ltd
Priority to CN201510483445.4A priority Critical patent/CN105142002B/en
Publication of CN105142002A publication Critical patent/CN105142002A/en
Application granted granted Critical
Publication of CN105142002B publication Critical patent/CN105142002B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331Caching operations, e.g. of an advertisement for later insertion during playback
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4335Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4341Demultiplexing of audio and video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439Processing of audio elementary streams
    • H04N21/4392Processing of audio elementary streams involving audio buffer management
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44004Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a control method for delay jitter removal based on a TCP live broadcasting system. The control method comprises the steps of: judging whether a current buffer region is empty; if not, shortening the data duration of the current buffer region when a current adjustment period arrives and if a difference between the maximum buffer region data duration and the minimum buffer region data duration in the current adjustment period is smaller than a preset buffer region duration jitter threshold and the minimum buffer region data duration is greater than a preset minimum buffer region duration threshold; if the current buffer region is empty, enlarging an initial buffer region data duration when the empty buffer region receives a first data packet, wherein the enlarged initial buffer region data duration is an increasing function of a second biggest delay jitter value, the initial buffer region data duration is the buffer data duration needed by the buffer region from being empty to play-recovering, and the second biggest delay jitter value is a second biggest value of time differences of reception of two continuous data packets from a period of time before the buffer region is empty to the time when the first data packet is received.

Description

Audio/video live broadcasting method, device and control method, device
Technical field
The present invention relates to network transmission technology field, particularly relate to control method, the device of a kind of audio/video live broadcasting method based on TCP live broadcast system elimination delay variation, device and correspondence.
Background technology
Video living transmission system is that the vision signal at scene (as camera, video camera) and voice signal (as microphone) are carried out Real-time Collection coding, and be sent to server by network and get on, user can use during network implementation and watch.A video living transmission system model the simplest is made up of five links: gather, encode, transmit, decode, play.Factor due to reality makes the realization of video living transmission system not be so simple, and it relates to a lot of professional technique.
Delay variation (jitter), as the important qos parameter of in video living transmission system, has direct impact to voice communication quality.Therefore need to adopt certain measure to prevent delay variation, thus ensure certain live quality.
The change that delay variation (jitter) refers to due to various delay causes the change of the packet arrival rate in network.Network delay refers to that an IP wraps in the time needed for transmission over networks, and it comprises algorithm time delay, coding delay, network transfer delay and jitter buffer time delay.Network delay shake is exactly that network delay is for a moment large for a moment little, in this case, even if transmit leg is timed sending packet, and the reception of recipient also cannot equally regularly, sometimes in one-period, a bag does not receive yet, and sometimes receives several bags in one-period.If network jitter is comparatively serious, packet also likely abandons.So, sound one card causing recipient to hear.
In order to eliminate delay variation, usual employing jitter buffer technology (JitterBuffer), namely a buffering area is set recipient, first datagram enters Buffer Pool and keeps in when arriving, packet takes out with level and smooth speed by system again from Buffer Pool subsequently, after decompress(ion), play to user.This buffer technology effectively can eliminate voice jitter in certain limit, and improves tonequality.
The buffer depth of JitterBuffer depends on the degree of network jitter, and network jitter is larger, and buffer depth is larger, and the delay of broadcasting is larger.So JitterBuffer make use of higher delay to exchange for smooth broadcasting, compare sound one card, delay larger a little but more smooth effect, its subjective experience wants better.The buffer depth of certain JitterBuffer is not necessarily always constant, can be the change according to network jitter degree and dynamic conditioning.When network recovery to very steady unobstructed time, buffer depth can be very little, like this postpone just can reduce.
Current buffer control method mainly contains two kinds: static jitter buffer control method and self-adapted jitter buffer control method.
In static jitter buffer control method, the size of buffering area is fixed value after transmission channel is set up always, and data shake being exceeded to buffer size will be dropped.The advantage of the method is: model is simple, be easy to realize, reliability is high; Shortcoming is: when network jitter is larger, and packet loss is high, and network delay and shake very little time, postpone comparatively large, applicability is not very strong.
In self-adapted jitter buffer control method, buffer size changes with the jitter conditions of real network.Compared with static jitter buffer control method, the advantage of self-adapted jitter buffer control method is: when network jitter is larger, packet loss is relatively low, and network delay and shake hour, postpone relatively little.But the self-adapted jitter buffer control method that prior art uses increases according to the jitter conditions of real network or reduce buffer size, and the value definite value often increasing or reduce.That is, for the jitter conditions needing the network increasing/reduce buffering area, be all increase current buffer/reduce a definite value, and ignore the difference needing to exist between the jitter conditions of the network of increase/minimizing buffering area, thus make dynamic conditioning level and smooth not.Therefore, for also inapplicable the higher video living transmission system of smoothness requirements.
Summary of the invention
The object of the embodiment of the present invention is to provide a kind of audio/video player method based on TCP live broadcast system elimination delay variation, device and control method, device, dynamically effectively can adjust buffer size according to network state, and ensure to make time delay as far as possible little under prerequisite that is level and smooth, that guarantee broadcasting smoothness.
Embodiments provide a kind of control method eliminating delay variation based on TCP live broadcast system, comprising:
Judge whether current buffer is empty;
When buffering area is not empty, when current adjustment cycle arrives, if the difference of the maximum buffer data duration in current adjustment cycle and minimal buffering district data duration is less than default buffering area duration dithering threshold, and described minimal buffering district data duration is greater than default minimal buffering district duration threshold value, then current buffer data duration is shortened, and the value shortened is less than or equal to the difference between described minimal buffering district data duration and described default minimal buffering district duration threshold value;
When buffering area is empty, initial buffer district data duration, to when receiving first packet then increases for empty by district to be buffered, and the initial buffer district data duration after increasing is the increasing function of second largest delay variation value; Wherein, described initial buffer district data duration is buffering area is the empty buffered data duration to recovering required for broadcasting; Described second largest delay variation value is buffering area is that unprecedented a period of time is to the Second Largest Value in the time difference receiving continuous print two packets received in the time of described first packet.
As the improvement of such scheme, when the initial buffer district data duration after described increase is greater than default maximum buffer duration threshold value, using described default maximum buffer duration threshold value as the initial buffer district data duration after described increase; When the initial buffer district data duration after described increase is less than default minimal buffering district duration threshold value, using described default minimal buffering district duration threshold value as the initial buffer district data duration after described increase.
As the improvement of such scheme, when described second largest delay variation value is less than default minimal time delay jitter value, using described default minimal time delay jitter value as described second largest delay variation value; When described second largest delay variation value is greater than default maximum delay jitter value, using described default maximum delay jitter value as described second largest delay variation value.
As the improvement of such scheme, also comprise: by the time difference stacked preservation of two packets received continuously.
As the improvement of such scheme, the initial buffer district data duration after described increase is the linear increasing function of second largest delay variation value.
As the improvement of such scheme, the initial buffer district data duration after described increase is the piecewise linearity increasing function of second largest delay variation value.
As the improvement of such scheme, described a period of time is 1/3 ~ 1/2 of a complete adjustment cycle.
The embodiment of the present invention is also corresponding discloses a kind of control device eliminating delay variation based on TCP live broadcast system, comprising:
Judge module, for judging whether current buffer is empty;
First adjusting module, for when buffering area be sky, when current adjustment cycle arrives, if the difference of the maximum buffer data duration in current adjustment cycle and minimal buffering district data duration is less than default buffering area duration dithering threshold, and described minimal buffering district data duration is greater than default minimal buffering district duration threshold value, then current buffer data duration is shortened, and the value shortened is less than or equal to the difference between described minimal buffering district data duration and described default minimal buffering district duration threshold value;
Second adjusting module, for being empty when buffering area, then increases initial buffer district data duration, and the initial buffer district data duration after increasing is the increasing function of second largest delay variation value; Wherein, described initial buffer district data duration is buffering area is the empty buffered data duration to recovering required for broadcasting; Second Largest Value in the time difference of continuous print two packets received in described second largest delay variation value to be buffering area be unprecedented a period of time.
As the improvement of such scheme, described second adjusting module also for: when the initial buffer district data duration after described increase is greater than default maximum buffer duration threshold value, using described default maximum buffer duration threshold value as the initial buffer district data duration after described increase; When the initial buffer district data duration after described increase is less than default minimal buffering district duration threshold value, using described default minimal buffering district duration threshold value as the initial buffer district data duration after described increase.
As the improvement of such scheme, described second adjusting module also for: when described second largest delay variation value is less than default minimal time delay jitter value, using described default minimal time delay jitter value as described second largest delay variation value; When described second largest delay variation value is greater than default maximum delay jitter value, using described default maximum delay jitter value as described second largest delay variation value.
As the improvement of such scheme, also comprise:
Stacked preservation module, for the time difference stacked preservation of two packets that will receive continuously.
As the improvement of such scheme, the initial buffer district data duration after described increase is the linear increasing function of second largest delay variation value.
As the improvement of such scheme, the initial buffer district data duration after described increase is the piecewise linearity increasing function of second largest delay variation value.
As the improvement of such scheme, described a period of time is 1/3 ~ 1/2 of a complete adjustment cycle.
The embodiment of the invention also discloses a kind of audio/video live broadcasting method based on TCP, comprising:
Receive packet by described data pack buffer in buffering area;
The buffer data duration of described buffering area is adjusted according to control method as above;
Packet is obtained and broadcasting of decoding based in the buffering area after the adjustment of buffer data duration.
The embodiment of the invention also discloses a kind of audio/video live broadcast device based on TCP, comprising:
Data reception module, for receiving packet and by described data pack buffer in buffering area;
Control device as above, for the buffer data duration to the described buffering area of adjustment; And
Decoding playing module, obtains packet and broadcasting of decoding based in the buffering area after the adjustment of buffer data duration.
Compared with prior art, audio/video live broadcasting method based on TCP live broadcast system elimination delay variation disclosed by the invention, device and control method, device, effectively can solve the problem of delay time and jitter in video living transmission system, in self-adapted jitter buffer control method, by judging whether current buffer is empty, and in buffering area be not empty/for empty, respectively buffer data duration is shortened/increase adjustment according to current network state, and the value of adjustment is the value changed according to network state.Therefore, dynamically effectively can adjust buffer size according to network state, and ensure to make time delay as far as possible little under prerequisite that is level and smooth, that guarantee broadcasting smoothness.
Accompanying drawing explanation
Fig. 1 is a kind of schematic flow sheet eliminating the control method of delay variation based on TCP live broadcast system in the embodiment of the present invention.
Fig. 2 is the buffering area change schematic diagram do not existed in current adjustment cycle when buffering area is empty.
Fig. 3 is the idiographic flow schematic diagram of the step S102 shown in Fig. 1.
Fig. 4 is the buffering area change schematic diagram existed in current adjustment cycle when buffering area is empty.
Fig. 5 is the schematic diagram of initial buffer district data duration after increasing and second largest delay variation value linearly relationships of increase function.
Fig. 6 is a kind of structural representation eliminating the control device of delay variation based on TCP live broadcast system in the embodiment of the present invention.
Fig. 7 is a kind of schematic flow sheet eliminating the video broadcasting method of delay variation based on TCP live broadcast system in the embodiment of the present invention.
Fig. 8 is the structural representation of the live receiving system of a kind of audio/video based on TCP in the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
See Fig. 1, it is a kind of schematic flow sheet eliminating the control method of delay variation based on TCP live broadcast system that the embodiment of the present invention provides.This control method comprises step:
S101, judge that whether current buffer is empty;
S102, when buffering area be not empty, when current adjustment cycle arrives, if the difference of the maximum buffer data duration in current adjustment cycle and minimal buffering district data duration is less than default buffering area duration dithering threshold, and described minimal buffering district data duration is when being greater than default minimal buffering district duration threshold value, then current buffer data duration is shortened, and the value shortened is less than or equal to the difference between described minimal buffering district data duration and described default minimal buffering district duration threshold value;
S103, when buffering area is empty, initial buffer district data duration, to when receiving first packet then increases for empty by district to be buffered, and the initial buffer district data duration after increasing is the increasing function of second largest delay variation value; Wherein, described initial buffer district data duration is buffering area is the empty buffered data duration to recovering required for broadcasting; Described second largest delay variation value is buffering area is that unprecedented a period of time is to the Second Largest Value in the time difference receiving continuous print two packets received in the time of described first packet.
The present embodiment is used for, when the data of reception audio/video live broadcast system are to play, effectively to eliminate delay variation.Because video existing in video living transmission system has audio frequency again, the present embodiment, for video living transmission system, describes the implementation procedure of the present embodiment in detail.For video living transmission system, audio pack is less than the data volume of video packets, in live broadcast system, first to ensure interactive voice quality, therefore the present embodiment adopt VoP in a network rate variation define delay variation value jitter and control buffer size.
Three threads are often created, respectively: read thread, audio frequency thread, video thread in video living transmission system.Read thread to be responsible for from network reception data, audio, video data is placed in Voice & Video buffering area respectively.Audio frequency thread fetches data and decodes from audio buffer, then plays sound.Video thread fetches data and decodes from screen buffer, then displaying video.Therefore in video living transmission system, there are two buffering areas, for the sake of simplicity, the present embodiment adopts unified control method to control two buffer size.
Because packet does not have synchronised clock between transmitting terminal and receiving terminal, cannot the real time delay of calculated data transmission, therefore the present embodiment is when designing jitter-buffer size, with receiving terminal time delay weigh network state for standard.
The video living transmission system of the present embodiment is based on Transmission Control Protocol, and this agreement is a kind of reliable protocol, there is not Network Packet Loss.The data packet number fluctuation arrived in order in a time cycle in the stable situation of network is smaller.But when network jitter is serious, the data packet number that receiving terminal receives can decline rapidly, causes buffer data to reduce rapidly, is even empty.When network recovery is normal time, receiving terminal can receive a large amount of packets in a flash, and therefore the data of buffering area are increased sharply or overflowed.
Suppose there are two continuous print audio/video bags, its sequence number is i and i+1, then agree as follows respectively:
(1) T i, T i+1: the internal time stamp of packet i and i+1;
(2) R i, R i+1: packet i and i+1 reaches the time of receiving terminal;
(3) P i, P i+1: the time that packet i and i+1 plays in this locality;
(4) Jitter i=R i+1-R i: packet i and i+1 reaches the time difference of receiving terminal, is determined by network state.
If there is no network delay shake, then have arbitrary i and j: Jitter i=Jitter jand R i+1-R i=T i+1-T i, in this ideal case, packet just can be play-overed without any need for buffering, a demand fulfillment P i+1-P i=Jitter i.But in a practical situation, owing to there is delay variation, will produce voice distortion if now play-overed, video shows as continuous-form card and pauses, so must play after audio/video bag buffer memory a period of time, voice so just can be made true and video links up.It can thus be appreciated that the target that the present embodiment carries out jitter buffer is exactly: make the reproduction time interval of audio/video bag equal their internal time stamp interval, and reduce card number of times and delay as far as possible.
Utilize the control method based on TCP live broadcast system elimination delay variation of the present embodiment to adjust buffer size, first introduce several parametric variable:
(1) min_buffer_time: default minimal buffering district duration threshold value, i.e. the jitter-buffer minimum time needing buffering, cushion this time first and namely can play.But each buffering reaches the minimum time and just can play after not representing.This parameter determines the minimal time delay of normal smooth playing.
(2) max_buffer_time: default maximum buffer duration threshold value, i.e. the maximum cushioning time of jitter-buffer, the packet exceeding this time can be dropped.This parameter determines the maximum delay that can allow.
(3) cur_buffer_time: current buffer data duration.This variable is data duration in reaction current buffer, this value is at every moment in change, therefore the max (maximum buffer data duration) that this value and current time got and min (minimal buffering district data duration) is needed to compare, thus upgrade the value of max (maximum buffer data duration) and min (minimal buffering district data duration), just can obtain final max (maximum buffer data duration) and the value of min (minimal buffering district data duration) after treating this end cycle.
(3.1) the maximum buffer data duration in max: one adjustment cycle.The reflection of this value be the maximum of data duration in the buffering area of moment change in an adjustment cycle.Mode by above-mentioned (3) obtains the final max value in an adjustment cycle.
(3.2) the minimal buffering district data duration in min: one adjustment cycle.The reflection of this value be the minimum value of data duration in the buffering area of moment change in an adjustment cycle.Mode by above-mentioned (3) obtains the final min value in an adjustment cycle.
(4) replay_buffer_time: initial buffer district data duration, refers to and arrives the buffered data duration recovered required for broadcasting after net cast card pauses (namely buffering area is for empty).This value is between min_buffer_time and max_buffer_time.Min_buffer_time got by the initial value of this value.The size of this value follow-up is dynamically determined by jitter value described below.
(5) jitter: delay variation value, namely continuous print two audio pack reach the time difference of receiving terminal, stackedly to save by it whenever calculating a jitter value.Jitter value in a period of time can reflect the dither state of network.
(6) lower than this value, min_jitter: default minimal time delay dithering threshold, can think that network jitter is very little, network is smooth.
(7) max_jitter: default maximum delay shake threshold r value, exceedes this value and can think that network jitter is very severe.
(8) Th: default buffering area duration dithering threshold, i.e. the threshold value of one-period internal buffer duration shake, is less than this threshold value and represents that network jitter is very little.
After defining above parametric variable, then the control method based on TCP live broadcast system elimination delay variation by the present embodiment adjusts buffer size.The control method of the present embodiment carries out different adjustment according to current network state is good/poor to buffer size, wherein, by judging whether current buffer is that sky tentatively distinguishes current network state as well still poor (above-mentioned steps S101), namely, when current buffer is not empty, can think that network state is relatively good; When current buffer is empty, can think that network state is poor.
Consider the reasonable situation of network state, namely the value fluctuation of delay variation value Jitter is very little, this time buffer data duration change as shown in Figure 2: adjustment cycle internal buffer, fluctuation is smaller, in Fig. 2, highs and lows is designated as max (maximum buffer data duration) and min (minimal buffering district data duration) respectively, buffering area is actual has in this case cushioned too much data, causes delay to be not less than min (minimal buffering district data duration).In fact optimal buffer size best_buffer_time=max – min under this situation.Therefore forwarding operation can be performed, discard a batch data the oldest in buffering area, the data duration abandoned is no more than the difference between min (minimal buffering district data duration) or min (minimal buffering district data duration) and min_buffer_time (default minimal buffering district duration threshold value), the countless certificate in buffering area can not be caused like this and to block time, after contrary F.F., live delay can be reduced.
With reference to figure 3, in above-mentioned steps S102, when judging current buffer not as when empty (network state is relatively good), adjust buffer data duration especially by following steps:
Maximum buffer data duration max in the current adjustment cycle of step S1021, real-time update and minimal buffering district data duration min; Concrete update mode is that (frequency acquisition can pre-set Real-time Collection, collection period is far fewer than adjustment cycle) current buffer data duration cur_buffer_time, and current buffer data duration cur_buffer_time and the maximum buffer data duration max got and minimal buffering district data duration min are compared, thus the value of real-time update maximum buffer data duration max and minimal buffering district data duration min;
Step S1022, judge whether current (complete) adjustment cycle arrives, if so, then perform step S1023; Otherwise return and perform step S1021;
Step S1023, (now can obtain the maximum buffer data duration max of current adjustment cycle and the value of minimal buffering district data duration min) when current (complete) adjustment cycle arrives, calculate relevant value by following formula (1), (2):
d 1=max-min;(1)
d 2=min-min_buffer_time(2)
Wherein, max is the maximum buffer data duration in current adjustment cycle, and min is the maximum buffer data duration in current adjustment cycle, d 1for the difference of the maximum buffer data duration max in current adjustment cycle and minimal buffering district data duration min; Min_buffer_time is default minimal buffering district duration threshold value, d 2for the difference between minimal buffering district data duration min and default minimal buffering district duration threshold value min_buffer_time.
The value d that step S1024, determining step S1023 calculate 1and d 2whether meet following formula (3), if so, then perform step S1025; Otherwise buffer data duration is not adjusted.
d 1<Th&d 2>0(3)
Wherein, Th is default buffering area duration dithering threshold.
Step S1025, by the current buffer data duration cur_buffer_time of current (i.e. the adjustment cycle due in) that collect shorten (being equivalent to F.F.) a period of time (data duration), and shorten data duration be less than or equal d 2.
In formula (3), when the difference of the maximum buffer data duration max calculated in current adjustment cycle and minimal buffering district data duration min, represent that network jitter is very little when this difference is less than default buffering area duration dithering threshold Th, therefore, if data duration min in minimal buffering district is larger than the minimal buffering district duration threshold value min_buffer_time preset this moment, then current buffer data duration can be shortened (being equivalent to perform forwarding operation), can time delay be reduced like this.In addition, the buffer data duration (or data duration of F.F.) shortened is less than or equal to difference between described minimal buffering district data duration min and described default minimal buffering district duration threshold value min_buffer_time, can not cause the countless certificate in buffering area like this and to block time.
When network state is poor time, the value fluctuation ratio of delay variation value Jitter is comparatively large, and receiving terminal receives data and sharply declines, but broadcasting end still fetches data according to fixing frequency to be play, therefore the data of buffering area can sharply decline until be zero, cause live card to pause.By the time, time network recovery is normal, receive suddenly a large amount of data at the receiving terminal at that moment recovered, then network state returns to situation shown in Fig. 2.As shown in Figure 4, in Fig. 4, peak is designated as max (maximum buffer data duration), and buffering area is empty to again there being the time interval of data to be designated as gap.In this case desirable buffer size is best_buffer_time=max+gap.Because delay variation value Jitter fluctuation causes more greatly card, therefore the size strengthening initial buffer district data duration (namely net cast card arrives the buffered data duration recovered required for broadcasting after pausing) is needed, the namely size of replay_buffer_time.Video living transmission system is due to few with user speech interaction, so high unlike other simple speech communication systems to the delay requirement of voice, allows higher time delay (such as 1 second).
In above-mentioned steps S103, when judging that current buffer is empty (network state is poor), then first wait for until buffering area is for sky is to when receiving first packet, immediately by calculating to be increased by initial buffer district data duration, and initial buffer district data duration (i.e. replay_buffer_time) after increasing is the increasing function of second largest delay variation value.
Concrete, when receiving terminal receives packet, time difference (delay variation value Jitter) the stacked preservation of two packets that can will receive continuously.Wherein, delay variation value Jitter falls into a trap at digital independent thread and obtains, and is kept in a stack, and the delay variation value Jitter of the always nearest a period of time of therefore depositing in this stack, can reflect the network state of nearest a period of time.In the present embodiment, when judging current buffer as empty to when receiving first packet, by the buffering area Second Largest Value that to be unprecedented a period of time (being preferably the time of 1/3 ~ 1/2 of a complete adjustment cycle) preserve in stack in the time receiving described first packet as second largest delay variation value Jitter, and select second largest delay variation value Jitter in this stack as the adjustment benchmark of the initial buffer district data duration after increasing, the initial buffer district data duration after namely increasing is the increasing function of described second largest delay variation value Jitter.
As shown in Figure 5, preferably, initial buffer district data duration (replay_buffer_time) after the present embodiment adjustment (increase) is the linear increasing function of second largest delay variation value Jitter.In the diagram, sec_jitter represents second largest delay variation value jitter, if this value is greater than max_jitter (the maximum jitter value of expectation), then and value max_jitter; If this value is less than min_jitter (the minimum jitter value of expectation), then value min_jitter.Fig. 4 shows when linear coefficient r is respectively: initial buffer district data duration (replay_buffer_time) after increasing in r<1, r=1 and r>1 tri-kinds of situations and the linear relationship of second largest delay variation value sec_jitter.In addition, initial buffer district data duration (replay_buffer_time) value after increase is between min_buffer_time and max_buffer_time.Namely when initial buffer district data duration (replay_buffer_time) after described increase is greater than default maximum buffer duration threshold value (max_buffer_time), using described default maximum buffer duration threshold value (max_buffer_time) as the initial buffer district data duration after described increase; When initial buffer district data duration (replay_buffer_time) after described increase is less than default minimal buffering district duration threshold value (min_buffer_time), using described default minimal buffering district duration threshold value (min_buffer_time) as the initial buffer district data duration after described increase.
Understandable, initial buffer district data duration (i.e. replay_buffer_time) after increase is not unique with the relationships of increase function of second largest delay variation value sec_jitter, can adopt other increasing functions such as piecewise linear function yet.
Visible, the control method eliminating delay variation based on TCP live broadcast system of the present embodiment, in self-adapted jitter buffer control method, by judging whether current buffer is empty, and when current buffer is not/is empty, respectively buffer data duration is shortened/increase adjustment according to current network state, and the value of adjustment is the value changed according to network state.Therefore, dynamically effectively can adjust buffer size according to network state, and ensure to make time delay as far as possible little under prerequisite that is level and smooth, that guarantee broadcasting smoothness.
With reference to figure 6, it is a kind of structural representation eliminating the control device of delay variation based on TCP live broadcast system in the embodiment of the present invention.This control device comprises:
Judge module 21, for judging whether current buffer is empty;
First adjusting module 22, for when buffering area be sky, when current adjustment cycle arrives, if the difference of the maximum buffer data duration in current adjustment cycle and minimal buffering district data duration is less than default buffering area duration dithering threshold, and described minimal buffering district data duration is greater than default minimal buffering district duration threshold value, then current buffer data duration is shortened, and the value shortened is less than or equal to the difference between described minimal buffering district data duration and described default minimal buffering district duration threshold value; And
Second adjusting module 23, for when buffering area be sky, district to be buffered is for sky is to when receiving first packet, then initial buffer district data duration is increased, and the initial buffer district data duration after increasing is the increasing function (comprising linear increasing function, piecewise linearity increasing function etc.) of second largest delay variation value; Wherein, described initial buffer district data duration is buffering area is the empty buffered data duration to recovering required for broadcasting; Described second largest delay variation value is buffering area is that unprecedented a period of time (being preferably the time of 1/3 ~ 1/2 of a complete adjustment cycle) is to the Second Largest Value in the time difference receiving continuous print two packets received in the time of described first packet.
Preferably, second adjusting module 23 of the present embodiment also for: when the initial buffer district data duration after described increase is greater than default maximum buffer duration threshold value, using described default maximum buffer duration threshold value as the initial buffer district data duration after described increase; When the initial buffer district data duration after described increase is less than default minimal buffering district duration threshold value, using described default minimal buffering district duration threshold value as the initial buffer district data duration after described increase.
Preferably, second adjusting module 23 of the present embodiment also for: when described second largest delay variation value is less than default minimal time delay jitter value, using described default minimal time delay jitter value as described second largest delay variation value; When described second largest delay variation value is greater than default maximum delay jitter value, using described default maximum delay jitter value as described second largest delay variation value.
Preferably, the control device of the present embodiment also comprises: stacked preservation module (not shown), for the time difference stacked preservation of two packets that will receive continuously.
The course of work of the control device based on TCP live broadcast system elimination delay variation of the present embodiment with reference to above-described embodiment, can not repeat them here.
With reference to figure 7, it is the schematic flow sheet of a kind of audio/video player method based on TCP in the embodiment of the present invention.This audio/video player method comprises step:
S301, receive packet by described data pack buffer in buffering area;
S302, utilize and eliminate based on TCP live broadcast system the buffer data duration that the control method of delay variation adjusts described buffering area shown in Fig. 1;
S303, based on buffer data duration adjustment after buffering area in obtain packet and broadcasting of decoding.
Wherein, the specific implementation of step S301 and step S303 can adopt the corresponded manner in existing audio/video player method to realize, and the specific implementation of step 302 with reference to the embodiment of figure 1 correspondence, can not repeat them here.
With reference to figure 8, it is the structural representation of a kind of audio/video live broadcast device based on TCP in the embodiment of the present invention.This audio/video live broadcast device comprises:
Data reception module 41, for receiving packet and by described data pack buffer in buffering area;
Control device 42, for the buffer data duration to the described buffering area of adjustment; And
Decoding playing module 43, obtains packet and broadcasting of decoding based in the buffering area after the adjustment of buffer data duration.
Wherein, the specific implementation of data reception module 41 and decoding playing module 43 can adopt the corresponded manner in existing video play device to realize, and the specific implementation of control device 42 can adopt the control device based on TCP live broadcast system elimination delay variation shown in Fig. 5 to realize, do not repeat them here.
In sum, video broadcasting method based on TCP live broadcast system elimination delay variation disclosed by the invention, device and control method, device, effectively can solve the problem of delay time and jitter in video living transmission system, in self-adapted jitter buffer control method, by judging whether current buffer is empty, and in buffering area be not empty/for empty, respectively buffer data duration is shortened/increases adjustment, and the value adjusted is the value changed according to network state.Therefore, buffer size can be adjusted dynamically according to network state, and ensure to make time delay as far as possible little under prerequisite that is level and smooth, that guarantee broadcasting smoothness.
The above is the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, under the premise without departing from the principles of the invention; can also make some improvements and modifications, these improvements and modifications are also considered as protection scope of the present invention.

Claims (16)

1. eliminate a control method for delay variation based on TCP live broadcast system, it is characterized in that, comprising:
Judge whether current buffering area is empty;
When buffering area is not empty, when current adjustment cycle arrives, if the difference of the maximum buffer data duration in current adjustment cycle and minimal buffering district data duration is less than default buffering area duration dithering threshold, and described minimal buffering district data duration is greater than default minimal buffering district duration threshold value, then current buffer data duration is shortened, and the value shortened is less than or equal to the difference between described minimal buffering district data duration and described default minimal buffering district duration threshold value;
When buffering area is empty, initial buffer district data duration, to when receiving first packet then increases for empty by district to be buffered, and the initial buffer district data duration after increasing is the increasing function of second largest delay variation value; Wherein, described initial buffer district data duration is buffering area is the empty buffered data duration to recovering required for broadcasting; Described second largest delay variation value is buffering area is that unprecedented a period of time is to the Second Largest Value in the time difference receiving continuous print two packets received in the time of described first packet.
2. the control method of delay variation is eliminated as claimed in claim 1 based on TCP live broadcast system, it is characterized in that, when the initial buffer district data duration after described increase is greater than default maximum buffer duration threshold value, using described default maximum buffer duration threshold value as the initial buffer district data duration after described increase; When the initial buffer district data duration after described increase is less than default minimal buffering district duration threshold value, using described default minimal buffering district duration threshold value as the initial buffer district data duration after described increase.
3. the control method of delay variation is eliminated as claimed in claim 1 based on TCP live broadcast system, it is characterized in that, when described second largest delay variation value is less than default minimal time delay jitter value, using described default minimal time delay jitter value as described second largest delay variation value; When described second largest delay variation value is greater than default maximum delay jitter value, using described default maximum delay jitter value as described second largest delay variation value.
4. eliminate the control method of delay variation as claimed in claim 1 based on TCP live broadcast system, it is characterized in that, also comprise:
By the time difference stacked preservation of two packets received continuously.
5. eliminate the control method of delay variation as claimed in claim 1 based on TCP live broadcast system, it is characterized in that, the initial buffer district data duration after described increase is the linear increasing function of second largest delay variation value.
6. eliminate the control method of delay variation as claimed in claim 1 based on TCP live broadcast system, it is characterized in that, the initial buffer district data duration after described increase is the piecewise linearity increasing function of second largest delay variation value.
7. eliminate the control method of delay variation as claimed in claim 1 based on TCP live broadcast system, it is characterized in that, described a period of time is 1/3 ~ 1/2 of a complete adjustment cycle.
8. eliminate a control device for delay variation based on TCP live broadcast system, it is characterized in that, comprising:
Judge module, for judging whether current buffer is empty;
First adjusting module, for when buffering area be sky, when current adjustment cycle arrives, if the difference of the maximum buffer data duration in current adjustment cycle and minimal buffering district data duration is less than default buffering area duration dithering threshold, and described minimal buffering district data duration is greater than default minimal buffering district duration threshold value, then current buffer data duration is shortened, and the value shortened is less than or equal to the difference between described minimal buffering district data duration and described default minimal buffering district duration threshold value;
Second adjusting module, for being empty when buffering area, initial buffer district data duration, to when receiving first packet then increases for empty by district to be buffered, and the initial buffer district data duration after increasing is the increasing function of second largest delay variation value; Wherein, described initial buffer district data duration is buffering area is the empty buffered data duration to recovering required for broadcasting; Described second largest delay variation value is buffering area is that unprecedented a period of time is to the Second Largest Value in the time difference receiving continuous print two packets received in the time of described first packet.
9. the control device of delay variation is eliminated as claimed in claim 8 based on TCP live broadcast system, it is characterized in that, described second adjusting module also for: when the initial buffer district data duration after described increase is greater than default maximum buffer duration threshold value, using described default maximum buffer duration threshold value as the initial buffer district data duration after described increase; When the initial buffer district data duration after described increase is less than default minimal buffering district duration threshold value, using described default minimal buffering district duration threshold value as the initial buffer district data duration after described increase.
10. the control device of delay variation is eliminated as claimed in claim 8 based on TCP live broadcast system, it is characterized in that, described second adjusting module also for: when described second largest delay variation value is less than default minimal time delay jitter value, using described default minimal time delay jitter value as described second largest delay variation value; When described second largest delay variation value is greater than default maximum delay jitter value, using described default maximum delay jitter value as described second largest delay variation value.
11., as claimed in claim 8 based on the control device of TCP live broadcast system elimination delay variation, is characterized in that, also comprise:
Stacked preservation module, for the time difference stacked preservation of two packets that will receive continuously.
12. as claimed in claim 8 based on the control device of TCP live broadcast system elimination delay variation, and it is characterized in that, the initial buffer district data duration after described increase is the linear increasing function of second largest delay variation value.
13. as claimed in claim 8 based on the control device of TCP live broadcast system elimination delay variation, and it is characterized in that, the initial buffer district data duration after described increase is the piecewise linearity increasing function of second largest delay variation value.
14. as claimed in claim 8 based on the control device of TCP live broadcast system elimination delay variation, and it is characterized in that, described a period of time is 1/3 ~ 1/2 of a complete adjustment cycle.
15. 1 kinds, based on the audio/video live broadcasting method of TCP, is characterized in that, comprising:
Receive packet by described data pack buffer in buffering area;
Control method according to any one of claim 1 ~ 7 adjusts the buffer data duration of described buffering area;
Packet is obtained and broadcasting of decoding based in the buffering area after the adjustment of buffer data duration.
16. 1 kinds, based on the audio/video live broadcast device of TCP, is characterized in that, comprising:
Data reception module, for receiving packet and by described data pack buffer in buffering area;
Control device as described in any one of claim 8 ~ 14, for the buffer data duration to the described buffering area of adjustment; And
Decoding playing module, obtains packet and broadcasting of decoding based in the buffering area after the adjustment of buffer data duration.
CN201510483445.4A 2015-08-07 2015-08-07 Audio/video live broadcasting method, device and control method, device Active CN105142002B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510483445.4A CN105142002B (en) 2015-08-07 2015-08-07 Audio/video live broadcasting method, device and control method, device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510483445.4A CN105142002B (en) 2015-08-07 2015-08-07 Audio/video live broadcasting method, device and control method, device

Publications (2)

Publication Number Publication Date
CN105142002A true CN105142002A (en) 2015-12-09
CN105142002B CN105142002B (en) 2018-06-22

Family

ID=54727187

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510483445.4A Active CN105142002B (en) 2015-08-07 2015-08-07 Audio/video live broadcasting method, device and control method, device

Country Status (1)

Country Link
CN (1) CN105142002B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106658049A (en) * 2016-12-31 2017-05-10 深圳市优必选科技有限公司 Video playing buffering method and system
CN106713941A (en) * 2017-01-04 2017-05-24 北京百度网讯科技有限公司 Audio and video live broadcast implementation method and server
CN108650521A (en) * 2018-05-17 2018-10-12 上海七牛信息技术有限公司 A kind of live broadcasting method, device and multimedia play system
CN109640160A (en) * 2018-12-28 2019-04-16 鸿视线科技(北京)有限公司 Sustainable synchronous control system and method based on cushioning control structure
CN110266611A (en) * 2019-07-08 2019-09-20 腾讯科技(深圳)有限公司 Processing method, the device and system of buffered data
CN110634511A (en) * 2019-09-27 2019-12-31 北京西山居互动娱乐科技有限公司 Audio data processing method and device
CN112822502A (en) * 2020-12-28 2021-05-18 阿里巴巴(中国)有限公司 Live broadcast jitter removal intelligent caching and live broadcast method, equipment and storage medium
CN113727186A (en) * 2021-08-26 2021-11-30 百果园技术(新加坡)有限公司 Video playing method and system for real-time video call

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101123571A (en) * 2006-08-07 2008-02-13 北京三星通信技术研究有限公司 Adjustment method for scheduling policy based on self-adapted jitter buffer
US7359324B1 (en) * 2004-03-09 2008-04-15 Nortel Networks Limited Adaptive jitter buffer control
US7916742B1 (en) * 2005-05-11 2011-03-29 Sprint Communications Company L.P. Dynamic jitter buffer calibration
CN102739503A (en) * 2011-04-01 2012-10-17 华平信息技术股份有限公司 Processing method of VoIP (voice over internet protocol) de-jitter buffer
CN103533451A (en) * 2013-09-30 2014-01-22 广州华多网络科技有限公司 Method and system for regulating jitter buffer

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7359324B1 (en) * 2004-03-09 2008-04-15 Nortel Networks Limited Adaptive jitter buffer control
US7916742B1 (en) * 2005-05-11 2011-03-29 Sprint Communications Company L.P. Dynamic jitter buffer calibration
CN101123571A (en) * 2006-08-07 2008-02-13 北京三星通信技术研究有限公司 Adjustment method for scheduling policy based on self-adapted jitter buffer
CN102739503A (en) * 2011-04-01 2012-10-17 华平信息技术股份有限公司 Processing method of VoIP (voice over internet protocol) de-jitter buffer
CN103533451A (en) * 2013-09-30 2014-01-22 广州华多网络科技有限公司 Method and system for regulating jitter buffer

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106658049A (en) * 2016-12-31 2017-05-10 深圳市优必选科技有限公司 Video playing buffering method and system
CN106658049B (en) * 2016-12-31 2019-08-30 深圳市优必选科技有限公司 Video playing buffering method and system
CN106713941A (en) * 2017-01-04 2017-05-24 北京百度网讯科技有限公司 Audio and video live broadcast implementation method and server
CN108650521A (en) * 2018-05-17 2018-10-12 上海七牛信息技术有限公司 A kind of live broadcasting method, device and multimedia play system
CN109640160A (en) * 2018-12-28 2019-04-16 鸿视线科技(北京)有限公司 Sustainable synchronous control system and method based on cushioning control structure
CN110266611A (en) * 2019-07-08 2019-09-20 腾讯科技(深圳)有限公司 Processing method, the device and system of buffered data
CN110266611B (en) * 2019-07-08 2023-06-23 腾讯科技(上海)有限公司 Method, device and system for processing buffered data
CN110634511A (en) * 2019-09-27 2019-12-31 北京西山居互动娱乐科技有限公司 Audio data processing method and device
CN112822502A (en) * 2020-12-28 2021-05-18 阿里巴巴(中国)有限公司 Live broadcast jitter removal intelligent caching and live broadcast method, equipment and storage medium
CN112822502B (en) * 2020-12-28 2022-06-07 阿里巴巴(中国)有限公司 Live broadcast jitter removal intelligent caching and live broadcast method, equipment and storage medium
CN113727186A (en) * 2021-08-26 2021-11-30 百果园技术(新加坡)有限公司 Video playing method and system for real-time video call
CN113727186B (en) * 2021-08-26 2024-05-17 百果园技术(新加坡)有限公司 Video playing method and system for real-time video call

Also Published As

Publication number Publication date
CN105142002B (en) 2018-06-22

Similar Documents

Publication Publication Date Title
CN105142002A (en) Audio/video live broadcasting method and device as well as control method and device
US11349900B2 (en) Voice encoding and sending method and apparatus
US7359324B1 (en) Adaptive jitter buffer control
US9736548B2 (en) Multipath rate adaptation
WO2017000719A1 (en) Congestion control method and device based on queue delay
US8995268B2 (en) Method for controlling a flow in a packet switching network
CN101854308B (en) Self-adaptation realizing method of high-tone quality service network of VoIP system
EP2936770A1 (en) Apparatus and methods for controlling jitter buffer
CN108494698B (en) Congestion control method based on transmission rate
US20110010625A1 (en) Method for Manually Optimizing Jitter, Delay and Synch Levels in Audio-Video Transmission
US7072291B1 (en) Devices, softwares and methods for redundantly encoding a data stream for network transmission with adjustable redundant-coding delay
TW200849842A (en) Method of regulating the transmission jitter within a reception terminal
CN109495660B (en) Audio data coding method, device, equipment and storage medium
CN104168218B (en) Jitter buffer method and device
CN102905128A (en) Code rate controlling method of coding and decoding processor in wireless video transmission process
CN110191060A (en) A kind of adjustment data packet sends method, apparatus, equipment and the system of code rate
CN108881931A (en) A kind of data buffering method and the network equipment
CN105791735A (en) Method and system for dynamically adjusting video call code streams
CN105472404A (en) Data processing method in video service
WO2001024398A1 (en) Method and apparatus for multiplexing voice and data packets in a cdma system
CN107979482B (en) Information processing method, device, sending end, jitter removal end and receiving end
CN101090369A (en) Method for controlling data packet sending speed in flow medium system
CN111212308B (en) Method for self-adaptive adjustment of wireless network
CN112786062A (en) Bandwidth adaptive network audio transmission method, device, medium and system
US20130142192A1 (en) Voice communication apparatus for intermittently discarding packets

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant