EP2404449A1 - Diffusion de vidéo en continu - Google Patents

Diffusion de vidéo en continu

Info

Publication number
EP2404449A1
EP2404449A1 EP10707637A EP10707637A EP2404449A1 EP 2404449 A1 EP2404449 A1 EP 2404449A1 EP 10707637 A EP10707637 A EP 10707637A EP 10707637 A EP10707637 A EP 10707637A EP 2404449 A1 EP2404449 A1 EP 2404449A1
Authority
EP
European Patent Office
Prior art keywords
quality
buffer
estimated
gop
bit rate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
EP10707637A
Other languages
German (de)
English (en)
Inventor
Michael Erling Nilsson
Rory Stewart Turnbull
Ian Barry Crabtree
Stephen Clifford Appleby
Patrick Joseph Mulroy
Steve Hoare
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
British Telecommunications PLC
Original Assignee
British Telecommunications PLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by British Telecommunications PLC filed Critical British Telecommunications PLC
Priority to EP10707637A priority Critical patent/EP2404449A1/fr
Publication of EP2404449A1 publication Critical patent/EP2404449A1/fr
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/15—Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164—Feedback from the receiver or from the transmission channel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • This invention relates to video streaming over networks, and is particularly useful in the case of networks with a non-deterministic bandwidth availability.
  • a situation is typical of the Internet, where packet delivery is by best-effort, or where the physical medium has an inherently non-deterministic behaviour, such as wireless connections.
  • This invention is applicable both in situations where a compressed asset is available in advance of delivery (such as Video on Demand), and where the complete asset is not available in advance (such as streaming a live event).
  • a compressed asset is available in advance of delivery (such as Video on Demand)
  • the complete asset is not available in advance (such as streaming a live event).
  • a server for streaming video includes a coding apparatus as shown in Figure 1 , comprising a video interface 1 that receives digitally coded video signals, in uncompressed form, at a video input 2.
  • a processor 3 operates under control of programs stored in disc storage 4 and has access to memory 5 and a video output buffer 6 that feeds a video output 7.
  • the programs include a general purpose operating system 40 and video coding software 41 which implements one of more of the coding methods shortly to be described.
  • the server transmits to a client, including a video decoder. This can be of conventional construction and is therefore not illustrated. It is however worth mentioning that the client includes a buffer for buffering received video data until it can be decoded.
  • each media asset is partitioned into time slices, and each time slice is encoded at a number of qualities.
  • the asset is streamed by sending time slices in temporal order from any of the different quality streams.
  • Time-slices will typically be coded independently (e.g. as a closed group of pictures, where each group begins with an I-frame) but may also support a switching picture framework to allow more bit efficient concatenation.
  • Stream switching as a mechanism to cope with dynamically changing bandwidth is a well known technique. Our objective here, however, is to select which quality to stream at each time-slice to maximise the perceptual quality of the overall stream, subject to the constraints of available bandwidth and client buffering.
  • each time slice is a group of pictures (GOP) and is encoded with a fixed set of qualities.
  • the invention is equally applicable to the case where a different set of qualities is used for each time slice.
  • all GOPs were encoded in advance, though if desired they could be encoded as required, with a sufficient look-ahead that the necessary results are available when needed.
  • Encoding at quality level j is preferably performed as described in our co-pending European patent application no. 08250815.1 (agent's ref.A31594) wherein each index j corresponds to a respective fixed perceptual quality.
  • each node represents the resulting buffer state (suitably quantised) after transmitting a time slice at a given quality, and each link represents a quality selection decision.
  • the buffer states in the trellis need to be quantised to such a level that there are sufficient number of states to find an optimum solution.
  • PCT/GB2008/003691 (agent's ref A31511) gives further details of a buffer state Viterbi trellis used for constant quality video encoding.
  • Evaluating the quality of a path through this lattice is not- as simple as taking the average of the qualities of the individual slices. For instance, it is well-known that constant quality is preferable to variable quality, even though the variable quality stream may have a higher average quality (See D. Hands & K. Cheng, Subject responses to constant and variable quality video, Human Vision and Electronic Imaging XIII 2008, SPIE Electronic Imaging, San Jose, California, USA) . It is also often suggested that over certain timescales, the perceived quality is biased towards the lower end of the qualities seen. Here, we assume that we have some quality measure that can rank paths through this lattice.
  • a buffer is used to provide some decoupling from the delivery time of (compressed) media samples, and their playout time. This buffering allows smoothing of a variable delivery rate of media samples.
  • the quality of the media playout is reduced.
  • the nature of the reduction will depend on a number of factors, including the masking ability of the client, the transport protocol used and whether buffer overflow/underflow occurs for audio or video or some other type of media.
  • a timeslice encoded at higher quality will produce more data than one encoded at lower quality. Therefore, the use of higher quality timeslices will cause data to be played out from the client buffer at a higher rate, and will need a higher rate of delivery over the network to prevent buffer underflow.
  • / and M are timeslice indices o, is the probability of buffer overflow just before we remove time slice i+l from the buffer, u, is the probability of buffer underflow just after we remove time slice i from the buffer.
  • I indicates the most recent GOP whose quality has been determined.
  • mi is the estimated bit rate, and is used to determine the quality of GOP 1+ 1.
  • B S i is the buffer level before GOP i is removed from the decoder buffer.
  • Q s, is the quality selected for GOP i-1.
  • C s, i is the cost before the decision for GOP i is made. The choice for GOP i affects the next
  • the whole media asset is encoded at each of the quality levels, and the number of bits used to encode each GOP i at each quality j, bj 0 , is recorded.
  • an initial estimate is made of the mean bit rate at which delivery through the network might be achieved. This may be derived from measurements made during preceding exchanges of information between the server and the client, in which, for example, the client requested the content; or it may be a value derived by the server based on how many other streams it is already delivering, the time of day or some other factor; or it may simply be a constant value.
  • this initial mean bit delivery rate as ⁇ I INIT - AS the quality of GOPs is determined and as they are transmitted, we will update this mean bit delivery rate, being referred to as In 1 at the time when the quality of GOP 1+1 is to be determined.
  • mi is not necessarily the actual bit rate at time I; rather, it is the most up- to date estimate of bit-rate available to be used in calculations concerning GOP 1+1.
  • 203 We determine a start up delay to be signalled to the receiver, indicating how long the receiver should wait between first receiving data to decode and removing all of the data representing the first GOP instantaneously from its buffer and starting to decode that data.
  • This value may be set to a fixed value, such as more or less than one GOP period, or may be set according to the video asset to be delivered, being longer for an asset for which the initial video scenes are particularly difficult to compress.
  • this start up delay figure as D.
  • s F * (number of quantised buffer states — 1) / maximum buffer size, where 7' indicates integer division with rounding.
  • the first GOP is transmitted at the selected quality level, and the value of the start up delay, D, is also transmitted.
  • the transmission of the GOP is monitored, and the statistics of the network throughput are updated to derive, if necessary, a new value of mean bit rate, mo, and parameters indicative of its variability.
  • the GOP (GOP I) is transmitted at the selected quality level. The transmission of the GOP is monitored, and the statistics of the network throughput are updated to derive, if necessary, a new value of mean bit rate, mi, and parameters indicative of its variability.
  • each node represents the resulting buffer state (suitably quantised) after transmitting a GOP at a given quality, and where each link represents a quality selection decision.
  • 309 Perform the actions in steps 110 to 119 for each value of quality index j and for each state s for which the state variable S s j is not marked as inactive.
  • s' F'* (number of quantised buffer states - I)/ maximum buffer size (with rounding).
  • Q s ⁇ is the quality at which GOP i-1 was coded on the path through the trellis to state s for GOP i.
  • o S ⁇ i j is the probability of buffer overflow just before we remove timeslice z+1 at when timeslice / has been encoded at quality y starting in state s
  • u Si i j is the probability of buffer underflow just after we remove timeslice i at quality y starting in state s.
  • step 315 If state variable S s ',i +1 is marked as active go to step 317, else go to step 316.
  • step 328 If there is only one possible path from GOP I to GOP 1+1 after the pruning process then exit to step 328.
  • step 307 If there are more GOPs to be processed then repeat for next GOP by returning to step 307; otherwise (325), from the set of paths arriving to the end of the file choose the path that leads to the best final cost, prune all other paths and exit to step 328. Note other choices for the best final state are possible: best cost after one GOP (or any number), highest lowest buffer level along trellis path etc. 328 Return the chosen quality transition path
  • step 313 we need to estimate the number of bits we would expect to be able to get through the network by the given time and a measure of the accuracy of that estimate.
  • Another option is to continually collect statistics as to bit rate and compute the actual standard deviation over a recent time window.
  • the probability given the standard deviation ⁇ * that the buffer fullness will actually reach zero - i.e. a deviation of (B s, i - bj j )/ ⁇ j times the standard deviation can be looked up in a Gaussian cumulative probability table. Alternatively it can be calculated from
  • the buffer would overflow if the fullness exceeds the buffer size B.
  • the probability of this can be found by looking up (F' -B)/ ⁇ j in a Gaussian cumulative probability table, or from
  • the method caters for variability in the actual bandwidth through the incorporation of the under and over flow probabilities. It can be further extended however by varying the bandwidth estimate as we propagate through the lattice based on other available information about future network bandwidth. There may be known events such as other streams about to end, which introduce a dependence on other streams but which would mean an imminent bandwidth increase. Downstairs rate curves were introduced .in the context of optimal bandwidth reservation for VBR coded video (See K. Sun & M. Ghanbari, An Algorithm for VBR video transmission scheme over the Internet, in
  • Any VBR asset will have a peak rate requirement to ensure no buffer starvation problems at the most difficult part of the content. Once this point is passed the next peak rate will be lower and so on.
  • This series of peak rates form a downstairs stepping profile and this future profile of all currently streamed assets may also be available. If streamed at a rate proportional to this rate requirement (e.g. using MuITCP and variable N) this would suggest with no new streams added there will be more rate and less contention moving forward in time.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Error Detection And Correction (AREA)

Abstract

Une séquence vidéo à coder est divisée en une pluralité de parties temporelles et analysée de manière à déterminer (au moins pour chaque partie après la première), selon une pluralité de réglages de qualité de codage, (i) une métrique de qualité de la partie et (ii) le nombre de bits générés en codant la partie selon ce réglage de qualité. Ces données sont analysées, par exemple à l'aide d'un treillis du genre Viterbi, de manière à choisir un ensemble de réglages de qualité, un par partie, qui tend à réduire au minimum un coût de qualité combiné de la séquence. Ce coût de qualité combiné est la somme de coûts de qualité individuels, chacun d'eux étant une fonction de la métrique de qualité de la partie codée respective. La séquence est codée à l'aide des réglages de qualité choisis. Afin de déterminer chaque coût de qualité individuel, en dépit du fait que l'on ne connaît pas de manière précise le débit de réseau qui sera disponible à un moment donné à l'avenir, on procède par estimation du degré de remplissage de la mémoire tampon du récepteur et de son écart type. A partir desdites destinations, on obtient la probabilité de soupassement et/ou de surpassement de la mémoire tampon, les coûts de qualité individuels étant également fonction de la probabilité de soupassement et/ou de surpassement.
EP10707637A 2009-03-05 2010-03-04 Diffusion de vidéo en continu Ceased EP2404449A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP10707637A EP2404449A1 (fr) 2009-03-05 2010-03-04 Diffusion de vidéo en continu

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP09250628A EP2227023A1 (fr) 2009-03-05 2009-03-05 Diffusion vidéo
EP10707637A EP2404449A1 (fr) 2009-03-05 2010-03-04 Diffusion de vidéo en continu
PCT/GB2010/000390 WO2010100427A1 (fr) 2009-03-05 2010-03-04 Diffusion de vidéo en continu

Publications (1)

Publication Number Publication Date
EP2404449A1 true EP2404449A1 (fr) 2012-01-11

Family

ID=41092107

Family Applications (2)

Application Number Title Priority Date Filing Date
EP09250628A Ceased EP2227023A1 (fr) 2009-03-05 2009-03-05 Diffusion vidéo
EP10707637A Ceased EP2404449A1 (fr) 2009-03-05 2010-03-04 Diffusion de vidéo en continu

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP09250628A Ceased EP2227023A1 (fr) 2009-03-05 2009-03-05 Diffusion vidéo

Country Status (3)

Country Link
EP (2) EP2227023A1 (fr)
CN (1) CN102369732B (fr)
WO (1) WO2010100427A1 (fr)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103428107B (zh) * 2012-05-14 2016-08-24 中国科学院声学研究所 一种基于缓存下溢概率估计的自适应码流切换方法及系统
AU2014224432B2 (en) 2013-03-08 2019-10-24 Oxford Nanopore Technologies Limited Enzyme stalling method
US11076187B2 (en) 2015-05-11 2021-07-27 Mediamelon, Inc. Systems and methods for performing quality based streaming
WO2016183251A1 (fr) 2015-05-11 2016-11-17 Mediamelon, Inc. Systèmes et procédés pour effectuer une diffusion en continu basée sur la qualité
US10911513B2 (en) 2018-07-16 2021-02-02 Netflix, Inc. Techniques for determining an upper bound on visual quality over a completed streaming session

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6490320B1 (en) * 2000-02-02 2002-12-03 Mitsubishi Electric Research Laboratories Inc. Adaptable bitstream video delivery system
WO2006078594A1 (fr) * 2005-01-19 2006-07-27 Thomson Licensing Procede et appareil de codage parallele en temps reel

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
None *
See also references of WO2010100427A1 *

Also Published As

Publication number Publication date
EP2227023A1 (fr) 2010-09-08
WO2010100427A1 (fr) 2010-09-10
WO2010100427A8 (fr) 2011-10-06
CN102369732A (zh) 2012-03-07
CN102369732B (zh) 2015-09-23

Similar Documents

Publication Publication Date Title
EP2612495B1 (fr) Diffusion en flux adaptative de vidéos de différents niveaux de qualité
US7054371B2 (en) System for real time transmission of variable bit rate MPEG video traffic with consistent quality
US20030165150A1 (en) Multi-threshold smoothing
US8996713B2 (en) Video streaming
CN104205769B (zh) 利用播放速率选择的改善的dash客户端和接收机
US7706384B2 (en) Packet scheduling with quality-aware frame dropping for video streaming
EP2364551B1 (fr) Lecture vidéo en transit multiplexée
US20030195977A1 (en) Streaming methods and systems
CN104584495B (zh) 用于无线网络上的自适应速率多媒体通信的装置和方法
KR20040041170A (ko) 혼잡 제어를 위한 전송률을 계산하기 위해 수신 버퍼 크기를 사용하는 데이터 통신 방법 및 시스템
JP2005538606A (ja) ビデオシーケンスのレートを制御する方法及び装置並びにビデオ符号化装置
Zahran et al. OSCAR: An optimized stall-cautious adaptive bitrate streaming algorithm for mobile networks
WO2010100427A1 (fr) Diffusion de vidéo en continu
Chou et al. Rate-distortion optimized receiver-driven streaming over best-effort networks
Peng et al. A hybrid control scheme for adaptive live streaming
CN112437321B (zh) 一种基于直播流媒体的自适应码率计算方法
CN110072130A (zh) 一种基于http/2的has视频切片推送方法
CN106921860B (zh) 一种端到端的视频发送方法及装置
US7533075B1 (en) System and method for controlling one or more signal sequences characteristics
AT&T NimbusSanL-Regu
Bouras et al. Evaluation of single rate multicast congestion control schemes for MPEG-4 video transmission
KR100782343B1 (ko) 영상 스트리밍 방법
Turaga et al. Adaptive live streaming over enterprise networks
ZHANG et al. Joint rate allocation and buffer management for robust transmission of VBR video
Tunali et al. Robust quality adaptation for internet video streaming

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20110930

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20121128

REG Reference to a national code

Ref country code: DE

Ref legal event code: R003

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED

18R Application refused

Effective date: 20170116