WO2004045216A1 - Video streaming device and method of control for switchable video streams - Google Patents

Video streaming device and method of control for switchable video streams Download PDF

Info

Publication number
WO2004045216A1
WO2004045216A1 PCT/EP2003/010929 EP0310929W WO2004045216A1 WO 2004045216 A1 WO2004045216 A1 WO 2004045216A1 EP 0310929 W EP0310929 W EP 0310929W WO 2004045216 A1 WO2004045216 A1 WO 2004045216A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
switching
video
method
streaming device
Prior art date
Application number
PCT/EP2003/010929
Other languages
French (fr)
Inventor
Jonathan Soon Yew Teh
Anthony Richard May
Original Assignee
Motorola Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to GB0226452A priority Critical patent/GB2395387B/en
Priority to GB0226452.1 priority
Application filed by Motorola Inc filed Critical Motorola Inc
Publication of WO2004045216A1 publication Critical patent/WO2004045216A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities

Abstract

A method of controlling a video streaming device, comprising a switching means for switching between alternative data rate versions of one of a plurality of video streams. The switching means employs a plurality of switching methods. Switching method selection is controlled by data encoded within selected frames of each of the alternative data rate versions of the video stream. The encoded data indicates which if any of the plurality of switching methods to select at and/or after said selected frames for the desired video stream switch. This enables the vide streaming device to employ different switching method for each frame, enabling the reduction both of prediction errors after switching and data storage requirements.

Description

Video streaming device and method of control for switchable video streams

Technical Field

This invention relates to the area of streaming video transmission .

Background

Streaming video involves transmitting video data from a server across a communications channel to a client, where it is played back in real-time as the video data is being received. This communications channel may include a wireless channel, or a network such as a LAN, or the

Internet. This channel may not always deliver sufficient data rate for the video data. As such, video streams are usually encoded at different data rates in order to accommodate varying channel conditions.

Several existing video streaming systems that work over the Internet either measure the available data rate before the video data is transmitted, or alternatively allow the client device to indicate the available data rate. These systems then transmit the appropriate stream, that being the stream which has a data rate less than or egual to the available data rate.

However, such systems are unable to cope when the available data rate changes during transmission of video data. When the channel data rate drops, the video is paused and the client has to wait for its buffer to fill up with video data before playback is resumed. There is a need for a video-streaming server to be able to switch between alternative data rate video streams responsively as channel conditions change. However, implementing such a switch is not a matter of simply changing to the alternative stream midway through transmission, as the different frame types used in hybrid video coding have to be taken into account to ensure that the resulting output at the client is of sufficient quality.

As a further consideration, the server has limited resources in terms of storage space and processing power. Any solution will preferably try to optimise both such resources and quality.

A number of video stream switching methods are known in the art, each corresponding to the different frame types present in a hybrid video codec such as MPEG-4 or H.26L. P- and I-Frames are defined as follows:

a) A P-frame is an intercoded frame that may predict macroblocks (segments of the image) from previous frame (s), using what is termed forward prediction. P-frames are thus sensitive to stream continuity.

b) An I-frame is an intracoded frame that only uses prediction within the same frame, and can essentially be thought of as a single still image.

Consequently, there are P-frame and I-frame switching methods as outlined below.

P-frame switching is illustrated in figure 1. In figure 1, video stream two 120 can be assumed to have a lower data rate and lower frame rate than stream one 110, and the figure illustrates switching from stream one to stream two. In this case when a stream-switching request 130 is received, the server simply selects the next nearest frame 124 from the new stream, resulting in the transmitted frame sequence 111, 112, 113, 124, 125, ...

P-frame switching as illustrated in figure 1 will result in display errors. This is because the next nearest frame 124 in the stream two is encoded using predictions based on the previous frame in stream two, 123, but due to the switch 140 must now use predictions from frame 113 of stream one, which does not contain identical information.

In most cases, the decoder would consequently suffer from drift' , which occurs when subsequent P-frames are received at the decoder and the reconstructed image drifts further away from the originally encoded image. The stream does eventually recover as intra-coded macroblocks exist in the video stream which causes the influence of the prediction errors incurred at the switch to recede in time, but the perception by the user is of a period of low-integrity imaging.

I-frame switching, by contrast, ensures that no errors occur when a stream switch takes place. I-frame switching requires that I-frames be inserted within each video stream, often at the start of a new scene. Because I-frames do not employ inter-frame prediction, it is possible to switch between streams upon reaching the next nearest I- frame in the new stream without incurring a prediction error. I-frame switching is illustrated in figure 2. In figure 2, stream two 220 can be assumed to have a lower data rate and lower frame rate than stream one 210, and the figure illustrates switching from stream one to stream two. In this case, when a stream-switching request 230 is received, the server must wait until an I-frame is due to occur in stream two 220. At this point the streams switch, 240, resulting in the transmitted frame sequence 211, 212, 213, 214, 215, 225, 226, ...

I-frame switching has a number of disadvantages. I-frames use proportionately more data and so necessarily reduce the overall quality of the encoded stream for a given data rate. Moreover, I-frames are typically only included every 5 or 10 seconds, and so the responsiveness of the system to stream switch requests is slow, increasing the likelihood of stream pauses at the client. A final disadvantage is that at low bitrates, H.263 and MPEG-4 encoders generally do not insert I-frames at all, and so an assumed use of I- frame switching for multiple streams would require modifications to the encoders.

A third method exists that attempts to overcome the problems of I- and P-frame switching.

S-frame switching utilises an additional supplementary data stream that bridges the two video streams during the switching process. The S video stream is generated to contain data for the nearest decompressed frame in a second stream, based on the previous input frame in a first stream. The result is that the switch from stream one to stream two does not incur prediction errors of the type seen with direct P-frame switching. S-frame switching is illustrated in figure 3. In figure 3, stream two 320 can be assumed to have a lower data rate and lower frame rate than stream one 310, and the figure illustrates switching from stream one to stream two. In this case when a stream-switching request 330 is received the server uses the S-frame corresponding to the frame in stream one as the basis for prediction of the next frame in stream two. This results in the transmitted frame sequence 311, 312, 313, 314, 304, 325, 326, ...

Whilst S-Frame switching produces the best quality reconstructed frames at the decoder for minimal delay, it necessitat.es the generation of a separate S-frame stream for each possible switching scenario, increasing the storage requirements at the server.

If one assumes that a stream switch is only permitted between adjacent data rate streams, then for n possible data rates the server requires 2 (n-1) S-frame streams. If one tries to mitigate the additional storage cost by only providing a corresponding S-frame every 10 P-frames, for example, this again reduces the responsiveness of the server to data rate changes.

The three switching methods above constitute 3 possible cost/quality trade-offs;

i. P-switching incurs no cost, but impacts quality when there is frame misalignment between streams due to the different data rates. ii. I-switching preserves switching quality, but requires additional data that impacts on overall quality for a given data rate. iii. S-switching preserves switching quality, but incurs cost in the server by requiring significant increases in storage, and cost during the encoding process by increasing the amount of computation required to produce the streams.

No one prior art method provides the ideal solution. Thus, there is a need to optimise this cost/quality trade-off, to improve the efficiency of video stream switching.

Summary of the Invention

In accordance with the present invention, there is provided a method of controlling a video streaming device, as claimed in claim 1, and a video streaming device, as claimed in claim 8.

Brief description of the drawings

FIG. 1 shows the switching strategy for the P-frame switching method.

FIG. 2 shows the switching strategy for the I-frame switching method.

FIG. 3 shows the switching strategy for the S-frame switching method.

FIG. 4 shows the switching strategy for the P-frame switching method when P-Frames between video streams are aligned.

Detailed description of the preferred embodiment In a preferred embodiment of the present invention, selected frames of each video stream are augmented with switching strategy recommendation data. Thus for each selected frame, data indicating the use of P-frame, I frame or S-frame or another form of switching as the best option for that frame may be encoded, for each desired switching scenario. This would require 2 or fewer bits of information per scenario, and thus would impose little cost when compared to the typical 2,800 bits of information per P- frame in a 36kbps video stream.

In a preferred embodiment of the present invention, the generation of switching strategy recommendation data would use the following rules:

i. Use P-frame switching when frames are aligned; ii. Use I-frame switching when the next available I-frame is an acceptably short distance from the current frame; iii. Use any alternative switching method supported by the streaming protocol as appropriate; iv. Use S-frame switching for remaining frames.

The provision of switching guidance need not be limited to the three methods of P-frame, I-frame or S-frame switching. Therefore rule iii above acknowledges that additional switching methods may benefit from control by the present invention.

A pair of P-frames in two separate video streams are aligned when they have identical time-stamps. Figure 4 illustrates this occurrence, in which switching from stream one 410 to stream two 420 minimises prediction error as frames 413 and 422 are substantially identical for the purposes of forward prediction by frame 423.

Evaluation

Evaluations in support of the present invention suggest that a significant number of P-frames are aligned. The table below is based on a 90 second test video sequence containing a number of different scenes:

Figure imgf000009_0001

Thus rules i. and iv. above may reduce the required number of S-frames by approximately a third for these four switching scenarios which are 36 to 22kbps, 22 to 36kbps, 22 to 15kbps and 15 to 22kbps. Moreover, the proportion of aligned P-Frames increases with data rate, and consequently the use of the present invention reduces the number of required S-frames as S-frames get larger.

Inclusion of I-frames in the video stream, for example at the start of each new scene, would further reduce the required number of S-frames by use of rule ii.

Alternative Embodiment

In an alternative embodiment, it may be decided that it is not necessary to be able to switch between streams at every possible rame. For example, if it is known that the client buffer is capable of storing 2 seconds of video, then a switch delay of 0.5 seconds may be acceptable. In the alternative embodiment, S-frames are therefore only required where there is more than (for example) a 0.5 second gap until the next pair of aligned P-frames or until the next I-frame.

Evaluation

Evaluations in support of the present invention suggest that this would additionally reduce the need for S-frames. The table below is based on a 90 second test video sequence containing a number of different scenes:

Figure imgf000010_0001

The final right-hand column suggests the likely proportion of required S-Frames relative to the number of P-frames. For the higher data rate example, fewer than 1 in 8 P- frames will require corresponding S-frames, as P-frame alignment occurs on average every half second. Where there are longer gaps, one frame in every 0.5s of the gap will have its switching strategy recommendation data encoded to provide an S-frame switch at that point, and a corresponding S-frame will be generated. For the lower data rate example, roughly 1 in 5 P-frames will require corresponding S-frames as P-frame alignment occurs on average every second, so requiring typically one S-frame between such alignments.

Again, the addition of I-frames will further reduce the S- fra e requirement . In summary, a preferred embodiment of the present invention incorporates switching strategy recommendation data within video frame data, for alternative data rate video streams. The switching strategy recommendation data indicates which of a plurality of alternative switching strategies to use for a given frame, based on rules concerning the location of frames that permit low error switching to occur, such as I-frames and aligned P-frames. The benefit of such an arrangement is to maintain quality whilst minimising cost, particularly in terms of S-frame generation and storage. An alternative embodiment proposes further reductions in S- frame. numbers, by only recommending S-frames if the temporal proximity of aligned P-frames or I-frames exceeds a stipulated duration.

Claims

Claims
1. A method of controlling a video streaming device, the video streaming device comprising a switching means for switching between alternative data rate versions of one of a plurality of video streams (110, 120, 210, 220, 310, 320, 410, 420) , the switching means employing a plurality of switching methods, the method of controlling comprising selecting the switching method in dependence on: the encoding of data within one or more selected frames of each of the plurality of alternative data rate versions of the video stream, the encoded data indicating which, if any, of the plurality of switching methods to select at and/or after said selected frame, in order to make the desired video stream switch.
2. A method of controlling a video streaming device according to claim 1, wherein: the switching means employs any of I-frame, P-frame or
S-frame switching; and the selection of I-frame, P-frame or S-frame switching is determined by the encoding of data within the selected frames indicating which, if any, of I-frame, P-frame or S- frame switching to select at and/or after said selected frames .
3. A method of controlling a video streaming device according to claim 1 or claim 2, wherein every video frame is selected for the encoding of data.
4. A method of controlling a video streaming device according to any previous claim, wherein switching method selection data is encoded to include method selection information only for the next higher alternative data rate video stream corresponding to the current video stream, where available, and next lower alternative data rate video stream corresponding to the current video stream, where available.
5. A method of controlling a video streaming device according to any previous claim, wherein S-frames that have been generated for the purpose of S-frame switching between a pair of alternative data rate versions of the video stream are only stored if the encoded data on either of the aforesaid pair of alternative data rate versions of the video stream indicates that the S-frame is required.
6. A method of controlling a video streaming device according to any previous claim, wherein the encoded data indicating which if any of the plurality of switching methods to select is based on the following rules; i. Use P-frame switching when frames are aligned; ii. Use I-frame switching when the next available I-frame is reached, if the next available I-frame is an acceptably short distance from the current frame for the purposes of the application; iii. Use any alternative switching method supported by the streaming protocol as appropriate; iv. Use S-frame switching for remaining frames.
7. A method of controlling a video streaming device according to any of claims 1 to 5, wherein the encoded data indicating which, if any, of the plurality of switching methods to select is based on the following rules; i. if the next available aligned P-frame is an acceptably short distance from the current frame for the purposes of the application, then use P-frame switching when the next available aligned P-frame is reached; ii. if the next available I-frame is an acceptably short distance from the current frame for the purposes of the application, then use I-frame switching when the next available I-frame is reached; iii. if the availability of said switching method occurs in an acceptably short time for the purposes of the application, then use any alternative switching method supported by the streaming protocol when said alternative switching method becomes available; iv. if rules i., ii. or iii. are not met, then use S-frame switching for each frame where the onset of the subsequent frame according to rule i, ii or iii would exceed the acceptable time following the last available switching position.
8. A video streaming device comprising a switching means for switching between alternative data rate versions of one of a plurality of video streams (110, 120, 210, 220, 310, 320, 410, 420), the switching means employing a plurality of switching methods, and the switching means being adapted to select the data rate version of a video stream in dependence on: encoded data within one or more selected frames of each of the plurality of alternative data rate versions of the video stream; the encoded data indicating which, if any, of the plurality of switching methods to select at and/or after said selected frames in order to make the desired video stream switch.
9. A video streaming device according to claim 8, wherein the video streaming device is a server.
PCT/EP2003/010929 2002-11-13 2003-09-29 Video streaming device and method of control for switchable video streams WO2004045216A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
GB0226452A GB2395387B (en) 2002-11-13 2002-11-13 Video streaming device and method of control for switchable video streams
GB0226452.1 2002-11-13

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
AU2003267427A AU2003267427A1 (en) 2002-11-13 2003-09-29 Video streaming device and method of control for switchable video streams

Publications (1)

Publication Number Publication Date
WO2004045216A1 true WO2004045216A1 (en) 2004-05-27

Family

ID=9947730

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2003/010929 WO2004045216A1 (en) 2002-11-13 2003-09-29 Video streaming device and method of control for switchable video streams

Country Status (4)

Country Link
AU (1) AU2003267427A1 (en)
GB (1) GB2395387B (en)
HK (1) HK1063402A1 (en)
WO (1) WO2004045216A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2522984C2 (en) * 2008-08-26 2014-07-20 Ксир Method of switching from source encoded video stream to destination encoded video stream
TWI561062B (en) * 2013-02-27 2016-12-01 Apple Inc Adaptive streaming techniques
US9571827B2 (en) 2012-06-08 2017-02-14 Apple Inc. Techniques for adaptive video streaming

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1638333A1 (en) * 2004-09-17 2006-03-22 Mitsubishi Electric Information Technology Centre Europe B.V. Rate adaptive video coding
US8635360B2 (en) 2007-10-19 2014-01-21 Google Inc. Media playback point seeking using data range requests
WO2009075766A2 (en) 2007-12-05 2009-06-18 Swarmcast, Inc. Dynamic bit rate scaling
US7979570B2 (en) 2008-05-12 2011-07-12 Swarmcast, Inc. Live media delivery over a packet-based computer network
WO2009155356A1 (en) 2008-06-18 2009-12-23 Onion Networks, KK Traffic and cost containment for internet access by adapting the coding rate when distributing- media content
WO2010065757A1 (en) * 2008-12-04 2010-06-10 Swarmcast, Inc. Adaptive playback rate with look-ahead
JP5267655B2 (en) 2009-03-19 2013-08-21 富士通株式会社 Image coding apparatus, image coding control method, and image coding program
US9948708B2 (en) 2009-06-01 2018-04-17 Google Llc Data retrieval based on bandwidth cost and delay

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020002708A1 (en) * 2000-06-27 2002-01-03 Bamboo Mediacasting, Inc Multicasting transmission of multimedia information
WO2002049343A1 (en) * 2000-12-15 2002-06-20 British Telecommunications Public Limited Company Transmission and reception of audio and/or video material
US6480541B1 (en) * 1996-11-27 2002-11-12 Realnetworks, Inc. Method and apparatus for providing scalable pre-compressed digital video with reduced quantization based artifacts

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7086077B2 (en) * 1999-04-01 2006-08-01 Sedna Patent Services, Llc Service rate change method and apparatus

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6480541B1 (en) * 1996-11-27 2002-11-12 Realnetworks, Inc. Method and apparatus for providing scalable pre-compressed digital video with reduced quantization based artifacts
US20020002708A1 (en) * 2000-06-27 2002-01-03 Bamboo Mediacasting, Inc Multicasting transmission of multimedia information
WO2002049343A1 (en) * 2000-12-15 2002-06-20 British Telecommunications Public Limited Company Transmission and reception of audio and/or video material

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
GIROD B ET AL: "Internet Video-Streaming" PUBLICATIONS OF THE UNIVERSITY OF ERLANGEN, 1998, XP002242167 Retrieved from the Internet: <URL:http://www.nt.e-technik.uni-erlangen. de/LNT_I/publications/pub_list/p ub_files/ln> [retrieved on 2003-05-22] *
HANNUKSELA M M: "H.26L FILE FORMAT" , ITU TELECOMMUNICATIONS STANDARDIZATION SECTOR VCEG-O44, XX, XX, PAGE(S) COMPLETE XP001148202 *
WALKER T ET AL: "FIRST IDEAS ON THE STORAGE OF AVC CONTENT IN MP4" , INTERNATIONAL STANDARD ISO/IEC, XX, XX, PAGE(S) 1-39 XP001089885 page 5, paragraph [3.1.2] Stream Switching page 8 - page 15, chapter 8 Stream Switching *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2522984C2 (en) * 2008-08-26 2014-07-20 Ксир Method of switching from source encoded video stream to destination encoded video stream
US9571827B2 (en) 2012-06-08 2017-02-14 Apple Inc. Techniques for adaptive video streaming
TWI561062B (en) * 2013-02-27 2016-12-01 Apple Inc Adaptive streaming techniques
US9992499B2 (en) 2013-02-27 2018-06-05 Apple Inc. Adaptive streaming techniques

Also Published As

Publication number Publication date
GB0226452D0 (en) 2002-12-18
GB2395387A8 (en) 2004-06-02
GB2395387B (en) 2005-02-16
AU2003267427A1 (en) 2004-06-03
HK1063402A1 (en) 2005-08-12
GB2395387A (en) 2004-05-19

Similar Documents

Publication Publication Date Title
JP5770345B2 (en) Video switching for streaming video data
KR101150102B1 (en) Fast channel change
EP2919453B1 (en) Video stream switching
CN1330185C (en) Multicast transmission system comprising a bandwidth scaler
EP1157562B1 (en) Video coding
KR100629158B1 (en) Method and system for buffering streamed data
JP4609753B2 (en) Method and system for staggered statistical multiplexing
KR100931915B1 (en) Grouping of image frames during video coding
JP4309098B2 (en) Method of transmitting layered video coding information
CN100592801C (en) Method and equipment for grouping of image frames in video coding
JP3788260B2 (en) Distribution system and its delivery method, and delivery program
JP4087852B2 (en) A method of transmitting video
JP5166021B2 (en) Method and system for enabling fast channel changes for DSL systems
US20020004840A1 (en) Streaming method and system for executing the same
Wenger Video redundancy coding in H. 263+
US9843844B2 (en) Network streaming of media data
US7693220B2 (en) Transmission of video information
JP3472115B2 (en) Video data transmission method and apparatus using a multi-channel
CN1476248B (en) Video-data transmitting and receiving system, apparatus and method
Apostolopoulos et al. Video streaming: Concepts, algorithms, and systems
US6327421B1 (en) Multiple speed fast forward/rewind compressed video delivery system
US8914835B2 (en) Streaming encoded video data
KR100868820B1 (en) A method and system for communicating a data stream and a method of controlling a data storage level
CN1242623C (en) Video coding method, decoding method and associated coder and decoder
US8245264B2 (en) Methods and systems to reduce channel selection transition delay in a digital network

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PH PL PT RO RU SC SD SE SG SK SL TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct app. not ent. europ. phase
NENP Non-entry into the national phase in:

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP