AU2011205017B2 - System and method for scalable video coding using telescopic mode flags - Google Patents

Abstract

C:4RPonblDCC\DERU77857 _l.DOC-27/07/2011 A system for decoding of scalable digital video, the system comprising: an input configured to receive a scalable digital video bitstream comprising slices 5 of a quality or spatial enhancement target layer and at least one additional layer in accordance with the SVC JD8 specification, the digital video bitstream containing control data (including prediction control data) associated with slices and control (including prediction), texture, or motion data associated with macroblocks or macroblock partitions; a decoder coupled to the input, wherein the decoder is configured to decode the 10 control data associated with a slice of the target layer and the at least one additional layer, and control, texture, or motion data associated with individual macroblock or macroblock partitions of the target layer and the at least one additional layer; a predictor coupled to the decoder, wherein the decoder is configured to generate prediction references for the control, texture, or motion data of a plurality of macroblocks 15 or macroblock partitions of the target layer as signaled prediction control data associated with a slice of the target layer or the at least one additional layer, or from prediction control data associated with individual macroblocks or macroblock partitions of the target layer or the at least one additional layer; and a combiner coupled to the predictor, wherein the combiner is configured to 20 combine the generated prediction references with the corresponding decoded control, texture, or motion data associated with the plurality of macroblocks or macroblock partitions of the target layer to produce portions of a decoded picture corresponding to the plurality of macroblocks or macroblock partitions of the target layer, 25 wherein the prediction control data associated with the slices of the target layer or the at least one additional layer include an adaptiveresidual_prediction flag parameter and, if the adaptiveresidualprediction flag parameter is not set, a defaultresidual_ predictionflag parameter and the prediction control data associated with individual macroblocks or macroblock partitions of the target layer or the at least one additional layer 30 include a residualpredictionflag parameter, and wherein the decoder is configured not to decode the residualjprediction flag parameter in macroblock or macroblock partitions of a C:\NR~tbi\DCC\DER\77557 II.DOC-27/07/2011I slice for which the adaptive residualpredictionflag parameter is set but decode instead the defaultresidual_predictionflag of the slice, and wherein the predictor is further configured to use the value of defaultresidual_prediction flag as the value for the residualprediction flag parameter for all macroblocks or macroblock partitions of the 5 slice.

Description

Regulation 3.2 AUSTRALIA Patents Act 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT (ORIGINAL) Name of Applicant: Vidyo, Inc., 433 Hackensack Avenue, 6th floor, Hackensack, New Jersey 07601, UNITED STATES OF AMERICA Actual Inventors: ELEFTHERIADIS, Alexandros HONG, Danny SHAPIRO, Ofer Address for Service: DAVIES COLLISON CAVE, Patent Attorneys, of I Nicholson Street, Melbourne 3000, Victoria, Australia Invention Title: System and method for scalable video coding using telescopic mode flags The following statement is a full description of this invention, including the best method of performing it known to us: C:\NRPotbl\DCC\DER\3778055_1.DOC - 27/7/i i SYSTEM AND METHOD FOR SCALABLE VIDEO CODING USING TELESCOPIC MODE FLAGS 5 SPECIFICATION CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of United States provisional patent application Serial No. 60/862,510 filed October 23, 2006. Further, this application is related to International patent application Nos. PCT/US06/028365, 10 PCT/US06/028366, PCT/US06/028367,, PCT/US06/027368, PCT/US06/061815, PCT/US06/62569, PCT/US07/80089, PCT/US07/062357, PCT/US07/65554, PCT/US07/065003, PCT/US07/63335, and PCT/USO7/81217. All of the aforementioned applications, which are commonly assigned, are hereby incorporated by reference herein in their entireties. 15 FIELD OF THE INVENTION The present invention relates to video communication systems. In particular, the invention relates to communication systems that use scalable video coding techniques in which two or more layers are used to represent a given video signal at corresponding levels of fidelity in terms of the signal's temporal, quality, or spatial 20 resolution. BACKGROUND OF THE INVENTION New digital video coding techniques, which are directed to general improvements in coding efficiency, have a number of new structural characteristics. Specifically, an important new characteristic is scalability. In scalable coding, an Specifically, an important new characteristic is scalability. In scalable coding, an original or source signal is represented using two or more hierarchically structured bitstreams. The hierarchical structure implies that decoding of a given bitstream depends on the availability of some or all. other bitstreams that are lower in the 5 hierarchy. Each bitstream, together with the bitstreams it depends on, offers a representation of the original signal at a particular temporal, quality (i.e., in terms of Signal-to-Noise Ratio, SNR), or spatial resolution. It is understood that term 'scalable' does not refer to magnitude or scale in terms of numbers, but rather to the ability of the encoding technique to offer a set of 10 different bitstreams corresponding to representations of the original or source signal at different 'scales' of resolutions or other qualities in general. The ITU-T H.264 Annex G specification, which is referred to as Scalable Video Coding (SVC), is an example of a video coding standard that offers video coding scalability in all of temporal, quality, or spatial dimensions. SVC is an extension of the H.264 standard (also 15 known as Advanced Video Coding or AVC). An example of an earlier standard, which also offered all three types of scalability, is ISO MPEG-2 (also published as ITU-T H.262). ITU G.729.1 (also known as G.729EV) is an example of a standard offering scalable audio coding. Scalable video coding techniques which are specifically designed for interactive video communication applications such as 20 videoconferencing are described in commonly assigned International patent application PCT/US06/028365. The concept of scalability was introduced in video and audio coding as a solution to distribution problems in streaming and broadcasting, and with a view to allow a given communication system to operate with varying access networks (e.g., 25 clients connected with different bandwidths), network conditions (e.g., bandwidth 2 fluctuation), and client devices (e.g., a personal computer that uses a large monitor vs. a handheld device with a much smaller screen). Commonly assigned International patent application PCT/US06/028365 describes the design of a new type of server called the Scalable Video 5 Communication Server (SVCS). SVCS can advantageously use scalable coded video for high-quality and low-delay video communication, and has a complexity, which is significantly reduced compared to traditional switching or transcoding Multipoint Control Units (MCUs). Similarly, commonly assigned International patent application PCT/US06/62569 describes a Compositing Scalable Video Coding Server 10 (CSVCS), which has the same benefits as an SVCS but produces a single coded output bit stream. International patent application PCTIUS07/80089 describes a Multicast Scalable Video Coding Server (MSVCS), which has the same benefits as an SVCS but utilizes available multicast communication channels. For convenience in the following description, the three different types of servers (SVCS, CSVCS, and 15 MSVCS) will be conmonly referred to as an SVCS, unless otherwise stated. The scalable video coding design and the SVCS architecture can be used in further advantageous ways, which are described, for example, in commonly assigned International patent applications PCT/US06/028367, PCT/US06/027368, PCT/US06/061815, PCT/US07/062357, and PCT/US07/063335. These applications 20 describe use of scalable coding techniques and SVCS architectures for effective trunking between servers, reduced jitter buffer delay, error resilience and random access, "thinning" of scalable video bitstreams to improve coding efficiency with reduced packet loss, and rate control, respectively. Further, commonly assigned International patent application PCT/US07/65554 describes techniques for 25 transcoding between scalable video coding formats and other formats. 3 The hierarchical coding process in a typical scalable video coding system follows a pyramidal design. A first base layer is constructed using a baseline encoding technique, suitable for single-layer coding. In the case of SVC, the base layer is encoded using H.264 AVC. Encoding the base layer in this way has the 5 benefit that the lowest scalability layer is backwards compatible with systems that are able to process only non-scalable video. Additional layers (referred to as enhancement layers) are constructed by further encoding of the difference between the original signal and the decoded output of a lower layer. The process is similar to successive approximation of the original signal. 10 Each additional enhancement layer improves the fidelity of the coded signal in one of three possible fidelity dimensions: temporal, quality (or SNR), or spatial. Temporal enhancement, when added to the base layer, provides a representation of the signal with a higher number of pictures or frames per second (fps). In quality scalability neither the spatial nor the temporal resolution is changed; rather, the 15 residual coding error is further encoded with finer quantization. Since finer quantization will result in a higher SNR, this form of scalability is often referred to as SNR scalability. SNR scalability is further subdivided into Coarse Grain and Fine Grain Scalability (CGS and FGS, respectively). Their difference is that in the latter case, an embedded coding scheme is used to encode the residual coding error, thus 20 allowing improvement of the lower layer SNR even if only a fraction of the enhancement layer is used in the decoding process. The more bits from the FGS layer are available, the better the improvement of the lower layer's SNR. For this reason the technique is also referred to as "progressive refinement." In CGS, the entire enhancement layer normally has to be available during the decoding process. Finally, 25 spatial enhancement provides a representation of the signal at a higher spatial 4 resolution (e.g., CIF vs. QCIF). It is noted that in order to construct the residual coding error, i.e., the difference between the original and the decoded output of the lower layer, the output of the lower layer has to be upsampled to the resolution of the original. 5 High coding efficiency in video coding is achieved by the utilization of effective models for predicting picture content, coupled with appropriate transformation of the resulting prediction errors (e.g., using the discrete cosine transform or integer approximations), quantization, and entropy coding of the resulting quantization levels and side information produced by the prediction and 10 coding process. A simple mechanism for producing multiple representations of a given video signal would be to create two or more separate encodings of corresponding subsets of the original signal. This technique is typically referred to as simulcasting. Scalable video coding achieves further coding gains compared with simulcasting by using lower layers as prediction references for the encoding of higher 15 layers. This inter-layer prediction exploits the inherent redundancy that exists across the three dimensions of a video signal. In a scalable video encoder, lower layer data is thus made available via additional prediction mode options. These additional options give the encoder more flexibility in its task of minimizing the distortion of the coded video signal while maintaining a given bit budget. At the same time, it is noted 20 that the additional options can make the encoding task more complex as more possibilities may be examined. An important feature of SVC is that single-loop decoding is used. This is achieved by limiting the encoder to use for inter-layer prediction only lower layer data that are available directly by parsing the bitstream. In other words, a decoder 25 decoding a higher layer does not have to fully decode a lower layer (i.e., reconstruct 5 H ader\!ntenovn\NRPonbl\DCC DER\$354878_l d-3017/2013 -6 the actual pixels) but instead needs only to parse the lower layer's bitstream data. This significantly reduces the computational requirements of decoders, and is a significant improvement compared to earlier scalable coding designs such as the one used in MPEG-2. 5 Even with single-loop decoding, however, the encoder's task in scalable coding is computationally demanding, as for every macroblock (MB) or macroblock partition of every layer it has to arrive at a decision in terms of the prediction mode, motion vector(s), and quantizer setting. The computational demand is even more pronounced in real-time applications such as videoconferencing, where pictures have to be processed within a given 10 amount of time and with very little delay. Consideration is now being given to improving scalable video coding systems so that computationally efficient encoding can be performed. In particular, attention is being directed to improving coding efficiency by appropriate signaling of prediction modes in the coded video signal. 15 SUMMARY OF THE INVENTION According to the present invention there is provided a system for decoding of scalable digital video, the system comprising: an input configured to receive a scalable digital video bitstream comprising groups 20 of coded units of a quality or spatial enhancement target layer and at least one additional layer, the digital video bitstream containing control data (including prediction control data) associated with a group of coded units and control (including prediction), texture, or motion data associated with individual coded units; a decoder coupled to the input, wherein the decoder is configured to decode the 25 control data associated with a group of coded units of the target layer and the at least one additional layer, and control, texture, or motion data associated with individual coded units of the target layer and the at least one additional layer; a predictor coupled to the decoder, wherein the decoder is configured to generate prediction references for the control, texture, or motion data of a plurality of coded units of 30 the target layer as signaled prediction control data associated with a group of coded units of the target layer or the at least one additional layer, or from prediction control data H d erIoenWRIonbl\DCCdoc-3007/2013 -7 associated with individual coded units of the target layer or the at least one additional layer; and a combiner coupled to the predictor, wherein the combiner is configured to combine the generated prediction references with the corresponding decoded control, 5 texture, or motion data associated with the plurality of coded units of the target layer to produce portions of a decoded picture corresponding to the plurality of coded units of the target layer, wherein the prediction control data associated with the groups of coded units of the target layer or the at least one additional layer and the prediction control data associated 10 with individual coded units of the target layer or the at least one additional layer include inter-layer prediction control data; wherein the predictor is configured to use values indicated by the inter-layer prediction control data associated with a group of coded units of the target layer when the corresponding inter-layer prediction control data associated with individual coded units of 15 the group of coded units of the target layer are not present in the digital video bitstream; wherein coded units correspond to macroblocks or macroblock partitions and groups of coded units correspond to slices; wherein the inter-layer prediction control data associated with a macroblock or macroblock partition comprise a residualpredictionflag parameter; 20 wherein the inter-layer prediction control data associated with a slice comprise an adaptiveresidual_prediction flag parameter, and wherein the decoder is further configured not to decode the residual_predictionflag parameter in macroblock or macrobock partitions of a slice for which the adaptive_residualprediction flag parameter is not set, and wherein the 25 predictor is further configured to assume a default value for the residualpredictionflag parameter for all macroblocks or macroblock partitions of the slice. The invention also provides a method for decoding of scalable digital video, the method comprising: at an input, receiving a scalable digital video bitstream comprising groups of coded 30 units of a quality or spatial enhancement target layer and at least one additional layer, the digital video bitstream containing control data (including prediction control data) HdeNnterwoven\NRortbl\DCC\DER\354878_Ldoc-3M7/2Ol3 -8 associated with a group of coded units and control (including prediction), texture, or motion data associated with individual coded units; at a decoder, decoding the received input by decoding the control data associated with a group of coded units of the target layer and the at least one additional layer, and 5 control, texture, or motion data associated with individual coded units of the target layer and the at least one additional layer; using a predictor coupled to the decoder, to generate prediction references for the control, texture, or motion data of a plurality of coded units of the target layer from signaled prediction control data associated with a group of coded units of the target layer 10 or the at least one additional layer, or from prediction control data associated with individual coded units of the target layer or the at least one additional layer; and at a combiner coupled to the predictor, combining using a processor the generated prediction references with the corresponding decoded control, texture, or motion data associated with the plurality of coded units of the target layer to produce portions of a 15 decoded picture corresponding to the plurality of coded units of the target layer, wherein the prediction control data associated with the groups of coded units of the target layer or the at least one additional layer and the prediction control data associated with individual coded units of the target layer or the at least one additional layer include inter-layer prediction control data, and wherein the predictor is configured to use values 20 indicated by the inter-layer prediction control data associated with a group of coded units of the target layer when the corresponding inter-layer prediction control data associated with individual coded units of the group of coded units of the target layer are not present in the digital video bitstream; wherein coded units correspond to macroblocks or macroblock partitions and 25 groups of coded units correspond to slices; wherein the inter-layer prediction control data associated with a macroblock or macroblock partition comprise a residual_prediction flag parameter; wherein the inter-layer prediction control data associated with a slice comprise an adaptiveresidual_prediction flag parameter; 30 the method further comprising at the decoder omitting decoding of the residualpredictionflag parameter in macroblock or macrobock partitions of a slice for H \derl neio n\NRPorblDCC 5 doc-30J7/20i3 -9 which the adaptiveresidualprediction flag parameter is not set, and at the predictor assuming a default value for the residual-prediction flag parameter for all macroblocks or macroblock partitions of the slice. 5 BRIEF DESCRIPTION OF THE DRAWINGS FIG. I is a schematic illustration of an exemplary architecture of a scalable video communication system. The system may have a design similar to conventional systems, but its components are further configured to accommodate telescopic mode operation, in accordance with the principles of the present invention. 10 FIG. 2 is a schematic illustration of the conventional structure of an AVC single layer video encoder. The system may have a block design similar to conventional systems, but which are further configured to accommodate telescopic mode operation, in accordance with the principles of the present invention. FIG. 3 is a schematic illustration of the structure of an AVC single-layer video 5 decoder. The system may have a block design similar to conventional systems, but which are further configured to accommodate telescopic mode operation, in accordance with the principles of the present invention. FIG. 4 is a schematic illustration of the structure of a conventional SVC scalable video encoder with three quality scalability layers. The system may have a 10 block design similar to conventional systems, but which is further configured to accommodate telescopic mode operation,.in accordance with the principles of the present invention. FIG. 5 is a schematic illustration of the structure of a conventional SVC scalable video encoder with three spatial scalability layers; 15 FIG. 6 is an illustration of exemplary syntax and semantics modifications for using the adaptive_prediction_flag in the SVC JD7 codec, in accordance with the principles of the present invention; and FIG. 7 is an illustration of exemplary syntax and semantics modifications for providing complete telescopic mode flag support in the SVC JD8 codec, in 20 accordance with the principles of the present invention. Throughout the figures the same reference numerals and characters, unless otherwise stated, are used to denote like features, elements, components or portions of the illustrated embodiments. Moreover, while the present invention will now be described in detail with reference to the Figures, it is done so in connection with the 25 illustrative embodiments. 10 DETAILED DESCRIPTION OF THE INVENTION Systems and methods for scalable video coding using special inter-layer prediction modes (called telescopic modes) are provided. FIG. I shows an exemplary architecture of a video communication system 100 5 which uses scalable coding. Communication system 100 includes a scalable video encoder 110, which communicates, over a communications network 120, with a scalable video decoder 130. In a preferred embodiment of communication system 100, the H.264 SVC coding format ('SVC') is used for video communication. (See, e.g., the SVC JD7 specification, T. Wiegand, G. Sullivan, J. Reichel, H. Schwarz, M. 10 Wien, eds., "Joint Draft 7: Scalable Video Coding," Joint Video Team, Doc. JVT T201, Klagenfurt, July 2006, which is incorporated by reference herein in its entirety). SVC is the scalable video coding extension (Annex G) of the H.264 video coding standard ('AVC'). An SVC coded bitstream can be structured into several components or layers. 15 A base layer offers a representation of the source signal at some basic fidelity. Additional layers (enhancement layers) provide information for improved representation of the signal in additional scalability dimensions (e.g., temporal, quality, or spatial). The layers of the coded bitstream are typically formed in a pyramidal structure, in which the decoding of a layer may require the presence of one 20 or more lower layers. It is noted that the AVC standard already supports temporal scalability through its use of reference picture lists and associated reference picture list reordering (or MMCO) commands. The lowest layer of an SVC stream is compliant with the AVC specification by design. l1 With continued reference to FIG. 1, the output of the Scalable Video Encoder contains two layers, Layer 0 and Layer 1, where Layer 0 is the base layer and Layer I is an enhancement layer. While FIG. I shows only two exemplary layers for simplicity, it will be understood that the present invention is applicable to situations 5 with any number of spatial or quality enhancement layers without limitation. Similarly, while FIG. I shows the two layers as being transported as separate streams over a single communications network 120, it will be understood that the streams corresponding to the two (or more) layers may be transported over any number of actual network connections in various possible configurations. The possible 10 configurations may, for example, include one in which all streams are multiplexed together for transport, and another in which more than the actual number of spatial or quality layers are transported when further demultiplexing of data takes place (e.g., when data partitioning is used, or when data is demultiplexed based on its underlying temporal layering). 15 FIG. 2 shows the design or architecture of an AVC single-layer video encoder 200 including blocks 202-218. The design shown is typical of block-based hybrid coding with motion compensation, which includes essentially all standard video codecs, although in an AVC encoder each block has considerably more operating options. The coded bitstream consists of multiplexed texture data (quantized 20 transform coefficients), motion data, and control data that are entropy coded for high coding efficiency. The texture data is the quantized residual prediction error that results after motion compensated or intra prediction is used, whereas the motion data is the data necessary to perform motion-compensated prediction (motion vector differences, etc.). As with any predictive coding scheme, the encoder contains a 12 decoder in its prediction loop. In FIG. 2, the decoder components of encoder 200 (i.e., blocks 206-214) are placed in decoder subunit 200A. FIG. 3 shows the design or architecture of a stand-alone AVC decoder 300. The design of AVC decoder 300 is identical to the decoder subunit 200A shown in 5 FIG. 2, with the only difference being the further inclusion of an entropy decoder/demultiplexer 310 (which is not necessary in an encoder). It is noted that positions of blocks 206-214 of decoder subunit 200A shown in FIG. 3 have been rearranged for improved readability. The key operating units that determine an encoder's efficiency and complexity 10 are its Coder Control and Motion Estimation units (e.g., Coder Control unit 202 and Motion Estimation unit 216, FIG. 2). The Coder Control unit is responsible for deciding appropriate modes of operation in order to maximize quality (or equivalently, minimize coding distortion) for a given bit rate. The Motion Estimation unit works in conjunction with the Coder Control unit, in that latter affects the estimation process of 15 the.former. Motion estimation is the most computationally intensive operation in video encoding, especially in modem codecs with quarter pel support. A scalable video codec can be constructed using a single-layer codec as the basis using a pyramidal structure. Scalable video coding techniques, which are specifically designed for interactive video communication applications such as 20 videoconferencing, are described in commonly assigned International patent application PCT/US06/028365. In a preferred embodiment of the present invention, where SVC is used as the scalable video codec, the enhancement layers are similarly built on top of an AVC base layer. FIG. 4 schematically shows the exemplary structure of a conventional SVC 25 encoder 400 for quality scalability with three quality layers. In SVC encoder 400, the 13 lowest layer is compliant with AVC (by design). The encoding process for the enhancement layers may use lower layer data for inter-layer prediction, which is indicated in the figure by vertical arrows labeled "Inter-layer prediction". The type of data that can be used for inter-layer prediction, in accordance with the single-loop 5 design principle of SVC, are intra, motion, and residual data. The resultant enhancement layer texture and motion data are coded similarly to the base (AVC) layer. While FIG. 4 shows the different layer data as multiplexed together in a single output bitstream, the different layer bitstreams may be transmitted on any number of channels (including a number higher than the number of layers if, e.g., data 10 partitioning is used, or if layers also contain temporal scalability components). FIG. 5 schematically shows the exemplary structure of a SVC encoder 500 for spatial scalability with three spatial layers. SVC encoder 500 for spatial scalability differs from SVC encoder 400 for quality scalability only in The only difference with FIG. 4 is that the input in the former is downsampled for constructing the two lower 15 spatial layers. Again, the lowest layer is compliant with AVC, and the inter-layer prediction process can use intra, motion, and residual data from lower layers, properly scaled or upsampled for the target resolution of the referencing layer. The decoders for quality and spatial scalability corresponding to encoders 400 and 500 shown in FIG. 4 and FIG. 5, respectively, are essentially the same as decoder 300 shown in FIG. 20 3, except for the added inter-layer prediction modes available, and the use of upsampling/scaling for texture/motion data in spatial scalability. A key engineering design problem in both single-layer and scalable video encoders such as encoders 400 and 500 is the computational complexity versus compression efficiency tradeoff. In all video coding standards, the standard 25 specification only specifies the structure of the bitstream and the decoding process; 14 the encoding procedure is left unspecified. Further, the video coding standards do not provide any guidance for effective encoding strategies in encoder design. The inventive systems and methods described herein are directed toward facilitating accelerated operation of a scalable video encoder, with appropriate 5 signaling to the decoder, and with improved coding efficiency. A video encoder (single-layer or scalable) must make several decisions while coding picture data. For every macroblock (MB), the encoder must make decisions regarding intra/inter coding, prediction mode selection, motion mode and vector selection, quantizer selection, etc. In the absence of other constraining factors, the 10 objective of an encoder is to make these decisions so as to minimize the distortion of the coded signal while keeping the output bit rate within given constraints. This process is called Rate-Distortion Optimization (RDO) in recognition that the encoder seeks to minimize distortion for a given rate. The dimension of the optimization problem is, however, extremely large. Typically fast, suboptimal algorithms and 15 heuristics are used in the design of effective encoders. It is noted that, in real video coding systems, perceptual factors may dictate decisions in the encoder that, in fact, increase the quantitative distortion, as expressed in terms of Peak Signal-to-Noise Ratio (PSNR), in order to obtain results that are more pleasing to human observers. These decisions are typically application specific. For example, different 20 configurations may be used for two encoders tuned for broadcast and videoconferencing applications, respectively, as both the subject matter and the operating bit rates of the two encoders are very different. The encoding process can be represented by a decision tree, which the encoder has to traverse in order to make final coding decisions for a particular MB, picture, or 25 group of pictures. For scalable video coding, in particular, the size of this tree is 15 significantly larger than that of a single layer encoder, since decisions have to be made not just for the base layer (equivalent to a single-layer encoder), but also for each of the enhancement layers present. The computational demands are thus considerable, and it is important to provide ways through which an encoder that may 5 not have enough computational resources to fully consider all coding choices, can derive a suitable simplified coding strategy. The implementation of a suitable coding strategy is particularly important for real-time, conversational services where both real-time and low-delay operation may be required. A suitable simplified coding strategy, for example, in the case of a live broadcast encoder may recognize that the 10 encoder has to operate in real-time, but its delay requirements are much more relaxed. Equally important for implementation, the encoder must be able to signal the result of this simplified coding strategy to the decoder, without adversely affecting coding performance. Computational demands have typically not been considered as a crucial design 15 factor in the development of coding standards, and the development of such codecs is performed using simulation software that is allowed to run as long as necessary in order to obtain optimized coding efficiency results (e.g., one hour or more per picture). As a result, conventional design of the bitstream according to the standards does not cater to the needs of accelerated encoder decision-making, in that the signaling of a 20 simplified coding strategy may incur substantial bit rate overhead. A specific type of accelerated decision-making for scalable video coding, according to the present invention, is a technique referred to herein as 'telescopic' operation. With this technique, accelerated encoder operation is achieved by forgoing full optimization of coding decisions for an enhancement layer, and using instead the 25 decisions that were made in the base layer. While further refinement of such 16 decisions is likely to achieve improved coding efficiency, they may already represent a very good computational complexity vs. coding efficiency tradeoff. The technique is particularly effective for videoconferencing material, where scene activity is generally low. 5 The codec in SVC JD7 is an extension of the AVC design (ITU-T Rec. H.264 I ISO/lEC 14496-10 version 4, "Advanced video coding for generic audiovisual services," 2005, incorporated herein by reference in its entirety, and referred to as the 'AVC specification'). The text of the AVC specification is in fact an integral part of the text of SVC JD7, as the SVC extension forms a new Annex G of the AVC 10 specification. As mentioned earlier, the base layer in an SVC bitstream is an AVC compliant bitstream by design. Sections 7.3.3 (Slice header syntax), 7.3.4 (Slice data syntax), and 7.3.5 (Macroblock layer syntax) of the AVC specification, including their subsections, define the detailed syntax of the coded picture data. The slice header (Section 7.3.3) includes parameters that affect the entire slice, such as an 15 indicator of the address of the first macroblock in the slice (first mb-in-slice), the slice type (1, P, B, SI, or SP), quantizer setting that affects all macroblocks in the slice (sliceqpdelta), as well as deblocking filter settings, weighting tables, and reference picture list reordering commands. The slice data (Section 7.3.4) syntax enable, among other things, skipping a number of initial macroblocks from the current slice. 20 The actual macroblock data are described in Section 7.3.5. The data includes the macroblock type (mbtype) which defines the prediction mode used for the current macroblock, the coded block pattern which signals which of the various chroma and luna blocks corresponding to the current macroblock have non-zero transform coefficients, as well as quantizer settings effective starting at the current macroblock 25 (mb_qp_delta). 17 The macroblock type is a crucial parameter, as it signals which of the many prediction modes the encoder chose to encode the current macroblock. SVC JD7 Tables 7.11 through 7.14 summarize the available macroblock types in AVC. For example, for a P slice (Table 7.13), mode I corresponds to prediction mode 5 'P_LO_L0l6x8', in which the 16x16 macroblock is split into two 16x8 regions, and two motion vectors are used to predict each part from a reference picture contained in the first reference picture list (LO). The actual motion vector data (mvd_0, coded as a difference) are shown in Section 7.3.5.1, and include an optional index to the reference picture used (ref idx_10) if it is not the default. The result of the prediction 10 process is the residual data, represented by the syntax described in Section 7.3.5.3 and associated subsections. The corresponding structures for enhancement layers in the SVC JD7 specification are defined in Sections G.7.3.3 through G.7.3.7. The structures are similar to the ones used in AVC, but with additional data that address the increased 15 options present in SVC. In general, their design is such that data that is present in lower layers are made available for the coding of the current layer. In other words, coded bitstream parameters can be reused in higher layers by directly referencing them, instead of coding them anew (inter-layer prediction). For single-loop decoding, this excludes data that can only be obtained by full decoding, as they would 20 necessitate the availability of complete decoding loops for layers lower than the current one, leading to multi-loop designs. Of particular importance is the slice header parameter base_idplusl (Section G.7.3.4), which identifies the reference layer for the current layer when using inter layer prediction. It is noted that the parameter baseid_plusl does not refer to the 25 base layer of the entire bitstream, but only to the particular layer that is used as the 18 basis or reference for predicting the current layer. The parameter jointly encodes the coordinates of the reference layer in terms of the spatial scalability layer, quality scalability layer, and fragment order (when FGS or progressive refinement slices are used). If the value of base idplusI is zero, then no inter-layer prediction is used 5 (e.g., in the lowest layer). When inter-layer prediction is used (i.e., when baselayer id-plusI is not zero), the parameter adaptivepredictionflag is present in the bitstream (Section G.7.3.4). The adaptive prediction flag affects how inter-layer prediction is used in the macroblock layer by controlling the presence and value of the parameter 10 basemodeflag (Section G.7.3.6). When basemodeflag is set, then the macroblock type, reference indices, and motion vectors, where applicable, are infened from the corresponding reference (base) layer macroblock. The presence of the basemodeflag for each macroblock of a slice is controlled at the slice level by the adaptivepredictionflag. When the 15 adaptivepredictionflag is not set, then the base mode flags do not have to be sent. The value one (set) is inferred for base_mode flag, thus forcing all macroblocks (within the crop window, i.e., when corresponding lower layer data are present in the base layer) to use the base layer mode and motion information. Hence by setting the adaptive_prediction flag to zero, a slice-wide telescopic mode prediction in which the 20 base layer decisions and data are reused in the enhancement layer can be achieved. The elimination of the base mode flags also results in a saving of approximately 12 Kbps for a CIF sequence at 30 fps using CAVLC entropy coding. A significant drawback, however,.of the SVC JD7 design is that when adaptivepredictionflag is not set, then the residual_predictionflag is inferred to be 25 set slice-wide for every macroblock, as seen in Section G.7.3.6.3. The 19 residual_predictionflag, relevant for non-intra macroblocks, indicates that the enhancement residual signal is predicted from the (possibly upsampled) reconstructed residual signal of the base macroblock (or sub-macroblock). Forcing residual prediction to be always used whenever adaptive prediction is not used, however, 5 decreases coding efficiency. Similarly, the alternative of forcing residual prediction to be always off whenever adaptive prediction is not used, again lowers the quality for a given bit rate. Extensive experimental results using standard test sequences indicate that, in all cases, the rate-distortion curves when setting residual prediction on a macroblock 10 basis are always higher than the corresponding curves when setting residual prediction to be always on or always off. Equivalently, for a given bit rate and when not using adaptive prediction (adaptive_prediction flag set to zero), being able to adaptively use residual prediction results in higher PSNR compared with the cases when residual prediction is either always on or always off. 15 The experimental results show that to enable an encoder to use and efficiently signal telescopic mode decisions to a decoder, the various prediction modalities must be a) signaled at the slice header for slice-wide use, and b) signaled by separate flags so that coupling of the different modalities is eliminated. Signaling mode decisions at the slice level allows the elimination of the corresponding signaling at the macroblock 20 or macroblock partition level, which results in increased compression efficiency. In an exemplary embodiment of the present invention, where SVC JD7 is used, the slice header may be augmented by a new flag, adaptive_residualpredictionflag. When this flag is not set (i.e., set to zero), then a default value for the residual_prediction_flag is assumed for every macroblock of the slice. The default 20 value for residualprediction_flag in an exemplary implementation of the present invention is set to one. FIG. 6 shows exemplary syntax and semantics 600 in a preferred embodiment of the invention as applied to the SVC JD7 codec. A close examination of the SVC 5 JD7 specification also reveals that the motion-related inter-layer prediction flags, motionprediction_flag_10 and motionpredictionflag_l (Sections G.7.3.6.1 and G.7.3.6.2) are also possible targets for accelerated encoder operation. These flags indicate (when set to 1) that the (possibly scaled) base motion vector(s) is used as motion vector predictor(s) for the list 0 (or list 1, respectively) motion vector(s) of the 10 macroblock (or macroblock partition) to which they belong. Similarly, for the residual_predictionflag, which refers to texture prediction, it is advantageous to allow inter-layer motion prediction to be set at the slice level for all macroblocks or macroblock partitions of the slice. In order to provide more flexibility for an encoder to turn inter-layer 15 prediction flags on or off for all macroblocks or macroblock partitions in a slice, it is advantageous to also encode at the slice level the default value that should be used when the corresponding flag is not set adaptively in each macroblock or macroblock partition. Such encoding at the slice level can be applied to both the basemode flag and the motionprediction flag. When the default basemodeflag is set to one, 20 however, the motionjprediction flag is useless and does not have to be transmitted. The combination of these flags provides full telescopic mode support to the encoder and decoder. FIG. 7 shows exemplary syntax and semantics 700 for a preferred embodiment of the invention as applied in the case where complete telescopic mode support is 25 added to the SVC JD8 codec (see T. Wiegand, G. Sullivan, J. Reichel, H. Schwarz, 21 M. Wien, eds., "Joint Draft 8: Scalable Video Coding," Joint Video Team, Doc. JVT U20 1, Hangzhou, October 2006, which is incorporated by reference herein in its entirety). The primary difference between SVC JD8 and JD7, for the purposes of this invention, is that in JD8 the NAL unit header flag layer baseflag specifies (when set 5 to one) that no inter-layer prediction (of coding mode, motion, sample value, and/or residual prediction) is used for the current slice, and that the parameter baseidplusl is renamed baseid. As an example, in order to signal to a decoder that an enhancement layer is coded by using only base layer information, the adaptive_predictionflag should be 10 set to 0, the default base_modeflag should be set to 1, and the adaptiveresidual_predictionflag should be set to 0. The flags adaptive motion_prediction flag and default motion_predictionflag are not used, and are not coded. The macroblocks of an enhancement layer slice (using the SVC JD8 specification) will only contain codedblockjpattern data indicating that no non 15 zero coefficients are present in any of their corresponding luma or chroma blocks. The need to transmit the coded block_pattern data may be eliminated by introducing an additional flag at the slice header, indicating that no macroblock data will be transmitted for the current slice, and that the above telescopic mode flag configuration is to be used. 20 Although the inventive techniques were described herein in the context of the SVC video coding standard, the principles of the present invention can be applied to any scalable video coding scheme that uses inter-layer prediction. Additionally, the inventive techniques may be applied to any coded unit of a video bitstreams, such as a picture or a group of pictures, and not exclusively the slice level. Similarly, the 25 technique is applicable when the smallest coded unit for which prediction modes are 22 C:WRPonbI\DCCDERU77571_I.DOC-27)07/2011 - 23 signaled is different than a macroblock (e.g., a block or any other structure, even of arbitrary shape, that comprises a coded unit). While there have been described what are believed to be the preferred embodiments of the present invention, those skilled in the art will recognize that other and 5 further changes and modifications may be made thereto without departing from the spirit of the invention, and it is intended to claim all such changes and modifications as fall within the true scope of the invention. It will be understood that in accordance with the present invention, the techniques described herein may be implemented using any suitable combination of hardware and 10 software. The software (i.e., instructions) for implementing and operating the aforementioned rate estimation and control techniques can be provided on computer readable media, which can include, without limitation, firmware, memory, storage devices, microcontrollers, microprocessors, integrated circuits, ASICs, on-line downloadable media, and other available media. 15 The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that that prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates. 20 Throughout this specification and the claims which follow, unless the context requires otherwise, the word "comprise", and variations such as "comprises" and "comprising", will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps. 25 The reference numerals or symbols in the following claims do not in any way limit the scope of the respective claims.

Claims (11)

1. A system for decoding of scalable digital video, the system comprising: an input configured to receive a scalable digital video bitstream comprising groups 5 of coded units of a quality or spatial enhancement target layer and at least one additional layer, the digital video bitstream containing control data (including prediction control data) associated with a group of coded units and control (including prediction), texture, or motion data associated with individual coded units; a decoder coupled to the input, wherein the decoder is configured to decode the 10 control data associated with a group of coded units of the target layer and the at least one additional layer, and control, texture, or motion data associated with individual coded units of the target layer and the at least one additional layer; a predictor coupled to the decoder, wherein the decoder is configured to generate prediction references for the control, texture, or motion data of a plurality of coded units of 15 the target layer as signaled prediction control data associated with a group of coded units of the target layer or the at least one additional layer, or from prediction control data associated with individual coded units of the target layer or the at least one additional layer; and a combiner coupled to the predictor, wherein the combiner is configured to 20 combine the generated prediction references with the corresponding decoded control, texture, or motion data associated with the plurality of coded units of the target layer to produce portions of a decoded picture corresponding to the plurality of coded units of the target layer, wherein the prediction control data associated with the groups of coded units of the 25 target layer or the at least one additional layer and the prediction control data associated with individual coded units of the target layer or the at least one additional layer include inter-layer prediction control data; wherein the predictor is configured to use values indicated by the inter-layer prediction control data associated with a group of coded units of the target layer when the 30 corresponding inter-layer prediction control data associated with individual coded units of the group of coded units of the target layer are not present in the digital video bitstream; H lenrwcownNRPonbl\DCC\DER\535487_ Ldoc-3007/20OI -25 wherein coded units correspond to macroblocks or macroblock partitions and groups of coded units correspond to slices; wherein the inter-layer prediction control data associated with a macroblock or macroblock partition comprise a residual_prediction flag parameter; 5 wherein the inter-layer prediction control data associated with a slice comprise an adaptive residual-prediction_flag parameter, and wherein the decoder is further configured not to decode the residual_prediction flag parameter in macroblock or macrobock partitions of a slice for which the adaptive residual_predictionflag parameter is not set, and wherein the 10 predictor is further configured to assume a default value for the residual-prediction flag parameter for all macroblocks or macroblock partitions of the slice.

2. The system of claim 1, wherein a received digital video bitstream conforms to an SVC specification. 15

3. The system of claim 1, wherein the inter-layer prediction control data associated with a further macroblock or macroblock partition comprise a basemodeflag parameter; the inter-layer prediction control data associated with a further slice comprise an 20 adaptiveprediction-flag parameter and a defaultbasemode flag parameter, wherein the decoder is further configured not to decode the basemodeflag parameter in macroblock or macrobock partitions of a slice for which the adaptivepredictionflag parameter is not set, and wherein the predictor is further configured to assume a value indicated by the defaultbasemodeflag parameter for the 25 base_modeflag parameter for all macroblocks or macroblock partitions of the slice.

4. The system of claim 1, wherein the inter-layer prediction control data associated with a further macroblock or macroblock partition comprise a motion-prediction_flag_10 parameter and a 30 motionprediction_flag_l I parameter; and the inter-layer prediction control data associated with a further slice comprise an H drcnenvoen\NPPortblICCDER\5354878I doc-30/07/2013 - 26 adaptive motion-prediction flag parameter and a default_motion prediction flag parameter, wherein the decoding means is further configured not to decode the motionprediction_flag_10 or motion_prediction_flag_l parameters in macroblock or 5 macrobock partitions of a slice for which the adaptive_motionypredictionflag parameter is not set, and wherein the predictor is further configured to assume a value indicated by the defaultmotionpredictionflag parameter for the motionpredictionflag 10 and motionprediction_flag_l parameters for all macroblocks or macroblock partitions of the slice. 10

5. A method for decoding of scalable digital video, the method comprising: at an input, receiving a scalable digital video bitstream comprising groups of coded units of a quality or spatial enhancement target layer and at least one additional layer, the digital video bitstream containing control data (including prediction control data) 15 associated with a group of coded units and control (including prediction), texture, or motion data associated with individual coded units; at a decoder, decoding the received input by decoding the control data associated with a group of coded units of the target layer and the at least one additional layer, and control, texture, or motion data associated with individual coded units of the target layer 20 and the at least one additional layer; using a predictor coupled to the decoder, to generate prediction references for the control, texture, or motion data of a plurality of coded units of the target layer from signaled prediction control data associated with a group of coded units of the target layer or the at least one additional layer, or from prediction control data associated with 25 individual coded units of the target layer or the at least one additional layer; and at a combiner coupled to the predictor, combining using a processor the generated prediction references with the corresponding decoded control, texture, or motion data associated with the plurality of coded units of the target layer to produce portions of a decoded picture corresponding to the plurality of coded units of the target layer, 30 wherein the prediction control data associated with the groups of coded units of the target layer or the at least one additional layer and the prediction control data associated H cle m onnNRPonbl\DC D IER\535 5_Ldoc-10AP/20I1 - 27 with individual coded units of the target layer or the at least one additional layer include inter-layer prediction control data, and wherein the predictor is configured to use values indicated by the inter-layer prediction control data associated with a group of coded units of the target layer when the corresponding inter-layer prediction control data associated 5 with individual coded units of the group of coded units of the target layer are not present in the digital video bitstream; wherein coded units correspond to macroblocks or macroblock partitions and groups of coded units correspond to slices; wherein the inter-layer prediction control data associated with a macroblock or 10 macroblock partition comprise a residualpredictionflag parameter; wherein the inter-layer prediction control data associated with a slice comprise an adaptiveresidualprediction flag parameter; the method further comprising at the decoder omitting decoding of the residualprediction flag parameter in macroblock or macrobock partitions of a slice for 15 which the adaptiveresidual_prediction flag parameter is not set, and at the predictor assuming a default value for the residual_prediction flag parameter for all macroblocks or macroblock partitions of the slice.

6. The method of claim 5, wherein 20 a received digital video bitstream conforms to an SVC specification.

7. The method of claim 5, wherein the inter-layer prediction control data associated with a further macroblock or macroblock partition comprise a basemodeflag parameter; 25 the inter-layer prediction control data associated with a further slice comprise an adaptivepredictionflag parameter and a defaultbase_modeflag parameter, the method further comprising at the decoder omitting decoding of the basemodeflag parameter in macroblock or macrobock partitions of a slice for which the adaptiveprediction flag parameter is not set, and at the predictor assuming a value 30 indicated by the defaultbase_mode_flag parameter for the basemode flag parameter for all macroblocks or macroblock partitions of the slice. H\4einc wnn\Po nbAtDCC\DER\35487 I doc.A30)7I II -28

8. The method of claim 5, wherein the inter-layer prediction control data associated with a further macroblock or macroblock partition comprise a motion_prediction flag_10 parameter and a motion_predictionflag_ll parameter; and 5 the inter-layer prediction control data associated with a further slice comprise an adaptive motionpredictionflag parameter and a default_motionpredictionflag parameter, the method further comprising at the decoder omitting decoding of the motionprediction flag_10 or motionprediction flag_l parameters in macroblock or 10 macrobock partitions of a slice for which the adaptive_motionpredictionflag parameter is not set, and at the predictor assuming a value indicated by the defaultmotionprediction_flag parameter for the motionprediction_flag 10 and motionprediction flag_l parameters for all macroblocks or macroblock partitions of the slice. 15

9. A computer program product, comprising a computer usable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement any one of claims 5 to 8. 20

10. A system for decoding of scalable digital video substantially as hereinbefore described with reference to the accompanying drawings.

11. A method for decoding of scalable digital video substantially as hereinbefore described with reference to the accompanying drawings. 25