Australian Patents Act 1990 - Regulation 3.2 ORIGINAL COMPLETE SPECIFICATION STANDARD PATENT Invention Title "Systems and methods for signaling and performing temporal level switching in scalable video coding" The following statement is a full description of this invention, including the best method of performing it known to me/us: P/00/O Il SYSTEMS AND METHODS FOR SIGNALING AND PERFORMING TEMPORAL LEVEL SWITCHING IN SCALABLE VIDEO CODING Cross-Reference To Related Applications 5 This application claims the benefit of United States provisional patent application Serial No. 60/829,609 filed October 16, 2006. Further, this application is related to International patent application Nos. PCT/US06/28365, PCT/US06/028366, PCT/US06/061815, PCT/US06/62569, PCT7US07/80089], PCT/US07/062357, PCT/US07/65554, 10 PCT/US07/065003, PCT/US06/028367, and PCT/US07/63335. All of the aforementioned applications, which are commonly assigned, are hereby incorporated by reference herein in their entireties. The disclosure of the complete specification of Australian Patent Application No. 2007311178, as originally filed, is incorporated herein by reference. 15 Field Of The Invention The present invention relates to a method for decoding media encoded using scalable coding with temporal scalability at a pre-determined maximum temporal level. The present invention relates to video communication systems. In particular, the invention relates to communication 20 systems that use temporally scalable video coding and in which a receiver or intermediate gateway switches from one temporal level to a higher or lower level to meet frame rate, bit rate, processing power, or other system requirements. Background Of The Invention 25 New digital video and audio 'scalable' coding techniques, which aim to generally improve coding efficiency, have a number of new structural characteristics (e.g., scalability). In scalable coding, an original or source signal is represented using two or more hierarchically structured bitstreams. The hierarchical structure implies that decoding of a given bitstream depends on 30 the availability of some or all other bitstreams that are lower in hierarchy. Each bitstream, together with the bitstreams it depends on, offer a representation of the original signal at a particular temporal, fidelity (e.g., in terms of signal-to-noise ratio(SNR)), or spatial resolution (for video).
WO 2008/048886 PCT/US2007/081217 It is understood that term 'scalable' does not refer to a numerical magnitude or scale, but refers to the ability of the encoding technique to offer a set of different bitstreams corresponding to efficient representations of the original or source signal at different 'scales' of resolutions or other signal qualities. The ITU-T H.264 5 Annex G specification, which is referred to as Scalable Video Coding (SVC), is an example of a video coding standard that offers video coding scalability in all of temporal, spatial, and fidelity dimensions. SVC is an extension of the H.264 standard (also known as Advanced Video Coding or AVC). An example of an earlier standard, which also offered all three types of scalability, is ISO MPEG-2 (also published as 10 ITU-T H.262). ITU G.729.1 (also known as G.729EV) is an example of a standard offering scalable audio coding. The concept of scalability was introduced in video and audio coding as a solution to distribution problems in streaming and broadcasting, and to allow a given communication system to operate with varying access networks (e.g., clients 15 connected with different bandwidths), under varying network conditions (e.g., bandwidth fluctuations), and with various client devices (e.g., a personal computer that uses a large monitor vs. a handheld device with a much smaller screen). Scalable video coding techniques, which are specifically designed for interactive video communication applications such as videoconferencing, are 20 described in commonly assigned International patent application PCT/US06/028365. Further, commonly assigned International patent application PCT/US06/028365 describes the design of a new type of server called the Scalable Video Communication Server (SVCS). SVCS can advantageously use scalable coded video for high-quality and low-delay video communication and has a complexity, which is 25 significantly reduced compared to traditional switching or transcoding Multipoint Control Units (MCUs). Similarly, commonly assigned International patent application PCT/US06/62569 describes a Compositing Scalable Video Coding Server (CSVCS), which has the same benefits as an SVCS but produces a single coded output bit stream. Furthermore, International patent application PCTIUS07/80089 30 describes a Multicast Scalable Video Coding Server (MSVCS), which has the same benefits as an SVCS but utilizes available multicast communication channels. The scalable video coding design and the SVCS/CSVCS architecture can be used in further advantageous ways, which are described, for example, in commonly assigned International patent applications PCT/US06/028367, PCT/US06/027368, -2- -3 PCT/US06/061815, PCT/US07/62357, and PCT/US07/63335. These applications describe the use of scalable coding techniques and SVCS/CVCS architecture for effective trunking between servers, reduced jitter buffer delay, error resilience and random access, "thinning" of scalable video bitstreams to improve coding efficiency with reduced packet loss, and 5 rate control, respectively. Further, commonly assigned International patent application PCT/US07/65554 describes techniques for transcoding between scalable video coding formats and other formats. Consideration is now being given to further improving video communication systems that 10 use scalable video coding. In such systems, a source may be a transmitting endpoint that encodes and transmits live video over a communication network, a streaming server that transmits pre-coded video, or a software module that provides access to a file stored in a mass storage or other access device. Similarly, a receiver may be a receiving endpoint that obtains the coded video or audio bit stream over a communication network, or directly 15 from a mass storage or other access device. An intermediate processing entity in the system may be an SVCS or a CSVCS. Attention is being directed toward improving the efficiency of switching between temporal layers by receivers and intermediate processing entities. 20 It is generally desirable to overcome or ameliorate one or more of the above described difficulties, or to at least provide a useful alternative. Summary of the Invention 25 According to the present invention, there is provided a method for decoding media encoded using scalable coding with temporal scalability at a pre-determined maximum temporal level, the method comprising: receiving a single bit flag temporallevel nestingflag in a sequence parameter set that is indicative of a temporally nested structure of layers and pertains to all frames of a 30 coded video sequence; and decoding with a decoding device those parts of the encoded media signal which correspond to temporal levels that are less than or equal to the maximum temporal level, -4 wherein the temporallevelnestingflag set to one indicates for any three frames picA, picB, picC included in the coded video sequence, that picB is not used for reference of picA: under a first condition that picB is of a temporal level lower or equal than 5 the temporal level of picA, under a second condition that the temporal level of picC is lower than the temporal level of picB, under a third condition that picC follows picB in decoding order, and under a fourth condition that picC precedes picA in decoding order. 10 According to the present invention, there is also provided a method for decoding media encoded using scalable coding with temporal scalability at a pre-determined maximum temporal level, the method comprising: receiving a temporallevel_nesting flag in a scalability information supplementary 15 enhancement information (SSEI) message that is indicative of a temporally nested structure of layers and pertains to a plurality of frames, and decoding with a decoding device those parts of the encoded media signal which correspond to temporal levels that are less than or equal to the maximum temporal level, wherein the scalable coding complies with H.264/SVC, and 20 wherein the temporallevelnesting flag set to one indicates for any three frames picA, picB, picC included in the coded video sequence, that picB is not used for reference of picA: under a first condition that picB is of a temporal level lower or equal than the temporal level of picA, 25 under a second condition that the temporal level of picC is lower than the temporal level of picB, under a third condition that picC follows picB in decoding order, and under a fourth condition that picC precedes picA in decoding order. 30 According to the present invention, there is also provided a non-transitory computer readable medium comprising a set of executable instructions to direct a processor to -4A perform the above-described methods. According to the present invention, there is also provided a decoding device configured to perform the above-described methods. 5 Systems and methods for signaling and temporal level switching in scalable video communication systems are provided. The systems and methods involve signaling select information, which enables temporal level switching in both lower and higher levels can be performed at arbitrary picture positions. The information is communicated as certain 10 constraints in the temporal prediction structure of the underlying video codec. The information can be used in intermediate processing systems as well as receivers in order to adapt to different system resources (e.g., frame rate, bit rate, processing power). Brief Description of the Drawings 15 Preferred embodiments of the present invention are hereafter described, by way of non limiting example only, with reference to the accompanying drawings, in which: Figure 1 is a schematic illustration of an exemplary architecture of a communication 20 system, in accordance with the principles of the present invention; Figures 2a-2c are schematic illustrations of examples of non-nested temporal layer prediction structures, in accordance with the principles of the present invention; Figure 3 is a schematic illustration of an example of a nested temporal layer prediction structure, in accordance with the principles of the present invention; 25 Figure 4 is an illustration of exemplary syntax modifications for temporal level nesting in SVCs Sequence Parameter Set, in accordance with the principles of the present invention; Figure 5 is an illustration of exemplary syntax modifications for temporal level nesting in SVCs Scalability Information SEI message, in accordance with the principles of the present invention; 30 Figure 6 is a schematic illustration of an exemplary architecture of a processing unit (encoder/server, gateway, or receiver), in accordance with the principles of the present invention; and -4B Figure 7 is a flow diagram illustrating an exemplary operation of an NAL Filtering Unit, in accordance with the principles of the present invention. Throughout the figures the same reference numerals and characters, unless otherwise 5 stated, are used to denote like features, elements, components or portions of the illustrated embodiments. Moreover, while the present invention will now be described in detail with reference to the figures, it is done so in connection with the illustrative embodiments. Detailed Description of Preferred Embodiments of the Invention 10 Systems and methods for "switching" signals in communication systems, which use scalable coding, are provided. The switching systems and methods are designed for communication systems with temporal scalability. 15 Figure 1 shows an exemplary architecture of a communication system 100, which uses scalable coding. Communication system 100 includes a media server or encoder 110 (e.g., a streaming server or a transmitting endpoint), which communicates video and/or audio signals with a client/receiver 120 over a network 130 through a media gateway 140. 20 The inventive "switching" systems and methods are described herein using communication system 100 as an example. For brevity, the description herein WO 2008/048886 PCT/US2007/081217 is limited to the video portion of communication system 100. It will be understood, however, that switching systems and methods also can be used for the scalable audio portions, with the understanding that no spatial scalability dimension can be provided to an audio signal, but multi-channel coding may additionally be used in audio signal 5 coding. Further the systems and methods describe herein also can be used for other multimedia data (e.g., graphics) which are coded in a scalable fashion. In a preferred embodiment of communication system 100, H.264 SVC coding format ('SVC') is used for video communication. (See, e.g., the SVC JD7 specification, T. Wiegand, G. Sullivan, J. Reichel, H. Schwarz, M. Wien, eds., "Joint 10 Draft 7: Scalable Video Coding," Joint Video Team, Doc. JVT-T201, Klagenfurt, July 2006, which is incorporated by reference herein in its entirety). SVC is the scalable video coding extension (Annex G) of the H.264 AVC video coding standard. The base layer of an SVC stream by design is compliant to the AVC specification. An SVC coded bitstream can be structured into several components or 15 layers. A base layer offers a representation of the source signal at some basic fidelity dimension or level. Additional layers (enhancement layers) provide information for improved representation of the signal in the additional scalability dimensions above the basic fidelity dimension. SVC offers considerable flexibility in creating bitstream structures with scalability in several dimensions, namely spatial, temporal, and 20 fidelity or quality dimensions. It is noted that the AVC standard already supports temporal scalability through its use of reference picture lists and associated reference picture list reordering commands. It is further noted that the layers in the coded bitstream are typically formed in a pyramidal structure, in which the decoding of a layer may require the 25 presence of one or more lower layers. Usually, the base layer is required for decoding of any of the enhancement layers in the pyramidal structure. However, all scalable encoding techniques do not have a pyramidal structure of the layers. For example, when scalability is provided through multiple description coding or simulcasting, independent decoding of some or all layers may be possible. Specifically for SVC, it 30 is possible to effectively implement simulcasting by turning all inter-layer prediction modes in the encoder off. The switching systems and methods described herein are applicable to all scalability formats including both pyramidal and non-pyramidal structures. -5- WO 2008/048886 PCT/US2007/081217 Scalability has features for addressing several system-level challenges, such as heterogeneous networks and/or clients, time-varying network performance, best-effort network delivery, etc. In order to be able to effectively use the scalability features, however, it is desirable that they are made accessible to system components 5 in addition to the video encoder and decoder. As previously noted, the switching systems and methods of the present invention are directed toward communication systems having temporal scalability (e.g., system 100). It is noted that use of media gateway 140 in system 100 is optional. The switching systems and methods of the present invention are also applicable when 10 instead of media gateway 140 a direct media server-to-client connection is used, or when the media server is replaced by a file that is directly accessible to the client on a mass storage or other access device, either directly or indirectly (e.g., a file access through a communication network). It is further noted that the systems and methods of the present invention remain the same when more than one media gateway 140 is 15 present in the path from the media server or encoder to the receiver. With renewed reference to FIG. 1, consider a simple operational scenario in which media server/encoder 110 (e.g., a streaming server or encoder a transmitting endpoint encoder) communicates scalable media with client/receiver 120 through media gateway 140. This simple scenario requires that a connection be made 20 between the media server and the client for transmitting an agreed-upon set of layers, which may, for example, be Remote Transport Protocol (RTP) encapsulated SVC Network Adaptation Layer (NAL) units. Furthermore, media gateway 140 has to be instructed, or has to decide on its own, how to best operationally utilize the incoming packets (e.g., the transmitted RTP-encapsulated SVC NAL units). In the case where 25 media gateway 140 has an SVCS/CSVCS architecture, this operational decision corresponds to a decision on which packets to drop and which to forward. Further, for proper decoder operation, client/receiver 120 must know or be able to deduce which set of layers it is supposed to receive through media gateway 140. To enable these operations, system 100 must represent and 30 communicate the scalability structure of the transmitted bit stream to the various system components. As an illustrative example, consider a video signal with two temporal resolutions, 15 and 30 fps, and two spatial resolutions, QCIF and CIF. Thus, the video signal has a four-layer scalability structure: layer LO containing the QCIF signal at 15 fps; layer LI containing the QCIF signal enhancement for 30 fps; layer SO -6- WO 2008/048886 PCT/US2007/081217 containing the CIF signal enhancement for 15 fps; and layer SI containing the CIF signal enhancement for 30 fps. The coding dependency in the four-layer scalability structure may, for example, be such that LO is the base layer, LI depends on LO, SO depends on LO, and Si depends on both LI and SO. System 100 must describe this 5 four-layer structure to the system components so that they can properly process the video signal. Supplemental Enhancement Information (SEI) messages, are data structures contained in an SVC bitstream that provide ancillary information about the coded video signal but are not necessary for the operation of the decoding process. 10 SVC offers a mechanism for describing the scalability structure of an SVC coded video bitstream through its "Scalability Information" SEI message (SSEI). The SSEI in Section G. 10.1.1 of the SVC JD7 specification is designed to enable capability negotiation (e.g., during a connection setup), stream adaptation (by video server or intermediate media gateways), and low-complexity processing (e.g., without inference 15 based on detailed bitstream parsing). The SSEI, defined in Section G.10.1.1 of the SVC JD7 specification, includes descriptive information about each layer (e.g., frame rate, profile information), and importantly, coding dependency information (i.e., which other layers a given layer depends on for proper decoding). Each layer is identified, within 20 the scope of the bitstream, by a unique 'layer id'. The coding dependency information for a particular layer is communicated by encoding the number of directly dependent layers (numdirectlydependent layers), and a series of difference values (directlydependent layer id delta), which when added to the particular layer's layer id identify the layer id's of the layers that the particular layer depends on for decoding. 25 Additionally, the "Scalability Information Layers Not Present" SEI message (SSEI-LNP) defined in G. 10.1.2, and the "Scalability Information Dependency Change" SEI message (SSEI-DC) defined in G.10.1.3 provide for in band our out-of-band signaling of dynamic changes in the transmitted bitstream, respectively. The former indicates which layers, comparing with the initial SSEI, are 30 not present in the bitstream from the point it is received, whereas the latter indicates inter-layer prediction dependency changes in the bitstream. International Patent Application No. PCT/US07/065003 describes these as well as additional systems and methods for managing scalability information. -7- WO 2008/048886 PCT/US2007/081217 Generally, the designs of the SSEI, SSEI-LNP, and SSEI-DC messages are such that when used in combination, the messages allow intermediate gateways or receivers to be continually informed about the overall structure of the bitstream transmitted from a server/encoder or gateway and to perform correct adaptation 5 functions. There are, however, important limitations in the designs, which become apparent upon close examination of different possible coding structures that may be used in real communication systems. For example, the SVC JD7 draft allows temporal structures, which contradict the pyramidal structure on which layering is being built, and which can be 10 problematic in real applications. Specifically, the only limitation that the SVC JD7 imposes on temporal levels is the following: "The decoding of any access unit with temporal_level equal to currTl shall be independent of all access units with temporal_level greater than currTl." (See G.7.4. 1, NAL unit SVC header extension semantics, p. 405). This limitation ensures that a given temporal level can be decoded 15 without access to information from higher temporal levels. It does not address, however, any dependencies that may exist within the particular temporal level as well as between the same and lower temporal levels. The SVC JD7 limitation ensures that a transition from a higher temporal level to a lower temporal level can be made immediately by simply discarding all access units with a higher temporal level. The 20 reverse operation, i.e., switching or transitioning from a lower temporal level to a higher temporal level, has a dependency problem. The problem can be understood with reference to FIGS. 2a and 2b, which show exemplary temporal layer picture prediction structures. FIGS. 2(a) shows a "temporally non-nested" structure 200a with two temporal layers, Layer 0 and Layer 25 1. The second layer (Layer 1) is formed as a completely separate "thread" that originates in the first frame (Layer 0). Since decoding of Layer 0 does not depend on Layer 1, this is a valid structure for SVC under the SVC JD7 draft. The problem transitioning from a lower temporal level to a higher temporal level with this structure is apparent for a receiver that receives only Layer 0 (at frames 0, 2, 4, etc.). The 30 receiver cannot add Layer 1 at will because the temporal extent of the dependency of Layer I from Layer 0 crosses over frames of Layer 0. If, for example, the receiver wishes to add Layer 1 at frame 2, it cannot do so by starting the decoding operation (for Layer 1) at the next frame (frame 3), since such decoding operation requires both frames 0 and 1, the latter of which was not received. -8- WO 2008/048886 PCTIUS2007/081217 FIG. 2(b) shows a similar temporally non-nested structure 200b, with a slightly more complicated coding structure of Layers 0 and 1. A receiver/decoder cannot switch to Layer 1 at frame 2, since frame 3 is predicted from frame 1. FIGS. 2a and 2b illustrate the problem of transitioning from a lower 5 temporal level to a higher temporal level using structures 200 and 202b, which for simplicity have only two layers each. It will be understood that the problem may exist with any number of temporal layers. FIG. 2c shows an exemplary structure 200c with three temporal layers, Layers 0-2. Structure 200c presents a similar transitioning problem because of the temporal extent of the layer dependencies. 10 It is noted temporally non-nested layer structures 200a-200c satisfy the requirements of G.7.4. 1, however, the use of the temporal scalability feature is seriously limited. In contrast, FIG. 3 shows a "temporally nested" layer structure 300, which satisfies the requirements of G.7.4.1 and also allows temporal switching from any layer to another. As shown in the figure, there is no instance of temporal nesting 15 in structure 300: for any frame i of layer N, there is no frame of a temporal level M<N that is inbetween frame i and any of its reference pictures in decoding order. Equivalently, no reference picture is used for inter prediction when a succeeding reference picture in decoding order has a lower temporal level value. This condition ensures that additional temporal layers to layer N can be added immediately after any 20 frame of layer N. The ability to easily add or remove temporal levels at the encoder/server, an intermediate gateway, or a receiver, is of fundamental importance in real-time, low-delay communications, as frame rate is one of the parameters that are directly available for rate bit rate and error control. It is noted that the exemplary 25 temporal prediction structures described in International Patent Application Nos. PCT/US06/28365, PCT/US06/028366, PCT/US06/061815, and PCT/US07/63335 are all nested. While the coding dependency information is explicitly encoded in the SSEI (and SSEI-DC), it does not capture the temporal extent of the dependency. For example, structures 200c and 300 have identical SSEI messages. 30 The systems and methods of the present invention include explicit information in the coded bitstream that (a) indicates the temporal extent of the dependency of temporal levels, and (b) provides the ability to enforce nested operation for specific application domains and profiles. -9- WO 2008/048886 PCT/US2007/081217 In one embodiment of the invention, the information consists of single bit flag, called "temporal levelnesting_flag," which is placed in SVC's Sequence Parameter Set. FIG. 4 shows modified syntax 400 for the relevant section of the JD7 5 text (Section G.7.3.2, Sequence parameter set SVC syntax) in accordance with the principles of the present invention. The added flag (temporallevelnestingflag) is the first one in the syntax structure. The semantics of the temporal_level_nestingflag (to be placed in G.7.4.2, Sequence parameter set SVC extension semantics in the JD7 text) are defined so that a value of 0 indicates that a 10 reference picture shall not be used for inter prediction if a succeeding reference picture in decoding order has a lower temporal level value, whereas a value of 1 indicates that no such restriction is placed. Alternative definitions of the semantics are also possible, without changing the limitation that it places on the structure of the bitstream. 15 In a second embodiment of the invention, the same temporal_levelnestingflag is placed in the SSEI (SVC JD7, Section G. 10.1.1), which has the additional benefit that all scalability information pertaining to a particular SVC bitstream is present in a single syntax structure. FIG. 5 shows modified syntax 500 for this case. The semantics for modified syntax 500 are 20 identical to the semantics applicable to syntax 400. The use of the temporal level nesting flag by a media server or encoder, media gateway, or receiver/decoder involves the same operations irrespective of whether the temporal level nestingflag is present in the SSEI or the Sequence Parameter Set. Since the operation is the same in both cases for all devices, for 25 convenience, all three different types of devices are referred to herein commonly as "Processing Units". FIG. 6 shows the architecture of an exemplary Processing Unit 600, as it relates to NAL filtering. Processing Unit 600 accepts SVC NAL units at each input, and produces copies of some or all of the input NAL units at its output. The decision 30 on which NAL units to forward to the output is performed at the NAL Filtering Unit 610. In a preferred architecture, NAL Filtering Unit 610 is controlled by a NAL Filter Configuration (NFC) table 620, which may be stored in RAM. NFC 620 is a three dimensional table, where the three dimensions T, D, and Q correspond to the temporallevel, dependencyid, and qualityid of a NAL. In FIG. 6, the table value is -10- WO 2008/048886 PCT/US2007/081217 shown in the PASS column. A value of 1 in a table entry with particular T, D, and Q values indicates that NAL Filtering.Unit 610 should forward an input NAL unit that has the same T, D, and Q values in its SVC header. Conversely, a value of 0 indicates that it should not forward the particular input NAL unit. Thus, according to NFC 620 5 shown in FIG. 6 the base layer (T=O, D=0, Q=O) is allowed to be forwarded to the output, but the higher temporal layer (T=1) is not. During set up, Processing Unit 600 obtains the SSEI, either in-band (from the SVC bitstream), through signaling, or other means. The SSEI is stored in RAM 640 to be used for later operations. NFC 620 may obtain its initial 10 configuration after the SSEI is obtained. The initial configuration may be, for example, such that all NAL units are passed on to the output (no filtering is applied). This is dependent on the specific application. Processing Unit 600 also sets an initial value to the TL memory 630, which stores the current operating temporal level. As shown in FIG. 6, Processing Unit 600 is also equipped with an 15 additional input, Temporal Level Switch Trigger 650. This input provides information to NAL Filtering Unit 610 on the desired temporal level of system operation. Temporal Level Switch Trigger 650 signal may, for example, have positive, zero, or negative integer values, indicating that after the current picture the temporal level should be increased, stay the same, or be reduced, respectively, by the 20 indicated amount. When NAL Filtering Unit 610 detects a negative value of Temporal Level Switch Trigger signal at a particular picture, it adds this value to the current operating temporal level value stored in TL memory 630 and reconfigures the NFC table 620 to reflect the desired new operating temporal level. If the addition results in 25 a negative value, a value of 0 is stored in TL memory 630. When NAL Filtering Unit 610 detects a positive Temporal Level Switch Trigger signal at a particular picture, it first checks the value of the temporallevelnestingflag. If the value is 0, then NAL Filtering Unit 610 cannot decide, in the absence of additional application-specific information, if it is possible to switch to the desired higher temporal level and no 30 action is taken. If the value of temporal_level_nestingflag is 1, then the value of the Temporal Level Switch Trigger signal is added to the TL memory, and the NFC table is reconfigured to reflect the desired new operating level. If the new value of the TL memory is higher than the maximum temporal level present in the bitstream, as reflected in the SSEI, then the TL is set to that maximum temporal level value. It is -11- WO 2008/048886 PCT/US2007/081217 noted that the maximum temporal level value can be obtained from the SSEI by parsing all the layer information contained in the SSEI, and storing the largest value of the temporallevel[i] syntax element. FIG. 7 shows a flow diagram 700 of the operation of NAL Filtering 5 Unit 610. In flow diagram 700, the legend 'TRIGGER' designates the value of Temporal Level Switch Trigger 650 signal of FIG. 6, while 'TL_MAX' designates the maximum temporal level value as obtained from the SSEI. Function NFC(T, D, Q) returns the value of NFC 620 for the particular combination of T, D, and Q values. It is noted that in systems where all components are purposefully 10 designed together, it may be possible to make a priori assumptions about the structure of the bitstream. In these cases, temporal level upswitching may be possible if certain criteria are satisfied by the T, D, and Q values. NAL Filtering Unit 610 may be configured to incorporate such criteria when attempting to perform temporal level upswitching, and to also elect to perform the temporal level upswitching at a later 15 picture, where presumably the application-specific conditions will be satisfied. While there have been described what are believed to be the preferred embodiments of the present invention, those skilled in the art will recognize that other and further changes and modifications may be made thereto without departing from the spirit of the invention, and it is intended to claim all such changes and 20 modifications as fall within the true scope of the invention. It will be understood that in accordance with the present invention, the techniques described herein may be implemented using any suitable combination of hardware and software. The software (i.e., instructions) for implementing and operating the aforementioned rate estimation and control techniques can be provided 25 on computer-readable media, which can include without limitation, firmware, memory, storage devices, microcontrollers, microprocessors, integrated circuits, ASICs, on-line downloadable media, and other available media. -12- - 13 Throughout this specification and the claims which follow, unless the context requires otherwise, the word "comprise", and variations such as "comprises" and "comprising", will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps. 5 The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that that prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of 10 endeavour to which this specification relates.