WO2006092671A1

WO2006092671A1 - Adaptive fallback system for mpeg decoders

Info

Publication number: WO2006092671A1
Application number: PCT/IB2005/050801
Authority: WO
Inventors: Johan Gerard Willem Janssen; Gene Farrel Pinkston; Torsten Fink; Gunther May
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2005-03-03
Filing date: 2005-03-03
Publication date: 2006-09-08
Also published as: JP2008532405A; CN1977538A

Abstract

Methods and apparatus in an adaptive fallback architecture provide for the preparsing (204) of an MPEG encoded video stream to enable the determination of when an MPEG decoder may be switched (208) into, or out of, modes of operation in order to reduce, for example, the computational, memory storage, and/or the memory bandwidth requirements of MPEG decoding. A pre-parser (104) evaluates at least one of the amount, type, and length of motion vectors in an MPEG stream, prior to decoding of that portion of the MPEG stream (304), and generates at least one control signal for communication to an MPEG decoder (106) which, responsive to the at least one control signal switches to a decoding mode that provides a peak memory bandwidth requirement below a predetermined threshold value.

Description

ADAPTIVE FALLBACK SYSTEM FOR MPEG DECODERS

The present invention relates generally to the decoding of MPEG encoded input streams, and more particularly relates to methods and apparatus for reducing the computational and memory requirements for at least portions of an MPEG decoding task.

Advances in semiconductor technologies and digital system architectures have resulted in great increases in computational power, while simultaneously resulting in reduced costs for such computational power. Low-cost high-performance computational resources have, in turn, enabled the implementation of a wide variety of consumer electronic products.

Among the popular consumer electronic products are those that can display that video data, and typically playback audio as well. Such consumer electronic products may receive video data to be displayed by means of, for example, wireless transmission, or by retrieving pre-recorded video data. With respect to video data, various industry standard specifications for the compression and decompression of video data have been developed. Such compression and decompression of video data are also sometimes referred to, respectively, as coding and decoding. By compressing video data for storage and/or transmission, the amount of storage and transmission capacity otherwise needed, can be reduced. One of the most commonly used family of standards for audio/video compression and decompression is referred to as MPEG.

MPEG compression, or encoding, performs a number of transformations upon an input video signal in order to represent that video signal with a smaller amount of data than the original video signal would have required by simply digitizing a video signal. MPEG encoding takes advantage of the fact that, typically from frame to frame, less than all of that frame data changes. The various changes between frames may be encoded as motion vectors which are applied to a base image frame. MPEG decoding essentially reverses the encoding process so as to reproduce the original video input.

The computational, memory storage, and memory bandwidth resources required for MPEG decoding can be quite substantial.

What is needed are methods and apparatus for reducing, even temporarily, one or more of the required computational, memory storage, and memory bandwidth resources required for MPEG decoding. Briefly, in an adaptive fallback architecture, a pre-parsing of an MPEG encoded video stream enables the determination of when an MPEG decoder may be switched into, or out of, modes of operation in order to reduce, for example, the computational, memory storage, and/or the memory bandwidth requirements of MPEG decoding. A pre-parser evaluates one or more of the amount, type, and length of motion vectors in an MPEG stream, prior to decoding of that portion of the MPEG stream, and generates at least one control signal for communication to an MPEG decoder which, responsive to the at least one control signal switches to a decoding mode that provides a peak memory bandwidth requirement below a predetermined threshold value. Fig. 1 is a high-level block diagram of a system including a demultiplexer, a pre- parser, and a hardware MPEG decoder, in accordance with the present invention.

Fig. 2 is a flow diagram illustrating a method in accordance with the present invention.

Fig. 3 is a flow diagram illustrating a method in accordance with the present invention.

Generally, the present invention relates to methods and apparatus for reducing the system load for MPEG decoding, at least during portions of decoding an MPEG video stream, depending upon various parameters in the MPEG video stream. More particularly, the present invention relates to methods and apparatus for dynamically adapting an MPEG decoding process so that that decoding process does not require an MPEG decoder to exceed a limit such as a predetermined peak memory bandwidth.

Reference herein to "one embodiment", "an embodiment", or similar formulations, means that a particular feature, structure, operation, or characteristic described in connection with the embodiment, is included in at least one embodiment of the present invention. Thus, the appearances of such phrases or formulations herein are not necessarily all referring to the same embodiment. Furthermore, various particular features, structures, operations, or characteristics may be combined in any suitable manner in one or more embodiments.

When decoding an MPEG video stream, the computational, memory storage, and memory bandwidth requirements, heavily depend on the contents of the MPEG stream. By means of looking ahead in the stream, information regarding such requirements can be determined before actual MPEG decoding takes place. This information can then be used to dynamically switch on/off fallback modes that can be built into, or are present in, the MPEG decoder, in order to temporarily reduce the system load (e.g., computational, memory storage, and memory bandwidth).

Fallback scenarios, or modes of operation, may be provided within an MPEG decoder to provide short cuts that reduce the load on the system. These short cuts are primarily used in software MPEG decoders. However, such MPEG decoders do not pre- parse the incoming video stream in order to determine the upcoming load on the MPEG decoder system.

Embodiments of the present invention include the ability to look ahead in an MPEG stream and then determine, depending on the value of one or more system parameters of the MPEG stream, whether fallback scenarios should be activated in the MPEG decoder. Without the ability to look ahead (i.e., pre-parse) provided by the present invention, a decision to switch to a fallback mode of operation in the MPEG decoder will, for some parameters, always be too late (e.g., if bandwidth needs to be regulated).

When being used, embodiments of the present invention typically perform several operations including receiving an MPEG stream; pre-parsing the MPEG stream in order to study the motion vectors; determining the peak memory. bandwidth that the MPEG decoder will consume for a given picture, or part of a picture, based, at least in part, upon the amount, type, and length of the motion vectors; and switching the MPEG decoder to a fallback mode of operation for the given picture, or part of a picture, if the determined peak memory bandwidth exceeds a predetermined value. In one embodiment, the fallback mode of operation only uses one of the two predictions of B-pictures, which results in a reduced requirement for memory bandwidth for the respective picture, or part of the picture.

Referring to Fig. 1, a high-level block diagram of an illustrative apparatus including a demultiplexer, a pre-parser, and a hardware MPEG decoder, in accordance with the present invention is shown. A demultiplexer 102 is coupled to receive a transport stream and produces as an output an elementary stream. In this case the transport stream includes encoded picture data. A pre-parser 104 is coupled to demultiplexers 102 so as to receive the elementary stream. It will be appreciated that the video stream referred herein may be a "high definition" MPEG stream, or any other video stream. Pre-parser 104 evaluates the contents of the elementary stream, and based upon, for example, the amount, type, and length of the motion vectors, determines a profile of the memory bandwidth requirements for decoding this portion of the video stream. If the memory bandwidth requirements exceed a predetermined value then pre-parser 104 outputs a control signal indicating that a fallback mode of operation should be used in decoding this particular portion of the video stream. This predetermined value may also be referred to as a limit or as a threshold. Pre- parser 104 also outputs a time-delayed version of the elementary stream. A decoder 106 is coupled to receive from pre-parser 104 both the time-delayed version of the elementary stream and the control signal. Decoder 106 operates in a mode suitable for remaining within certain memory bandwidth requirements depending on the state of the control signal provided by pre-parser 104 to decoder 106. In one embodiment, decoder 106 only uses one of the two predictions of B-pictures if pre-parser 104 determines that the decoding operation would otherwise exceed a predetermined memory bandwidth requirement. Decoder 106 is further adapted to output a decoded sequence. Decoding of encoded video sequences such as MPEG encoded video are well known, and the details of such decoding are not further described herein.

With respect to the predetermined value mentioned above, in various embodiments it may be either a fixed or a programmable value. Generally, such a predetermined value relates to a resource limit of the MPEG decoder, such as, for example, peak memory bandwidth. However, any resource limit may be a consideration in making a determination as to whether the MPEG decoder should operate in any particular fallback mode. In embodiments wherein the predetermined value is programmable, this limit may be dynamically set in response to a user's preference, the incoming video stream, or any suitable consideration or combination of considerations. In one illustrative embodiment in which the predetermined value relates to a peak memory bandwidth, this value may be chosen such that no frames are dropped by the MPEG decoder. In an alternative embodiment, the value may be chosen so as to maintain an average from drop rate. In still further alternative embodiments the predetermined value may be chosen on the basis of any suitable design consideration, or combination of design considerations. The present invention is not limited to any particular scheme for setting the predetermined value. In still further alternative embodiments, there may be two or more predetermined values relating to two or more resource limits (e.g., available memory storage capacity and peak memory bandwidth availability), and pre-parser 104 may make the determination regarding the MPEG decoder operating in a fallback mode based on such two or more limits.

Referring to Fig. 2, a flow diagram illustrating a method in accordance with the present invention is shown. The method shown in Fig. 2 reduces the system load for a video decoder, and includes receiving 202 an encoded video stream, pre-parsing 204 the encoded video stream, determining 206, based at least in part upon the pre-parsing of the encoded video stream, a system load requirement for decoding at least a first portion of a first picture in the encoded video stream, and switching 208 an operational mode of a video stream decoder to a fallback mode if the determined system load requirement exceeds a predetermined value. It will be appreciated that pre-parsing may be implemented in hardware, software, or a combination of hardware and software.

Referring to Fig. 3, a flow diagram illustrating a method in accordance with the present invention is shown. The method shown in Fig. 3 reduces the system load for an MPEG decoder, and includes providing 302 an MPEG encoded video stream to a pre- parser, prior to providing the MPEG encoded video stream to an MPEG decoder, evaluating 304 at least one of the amount, type, and length of a plurality of motion vectors in the MPEG encoded video stream, determining 306, based at least in part upon the evaluating of the MPEG encoded video stream, a peak memory bandwidth that will be required by the MPEG decoder operating in a non-fallback mode for decoding at least a first picture portion in the MPEG encoded video stream, switching 308 an operational mode of the MPEG decoder from the non-fallback mode to a fallback mode if the determined peak memory bandwidth requirement exceeds a predetermined value, and providing 310 the MPEG encoded video stream to an MPEG decoder. It will be appreciated that evaluating the motion vectors may be implemented by hardware, software, or a combination of hardware and software.

Various embodiments of the present invention include methods and apparatus for evaluating a portion of an encoded video stream prior to decoding of that portion of the encoded video stream, and based on that evaluation, switching a video decoder from a first mode of operation to a second mode of operation in order to keep the system loading requirements, for decoding that portion of the encoded video stream, below a predetermined value.

An advantage of some embodiments of the present invention, is that real-time decoding of the video stream can be maintained by reducing a system loading requirement, such as peak memory bandwidth, below a predetermined value. It is to be understood that the present invention is not limited to the embodiments described above, but encompasses any and all embodiments within the scope of the subjoined Claims.

Claims

CLAIMS What is claimed is:

1. A method of reducing the system load for a video decoder, comprising: receiving (202) an encoded video stream; pre-parsing (204) the encoded video stream; determining (206), based at least in part upon the pre-parsing of the encoded video stream, a system load requirement for decoding at least a first portion of a first picture in the encoded video stream; and switching (208) an operational mode of a video stream decoder to a fallback mode if the determined system load requirement exceeds a predetermined value.

2. The method of Claim 1, wherein pre-parsing comprises evaluating one or more motion vectors in the encoded video stream prior to decoding the video stream.

3. The method of Claim 2, wherein determining the system load requirement comprises evaluating at least one of the amount, type, and length of the pre-parsed motion vectors.

4. The method of Claim 3, wherein the system load requirement comprises a memory bandwidth requirement.

5. The method of Claim 4, wherein the memory bandwidth requirement is a peak memory bandwidth requirement.

6. The method of Claim 3, wherein encoded video stream is an MPEG encoded video stream; the decoder is an MPEG decoder; and the fallback mode of operation results in a reduced system load requirement.

7. The method of Claim 6, wherein the reduced system load requirement is a reduced memory bandwidth requirement.

8. The method of Claim 7, wherein the fallback mode of operation uses only one of the two predictions of B-pictures.

9. The method of Claim 8, further comprising switching the mode of operation of the MPEG decoder from a fallback mode to a non-fallback mode when a determination is made that the pre-determined peak memory bandwidth will not be exceeded by decoding at least a second portion of a second picture.

10. The method of Claim 9, wherein the first picture and the second picture are different pictures.

1 1. A method of reducing the system load for an MPEG decoder, comprising: providing (302) an MPEG encoded video stream to a pre-parser; prior to providing the MPEG encoded video stream to an MPEG decoder, evaluating (304) at least one of the amount, type, and length of a plurality of motion vectors in the MPEG encoded video stream; determining (306), based at least in part upon the evaluating of the MPEG encoded video stream, a peak memory bandwidth that will be required by the MPEG decoder operating in a non-fallback mode for decoding at least a first picture portion in the MPEG encoded video stream; switching (308) an operational mode of the MPEG decoder from the non- fallback mode to a fallback mode if the determined peak memory bandwidth requirement exceeds a predetermined value; and providing (310) the MPEG encoded video stream to an MPEG decoder.

12. The method of Claim 11 , wherein the fallback mode of operation only uses one of the two predictions of a B-picture.

13. The method of Claim 12, wherein switching the operational mode of the MPEG decoder to the fallback mode results in a reduced peak memory bandwidth requirement during decoding of the first picture portion.

14. The method of Claim 13, further comprising decoding the MPEG encoded video stream; and switching from the fallback mode to the non-fallback mode.

15. An apparatus for reducing the system loading requirements of decoding an encoded video stream, comprising: a demultiplexer (102) adapted to receive a first data stream, and to produce as an output a second data stream; a pre-parser (104), coupled to receive the second data stream, operable to produce as a first output a third data stream, and further operable to produce as a second output at least one control signal; and a decoder (106), coupled to receive the third data stream and the at least one control signal from the pre-parser, and to produce a decoded version of the time-delayed version of the second data stream, the decoder capable of operating in at least two decoding modes of operation, each mode of operation having different system loading requirements; wherein the first data stream includes encoded picture data, and the mode of operation of the decoder is selectable in response to the at least one control signal.

16. The apparatus of Claim 15, wherein the first data stream is a transport stream, the second data stream is an elementary stream, the third data stream is a time- delayed version of the elementary stream; and wherein the pre-parser is operable to produce the at least one control signal based at least in part on at least one predetermined decoder resource limit.

17. The apparatus of Claim 16, wherein the decoder is an MPEG decoder, and wherein one mode of operation of the MPEG decoder only uses one of two predictions of B-pictures in order to reduce the memory bandwidth required for decoding.

18. The apparatus of Claim 17, wherein the MPEG decoder is adapted to receive the at least one control signal from the pre-parser on a picture by picture basis.

19. The apparatus of Claim 17, wherein the MPEG decoder is operable to select one of a plurality of fallback modes of operation responsive to the at least one control signal.

20. The apparatus of Claim 19, wherein the predetermined resource limit is set based upon a predetermined frame drop rate.