KR20080082676A - A hardware multi-standard video decoder device - Google Patents

Abstract

A hardware multi-standard video decoder device. A command parser accesses a video stream and identifies a video encoding standard used for encoding the video stream. A plurality of hardware decoding blocks perform operations associated with decoding the video stream, wherein different subsets of the plurality of hardware decoding blocks are for decoding video streams encoded using different video encoding standards.

Description

Hardware multi-standard video decoder device {A HARDWARE MULTI-STANDARD VIDEO DECODER DEVICE}

The present invention relates to video decoding. Specifically, the present invention relates to a hardware multiple standard video decoder device.

Digital video streams are typically encoded using one of many different encoding standards. For example, a digital video stream can be compressed for conversion to data formats that require fewer bits. This compression is lossless so that the original video stream can be played back during decoding, or lossy, so that an exact copy of the original video stream cannot be played back, but the decoding of the compressed data is more efficient.

There are currently many video encoding standards, and new standards are often emerging. Examples of current video encoding standards are Joint Photographic Experts Group (JPEG), Moving Picture Experts Group (MPEG), MPEG-2, MPEG-3, MPEG-4, H.263, H.263 +, H.264 and Real Proprietary standards such as video and window media. In order to fully realize the benefits of digital video, the user needs access to decoders that can decode all common encoding standards.

Many important uses of video streaming are related to real time communication. For example, video telephony requires real-time video decoding in order to be able to synchronize with the corresponding audio signal. Accordingly, it is also desirable to provide users with real time video decoding to provide applications related to real time communication. Moreover, a situation arises in which a user requires decoding of multiple video streams. For example, a user who is currently on a video phone call receives an attachment image from the person he is talking to. In this example, real time decoding of the video telephony stream must be maintained while the image required for the conversation is decoded.

Currently, video decoding is performed using two available methods: single standard hardware video decoders and software based programmable cores capable of decoding the video stream in accordance with one or more video standards. Single standard hardware video decoders can provide real time decoding functionality. However, in order to decode a video stream encoded using a particular encoding standard, the user must have a hardware video decoder for that particular standard. Since there are a number of widely used video encoding standards, users will need many different single standard hardware video decoders to access digital video encoded using different video encoding standards, which requires significant financial expenditure on the user. do. Moreover, conventional computer systems do not have the ability to add multiple single standard hardware video decoders, further limiting the number of video streams that a user can access.

Current software based programmable core video decoders can be used to provide decoding using one or more video encoding standards. The programmable core video decoder may include hardware acceleration to accelerate the decoding function. However, the programmable core does all the decoding. Programmable core video decoders typically have high processing overhead, are less efficient, and consume much more power than a single standard hardware video decoder. Moreover, programmable core video decoders cannot consistently provide real-time video decoding, since decoding depends on the processing requirements of the entire computer system.

Thus, currently available digital video decoders cannot provide real-time video decoding for many widely used video encoding standards. In addition, currently available digital video decoders cannot provide simultaneous video decoding for multiple streams encoded using multiple video encoding standards that are widely used. Moreover, currently available digital video decoders cannot simultaneously decode multiple video streams in which at least one video stream requires real time decoding. Therefore, there is a need for a new digital video decoder that overcomes the limitations of the prior art. New digital video decoders must provide real-time video decoding for many different video standards. The new digital video decoder must also provide simultaneous video decoding functionality for a plurality of video streams encoded using a plurality of different video standards.

Summary of the Invention

Embodiments of the present invention provide a hardware multi-standard video decoder device for providing video decoding functionality for a plurality of different video encoding standards. Embodiments of the present invention may provide real time decoding for each of a plurality of video encoding standards.

In one embodiment, the present invention provides a hardware multiple standard video decoder device. The command parser of the hardware multi-standard video decoder device may be operable to access the video stream and identify the video encoding standard used to encode the video stream. The hardware multi-standard video decoder apparatus also includes a plurality of hardware decoding blocks for performing operations associated with decoding of the video stream, wherein different subsets of the plurality of hardware decoding blocks receive video streams encoded using different video encoding standards. For decoding. In one embodiment, a hardware multi-standard video decoder device is implemented in an integrated circuit coupled to a printed circuit board, the printed circuit board coupled to a connector for detachably coupling the printed circuit board to a computer system.

In one embodiment, the command parser may be operable to activate a first subset of the plurality of hardware decoding blocks used to decode the first identified video encoding standard used to encode the video stream, and thus the video. Hardware decoding blocks not associated with decoding of the stream are not activated. In one embodiment, the command parser may be operable to activate a second subset of the plurality of hardware decoding blocks used to decode the second identified video encoding standard used to encode the video stream, and thus the video. Hardware decoding blocks not associated with decoding of the stream are not activated.

In one embodiment, the plurality of hardware decoding blocks is implemented in a multi-stage macroblock level pipeline. In one embodiment, the command parser may be operable to deactivate hardware decoding blocks within this stage if no data in the video stream is received at one stage of the multi-stage macroblock level pipeline. In one embodiment, the hardware multi-standard video decoder device accesses the memory unit after fully decoding the video stream.

In one embodiment, the hardware multi-standard video decoder device further comprises a hardware post processing block for performing a post processing operation on the decoded video stream. In one embodiment, the command parser operates to deactivate a plurality of hardware decoding blocks when the video stream received at the command parser is a decoded video stream, such that the hardware post processing block performs a post processing operation on the decoded video stream. It can be done. In one embodiment, the hardware post processing block includes a filter.

In another embodiment, the present invention provides a method for decoding a video stream, which method is implemented using a hardware multiple standard video decoder device. The video stream is accessed. The video standard used to encode the video stream is identified. A subset of the hardware decoding blocks of the plurality of hardware decoding blocks of the hardware multi-standard video decoder device used to decode the video stream is determined, and different subsets of the plurality of hardware decoding blocks are encoded using different video encoding standards. To decode the video streams. The video stream is decoded using a subset of hardware decoding blocks.

In one embodiment, the plurality of registers comprises a memory surface pointer register and a frame level parameter register. In one embodiment, the hardware multi-stream multi-standard video decoder device further comprises a hardware post-processing block for performing a post processing operation on at least one decoded video stream.

In one embodiment, the plurality of video streams comprises at least one digital still image stream and a digital movie stream. In this embodiment, the portions of the plurality of video streams are macroblocks of the digital still image stream and the digital movie stream. In another embodiment, the plurality of video streams comprises a plurality of digital movie streams. In this embodiment, the portions of the plurality of video streams are macroblocks of the plurality of digital movie streams.

In another embodiment, the present invention provides a method for decoding a plurality of video streams, which method is implemented using a hardware multi-stream multi-standard video decoder device. Multiple video streams are accessed. Video standards used to encode video streams are identified. Subsets of the hardware decoding blocks of the plurality of hardware decoding blocks of the hardware multi-stream multi-standard video decoder device used to decode the video streams are determined, and the subsets of the plurality of hardware decoding blocks are encoded using different video encoding standards. To decode the video streams. The plurality of video streams are decoded using a subset of hardware decoding blocks. In one embodiment, the plurality of subsets of hardware decoding blocks is activated, so that hardware decoding blocks not associated with decoding of the plurality of video streams are not activated.

In general, the present invention discloses a hardware multiple standard video decoder device. The command parser identifies the video encoding standard used to access the video stream and encode the video stream. A plurality of hardware decoding blocks performs operations associated with decoding of a video stream, and different subsets of the plurality of hardware decoding blocks decode video streams encoded using different video encoding standards.

The invention is shown by way of example and not by way of limitation in the figures of the accompanying drawings in which like reference numerals indicate like elements.

1 is a schematic diagram of basic components of a computer system according to an embodiment of the present invention.

2A is a diagram of an exemplary hardware video decoder card implemented on a printed circuit board in accordance with one embodiment of the present invention.

2B is a diagram of an exemplary architecture that includes a hardware multi-standard video decoder device in accordance with an embodiment of the present invention.

3 is a block diagram illustrating internal components of a hardware multi-standard video decoder device according to an embodiment of the present invention.

4 is a block diagram illustrating internal components of an exemplary hardware multi-standard video decoder device according to an embodiment of the present invention.

5 is a flowchart of a video stream decoding method implemented using a hardware multi-standard video decoder device according to an embodiment of the present invention.

6 is a diagram illustrating internal components of a hardware multi-stream multi-standard video decoder device according to an embodiment of the present invention.

7A and 7B are diagrams illustrating exemplary interleaved portions of multiple video streams in accordance with one embodiment of the present invention.

8 is a flowchart of a method for decoding a plurality of video streams, implemented using a hardware multi-stream multi-standard video decoder device, in accordance with an embodiment of the present invention.

9 is a flowchart of a method for processing out-of-order macroblocks of a video stream, in accordance with an embodiment of the present invention.

10A and 10B are diagrams of exemplary rotation of macroblocks of frames in accordance with an embodiment of the present invention.

11 is a flowchart of a method for rotating macroblocks of a frame according to an embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Now, preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, are described in detail. While the invention has been described in connection with the preferred embodiments, it will be understood that they are not intended to limit the invention to these embodiments. On the contrary, the invention is intended to cover alternatives, modifications and equivalents that may be included within the spirit and scope of the invention as defined by the appended claims. In addition, in the following detailed description of embodiments of the invention, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, those skilled in the art will recognize that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of embodiments of the present invention.

Symbols and Nomenclature

Certain portions of the detailed description that follows are provided in connection with procedures, steps, logical blocks, processing, and other symbolic representations of operations on data bits in computer memory. These descriptions and representations are the means used by those in the data processing arts to convey their work to others in the field. Procedures, computer execution steps, logic blocks, processes, and the like are considered here and generally to be a consistent sequence of steps or instructions that lead to a desired result. The steps are those requiring physical manipulation of physical quantities. Generally, but not necessarily, these quantities take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and manipulated in a computer system. Sometimes, for general use reasons, it has proven convenient to refer to these signals as bits, values, elements, symbols, letters, terms, numbers, and the like.

However, it should be remembered that both these and similar terms should be associated with appropriate physical quantities and are merely convenient labels applied to these quantities. As will be apparent from the description below, unless otherwise stated, "identification" or "access" or "perform" or "decoding" or "activation" or "deactivation" or "determination" or "processing" throughout this invention. Or descriptions using terms such as "receive" or "buffering" or "array" or "transmit" or "parsing" or "interleaving" or "rotate" or "relocate" or "save" or the like. For example, the hardware multi-standard video decoder device 150 of FIG. 3, the hardware multi-stream multi-standard video decoder device (eg, the hardware multi-stream multi-standard video decoder device 600 of FIG. 6), the microcode engine (Eg, microcode engine 260 of FIG. 2B), a rotation engine (eg, rotation engine 450 of FIG. 4), or physical (electronic) quantities within registers and memories of a computer system. Refers to the operations and processes of a similar electronic computing device that manipulates the data represented as a computer system memories or registers or other data similarly represented as physical quantities in storage, transmission or display devices. I understand.

Computer system platform

1 illustrates an exemplary computer system in which embodiments of the present invention may be practiced. In general, computer system 100 is a bus 110 for communicating information, a processor 101 coupled to bus 110 to process information and instructions, and a bus 110 coupled to bus 110 to processor 101. Volatile memory 102, also referred to as random access memory (RAM) for storing information and instructions, and coupled to bus 110 to store static information and instructions for process 101, where read-only Non-volatile memory 103, also referred to as memory (ROM).

In one embodiment, computer system 100 optionally includes an optical data storage device 104 such as a magnetic or optical disk and a disk drive coupled to bus 110 for storing information and instructions. In one embodiment, computer system 100 is coupled to bus 110 to a user output device, such as display device 105 for displaying information to a computer user, coupled to bus 110 to process information and command selections. A user input device, such as an alphanumeric input device 106 including alphanumeric and function keys for communicating to 101, and / or coupled to the bus 110 to communicate user input information and command selections to the processor 101. A user input device, such as a cursor control device 107, as an option. In addition, an optional input / output (I / O) device 108 is used to couple the computer system 100 on a network, for example.

In one embodiment, computer system 100 is also a hardware multi-standard video decoder device 150, also referred to herein as decoder device 150, for decoding a video stream encoded using one of a number of video encoding standards. It includes. Decoder device 150 includes a plurality of hardware decoding blocks for performing decoding operations required by multiple video encoding standards. It should be understood that decoder device 150 may be configured to decode video according to any combination of video encoding standards, including digital still images and digital movies. For example, decoder device 150 may encode video using any of JPEG, MPEG-4, H.263, H.263 +, H.264, and Window Media (WMV9 / VC-1) formats. It can be configured to decode.

Decoder device 150 is a discrete graphics card, discrete integrated circuit die (eg, motherboard) designed to be coupled to computer system 100 via discrete components, connectors (eg, AGP slots, PCI-Express slots, etc.). It may be implemented as an integrated decoder device included in an integrated circuit die of a computer system chipset component. In addition, a local graphics memory may be included for data storage of the decoder device 150.

2A shows a diagram of an exemplary hardware video decoder card 200 implemented on a printed circuit board in accordance with one embodiment of the present invention. The hardware video decoder card 200 includes a printed circuit board (PCB) 210, an integrated circuit (IC) chip 220, a data line 225, and a connector 230. IC chip 220 includes a hardware multi-standard video decoder device 150. The connector 230 is configured to couple to a computer system (eg, the computer system 100 of FIG. 1) through a connector (eg, an AGP slot, a PCI-Express slot, etc.) of the computer system. Data line 225 is for communicating data (eg, a bit stream) between computer system and IC chip 220.

2B shows a diagram of an exemplary architecture 250 that includes a hardware multi-standard video decoder device 150 in accordance with an embodiment of the present invention. Architecture 250 includes a microcode engine 260, a hardware multi-standard video decoder device 150, and a memory 270. In one embodiment, microcode engine 260 controls the operation of hardware multiple standard video decoder device 150. The microcode engine 260 includes operations that the hardware multi-standard video decoder device 150 should perform, acting as a translation layer between the machine instructions and the hardware device decoder 150. In one embodiment, bit stream parsing and variable length decoding (VLD) are performed in the microcode engine 260. Memory 270 is used to perform decoding and post-processing operations on the video streams received by decoder device 150. One embodiment of the operating memory 270 is described in the memory 330 of FIG.

2B, in one embodiment, the present invention provides for rearranging macroblocks in the microcode engine 260. As shown in FIG. As described below, decoder device 150 may include in-loop deblocking (eg, in in-loop deblocking filter 440) and out-of-loop deblocking and / or de-ringing (eg, out-of-loop filter 442). Different post-processing operations, such as In various embodiments, in-loop deblocking requires macroblocks to be received at the in-loop deblocking filter in raster scan order. However, certain video standards, such as H.264, support the transmission and reception of macroblocks in non-raster scan order. Accordingly, the present invention provides for rearranging macroblocks in raster scan order to support in-loop deblocking for video standards that support the transmission and reception of macroblocks in non-raster scan order.

In one embodiment, preprocessing operations are performed in microcode engine 260. In one embodiment, bit stream parsing and variable length decoding (VLD) are performed in the microcode engine 260. The microcode engine 260 is configured to arrange the macroblocks before sending them to the hardware decoder device 150. The microcode engine 260 buffers one frame of compressed data. In one embodiment, microcode engine 260 buffers one frame of run length encoded compressed data. In one embodiment, the microcode engine 260 parses the incoming bit stream and then performs the VLD. When the microcode engine 260 detects disordered macroblocks, it buffers the data and waits for all macroblocks to be received. The microcode engine 260 then arranges the macroblocks in raster scan order and sends them to the hardware decoder device 150.

By buffering macroblocks while the macroblocks are still compressed data, the microcode engine 260 only needs to buffer a maximum of one frame of run length encoded compressed data much less than the decoded video data. In addition, buffering compressed macroblocks also saves power. Video streams received over the air also generate many errors. Partitioning of bit stream parsing to microcode engine 260 also has the advantage of improving error recovery.

Hardware Multi-Standard Video Decoder Device Architecture

3 illustrates internal components of a hardware multi-standard video decoder device 150 in accordance with an embodiment of the present invention. As shown in FIG. 3, the decoder device 150 includes a command parser 305, a plurality of hardware decoding blocks 310 to 318, a hardware post-processing block 320, and a memory 330. Decoder device 150 may decode a number of video encoding standards.

The command parser 305 is for accessing the video stream 302 (eg, bit stream). Video stream 302 is a compressed video stream encoded according to one of a number of video encoding standards. It should be understood that video stream 302 may include digital still image data (eg, JPEG encoded) or digital movie data (eg, MPEG-4). In one embodiment, video stream 302 is received from a microcode engine (eg, microcode engine 260 of FIG. 2B). Command parser 305 identifies the video encoding standard used to encode video stream 302. In one embodiment, bit stream parsing and variable length decoding (VLD) is performed before the command parser 305 accesses the video stream 302. Bit stream parsing and VLD may be performed by a host CPU (eg, processor 101 of FIG. 1) or by a microcode engine (eg, microcode engine 260 of FIG. 2B). The command parser 305 also controls the movement of data through the decoder device 150 by controlling clock cycles.

A plurality of hardware decoding blocks 310-318 are for performing operations associated with decoding of the video stream. It should be understood that the hardware decoding blocks 310-318 represent different decoding functions required to decode video streams in accordance with video standards implemented within the video decoder 150. Video encoding standards such as MPEG-4 require certain operations to be performed to decode a video stream, so that all MPEG-4 decoders can decode MPEG-4 video streams. It should be understood that the operations required to perform decoding in accordance with various standards are known to those skilled in the art.

In one embodiment, the hardware decoding blocks of decoder device 150 are configured to perform operations at the macroblock level (eg, 8x8 pixel macroblocks). However, it should be understood that the decoder device 150 may include hardware decoding blocks that perform operations at levels of other dimensions, such as the frame level.

Different subsets of the hardware decoding blocks 310-318 are for decoding video streams encoded using different video encoding standards. For example, the first exemplary video standard requires the use of hardware decoding blocks 312 and 316 in decoding the video stream. The second exemplary video standard requires the use of hardware decoding blocks 310, 312, 314, 318 in decoding a video stream. Thus, in various embodiments of the present invention, only the hardware decoding blocks needed to decode the video stream are used in decoding the video stream encoded using the identified video standard.

In one embodiment, the command parser 305 may operate to activate only the hardware decoding blocks needed for decoding the received video stream, so that hardware decoding blocks not associated with decoding of the video stream are not activated. For example, a first subset of hardware decoding blocks (e.g., hardware decoding blocks 312, 316) used to decode the first identified video encoding standard is activated and thus related to the decoding of the video stream. Hardware decoding blocks that are not (eg, hardware decoding blocks 310, 314, 318) are not activated. In another example, a second subset of hardware decoding blocks (eg, hardware decoding blocks 310, 312, 314, 318) used to decode the second identified video encoding standard is activated and thus the video stream. A hardware decoding block (eg, hardware decoding block 316) not associated with the decoding of is not activated. In one embodiment, the command parser 305 is a component of the only decoder device 150 that is active. Hardware decoding blocks are activated as needed according to the identified video standard and data flow.

In one embodiment, the hardware decoding blocks of decoder device 150 are implemented in a multi-stage macroblock level pipeline. As shown in FIG. 3, the decoder device 150 includes a pipeline stage 1 that includes hardware decoding blocks 310 and 312, and a pipeline stage 2 that includes hardware decoding blocks 314, 316, and 318. It is implemented as a three stage macroblock level pipeline comprising: In one embodiment, command parser 305 directs the macroblock of video stream 302 to the hardware decoding blocks of pipeline stage 1. In one embodiment, two or more macroblocks may be located in pipeline stage 1, while pipeline stages 2 and 3 are limited to having only one macroblock. In one embodiment, the hardware decoding blocks 312, 316, 318 are in the residual data path and the hardware decoding blocks 310, 314 are in the prediction data path. In one embodiment, the residual data path handles error or differential data, and the prediction path accesses data associated with the previous frame or macroblock.

In one embodiment, the command parser 305 may be operable to deactivate hardware decoding blocks within the stage if no data in the video stream is received at one stage of a multi-stage macroblock level pipeline. For example, when decoding the video stream 302, when the final data of the video stream 302 leaves pipeline stage 1 and no data is received in pipeline stage 1, all hardware decoding of pipeline stage 1 The block is deactivated. Thus, additional power savings are achieved by deactivating all hardware decoding blocks of the pipeline stage even if hardware decoding blocks are needed for the video standard associated with the video stream 302.

In one embodiment, video stream 302 does not enter or exit memory 330 until fully decoded. The memory 330 may be an external memory unit (eg, the volatile memory 102 or the nonvolatile memory 103 of FIG. 1) or an internal memory unit of the decoder device 150. By not accessing the memory 330 until after the video stream 302 has been fully decoded, the decoder device 150 uses less power.

In one embodiment, the decoder device 150 further includes a hardware post-processing block 320 for performing a post-processing operation on the decoded video stream. In one embodiment, hardware post-processing block 320 includes a deblocking filter. It should be understood that the deblocking filter may be an in-loop deblocking filter or an out-of-loop deblocking and / or deringing filter. In-loop deblocking filter performs deblocking operations before accessing memory 330. The out-of-loop deblocking and deringing filter performs deblocking and deringing operations on the data accessed from the memory 330. However, it should be understood that hardware post-processing block 320 may perform any type of post-processing operation. Moreover, there may be any number of hardware post processing blocks 320 for performing multiple post processing operations.

In one embodiment, the command parser 305 may operate to deactivate all hardware decoding blocks if the video stream 302 is a decoded video stream, such that the hardware post processing block 320 may be decoded in the decoded video stream. It is possible to perform a post-processing operation on the. In other words, the decoder device 150 may be used only as a hardware post-processing device. When the decoded video stream is received at the decoder device 150, all hardware decoding blocks are deactivated, and a post processing operation on the decoded video stream is performed.

4 is a block diagram illustrating internal components of an exemplary hardware multi-standard video decoder device 400, also referred to as decoder device 400, in accordance with an embodiment of the present invention. Decoder device 400 is configured to operate as any one of JPEG, MPEG-4, H.263, H.263 +, H.264 or WMC9 / V3-1 decoders. Thus, the decoder device 400 is required to decode video streams encoded using any of the JPEG, MPEG-4, H.263, H.263 +, H.264 or WMC9 / V3-1 standards. Hardware decoding blocks for performing a decoding operation. However, it should be understood that the present invention is flexible to support of other video standards, and the present invention is not intended to be limited to the embodiment described in FIG.

As shown in FIG. 4, the decoder device 400 includes a command parser 402, a plurality of hardware decoding blocks, a plurality of hardware post-processing blocks, and a memory 460. The command parser 402 is for accessing the video stream 401 (eg, bit stream). It should be understood that video stream 401 may include digital still image data (eg, JPEG encoded) or digital movie data (eg, MPEG-4). In one embodiment, video stream 401 is received from a microcode engine (eg, microcode engine 260 of FIG. 2B). Video stream 401 is a compressed video stream encoded according to one of a number of video encoding standards. Command parser 402 identifies the video encoding standard used to encode video strip 401. In one embodiment, before the command parser 402 accesses the video stream 401, bit stream parsing and variable length decoding (VLD) are performed. Bit stream parsing and VLD may be performed by a host CPU (eg, processor 101 of FIG. 1) or a microcode engine. It should be understood that if the video stream 401 has been encoded using a different video standard than those that the decoder device 400 is configured to decode, then no decoding operation is performed. In one embodiment, the command parser 402 sends an indication to the computer system indicating that decoding cannot be performed on a video stream encoded using an unsupported standard.

In identifying the video standard used to encode the video stream 401, the command parser 402 directs macroblocks of the video stream 401 to hardware decoding blocks suitable for the identified video standard. In one embodiment, the command parser activates hardware decoding blocks suitable for the identified video standard, so that hardware decoding blocks that are not needed for the identified video standard are deactivated. Command parser 402 also controls the movement of data through decoder device 400 by controlling clock cycles. In one embodiment, the command parser 402 is a component of the only decoder device 400 that is active. Hardware decoding blocks are activated as needed according to the identified video standard and data flow.

The hardware decoding blocks of the decoder device 400 include the intra prediction mode engine 404, the motion vector (MV) prediction engine 406, the coefficient (eg, execution length (RD) or inverse quantization) engine 408, AC / DC (eg, AC / DC prediction or inverse quantization) prediction engine 410, intra prediction engine 414, rotation engine 415, motion compensation engine 416, 4x4 inverse transform engine 418, 8x8 inverse Discrete Cosine Transform (IDCT) engine 420, IDCT format converter engine 422, intra prediction buffer 432, prediction sample 434, and residual block 436. Decoder device 400 further includes multiplexers 405, 409, 417, 419, 439 and adder 435. Decoder device 400 also optionally includes hardware post-processing blocks, namely an in-loop deblocking filter 440, an out-of-loop filter 442, and a rotation engine 450.

Decoder device 400 is implemented in a three stage macroblock level pipeline with a residual path and a prediction path. In one embodiment, two or more macroblocks may be located in pipeline stage 1, while pipeline stages 2 and 3 are limited to having only one macroblock. The residual path includes a coefficient engine 408, an AC / DC prediction engine 410, a 4x4 inverse transform engine 418, an 8x8 IDCT engine 420, an IDCT format converter engine 422, and a residual block 436. The prediction path is intra prediction mode engine 404, MV prediction engine 406, intra prediction engine 414, rotation engine 415, motion compensation engine 416, intra prediction buffer 432, and prediction sample 434. It includes.

As mentioned above, the decoder device 400 may be operable to decode video streams in accordance with any of the JPEG, MPEG-4, H.263, H.263 +, H.264 or WMC9 / V3-1 standards. have. The described hardware decoding blocks perform all the decoding operations required according to the supported standards. Specific operations of hardware decoding blocks are described in each of the standards and are therefore known and understood by those skilled in the art. Thus, certain operations of hardware decoding blocks are not described in detail herein.

In one embodiment, MV parameters and intra prediction parameters are passed to MV prediction engine 406 and intra prediction mode engine 404 in the prediction path, respectively. These engines calculate the actual motion vectors or intra prediction mode based on the programmed video standard and pass them to the motion compensation engine 416 or intra prediction engine 414 respectively. Motion compensation engine 416 or intra prediction engine 414 calculates prediction data. In one embodiment, motion compensation engine 416 includes a rotation engine 415. The rotation engine 415 is for rotating the reference frame to align with the incoming video frame. Rotation engine 415 is activated whenever a motion compensation engine is used to encode a video stream. Error data, on the other hand, is processed in the required subset of coefficient engine 408, AC / DC prediction engine 410, 4x4 inverse transform engine 418, 8x8 IDCT engine 420, and IDCT format converter engine 422.

The recovered error data is added to the prediction data and then passed to pipeline stage 3. The resulting data is further processed as needed and written to and displayed in memory 460. In-loop deblocking filter is used in H264 and WMV9 / VC-1 modes. In WMV9 / VC-1 mode, in-loop deblocking filter 440 is used to implement the overlap smoothing filter. The out of loop filter 442 can be used on any video stream to improve the quality of the decoded image. In one embodiment, the out-of-loop filter 442 is executed concurrently with the rest of the decoder device 400. The out-of-loop filter 442 should be triggered after the frame is decoded into memory 460. The decoded image may also be rotated before it is written to the memory 460 in pipeline stage 3 in the rotation engine 450.

Exemplary Operation of a Hardware Multi-Standard Video Decoder Device for Supported Video Standards

The following embodiments describe the operation of the decoder device 400 for each of the supported video standards.

JPEG : JPEG decoding does not require hardware decoding blocks of the prediction path because JPEG video streams are for playing digital still images. Thus, intra prediction mode engine 404, MV prediction engine 406, intra prediction engine 414, rotation engine 415, motion compensation engine 416, intra prediction buffer 432 and prediction sample 434 All are disabled for JPEG decoding. Also, JPEG decoding does not require a 4x4 inverse transform engine 418, which is thus deactivated. The command parser 402 uses the coefficient engine 408, the AC / DC prediction engine 410, the 8x8 IDCT engine 420, the decimal IDCT engine 438, the IDCT format converter engine 422, and the residual block 436. Activate it. Command parser 402 routes data from video stream 401 through active hardware decoding blocks for decoding a JPEG encoded video stream. It should be understood that the operations and sequence of operations performed by the hardware decoding blocks are required by the JPEG standard.

JPEG decoding requires only the use of an 8x8 IDCT engine 420 and a decimal IDCT engine 438. In one embodiment, the command parser 402 may be operative to identify which of the 8x8 IDCT engine 420 and the decimated IDCT engine 438 is activated for the video stream. The 8x8 IDCT engine 420 is activated to fully decode the video stream, while the decimal IDCT engine 438 is activated when the video stream indicates decimation. The IDCT format converter engine 422 may perform format conversion. For example, the IDCT format converter engine 422 converts the format between any of the YUV 4: 4: 4, YUV 4: 2: 2, YUV 4: 2: 2R, and YUV 4: 2: 0 formats. Can be done. It is to be understood that other format conversions may be performed, and that the IDCT format converter engine 422 is not limited to the formats listed.

The decoded JPEG video stream is in pipeline stage 2. In one embodiment, the decoded JPEG video stream is stored in memory 330. In another embodiment, post processing operations are performed on the decoded JPEG video streams prior to storing them in the memory 330.

MPEG- 4 / H.263 : MPEG-4 and H.263 decoding are very similar to each other for the purpose of the decoder apparatus 400. Specifically, the MPEG-4 standard requires MPEG-4 decoders to be able to decode H.263 encoded video streams. MPEG-4 and H.263 decoding does not require intra prediction mode engine 404, intra prediction engine 414, IDCT format converter engine 422, and 4x4 inverse transform engine 418, which are deactivated. In-loop deblocking filter 440 is also deactivated for post processing operations. Thus, the command parser may include an MV prediction engine 406, a coefficient engine 408, an AC / DC prediction engine 410, a rotation engine 415, a motion compensation engine 416, an 8x8 IDCT engine 420, an intra prediction buffer. 432, activate the prediction sample 434 and the residual block 436. Command parser 402 routes data from video stream 401 through active hardware decoding blocks for decoding an MPEG-4 or H.263 encoded video stream. It should be understood that the operations and sequence of operations performed by the hardware decoding blocks are required by the MPEG-4 and H.263 standards.

Command parser 402 may direct macroblocks to appropriate residual path or prediction path hardware decoding blocks. In one embodiment, while the prediction frames (P frames) are processed in the MV prediction engine 406 in pipeline stage 1, the intra frames (I frames) are added to the coefficient engine 408 and AC of the remaining path. / DC prediction engine 410 may be processed. I frames and P frames are synchronized in pipeline stage 2. Command parser 402 may also activate appropriate hardware decoding blocks of 8x8 IDCT engine 420.

The decoded MPEG-4 / H.263 video stream is in pipeline stage 2. In one embodiment, the decoded MPEG-4 / H.263 video stream is stored in memory 330. In another embodiment, post-processing operations are performed on the decoded MPEG-4 / H.263 video streams prior to storing them in the memory 330. In another embodiment, post processing operations are performed on the MPEG-4 / H.263 video stream encoded in the out-of-loop filter 442. In one embodiment, out-of-loop filter 442 is a deblocking filter. In another embodiment, out-of-loop filter 442 is a deringing filter. In another embodiment, out-of-loop filter 442 is both a deblocking filter and a deringing filter. It should be understood that the out-of-loop filter 442 can be implemented as any deblocking and / or deringing filter.

H.263 + : H.263 + decoding is similar to MPEG-4 / H.263 decoding as described above. H.263 + shifts some of the decoding operation into the VLD, which is performed before the command parser 402 accesses the video stream 401. In addition to not requiring an intra prediction mode engine 404, an intra prediction engine 414, a 4x4 inverse transform engine 418, and an out-of-loop engine 442 and thus deactivating, the command parser 402 also has a counting engine ( 408 and deactivate the AC / DC prediction engine 410. In other respects, H.263 + decoding is similar to MPEG-4 / H.263 decoding as described above. It should be understood that the operations and sequence of operations performed by the hardware decoding blocks are required by the H.263 + standard.

H.264 : H.264 decoding does not require AC / DC prediction engine 410, 8x8 IDCT engine 420, and IDCT format converter engine 422, which are deactivated. Accordingly, the command parser 402 may include the intra prediction mode engine 404, the MV prediction engine 406, the coefficient engine 408, the intra prediction engine 414, the rotation engine 415, the motion compensation engine 416, 4x4. Inverse transform engine 418, intra prediction buffer 432, predictive sample 434, and residual block 436 are activated. Intra prediction buffer 432 can store the top row of pixels from the previous macroblock, so intra prediction engine 414 can access previous "leveling" pixels when processing the next row of macroblocks. The command parser 402 routes data from the video stream 401 through active hardware decoding blocks for decoding the H.264 encoded video stream. It should be understood that the operations and sequence of operations performed by the hardware decoding blocks are required by the H.264 standard.

Command parser 402 may direct macroblocks to appropriate residual path or prediction path hardware decoding blocks. In one embodiment, the frames may be processed simultaneously in the residual path and the prediction path in pipeline stage 1. The frames are synchronized in pipeline stage 2.

The decoded H.264 video stream is in pipeline stage 2. In one embodiment, in-loop post-processing operations are performed on them before storing the decoded H.264 video stream in memory 330. In another embodiment, out-of-loop post-processing operations are performed on the H.264 video stream decoded in out-of-loop filter 442. It should be understood that the out-of-loop filter 442 can be implemented as any deblocking filter and / or deringing filter.

WMV9 / VC- 1 : WMV9 / VC-1 decoding does not require intra prediction mode engine 404 and intra prediction engine 414, which are deactivated. Accordingly, the command parser 402 may include the MV prediction engine 406, the coefficient engine 408, the AC / DC prediction engine 410, the rotation engine 415, the motion compensation engine 416, the 4 × 4 inverse transform engine 418, Activate the 8x8 IDCT engine 420, intra prediction buffer 432, prediction sample 434, and residual block 438. Command parser 402 routes data from video stream 401 through active hardware decoding blocks for decoding a WMV9 / VC-1 encoded video stream. It should be understood that the operations and sequence of operations performed by the hardware decoding blocks are required by the WMV9 / VC-1 standard.

The command parser 402 can direct the macroblocks to the appropriate residual path or prediction path hardware decoding blocks. In one embodiment, the frames may be processed simultaneously in the residual path and the prediction path in pipeline stage 1. The frames are synchronized in pipeline stage 2.

The decoded WMV9 / VC-1 video stream is in pipeline stage 2. In one embodiment, in-loop post-processing operations are performed on them before storing the decoded WMV9 / VC-1 video stream in memory 330. In one embodiment, in-loop deblocking filter 440 is used to implement the overlap smoothing filter. In another embodiment, post-processing operations are performed on the WMV9 / VC-1 video stream decoded in out-of-loop filter 442. It should be understood that the out-of-loop filter 442 can be implemented as any deblocking filter and / or deringing filter.

Post Processing Actions

Pipeline stage 3 of decoder device 400 includes three hardware post-processing blocks: an in-loop deblocking filter 440, an out-of-loop filter 442, and a rotation engine 450. In-loop deblocking filter 440 is used in H.264 and WMV9 / VC-1 modes. In one embodiment, in WMV9 / VC-1 mode, in-loop deblocking filter 440 is used to implement the overlap smoothing filter.

Out-of-loop filter 442 may be used for any video stream to improve the quality of the decoded image. In one embodiment, the out-of-loop filter 442 is executed concurrently with the rest of the decoder device 400. The out-of-loop filter 442 should be triggered after the frame is decoded into memory 460.

It should be understood that any deblocking and / or deringing filter may be used as the out-of-loop filter 442. For example, the International Organization for Standardization (ISO), an organization for overseeing many video standards that may be implemented in device 150, often includes the deblocking filters proposed in the standardization publication. For example, the out-of-loop filter 442 may include a deblocking filter described in ISO / IEC 14496-2: 2001, section F.3.1 of the ISO publication.

The decoded image may also be rotated before it is written to the memory 460 in pipeline stage 3 in the rotation engine 450. The rotation engine 450 is configured to provide on-the-fly macroblock rotation in which individual macroblocks are rotated based on the indicated degree of rotation and placed in a new position of the frame. See FIGS. 10A, 10B and 11 below for a detailed description of the operation of the rotary engine 450.

Video using hardware multi-standard video decoder device Stream  How to decode

5 shows a flowchart of a video stream decoding method 500 implemented using a hardware multi-standard video decoder device in accordance with an embodiment of the present invention. Although certain steps are disclosed in method 500, these steps are illustrative. That is, embodiments of the present invention are suitable for carrying out various other steps or variations of the steps described in FIG. In one embodiment, the method 500 is performed by the decoder device 150 of FIG.

In step 510 of process 500, the video stream is accessed. In step 520, the video standard used to encode the video stream is identified. The hardware multi-standard video decoder device is configured to decode a video stream in accordance with a plurality of video standards.

In step 530, a subset of the hardware decoding blocks of the plurality of hardware decoding blocks of the hardware multi-standard video decoder device used to decode the video stream is determined. Different subsets of the plurality of hardware decoding blocks can decode video streams encoded using different video encoding standards. In one embodiment, as shown in step 540, the subset of hardware decoding blocks is activated, so that hardware decoding blocks not associated with decoding of the video stream are deactivated.

At step 550, the video stream is decoded using a subset of hardware decoding blocks. In one embodiment, as shown in step 560, the hardware decoding blocks in one stage of the multi-stage macroblock level pipeline are deactivated if no data in the video stream is received at that stage. It should be understood that steps 540 and 560 provide additional power savings and are optional.

In step 570, following decoding of the video stream, the memory unit is accessed. In one embodiment, the decoded video stream is stored in memory for display. In one embodiment, as shown in step 580, a post processing operation is performed on the decoded video stream. It should be understood that the post processing operation may be performed before or after step 570 is performed. In one embodiment, the decoded video stream is rotated. In another embodiment, an in-loop deblocking filter is applied to the decoded video stream. Rotation and in-loop deblocking are performed before the memory unit is accessed. In one embodiment, out of loop deblocking and deringing filters are applied to the decoded video stream after the memory unit is accessed.

Multiple encoded using different video standards Of stream  Decoding Using Hardware Multi-Standard Video Decoder Devices

Embodiments of the hardware multi-standard video decoder device of the present invention can also decode multiple video streams simultaneously. Portions of video streams such as macroblocks or frames are interleaved. The decoder device sequentially accesses the interleaved portions. Thus, the decoder device performs decoding operations on the interleaved portions. For example, a decoding operation may be performed on macroblocks of two video streams. The video streams are interleaved so that macroblocks of the video streams alternate. Every clock cycle, a decoding operation may be performed on alternate video streams.

6 is a diagram illustrating internal components of a hardware multi-stream multi-standard video decoder device 600 according to an embodiment of the present invention. As shown in FIG. 6, the decoder device 600 includes a video stream interleaver 605, a command parser 305, a plurality of hardware decoding blocks 310-318, a hardware post-processing block 320, and a memory 330. Register set 610 and register set 620. Decoder device 600 can decode a number of video encoding standards and operates very similarly to decoder device 150 of FIG. Decoder device 600 differs from decoder device 150 in that register sets 610, 620 allow decoder device 600 to decode multiple video streams simultaneously.

Video stream interleaver 605 may access multiple video streams to interleave portions of the video streams. As shown, video stream interleaver 605 accesses video streams 601, 602. However, it should be understood that the video stream interleaver 605 may receive any number of video streams and is not limited to the embodiment shown in FIG. 6. In one embodiment, video streams 601, 602 are received from a microcode engine (eg, microcode engine 260 of FIG. 2B).

7A and 7B are diagrams illustrating exemplary interleaved portions of multiple video streams in accordance with embodiments of the present invention. Referring to FIG. 7A, two interleaved video streams are shown, one stream being a still image video stream (eg, JPEG), and the other stream being a digital movie stream (eg, MPEG-4). )to be. As shown, if the video streams contain only one digital movie stream, the video streams may be interleaved at the macroblock level. Specifically, still image macroblocks 704 and 708 are interleaved with digital movie macroblocks 702 and 706, so that the macroblocks from each video stream alternate in the interleaved stream 700. When the video streams are interleaved at the macroblock level, the software driver of the decoder device 600 buffers the macroblock data in system memory to manage the decoding of the interleaved video streams.

7B, two interleaved video streams are shown, both streams being digital movie streams. As shown, when the video streams include multiple digital movie streams, the video streams are interleaved at the frame level. Specifically, the first digital movie frames 752 and 756 are interleaved with the second digital movie frames 754 and 758, so that frames from each video stream alternate in the interleaved stream 750. When the video streams are interleaved at the frame level, the software driver of the decoder device 600 buffers the frame data in system memory to manage decoding of the interleaved video streams.

Referring to FIG. 6, the command parser 305, the hardware decoding blocks 310-318, the hardware post-processing block 320, and the memory 330 operate as shown in FIG. 3. The resulting data and other decoder parameters are passed through the command parser 305 to the decoder device. Data from the command parser 305 may be routed to the residual path (hardware decoding blocks 312, 316, 318) or the prediction path (hardware decoding blocks 310, 314). The remaining path handles error or differential data, while the prediction path will prepare / fetch data of the previous frame or previous macroblock.

In order to manage the decoding of interleaved video streams, two register sets 610 and 620 are maintained in pipeline stage 1. In one embodiment, register sets 610 and 620 store memory surface pointers 612 and 622 and frame level parameters 614 and 624 respectively. Each of the register sets is used to store parameters associated with one of the video streams. For example, register set 610 is used to store parameters associated with video stream 601 and register set 620 is used to store parameters associated with video stream 602. If a portion of one video stream is processed in pipeline stage 1, the appropriate parameters are passed to downstream pipeline stages 2 and 3 in the form of packets with residual or predicted data. The decoded data will be routed to the appropriate area in the memory based on whether the macroblock is a still image or a digital movie type. It should be understood that the decoder device 600 may be configured to decode any number of video streams by adding the appropriate number of register sets such that each stream to be decoded has an associated register set.

8 shows a flowchart of a method 800 for decoding multiple video streams, implemented using a hardware multi-stream multi-standard video decoder device, in accordance with an embodiment of the present invention. Although certain steps are disclosed in method 800, these steps are illustrative. That is, embodiments of the present invention are suitable for carrying out various other steps or variations of the steps described in FIG. In one embodiment, the method 800 is performed by the decoder device 600 of FIG.

In step 810 of process 800, a plurality of video streams are accessed. In step 820, video standards used to encode video streams are identified. The hardware multi-stream multi-standard video decoder device is configured to decode video streams according to a plurality of video standards. In step 830, portions of the video streams are interleaved. In one embodiment, if the video streams contain only one digital movie stream, macroblocks of the video streams are interleaved. In another embodiment, if the video streams comprise multiple digital movie streams, the frames of the video streams are interleaved. It should be understood that steps 820 and 830 may be performed in any order.

In step 840, subsets of hardware decoding blocks of the plurality of hardware decoding blocks of the hardware multi-standard video decoder apparatus used to decode the plurality of video streams are determined. Different subsets of the plurality of hardware decoding blocks may be operable to decode video streams encoded using different video encoding standards. In one embodiment, as shown in step 850, the subsets of hardware decoding blocks are activated, so that a hardware decoding block that is not associated with decoding of video streams is not activated.

In step 860, the video streams are decoded using subsets of hardware decoding blocks. In step 870, following decoding of the video streams, the memory unit is accessed. In one embodiment, the decoded video stream is stored in memory for display. In one embodiment, a post-processing operation is performed on at least one decoded video stream, as shown at step 880. It should be understood that the post processing operation may be performed before or after step 870 is performed. In one embodiment, the decoded video stream is rotated. In another embodiment, an in-loop deblocking filter is applied to the decoded video stream. Rotation and in-loop deblocking are performed before the memory unit is accessed. In one embodiment, out of loop deblocking and deringing filters are applied to the decoded video stream after the memory unit is accessed.

video Of stream  Processing disordered macroblocks

Referring to FIG. 2B, in one embodiment, the present invention provides for buffering and rearranging macroblocks in the microcode engine 260. The present invention provides for arranging macroblocks in raster scan order to support in-loop deblocking for video standards that support sending and receiving macroblocks in non-raster scan order. Microcode engine 260 is configured to receive compressed data representing macroblocks of a frame of the video stream. In one embodiment, at least one macroblock is received disorderly. The microcode engine 260 is configured to buffer the compressed data and to arrange the macroblocks of the frame in raster scan order.

9 shows a flowchart of a method 900 for processing disordered macroblocks of a video stream in accordance with an embodiment of the present invention. Although certain steps are disclosed in method 900, these steps are illustrative. That is, embodiments of the present invention are suitable for carrying out various other steps or variations of the steps described in FIG. In one embodiment, the method 900 is performed by the microcode engine 260 of FIG. 2B.

In step 910 of process 900, compressed data is received that represents macroblocks of a frame of a video stream, wherein at least one macroblock is received disorderly. At step 920, the compressed data is buffered. In one embodiment, the compressed data is buffered in a buffer of the microcode engine 260. In step 930, the video stream is parsed and the VLD is performed on the video stream. It should be understood that step 930 is optional and that video stream parsing and VLD may be performed by a hardware decoder device. It should also be understood that other or additional preprocessing operations may be performed on the video stream at step 930.

In step 935, it is determined whether the video stream requires in-loop deblocking. In one embodiment, the compressed data includes an indication as to whether intra-loop blocking should be performed on the video stream. If in-loop deblocking is needed, the macroblocks of the frame are arranged in raster scan order as shown in step 940. In one embodiment, all macroblocks of a frame are buffered before the macroblocks are arranged in raster scan order. The method 900 then proceeds to step 950. Alternatively, if no in-loop deblocking is needed, the method 900 proceeds directly to step 950.

In step 950, the video stream is decoded. In one embodiment, macroblocks are decoded in raster scan order. In one embodiment, the video stream is decoded by a hardware multiple standard video decoder device (eg, decoder device 150 of FIG. 3 or decoder device 400 of FIG. 4). In one embodiment, the video stream is decoded according to the method 500 of FIG. 5.

In step 960, deblocking in a macroblock level loop is performed on the decoded macroblock. In step 970, the memory unit is accessed. In one embodiment, the deblocked and decoded video stream is stored in memory for display.

In step 980, out-of-frame post-processing is performed on the decoded frame. In one embodiment, the out-of-loop postprocessing includes deblocking and deringing operations. It should be understood that step 980 is optional. The method 900 then returns to step 970 where the memory unit is accessed. In one embodiment, the deblocked, deringed, and decoded video stream is stored in memory for display.

By buffering macroblocks while the macroblocks are still compressed data, the microcode engine 260 only needs to buffer a maximum of one frame of run length encoded compressed data much less than the decoded video data. In addition, buffering the compressed macroblocks saves power. Video streams received wirelessly also generate many errors. Partitioning of bit stream parsing to microcode engine 260 also has the advantage of improving error recovery.

video Of stream  On-the-Fly Rotation of Macroblocks

Embodiments of the present invention provide a rotation engine for rotating a bit stream in an "on the fly" manner before the video stream is written to memory. Embodiments of the present invention may rotate a video stream by rotating them when macroblocks of the video stream are received, and rearrange macroblocks within a frame based on the rotation. Embodiments of the present invention can rotate video streams without having to deliver decoded frames secondary by operating on the macroblocks before writing the decoded macroblocks to memory.

In one embodiment, the present invention provides a rotation engine configured to rotate a macroblock of a frame of a video stream according to the degree of rotation and to relocate the macroblock to a new location within the frame based on the degree of rotation. In one embodiment, the video decoder device further includes a memory for storing the macroblock for display. In one embodiment, the rotation engine is configured to rotate the macroblock and rearrange the macroblock within the frame before accessing the memory.

10A and 10B show diagrams of exemplary rotation of macroblocks of frames in accordance with one embodiment of the present invention. 10A and 10B describe the operation of the rotation engine 450 of FIG. 4, the embodiments described may be implemented in any type of video decoder device, using the hardware multi-standard video decoder device 400 of FIG. 4. It should be understood that it is not limited to. For example, the rotation engine may be included within a single standard hardware decoder or software decoder.

Referring to FIG. 10A, diagram 1000 illustrates the rotation of frame 1010 using the rotation engine 450 of FIG. 4. Frame 1010 includes a number of macroblocks. Macroblock 1012 is shown as a first macroblock that is received at rotation engine 450. In one embodiment, the macroblocks are received in raster scan order, where macroblock 1012 is the upper left macroblock and is therefore the first macroblock received.

The rotation engine 450 is configured to rotate the macroblock 1012 and to relocate the macroblock 1012 to a new location within the frame 1010. Rotation and relocation are based on the degree of rotation associated with the video stream. Rotation degree indicates how much the video stream should be rotated. For example, the degree of rotation can be 90 degrees clockwise, 90 degrees, 180 degrees, or any other degree of rotation in the counterclockwise direction.

Diagram 1000 illustrates the operation of rotary engine 450 using 90 degrees of rotation in a clockwise direction. The macroblock 1012 is rotated 90 degrees clockwise. Rotation engine 450 also rearranges macroblock 1012 so that rotated macroblock 1012, shown as macroblock 1022 in rotated frame 1020, is used for all other macroblocks in frame 1020. Be in the same location.

Embodiments of the present invention also provide for rotating frames at macroblock level where macroblocks are received randomly. Referring to FIG. 10B, diagram 1050 illustrates the rotation of frame 1060 using the rotation engine 450 of FIG. 4. Macroblock 1062 is shown as a first macroblock received at rotation engine 450. In this embodiment, the macroblock 1062 is the first macroblock to be received, but since it is not the upper left macroblock, the macroblocks are not received in raster scan order.

The rotation engine 450 is configured to rotate the macroblock 1062 and to relocate the macroblock 1062 to a new location within the frame 1060. Diagram 1050 illustrates the operation of rotary engine 450 using 90 degrees of rotation in the clockwise direction. Macroblock 1062 is rotated 90 degrees clockwise. Rotation engine 450 also rearranges macroblock 1062 so that rotated macroblock 1062, shown as macroblock 1072 in rotated frame 1070, for all other macroblocks in frame 1070. Be in the same location.

11 shows a flowchart of a method 1100 for rotating macroblocks of a frame according to one embodiment of the present invention. Although certain steps are disclosed in method 1100, these steps are illustrative. That is, embodiments of the present invention are suitable for carrying out various other steps or variations of the steps described in FIG. In one embodiment, the method 1100 is performed by the rotation engine 450 of FIG. 4.

In step 1110, the video stream is decoded. In one embodiment, the video stream is decoded by a hardware multiple standard video decoder device (eg, decoder device 150 of FIG. 3 or decoder device 400 of FIG. 4). In one embodiment, the video stream is decoded according to the method 500 of FIG. 5. It should be understood that step 1110 is optional and that the video stream can already be decoded before processing.

In step 1120, the rotation rate for the video stream is accessed. In one embodiment, the degree of rotation is one of 90 degrees clockwise, 90 degrees counterclockwise, and 180 degrees. However, it should be understood that any rotation can be used. In step 1130, the macroblock of the video stream is accessed.

In step 1140, the macroblock is rotated according to the degree of rotation. At step 1150, the macroblock is relocated to a new location within the frame, which new location is based on the degree of rotation. It should be understood that the macroblocks are rearranged to be in the same position for all other macroblocks in the frame once rotated. In one embodiment, the rotation of the macroblocks and the relocation of the macroblocks are performed before accessing the memory.

In step 1160, the macroblock is stored in memory for display. In one embodiment, as shown in step 1170, a deblocking operation is performed on the decoded macroblock. It should be understood that step 1170 is optional. Moreover, it should be understood that step 1170 may include performing in-loop deblocking or out-of-loop nonblocking and deringing.

As such, embodiments of the present invention provide a new hardware multi-standard video decoder device architecture that supports hardware-based decoding of video streams in accordance with multiple video standards. Embodiments of the present invention may provide real time decoding for each of a plurality of video encoding standards. Embodiments of the present invention provide post processing operations on decoded video streams. One embodiment of the present invention provides video decoding for video streams using any of the JPEG, MPEG-4, H.263, H.263 +, H.264, and WMV9 / VC-1 video standards. Provided is a hardware decoder device.

Embodiments of the present invention provide a hardware multi-stream multi-standard video decoder device for providing simultaneous video decoding functionality for a plurality of different video encoding standards. Embodiments of the present invention can decode multiple interleaved video streams simultaneously.

Embodiments of the present invention provide a video decoder architecture for providing in-loop deblocking of a video stream without the need for additional memory to arrange macroblocks in raster scan order. Embodiments of the present invention may arrange macroblocks of a video stream in a microcode engine. Embodiments of the present invention may provide decoding and out-of-loop deblocking and / or deringing for a video stream encoded using one of a plurality of supported video standards.

Embodiments of the present invention provide a rotation engine for rotating a video stream in an "on the fly" manner before the video stream is written to memory. Embodiments of the present invention may rotate a video stream by rotating them when macroblocks of the video stream are received, and relocate the macroblocks within a frame based on the rotation. Embodiments of the present invention can rotate video streams without the need for secondary delivery of decoded frames by operating on macroblocks before writing the decoded macroblocks to memory.

The foregoing descriptions of specific embodiments of the present invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and many modifications and variations are possible in light of the above teaching. The embodiments are chosen to best illustrate the principles of the present invention and their practical applications and to enable those skilled in the art to best utilize it with various modifications as are suited to the particular use contemplated by the present invention and various embodiments. Was explained. It is intended that the scope of the invention be defined by the claims appended hereto and their equivalents.

Claims (10)

A decoding method implemented using a hardware multi-standard video decoder device, Accessing a first video stream, wherein the first video stream is one of a plurality of video streams; Identifying a first video standard used to encode the first video stream; Determining a first subset of hardware decoding blocks of a plurality of hardware decoding blocks of the hardware multi-standard video decoder device used to decode the first video stream, wherein different subsets of the plurality of hardware decoding blocks are different. Operable to decode video streams encoded using video encoding standards; And Decoding the first video stream using the first subset of hardware decoding blocks. Decoding method comprising a. The method of claim 1, Activating a subset of the hardware decoding blocks such that a hardware decoding block not associated with decoding the first video stream is not activated. The method of claim 1, Wherein the plurality of hardware decoding blocks are implemented within a multi-stage macroblock level pipeline. The method of claim 3, If no data of the first video stream is received at the stage, deactivating hardware decoding blocks within one of the multi-stage macroblock level pipelines. The method of claim 1, Accessing a memory unit following said decoding of said first video stream. The method of claim 1, Accessing the plurality of video streams; Identifying a second video standard used for video streams of the plurality of video streams; Interleaving portions of the plurality of video streams; Determining a plurality of subsets of hardware decoding blocks of the plurality of hardware decoding blocks; And Decoding the plurality of video streams using the plurality of subsets of the hardware decoding blocks. Decoding method further comprising. The method of claim 6, Wherein the plurality of video streams comprises at least one digital still image stream and a digital movie stream. The method of claim 7, wherein And said portions of said plurality of video streams are frames of said digital still image stream and said digital movie stream. The method of claim 6, And the plurality of video streams comprises a plurality of digital movie streams. The method of claim 9, And said portions of said plurality of video streams are macroblocks of said plurality of digital movie streams.

Images