JP2009518960A - Hardware multi-standard video decoder device - Google Patents

Abstract

A hardware multi-standard video decoder device. The instruction parser accesses the video stream and identifies the video coding standard that was used to encode the video stream. Multiple hardware decoding blocks perform operations related to decoding a video stream, and different subsets of multiple hardware decoding blocks decode video streams encoded using different video encoding standards. It is for making it.
[Selection] Figure 1

Description

Field

  [001] The technical field described herein relates to video decoding. More specifically, the present specification relates to a hardware multi-standard video decoder device.

background

  [002] In general, a digital video stream is encoded using one of many different encoding standards. For example, a digital video stream can be compressed for conversion to a data format that requires fewer bits. This compression can be reversible so that the original video stream can be reproduced at the time of decoding, or an exact duplicate of the original video stream cannot be reproduced, but the compressed data Can be irreversible so that the decoding of is more efficient.

  [003] Currently, there are many video coding standards, and new standards frequently appear. Examples of current video coding standards include JPEG (Joint Photographic Experts Group), MPEG (Moving Pictures Experts Group), MPEG-2, MPEG-3, MPEG-4, and H.264. 263, H.M. 263+, H.H. H.264, and proprietary standards such as Real Video and Windows Media. In order to fully benefit from digital video, the user needs access to a decoder that can decode all common coding standards.

  [004] Many important applications for streaming video are related to real-time communications. For example, a videophone requires real-time video decoding so that the videophone can be synchronized with the corresponding audio signal. Accordingly, it is still desirable to provide real-time video decoding to users to provide applications related to real-time communications. Furthermore, a situation arises where the user requests decoding of multiple video streams. For example, a user who is currently talking on a videophone receives an attached image from a person who is speaking. In this example, real-time decoding of the videophone stream must be maintained as the images required for the conversation are decoded.

  [005] Currently, video decoding is a software-based program that can decode a video stream according to two available methods: a single standard capable hardware video decoder and one or more video standards. It is executed using one of the possible cores. A single-standard hardware video decoder can provide real-time decoding capabilities. However, in order to decode a video stream encoded using a specific encoding standard, the user must have a hardware video decoder for that specific standard. Since there are a number of widely used video coding standards, users have much to access digital video encoded using different video coding standards, which is costly to the user. Requires a different single-standard hardware video decoder. Furthermore, typical computer systems cannot add many single-standard hardware video decoders, and further limit the number of video streams that a user can access.

  [006] Current software-based programmable core video decoders can be utilized to provide encoding using one or more video encoding standards. A programmable core video decoder may include hardware acceleration to speed up the decoding function. However, the programmable core performs all decryption. In general, a programmable core video decoder has a large processing overhead, is less efficient, and consumes much more power than a single-standard hardware video decoder. Furthermore, programmable core video decoders cannot always provide real-time video decoding because the decoding is affected by the processing requirements of the entire computer system.

  [007] Thus, currently available digital video decoders cannot provide real-time video decoding for a number of widely used video coding standards. In addition, currently available digital video decoders cannot simultaneously provide video decoding for multiple streams encoded using a number of widely used video encoding standards. Furthermore, currently available digital video decoders cannot simultaneously decode multiple video streams where at least one video stream requires real-time decoding. Therefore, what is needed is a new digital video decoder that overcomes the limitations of the prior art. New digital video decoders should provide real-time video decoding capabilities for multiple different video standards. A new digital video decoder should also provide video decoding capabilities for multiple video streams encoded using multiple different video standards simultaneously.

Overview

  [008] Embodiments of the present invention provide a hardware multi-standard video decoder device for providing video decoding functionality for a plurality of different video encoding standards. Embodiments of the present invention can provide real-time decoding for each of a plurality of video coding standards.

  [009] In one embodiment, the present invention provides a hardware multi-standard video decoder device. The instruction parser of the hardware multi-standard video decoder device is operable to access the video stream and is operable to identify the video encoding standard used to encode the video stream. The hardware multi-standard video decoder apparatus also includes a plurality of hardware decoding blocks for performing operations related to decoding of the video stream, wherein different subsets of the plurality of hardware decoding blocks are different video encodings. For decoding a video stream encoded using a standard. In one embodiment, a hardware multi-standard video decoder device is implemented in an integrated circuit coupled to a printed circuit board, the printed circuit board being a connector for releasably coupling the printed circuit board to a computer system. Combined with

  [010] In one embodiment, the instruction parser uses a first identified video code used to encode the video stream such that hardware decoding blocks not involved in decoding the video stream are not activated. Operable to operate a first subset of a plurality of hardware decoding blocks used to decode the encoding standard. In one embodiment, the instruction parser determines the second identified video encoding standard used to encode the video stream such that hardware decoding blocks not involved in decoding the video stream are not activated. Operable to operate a second subset of the plurality of hardware decoding blocks used to decode.

  [011] In one embodiment, the plurality of hardware decoding blocks are implemented in a multi-stage macroblock level pipeline. In one embodiment, the instruction parser is operable to stop the hardware decoding block in a stage when no data in the video stream is received in one stage of the multi-stage macroblock level pipeline. In one embodiment, the hardware multi-standard video decoder device accesses the memory unit after fully decoding the video stream.

  [012] In one embodiment, the hardware multi-standard video decoder device further includes a hardware post-processing block for performing post-processing operations on the decoded video stream. In one embodiment, the instruction parser is a video stream obtained by decoding the video stream received at the instruction parser so that the hardware post-processing block performs a post-processing operation on the decoded video stream. In some cases, it is operable to stop a plurality of hardware decoding blocks. In one embodiment, the hardware post-processing block includes a filter.

  [013] In another embodiment, the present invention provides a method for decoding a video stream, and the method is performed using a hardware multi-standard video decoder device. A video stream is accessed. The video standard used to encode the video stream is identified. A subset of the hardware decoding blocks of the plurality of hardware decoding blocks of the hardware multi-standard video decoder device used to decode the video stream is determined, and the plurality of hardware decoding blocks are different. The subset is operable to decode video streams encoded using different video encoding standards. The video stream is decoded using a subset of hardware decoding blocks.

  [014] In one embodiment, the plurality of registers includes a memory surface pointer register and a frame level parameter register. In one embodiment, the hardware multi-stream multi-standard video decoder device further includes a hardware post-processing block for performing post-processing operations on the at least one decoded video stream.

  [015] In one embodiment, the plurality of video streams includes at least one digital still image stream and a digital movie stream. In this embodiment, some of the plurality of video streams are macroblocks of a digital still image stream and a digital movie stream. In another embodiment, the plurality of video streams includes a plurality of digital movie streams. In this embodiment, some of the multiple video streams are macroblocks of multiple digital movie streams.

  [016] In another embodiment, the present invention provides a method for decoding a plurality of video streams, the method being performed using a hardware multi-stream multi-standard video decoder device. Multiple video streams are accessed. The video standard used to encode the video stream is identified. A subset of hardware decoding blocks of a plurality of hardware decoding blocks of a hardware multi-stream multi-standard video decoder device used to decode a video stream is determined, and the plurality of hardware decoding blocks The different subsets are operable to decode video streams encoded using different video encoding standards. Multiple video streams are decoded using a subset of the hardware decoding blocks. In one embodiment, a plurality of subsets of hardware decoding blocks are activated such that hardware decoding blocks not involved in decoding the plurality of video streams are not activated.

  In general, this specification discloses a hardware multi-standard video decoder device. The instruction parser accesses the video stream and identifies the video coding standard that was used to encode the video stream. Multiple hardware decoding blocks perform operations related to decoding a video stream, and different subsets of multiple hardware decoding blocks decode video streams encoded using different video encoding standards. It is for making it.

  [017] The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings in which like reference numerals refer to like elements.

Detailed description

  [030] Reference will now be made in detail to the preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings. While the invention will be described in conjunction with the preferred embodiments, it will be understood that these embodiments are not intended to limit the invention to these embodiments. Rather, the present invention is intended to embrace alterations, modifications, and equivalents that may be included within the spirit and scope of the present invention as defined by the appended claims. Furthermore, in the following detailed description of embodiments of the present invention, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be understood by one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the embodiments of the present invention.

Notation and terminology
[031] Some of the detailed description that follows is presented in terms of procedures, steps, logical blocks, processing, and other symbolic representations of operations on data bits in computer memory. These descriptions and representations are the means used by those familiar with data processing technology to most effectively convey the content of their performance to others familiar with the technology. Procedures, computer-executed steps, logic blocks, processes, etc. are here and generally considered to be a self-consistent sequence of steps or instructions that produce the desired result. A step is a step that requires physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. It has proven convenient to refer to these signals as bits, values, elements, symbols, characters, words, numbers, etc. mainly for common use reasons.

  [032] It should be noted, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. As will be apparent from the discussion below, unless otherwise specified, throughout the present invention “identify” or “access” or “execute” or “decrypt” or “activate” "Or" "stop" or "determine" or "process" or "receive" or "buffer" or "align" or "forward" or "parse" , Or “interleave”, “rotate”, “relocate”, or “store” terms are considered as physical (electronic) quantities in the computer system registers and memory. Manipulate the data represented, and other data represented as physical quantities in the computer system's memory or registers, or other such information storage, transmission, or display devices as well. Hardware multi-standard video decoder device (for example, hardware multi-standard video decoder device 150 in FIG. 3), hardware multi-stream multi-standard video decoder device (for example, hardware multi-stream video in FIG. 6). Standard-compliant video decoder apparatus 600), microcode engine (eg, microcode engine 260 of FIG. 2B), rotation engine (eg, rotation engine 450 of FIG. 4), or similar electronic computing device operation and process. To be understood.

Computer system platform
[033] FIG. 1 illustrates an exemplary computer system 100 in which embodiments of the present invention may be implemented. In general, computer system 100 stores a bus 110 for communicating information, a processing device 101 coupled to bus 110 for processing information and instructions, and information and instructions for processing device 101. A volatile memory 102, also referred to as a random access memory (RAM), coupled to the bus 110, and a bus 110 for storing static information and instructions for the processing unit 101. The book includes a non-volatile memory 103 also called a read-only memory (ROM).

  [034] In one embodiment, the computer system 100 includes an optional data storage device 104, such as a magnetic or optical disk and disk drive, coupled to the bus 110 for storing information and instructions. In one embodiment, the computer system 100 allows the processing device 101 to select an optional user output device, such as a display device 105 coupled to the bus 110, and information and instructions for displaying information to a computer user. To communicate the selection of optional user input devices, such as alphanumeric input device 106 including alphanumeric and function keys coupled to bus 110, and / or user input information and command selection to processing device 101. Optional user input devices such as cursor control device 107 coupled to bus 110. In addition, an optional input / output (I / O) device 108 is used to couple the computer system 100 to, for example, a network.

  [035] In one embodiment, the computer system 100 is also referred to herein as a decoder device 150 for decoding a video stream that has been encoded using one of a plurality of video encoding standards. A hardware multi-standard video decoder device 150 is also included. The decoder device 150 includes a plurality of hardware decoding blocks for performing decoding operations required by a plurality of video encoding standards. It should be understood that the decoder device 150 can be configured to decode video according to any combination of video encoding standards, including digital still images and digital movies. For example, the decoder device 150 may be JPEG, MPEG-4, H.264, or the like. 263, H.M. 263+, H.H. H.264, and can be configured to decode video encoded using any of the Windows Media (WMV9 / VC-1) formats.

  [036] The decoder device 150 is a separate graphics card (eg, on a motherboard) designed to couple to the computer system 100 via a separate component, connector (eg, AGP slot, PCI-Express slot, etc.). It should be understood that it can be implemented as a separate integrated circuit die (directly attached) or as an integrated decoder device included in an integrated circuit die of a chipset component of a computer system. Further, a local graphics memory can be included in the decoder device 150 for data storage.

  [037] FIG. 2A shows a diagram of an exemplary hardware video decoder card 200 mounted on a printed circuit board, according to one embodiment of the invention. The hardware video decoder card 200 includes a printed circuit board (PCB) 210, an integrated circuit (IC) chip 220, data wiring 225, and a connector 230. The IC chip 220 includes a hardware multi-standard video decoder device 150. Connector 230 is configured to couple to a computer system (eg, computer system 100 of FIG. 1) via a computer system connector (eg, an AGP slot, a PCI-Express slot, etc.). The data wiring 225 is for transmitting data (for example, a bit stream) between the computer system and the IC chip 220.

  [038] FIG. 2B shows a diagram of an exemplary architecture 250 that includes a hardware multi-standard video decoder device 150, according to one embodiment of the invention. The architecture 250 includes a microcode engine 260, a hardware multi-standard video decoder device 150, and a memory 270. In one embodiment, the microcode engine 260 controls the operation of the hardware multi-standard video decoder device 150. The microcode engine 260 includes the operations that the hardware multi-standard video decoder device 150 must perform and acts as a translation layer between machine instructions and the hardware device decoder 150. In one embodiment, parsing and variable length decoding (VLD) of the bitstream is performed in the microcode engine 260. Memory 270 is used by decoder device 150 to perform decoding and post-processing operations on the received video stream. One embodiment of operational memory 270 is shown in memory 330 of FIG.

  [039] Referring to FIG. 2B, in one embodiment, the present invention performs macroblock reordering in the microcode engine 260. As described below, decoder device 150 may perform various operations such as in-loop deblocking (eg, in in-loop deblocking filter 440) and out-of-loop deblocking and / or deringing (eg, in out-of-loop filter 442). Support post-processing operations. In various embodiments, in-loop deblocking requires macroblocks to be received in raster scan order at an in-loop deblocking filter. However, H. Certain video standards, such as H.264, support transmission and reception of macroblocks in non-raster scan order. Thus, the present invention provides for aligning macroblocks in raster scan order to support in-loop deblocking for video standards that support transmission and reception of macroblocks in non-raster scan order.

  [040] In one embodiment, pre-processing operations are performed in the microcode engine 260. In one embodiment, parsing and variable length decoding (VLD) of the bitstream is performed in the microcode engine 260. The microcode engine 260 is configured to align the macroblocks before sending them to the hardware decoder device 150. The microcode engine 260 buffers one frame of compressed data. In one embodiment, the microcode engine 260 buffers one frame of run-length encoded compressed data. In one embodiment, the microcode engine 260 parses the received bitstream and then performs VLD. If the microcode engine 260 detects an out-of-order macroblock, the microcode engine 260 buffers the data and waits for all macroblocks to be received. Next, the microcode engine 260 arranges the macroblocks in the raster scan order and transmits the macroblocks to the hardware decoder device 150.

  [041] By buffering a macroblock while the macroblock is still compressed data, the microcode engine 260 buffers up to one frame of run-length encoded compressed data that is much less than the decoded video data. You only need to ring. Furthermore, compressed macroblock buffering also saves power. Also, video streams received over the air are likely to contain many errors. Also, the parsing of the bitstream for the microcode engine 260 has the advantage of improving error recovery.

Hardware multi-standard video decoder device architecture
[042] FIG. 3 shows a diagram illustrating the internal components of a hardware multi-standard video decoder device 150 according to one embodiment of the invention. As shown in FIG. 3, the decoder device 150 includes an instruction parser 305, a plurality of hardware decoding blocks 310 to 318, a hardware post-processing block 320, and a memory 330. Decoder device 150 is operable to decode a plurality of video encoding standards.

  [043] The instruction parser 305 is for accessing a video stream 302 (eg, a bitstream). Video stream 302 is a compressed video stream encoded according to one of a plurality of video encoding standards. It should be understood that the video stream 302 may include digital still image data (eg, encoded with JPEG) or digital movie data (eg, MPEG-4). In one embodiment, video stream 302 is received from a microcode engine (eg, microcode engine 260 of FIG. 2B). The instruction parser 305 identifies the video encoding standard that was used to encode the video stream 302. In one embodiment, parsing and variable length decoding (VLD) of the bitstream is performed before the instruction parser 305 accesses the video stream 302. Bitstream parsing and VLD can be performed by a host CPU (eg, processing unit 101 of FIG. 1) or a microcode engine (eg, microcode engine 260 of FIG. 2B). The instruction parser 305 also controls the movement of data through the decoder device 150 by controlling the clock cycle.

  [044] A plurality of hardware decoding blocks 310-318 are for performing operations associated with decoding the video stream. It should be understood that the hardware decoding blocks 310 to 318 represent the different decoding functions required to decode a video stream according to the video standard implemented in the video decoder 150. Video encoding standards such as MPEG-4 indicate that certain operations are performed to decode a video stream so that all MPEG-4 decoders can decode the MPEG-4 video stream. I need. It should be understood that the operations required to perform decoding according to various standards are well known to those skilled in the art.

  [045] In one embodiment, the hardware decoding block of decoder device 150 is configured to perform operations at a macroblock level (eg, an 8x8 pixel macroblock). However, it should be understood that the decoder device 150 may include hardware decoding blocks that perform operations at other dimensional levels, such as the frame level.

  [046] Different subsets of hardware decoding blocks 310 to 318 are for decoding video streams encoded using different video encoding standards. For example, the first exemplary video standard requires the use of hardware decoding blocks 312 and 316 in decoding the video stream. The second exemplary video standard requires the use of hardware decoding blocks 310, 312, 314, and 318 in decoding the video stream. Thus, in various embodiments of the present invention, only the hardware decoding blocks required to decode a video stream are used to decode a video stream encoded using a specified video standard. used.

  [047] In one embodiment, the instruction parser 305 determines the hardware decoding required to decode the video stream such that hardware decoding blocks not involved in decoding the received video stream are not activated. It is operable to actuate only the block. For example, the first identified video coding standard is decoded such that hardware decoding blocks not related to decoding of the video stream (eg, hardware decoding blocks 310, 314, and 318) are not activated. A first subset of hardware decoding blocks used to do this (eg, hardware decoding blocks 312 and 316) is activated. In another example, a second identified video coding standard is decoded such that a hardware decoding block not related to decoding of the video stream (eg, hardware decoding block 316) is not activated. A second subset of decoding blocks used for the purpose (eg, hardware decoding blocks 310, 312, 314, and 318) is activated. In one embodiment, instruction parser 305 is the only active component of decoder device 150. The hardware decoding block is activated as needed according to the specified video standard and data flow.

  [048] In one embodiment, the hardware decoding block of decoder device 150 is implemented in a multi-stage macroblock level pipeline. As shown in FIG. 3, the decoder device 150 includes a pipeline stage 1 that includes hardware decoding blocks 310 and 312 and a pipeline stage 2 that includes hardware decoding blocks 314, 316, and 318. It is implemented as a three stage macroblock level pipeline containing. In one embodiment, the instruction parser 305 directs the macroblock of the video stream 302 to the pipeline stage 1 hardware decoding block. In one embodiment, more than one macroblock may exist in pipeline stage 1, while pipeline stages 2 and 3 are limited to having only one macroblock. In one embodiment, hardware decoding blocks 312, 316, and 318 are in the residual data path, and hardware decoding blocks 310 and 314 are in the prediction data path. In one embodiment, the residual data path handles error or difference data, and the prediction path accesses data associated with the previous frame or macroblock.

  [049] In one embodiment, the instruction parser 305 may stop the hardware decoding block in a stage when the data of the video stream is not received in one stage of the multi-stage macroblock level pipeline. It is possible to operate. For example, in decoding video stream 302, all hardware decoding in pipeline stage 1 when the last data for video stream 302 exits pipeline stage 1 and no data is received in pipeline stage 1 The block is stopped. Thus, even if hardware decoding blocks are required for the video standard associated with video stream 302, further power savings are achieved by stopping all hardware decoding blocks in the pipeline stage. The

  [050] In one embodiment, the video stream 302 does not enter or leave the memory 330 until the video stream 302 is fully decoded. It should be understood that the memory 330 may be an external memory unit (eg, the volatile memory 102 or the non-volatile memory 103 of FIG. 1) or a built-in memory unit of the decoder device 150. By not accessing the memory 330 until the video stream 302 is fully decoded, the decoder device 150 uses less power.

  [051] In one embodiment, the decoder device 150 further includes a hardware post-processing block 320 for performing post-processing operations on the decoded video stream. In one embodiment, hardware post-processing block 320 includes a deblocking filter. It should be understood that the deblocking filter may be an in-loop deblocking filter or an out-of-loop deblocking and / or deringing filter. The in-loop deblocking filter performs a deblocking operation before accessing the memory 330. The out-of-loop deblocking and deringing filter performs deblocking and deringing operations on data accessed from the memory 330. However, it should be understood that the hardware post-processing block 320 can perform any type of post-processing operation. In addition, there can be any number of hardware post-processing blocks 320 to perform multiple post-processing operations.

  [052] In one embodiment, the instruction parser 305 is a video stream from which the video stream 302 has been decoded such that the hardware post-processing block 320 performs post-processing operations on the decoded video stream. In some cases, it is operable to stop all hardware decoding blocks. In other words, the decoder device 150 can be used only as a hardware post-processing device. When the decoded video stream is received at the decoder device 150, all hardware decoding blocks are stopped and post-processing operations on the decoded video stream.

  [053] FIG. 4 shows a block diagram illustrating the internal components of an exemplary hardware multi-standard video decoder device 400, also referred to as decoder device 400, according to one embodiment of the invention. The decoder device 400 is JPEG, MPEG-4, H.264, or the like. 263, H.M. 263+, H.H. It is configured to operate as any one of H.264 or WMV9 / VC-1 decoders. Therefore, the decoder device 400 is compatible with JPEG, MPEG-4, H.264. 263, H.M. 263+, H.H. H.264, or a hardware decoding block for performing all decoding operations necessary to decode a video stream encoded using any one of the WMV9 / VC-1 standards Including. However, it should be understood that the present invention flexibly supports other video standards, and that the present invention is not intended to be limited to the embodiment shown in FIG.

  [054] As shown in FIG. 4, the decoder device 400 includes an instruction parser 402, a plurality of hardware decoding blocks, a plurality of hardware post-processing blocks, and a memory 460. The instruction parser 402 is for accessing a video stream 401 (for example, a bit stream). It should be understood that the video stream 401 may include digital still image data (eg, encoded with JPEG) or digital movie data (eg, MPEG-4). In one embodiment, video stream 401 is received from a microcode engine (eg, microcode engine 260 of FIG. 2B). Video stream 401 is a compressed video stream that is encoded according to one of a plurality of video encoding standards. The instruction parser 402 identifies the video encoding standard that was used to encode the video stream 401. In one embodiment, parsing and variable length decoding (VLD) of the bitstream is performed before the instruction parser 402 accesses the video stream 401. Bitstream parsing and VLD can be performed by a host CPU (eg, processing unit 101 of FIG. 1) or a microcode engine. It should be noted that if the video stream 401 is encoded using a video standard that is different from the video standard that the decoder device 400 is configured to decode, the decoding operation is not performed. In one embodiment, the instruction parser 402 sends an indication to the computer system indicating that decoding cannot be performed on a video stream encoded using an unsupported standard.

  [055] Once the video standard used to encode the video stream 401 is identified, the instruction parser 402 directs the macroblocks of the video stream 401 to the appropriate hardware decoding block for the identified video standard. . In one embodiment, the instruction parser operates the appropriate hardware decoding block for the identified video standard such that hardware decoding blocks that are not required for the identified video standard are stopped. The instruction parser 402 also controls the movement of data through the decoder device 400 by controlling the clock cycle. In one embodiment, instruction parser 402 is the only active component of decoder device 400. The hardware decoding block is activated as needed according to the specified video standard and data flow.

  [056] The hardware decoding block of the decoder device 400 includes an intra prediction mode engine 404, a motion vector (MV) prediction engine 406, a coefficient (eg, run length (RD) or inverse quantization) engine 408, AC / DC ( For example, AC / DC prediction or inverse quantization) prediction engine 410, intra prediction engine 414, rotation engine 415, motion compensation engine 416, 4 × 4 inverse transform engine 418, 8 × 8 inverse discrete cosine transform (IDCT) engine 420, An IDCT format converter engine 422, an intra prediction buffer 432, a prediction sample 434, and a residual block 436 are included. The decoder device 400 further includes multiplexing devices 405, 409, 417, 419, 439 and an adder 435. The decoder device 400 also optionally includes hardware post-processing blocks, ie, an in-loop deblocking filter 440, an out-of-loop filter 442, and a rotation engine 450.

  [057] The decoder device 400 is implemented in a three-stage macroblock level pipeline having a residual path and a prediction path. In one embodiment, more than one macroblock may exist in pipeline stage 1, while pipeline stages 2 and 3 are limited to having only one macroblock. The residual path includes a coefficient engine 408, an AC / DC prediction engine 410, a 4 × 4 inverse transform engine 418, an 8 × 8 IDCT engine 420, an IDCT format converter engine 422, and a residual block 436. The prediction path includes an intra prediction mode engine 404, an MV prediction engine 406, an intra prediction engine 414, a rotation engine 415, a motion compensation engine 416, an intra prediction buffer 432, and a prediction sample 434.

  [058] As described above, the decoder device 400 is compatible with JPEG, MPEG-4, H.264. 263, H.M. 263+, H.H. H.264, or WMV9 / VC-1 standard, is operable to decode a video stream. The hardware decoding block described above performs all the decoding operations required by the supported standards. Since the specific operation of the hardware decoding block is described in each of the standards, their operation is well known and understood by those skilled in the art. Accordingly, the specific operation of the hardware decoding block will not be described in detail herein.

  [059] In one embodiment, MV parameters and intra prediction parameters are passed to the MV prediction engine 406 and intra prediction mode engine 404, respectively, in the prediction path. These engines calculate actual motion vectors or intra prediction modes based on programmed video standards and pass them to motion compensation engine 416 or intra prediction engine 414, respectively. The motion compensation engine 416 or the intra prediction engine 414 calculates prediction data. In one embodiment, motion compensation engine 416 includes a rotation engine 415. The rotation engine 415 is for rotating the reference frame so that the position matches the received video frame. The rotation engine 415 is activated each time the motion compensation engine is used to decode the video stream. On the other hand, the error data is processed in the required subset of coefficient engine 408, AC / DC prediction engine 410, 4x4 inverse transform engine 418, 8x8 IDCT engine 420, and IDCT format converter engine 422.

  [060] The recovered error data is added to the prediction data and then passed further to stage 3 of the pipeline. The resulting data is further processed as needed and written to memory 460 for display. In-loop deblocking filter is used in H264 and WMV9 / VC-1 modes. In WMV9 / VC-1 mode, the in-loop deblocking filter 440 is used to implement an overlap smoothing filter. An out-of-loop filter 442 can be used for any video stream to improve the quality of the decoded image. In one embodiment, the out-of-loop filter 442 is performed concurrently with the rest of the decoder device 400. Out-of-loop filter 442 should be activated after the frame is decoded into memory 460. The decoded image can also be rotated by rotation engine 450 before writing to memory 460 at stage 3 of the pipeline.

Exemplary operation of a hardware multi-standard video decoder device for supported video standards
[061] The following embodiments describe the operation of the decoder device 400 for each of the supported video standards.

  [062] JPEG: Since the JPEG video stream is for reproducing a digital still image, JPEG decoding does not require a hardware decoding block in the prediction path. Accordingly, the intra prediction mode engine 404, the MV prediction engine 406, the intra prediction engine 414, the rotation engine 415, the motion compensation engine 416, the intra prediction buffer 432, and the prediction sample 434 are all stopped for JPEG decoding. Also, JPEG decoding does not require the 4x4 inverse transform engine 418, and therefore the 4x4 inverse transform engine 418 is stopped. The instruction parser 402 operates a coefficient engine 408, an AC / DC prediction engine 410, an 8 × 8 IDCT engine 420, a decimation IDCT engine 438, an IDCT format converter engine 422, and a residual block 436. The instruction parser 402 routes data from the video stream 401 through an active hardware decoding block for decoding a JPEG encoded video stream. It should be understood that the operations performed by the hardware decoding block and the order of operations are defined by the JPEG standard.

  [063] JPEG decoding only requires the use of one of the 8x8 IDCT engine 420 and the decimation IDCT engine 438. In one embodiment, the instruction parser 402 is operable to identify which of the 8 × 8 IDCT engine 420 and the decimation IDCT engine 438 is activated for the video stream. The 8x8 IDCT engine 420 is activated to fully decode the video stream, while the decimation IDCT engine 438 is activated when the video stream indicates decimation. The IDCT format converter engine 422 is operable to perform format conversion. For example, the IDCT format converter engine 422 performs format conversion between any of the following formats: YUV4: 4: 4, YUV4: 2: 2, YUV4: 2: 2R, and YUV4: 2: 0. Can be executed. It should be understood that other format conversions can be performed and that the IDCT format converter engine 422 is not limited to the listed formats.

  [064] The decoded JPEG video stream exits stage 2 of the pipeline. In one embodiment, the decoded JPEG video stream is stored in memory 330. In another embodiment, post-processing operations are performed on the decoded JPEG video stream before storing it in memory 330.

  [065] MPEG-4 / H. 263: MPEG-4 and H.264. The decoding of H.263 is very similar to each other for the purpose of the decoder device 400. Specifically, the MPEG-4 standard is based on the MPEG-4 decoder. It is required to be operable to decode a video stream encoded at H.263. MPEG-4 and H.264. The H.263 decoding does not require the intra prediction mode engine 404, the intra prediction engine 414, the IDCT format converter engine 422, and the 4 × 4 inverse transform engine 418, which are stopped. Furthermore, the in-loop deblocking filter 440 is also stopped for post-processing operations. Thus, the instruction parser includes an MV prediction engine 406, a coefficient engine 408, an AC / DC prediction engine 410, a rotation engine 415, a motion compensation engine 416, an 8 × 8 IDCT engine 420, an intra prediction buffer 432, a prediction sample 434, and a residual block. 436 is activated. The instruction parser 402 is MPEG-4 or H.264. Data from the video stream 401 is routed through an active hardware decoding block for decoding the H.263 encoded video stream. The operations performed by the hardware decoding block and the order of the operations are MPEG-4 and H.264. It should be understood that it is defined by the H.263 standard.

  [066] The instruction parser 402 is operable to direct the macroblock to the hardware decoding block of the appropriate residual path or prediction path. In one embodiment, an intra frame (I-frame) is a residual path coefficient engine 408 and AC / AC at the same time that a predicted frame (P-frame) is processed in the MV prediction engine 406 in stage 1 of the pipeline. It can be processed in the DC prediction engine 410. I-frames and P-frames are synchronized in stage 2 of the pipeline. The instruction parser 402 is also operable to operate the appropriate hardware decoding block of the 8 × 8 IDCT engine 420.

  [067] The decoded MPEG-4 / H. The H.263 video stream exits stage 2 of the pipeline. In one embodiment, the decoded MPEG-4 / H. The H.263 video stream is stored in the memory 330. In another embodiment, prior to storing in memory 330, the decoded MPEG-4 / H. A post-processing operation is performed on the H.263 video stream. In another embodiment, the decoded MPEG-4 / H. A post-processing operation is performed on the out-of-loop filter 442 on the H.263 video stream. In one embodiment, the out-of-loop filter 442 is a deblocking filter. In another embodiment, the out-of-loop filter 442 is a deringing filter. In another embodiment, the out-of-loop filter 442 is both a deblocking filter and a deringing filter. It should be understood that the out-of-loop filter 442 can be implemented as any deblocking and / or deringing filter.

  [068] H.C. 263+: H. The H.263 + decoding is based on MPEG-4 / H. Similar to H.263 decoding. H. H.263 + transfers part of the decoding operation to a VLD that is executed before the instruction parser 402 accesses the video stream 401. Does not require intra prediction mode engine 404, intra prediction engine 414, 4x4 inverse transform engine 418, and out-of-loop filter 442, thus intra prediction mode engine 404, intra prediction engine 414, 4x4 inverse transform engine 418, and In addition to stopping the out-of-loop filter 442, the instruction parser 402 also stops the coefficient engine 408 and the AC / DC prediction engine 410. In other respects, H.C. The H.263 + decoding is based on MPEG-4 / H. Similar to H.263 decoding. The operations performed by the hardware decoding block and the order of the operations are H.264. It should be understood that it is defined by the H.263 + standard.

  [069] H. 264: H. H.264 decoding does not require the AC / DC prediction engine 410, the 8x8 IDCT engine 420, and the IDCT format converter engine 422, which are stopped. Therefore, the instruction parser 402 includes an intra prediction mode engine 404, an MV prediction engine 406, a coefficient engine 408, an intra prediction engine 414, a rotation engine 415, a motion compensation engine 416, a 4 × 4 inverse transform engine 418, an intra prediction buffer 432, a prediction The sample 434 and the residual block 436 are activated. The intra prediction buffer 432 stores the first row of pixels from the previous macroblock so that the intra prediction engine 414 can access the previous “leveled” pixels when processing the next row of the macroblock. It is possible to operate. The instruction parser 402 The data from the video stream 401 is routed through an active hardware decoding block for decoding the H.264 encoded video stream. The operations performed by the hardware decoding block and the order of the operations are H.264. It should be understood that it is defined by the H.264 standard.

  [070] The instruction parser 402 is operable to direct the macroblock to the hardware decoding block of the appropriate residual path or prediction path. In one embodiment, the frame can be processed simultaneously in the residual path and the prediction path within stage 1 of the pipeline. The frames are synchronized in stage 2 of the pipeline.

  [071] Decrypted H.264. The H.264 video stream exits stage 2 of the pipeline. In one embodiment, the decrypted H.264 file is stored prior to storage in memory 330. An in-loop post-processing operation is performed on the H.264 video stream. In another embodiment, the decrypted H.264 An out-of-loop post-processing operation is performed on the H.264 video stream by the out-of-loop filter 442. It should be understood that the out-of-loop filter 442 can be implemented as any deblocking filter and / or deringing filter.

  [072] WMV9 / VC-1: Decoding of WMV9 / VC-1 does not require the intra prediction mode engine 404 and the intra prediction engine 414, which are stopped. Therefore, the instruction parser 402 includes the MV prediction engine 406, the coefficient engine 408, the AC / DC prediction engine 410, the rotation engine 415, the motion compensation engine 416, the 4 × 4 inverse transform engine 418, the 8 × 8 IDCT engine 420, and the intra prediction buffer 432. , Activate the prediction sample 434 and the residual block 436. The instruction parser 402 routes data from the video stream 401 through an active hardware decoding block for decoding the WMV9 / VC-1 encoded video stream. It should be understood that the operations performed by the hardware decoding block and the order of operations are defined by the WMV9 / VC-1 standard.

  [073] The instruction parser 402 is operable to direct the macroblock to the hardware decoding block of the appropriate residual path or prediction path. In one embodiment, the frame can be processed simultaneously in the residual path and the prediction path within stage 1 of the pipeline. The frames are synchronized in stage 2 of the pipeline.

  [074] The decoded WMV9 / VC-1 video stream exits stage 2 of the pipeline. In one embodiment, in-loop post-processing operations are performed on the decoded WMV9 / VC-1 video stream before storing it in memory 330. In one embodiment, the in-loop deblocking filter 440 is used to implement an overlap smoothing filter. In another embodiment, a post-processing operation is performed on the decoded WMV9 / VC-1 video stream with an out-of-loop filter 442. It should be understood that the out-of-loop filter 442 can be implemented as any deblocking and / or deringing filter.

Post-processing operation
[075] Stage 3 of the pipeline of the decoder device 400 includes three hardware post-processing blocks: an in-loop deblocking filter 440, an out-of-loop filter 442, and a rotation engine 450. The in-loop deblocking filter 440 is an H.264 filter. Used in H.264 and WMV9 / VC-1 modes. In one embodiment, in WMV9 / VC-1 mode, the in-loop deblocking filter 440 is used to implement an overlap smoothing filter.

  [076] Out-loop filter 442 can be used for any video stream to improve the quality of the decoded image. In one embodiment, the out-of-loop filter 442 is performed concurrently with the rest of the decoder device 400. Out-of-loop filter 442 should be activated after the frame is decoded into memory 460.

  [077] It should be understood that any deblocking and / or deringing filter may be used for the out-of-loop filter 442. For example, the International Organization for Standardization (ISO), an organization that manages many of the video standards that can be implemented on the device 150, often includes proposals for deblocking filters in standardization publications. For example, the out-of-loop filter 442 can be found in ISO publication ISO / IEC 14496-2: 2001, section F.1. The deblocking filter described in 3.1 can be included.

  [078] The decoded image may also be rotated by rotation engine 450 before writing to memory 460 at stage 3 of the pipeline. The rotation engine 450 is configured to provide rotation of on-the-fly macroblocks, and each macroblock is rotated based on the indicated degree of rotation and placed at a new position in the frame. For a detailed discussion of the operation of the rotation engine 450, see the discussion of FIGS. 10A, 10B, and 11 below.

Method for decoding a video stream using a hardware multi-standard video decoder device
[079] FIG. 5 shows a flowchart of a method 500 for decoding a video stream, which is performed using a hardware multi-standard video decoder device according to an embodiment of the present invention. Although specific steps are disclosed in method 500, such steps are exemplary. That is, embodiments of the present invention are well suited to performing various other steps or variations of the steps described in FIG. In one embodiment, the method 500 is performed by the decoder device 150 of FIG.

  [080] In step 510 of process 500, a video stream is accessed. In step 520, the video standard used to encode the video stream is identified. The hardware multi-standard video decoder device is configured to decode a video stream according to a plurality of video standards.

  [081] At step 530, a subset of the hardware decoding blocks of the plurality of hardware decoding blocks of the hardware multi-standard video decoder device used to decode the video stream is determined. Different subsets of the plurality of hardware decoding blocks are operable to decode video streams encoded using different video encoding standards. In one embodiment, as shown in step 540, a subset of hardware decoding blocks are activated so that hardware decoding blocks not involved in decoding the video stream are not activated.

  [082] At step 550, the video stream is decoded using a subset of the hardware decoding blocks. In one embodiment, as shown in step 560, if video stream data is not received in one stage of the multi-stage macroblock level pipeline, the hardware decoding blocks in that stage are stopped. It should be understood that steps 540 and 560 provide additional power savings and are optional.

  [083] In step 570, the memory unit is accessed after decoding the video stream. In one embodiment, the decoded video stream is stored in memory for display. In one embodiment, a post-processing operation on the decoded video stream, as shown in step 580. It should be understood that post-processing operations can be performed before step 570 is performed or after step 570 is performed. In one embodiment, the decoded video stream is rotated. In another embodiment, an in-loop deblocking filter is applied to the decoded video stream. Rotation and in-loop deblocking are performed before the memory unit is accessed. In one embodiment, out-of-loop deblocking and deringing filters are applied to the decoded video stream after the memory unit is accessed.

Decode multiple streams encoded using different video standards using a hardware multi-standard video decoder device
[084] Embodiments of the hardware multi-standard video decoder device of the present invention are also operable to decode multiple video streams simultaneously. Some video streams, such as macroblocks or frames, are interleaved. The decoder device sequentially accesses the interleaved parts. Therefore, the decoder device performs a decoding operation on the interleaved part. For example, a decoding operation can be performed on macroblocks of two video streams. The video stream is interleaved so that macroblocks of the video stream appear alternately. At each clock cycle, a decoding operation can be performed on the alternating video stream.

  [085] FIG. 6 shows a diagram illustrating the internal components of a hardware multi-stream multi-standard video decoder device 600 according to one embodiment of the invention. As shown in FIG. 6, the decoder apparatus 600 includes a video stream interleaver 605, an instruction parser 305, a plurality of hardware decoding blocks 310 to 318, a hardware post-processing block 320, a memory 330, a register set 610, and a register. Set 620 is included. Decoder device 600 is operable to decode a plurality of video coding standards and operates in many ways similar to decoder device 150 of FIG. Decoder device 600 differs from decoder device 150 in that register sets 610 and 620 enable decoder device 600 to decode multiple video streams simultaneously.

  [086] Video stream interleaver 605 is operable to access multiple video streams and to interleave portions of the video streams. As shown, video stream interleaver 605 accesses video streams 601 and 602. However, it should be understood that the video stream interleaver 605 is operable to receive any number of video streams and is not limited to the embodiment shown in FIG. In one embodiment, video streams 601 and 602 are received from a microcode engine (eg, microcode engine 260 of FIG. 2B).

  [087] FIGS. 7A and 7B show diagrams illustrating exemplary interleaved portions of multiple video streams, in accordance with an embodiment of the present invention. Referring to FIG. 7A, two interleaved video streams are shown, one stream is a still image video stream (eg, JPEG) and the other stream is a digital movie stream (eg, MPEG-4). . As shown, if the video stream contains only one digital movie stream, the video stream can be interleaved at the macroblock level. Specifically, still image macroblocks 704 and 708 are interleaved with digital movie macroblocks 702 and 706 so that macroblocks from each video stream appear alternately in interleaved stream 700. If the video stream is interleaved at the macroblock level, the software driver of the decoder device 600 buffers the macroblock data in system memory to manage the decoding of the interleaved video stream.

  [088] Referring to FIG. 7B, two interleaved video streams are shown, both streams being digital movie streams. As shown, if the video stream includes multiple digital movie streams, the video stream is interleaved at the frame level. Specifically, the first digital movie frames 752 and 756 are interleaved with the second digital movie frames 754 and 758 such that the frames from each video stream appear alternately in the interleaved stream 750. . When the video stream is interleaved at the frame level, the software driver of the decoder device 600 buffers the frame data in the system memory to manage the decoding of the interleaved video stream.

  [089] Referring to FIG. 6, the instruction parser 305, hardware decoding blocks 310-318, hardware post-processing block 320, and memory 330 operate as shown in FIG. Residual data and other decoder parameters are passed to the decoder device through the instruction parser 305. Data from the instruction parser 305 is routed to either the residual path (hardware decoding blocks 312, 316, and 318) or the prediction path (hardware decoding blocks 310 and 314). The residual path processes error or difference data, while the prediction path prepares / acquires previous frame or previous macroblock data.

  [090] In order to manage the decoding of the interleaved video stream, two register sets 610 and 620 are maintained in stage 1 of the pipeline. In one embodiment, register sets 610 and 620 store memory surface pointers 612 and 622, respectively, and store frame level parameters 614 and 624, respectively. Each of the register sets is used to store parameters associated with one of the video streams. For example, register set 610 is used to store parameters associated with video stream 601 and register set 620 is used to store parameters associated with video stream 602. When any part of one video stream is processed in pipeline stage 1, the appropriate parameters are passed in the form of packets to downstream pipeline stages 2 and 3 along with residual or predicted data. The decoded data is routed to the appropriate area in memory based on whether the macroblock is in the form of a still image or a digital movie. The decoder device 600 can be configured to decode any number of video streams by adding an appropriate number of register sets such that each stream to be decoded has an associated register set. .

  [091] FIG. 8 shows a flowchart of a method 800 for decoding a plurality of video streams, which is performed using a hardware multi-stream multi-standard video decoder device according to an embodiment of the present invention. . Although specific steps are disclosed in method 800, such steps are exemplary. That is, embodiments of the present invention are well suited to performing various other steps or variations of the steps described in FIG. In one embodiment, the method 800 is performed by the decoder device 600 of FIG.

  [092] At step 810 of process 800, multiple video streams are accessed. In step 820, the video standard used to encode the video stream is identified. The hardware multi-stream multi-standard video decoder device is configured to decode video streams according to a plurality of video standards. In step 830, a portion of the video stream is interleaved. In one embodiment, if the video stream contains only one digital movie stream, the macroblocks of the video stream are interleaved. In another embodiment, if the video stream includes multiple digital movie streams, the frames of the video stream are interleaved. It should be understood that steps 820 and 830 can be performed in any order.

  [093] At step 840, a subset of the hardware decoding blocks of the plurality of hardware decoding blocks of the hardware multi-standard video decoder device used to decode the plurality of video streams is determined. . Different subsets of the plurality of hardware decoding blocks are operable to decode video streams encoded using different video encoding standards. In one embodiment, as shown in step 850, a subset of hardware decoding blocks are activated such that hardware decoding blocks not related to decoding of the video stream are not activated.

  [094] At step 860, the video stream is decoded using a subset of the hardware decoding blocks. In step 870, the memory unit is accessed after decoding the video stream. In one embodiment, the decoded video stream is stored in memory for display. In one embodiment, a post-processing operation on at least one decoded video stream, as shown in step 880. It should be understood that the post-processing operation can be performed before step 870 is performed or after step 870 is performed. In one embodiment, the decoded video stream is rotated. In another embodiment, an in-loop deblocking filter is applied to the decoded video stream. Rotation and in-loop deblocking are performed before the memory unit is accessed. In one embodiment, out-of-loop deblocking and deringing filters are applied to the decoded video stream after the memory unit is accessed.

Handling macroblocks with out-of-order video streams
[095] Referring to Figure 2B, in one embodiment, the present invention performs macroblock buffering and reordering in the microcode engine 260. The present invention provides for aligning macroblocks in raster scan order to support in-loop deblocking for video standards that support transmission and reception of macroblocks in non-raster scan order. Microcode engine 260 is configured to receive compressed data representing macroblocks of frames of a video stream. In one embodiment, at least one macroblock is received out of order. The microcode engine 260 is configured to buffer the compressed data and is configured to align the macroblocks of the frame in raster scan order.

  [096] FIG. 9 shows a flowchart of a method 900 for processing out-of-order macroblocks in a video stream, according to an embodiment of the present invention. Although specific steps are disclosed in method 900, such steps are exemplary. That is, embodiments of the present invention are well suited to performing various other steps or variations of the steps described in FIG. In one embodiment, the method 900 is performed by the microcode engine 260 of FIG. 2B.

  [097] At step 910 of method 900, compressed data representing a macroblock of a frame of a video stream is received, and at least one macroblock is received out of order. In step 920, the compressed data is buffered. In one embodiment, the compressed data is buffered in a microcode engine 260 buffer. In step 930, the video stream is parsed and VLD is performed on the video stream. It should be understood that step 930 is optional and that parsing and VLD of the video stream can be performed by a hardware decoder device. It should further be appreciated that other or further pre-processing operations can be performed on the video stream at step 930.

  [098] At step 935, it is determined whether the video stream requires in-loop deblocking. In one embodiment, the compressed data includes an indication of whether in-loop deblocking should be performed on the video stream. If in-loop deblocking is required, as shown in step 940, the macroblocks of the frame are aligned in raster scan order. In one embodiment, all macroblocks in the frame are buffered before the macroblocks are aligned in raster scan order. The method 900 then proceeds to step 950. Alternatively, if in-loop deblocking is not required, method 900 proceeds directly to step 950.

  [099] In step 950, the video stream is decoded. In one embodiment, the macroblocks are decoded in raster scan order. In one embodiment, the video stream is decoded by a hardware multi-standard video decoder device (eg, decoder device 150 of FIG. 3 or decoder device 400 of FIG. 4). In one embodiment, the video stream is decoded according to the method 500 of FIG.

  [0100] In step 960, macroblock-level intra-loop deblocking is performed on the decoded macroblock. In step 970, the memory unit is accessed. In one embodiment, the deblocked and decoded video stream is stored in memory for display.

  [0101] In step 980, frame-level out-of-loop post processing is performed on the decoded frame. In one embodiment, the out-of-loop post processing includes deblocking and deringing operations. It should be understood that step 980 is optional. The method 900 then returns to step 970 and the memory unit is accessed. In one embodiment, the deblocked, deringed and decoded video stream is stored in memory for display.

  [0102] By buffering the macroblock while the macroblock is still compressed data, the microcode engine 260 allows a maximum of one frame of run-length encoded compressed data to be much less than the decoded video data. Only need to be buffered. Furthermore, the compressed macroblock buffer also saves power. Also, video streams received over the air are likely to contain many errors. Also, the parsing of the bitstream for the microcode engine 260 has the advantage of improving error recovery.

Rotate video stream macroblocks on the fly
[0103] Embodiments of the present invention provide a rotation engine for rotating a video stream "on the fly" before the video stream is written to memory. Embodiments of the present invention can rotate video streams by rotating the macroblocks of those video streams as they are received and rearranging the macroblocks within the frame based on the rotation. Embodiments of the present invention operate on a macroblock before writing the decoded macroblock to memory, thereby rotating the video stream without requiring a second pass in the decoded frame. can do.

  [0104] In one embodiment, the present invention provides a rotation engine configured to rotate a macroblock of a frame of a video stream according to the degree of rotation and to relocate the macroblock to a new position within the frame. Provide new position based on the degree of rotation. In one embodiment, the video decoder device further includes a memory for storing macroblocks for display. In one embodiment, the rotation engine is configured to rotate the macroblock and reposition the macroblock within the frame before accessing the memory.

  [0105] FIGS. 10A and 10B show exemplary rotation diagrams of macroblocks of a frame, according to embodiments of the invention. 10A and 10B illustrate the operation of the rotation engine 450 of FIG. 4, the illustrated embodiment can be implemented in any type of video decoder device and the hardware multi-standard video decoder of FIG. It should be understood that the invention is not limited to the use of device 400. For example, the rotation engine can be included in a single standard hardware decoder or software decoder.

  [0106] Referring to FIG. 10A, FIG. 1000 illustrates rotation of the frame 1010 using the rotation engine 450 of FIG. Frame 1010 includes a number of macroblocks. Macroblock 1012 is shown as the first macroblock received at rotation engine 450. In one embodiment, the macroblocks are received in raster scan order in which the macroblock 1012 is the first received macroblock, such that the macroblock 1012 is the upper left macroblock.

  [0107] The rotation engine 450 is configured to rotate the macroblock 1012 and reposition the macroblock 1012 to a new location within the frame 1010. The rotation and rearrangement is based on the degree of rotation associated with the video stream. The degree of rotation indicates how the video stream should be rotated. For example, the degree of rotation may be 90 degrees clockwise, 90 degrees, 180 degrees counterclockwise, or any other degree of rotation.

  [0108] FIG. 1000 illustrates the operation of the rotation engine 450 using a degree of rotation of 90 degrees clockwise. Macroblock 1012 is rotated 90 degrees clockwise. The rotation engine 450 is configured so that the rotated macroblock 1012, shown as the macroblock 1022 in the rotated frame 1020, is in the same position relative to all other macroblocks in the frame 1020. Is also rearranged.

  [0109] Embodiments of the present invention also provide frame rotation at the macroblock level where macroblocks are received out of order. Referring to FIG. 10B, FIG. 1050 illustrates rotation of the frame 1060 using the rotation engine 450 of FIG. Macroblock 1062 is shown as the first macroblock received at rotation engine 450. In this embodiment, the macroblocks are received in an order that is not in raster scan order so that macroblock 1062 is the first received macroblock but not the upper left macroblock.

  [0110] The rotation engine 450 is configured to rotate the macroblock 1062 and reposition the macroblock 1062 to a new location within the frame 1060. FIG. 1050 shows the operation of the rotation engine 450 using a degree of rotation of 90 degrees clockwise. Macroblock 1062 is rotated 90 degrees clockwise. The rotation engine 450 is configured so that the rotated macroblock 1062, shown as the macroblock 1072 in the rotated frame 1070, is in the same position relative to all other macroblocks in the frame 1070. Is also rearranged.

  [0111] FIG. 11 shows a flowchart of a method 1100 for rotating a macroblock of a frame according to an embodiment of the present invention. Although specific steps are disclosed in method 1100, such steps are exemplary. That is, embodiments of the present invention are well suited to performing various other steps or variations of the steps described in FIG. In one embodiment, the method 1100 is performed by the microcode engine 450 of FIG.

  [0112] In step 1110, the video stream is decoded. In one embodiment, the video stream is decoded by a hardware multi-standard video decoder device (eg, decoder device 150 of FIG. 3 or decoder device 400 of FIG. 4). In one embodiment, the video stream is decoded according to the method 500 of FIG. It should be understood that step 1110 is optional and that the video stream has already been decoded prior to processing.

  [0113] In step 1120, the degree of rotation for the video stream is accessed. In one embodiment, the degree of rotation is one of 90 degrees clockwise, 90 degrees counterclockwise, and 180 degrees. However, it should be understood that any degree of rotation may be used. In step 1130, a macroblock of the video stream is accessed.

  [0114] In step 1140, the macroblock is rotated according to the degree of rotation. In step 1150, the macroblock is relocated to a new position in the frame, and the new position is based on the degree of rotation. It should be understood that a macroblock is rearranged so that it is in the same relative position with respect to all other macroblocks in the frame once it has been rotated. In one embodiment, macroblock rotation and macroblock relocation are performed before accessing the memory.

  [0115] In step 1160, the macroblock is stored in memory for display. In one embodiment, as shown in step 1170, a deblocking operation is performed on the decoded macroblock. It should be understood that step 1170 is optional. Further, it should be understood that step 1170 can include performing in-loop deblocking, or out-of-loop deblocking and deringing.

  [0116] Thus, embodiments of the present invention provide a new hardware multi-standard video decoder device architecture that supports hardware-based decoding of video streams according to multiple video standards. Embodiments of the present invention can provide real-time decoding for each of a plurality of video coding standards. Embodiments of the present invention provide post-processing operations on the decoded video stream. One embodiment of the present invention is JPEG, MPEG-4, H.264. 263, H.M. 263+, H.H. A hardware decoder device that provides video decoding for video streams using any of H.264 and WMV9 / VC-1 video standards.

  [0117] Embodiments of the present invention provide a hardware multi-stream multi-standard video decoder device for simultaneously providing video decoding functions for a plurality of different video coding standards. Embodiments of the present invention can simultaneously decode multiple interleaved video streams.

  [0118] Embodiments of the present invention provide a video decoder architecture for providing in-loop deblocking of a video stream without requiring additional memory to align macroblocks in raster scan order. Embodiments of the present invention can align macroblocks of a video stream in a microcode engine. Embodiments of the present invention can provide decoding and out-of-loop deblocking and / or deringing for a video stream encoded using one of multiple supported video standards.

  [0119] Embodiments of the present invention provide a rotation engine for rotating a video stream "on the fly" before the video stream is written to memory. Embodiments of the present invention can rotate video streams by rotating the macroblocks of those video streams as they are received and rearranging the macroblocks within the frame based on the rotation. Embodiments of the present invention can rotate a video stream without requiring a second pass in the decoded frame by operating on the macroblock before writing the decoded macroblock to memory. .

  [0120] The foregoing descriptions of specific embodiments of the present invention have been presented for purposes of illustration and description. These embodiments are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and many modifications and variations are possible in light of the above teaching. is there. The embodiments best describe the principles of the invention and its practical application so that those skilled in the art can best utilize the invention and various embodiments with various modifications suitable for the particular application envisaged. Selected and explained to make it possible. It is intended that the scope of the invention be defined by the claims appended hereto and their equivalents.

FIG. 2 is a schematic diagram of basic components of a computer system according to an embodiment of the invention. FIG. 2 is a diagram of an exemplary hardware video decoder card mounted on a printed circuit board, according to one embodiment of the invention. FIG. 3 is an exemplary architecture including a hardware multi-standard video decoder device, according to one embodiment of the present invention. FIG. 2 is a block diagram illustrating internal components of a hardware multi-standard video decoder device according to an embodiment of the present invention. FIG. 3 is a block diagram illustrating internal components of an exemplary hardware multi-standard video decoder device according to an embodiment of the present invention. Fig. 4 is a flowchart of a method for decoding a video stream, the method being performed using a hardware multi-standard video decoder device according to an embodiment of the present invention. FIG. 3 illustrates internal components of a hardware multi-stream multi-standard video decoder device according to an embodiment of the present invention. FIG. 3 illustrates an exemplary interleaved portion of multiple video streams, in accordance with an embodiment of the present invention. FIG. 3 illustrates an exemplary interleaved portion of multiple video streams, in accordance with an embodiment of the present invention. FIG. 5 is a flowchart of a method for decoding a plurality of video streams, the method being performed using a hardware multi-stream multi-standard video decoder device according to an embodiment of the present invention. 4 is a flowchart of a method for processing out-of-order macroblocks in a video stream, according to an embodiment of the present invention. FIG. 3 is an exemplary rotation diagram of a macroblock of a frame according to an embodiment of the present invention. FIG. 3 is an exemplary rotation diagram of a macroblock of a frame according to an embodiment of the present invention. 4 is a flowchart of a method for rotating a macroblock of a frame according to an embodiment of the present invention.

Claims (10)

A method for decoding, performed using a hardware multi-standard video decoder device, comprising:
Accessing a first video stream that is one of a plurality of video streams;
Identifying a first video standard used to encode the first video stream;
Determining a first subset of hardware decoding blocks of a plurality of hardware decoding blocks of the hardware multi-standard video decoder device used to decode the first video stream; The determining step, wherein different subsets of the plurality of hardware decoding blocks are operable to decode video streams encoded using different video encoding standards;
Decoding the first video stream using the first subset of hardware decoding blocks.   The method of claim 1, further comprising operating the subset of hardware decoding blocks such that hardware decoding blocks not related to decoding of the first video stream are not activated.   The method of claim 1, wherein the plurality of hardware decoding blocks are implemented in a multi-stage macroblock level pipeline.   The method of claim 3, further comprising stopping a hardware decoding block in the stage when data of the first video stream is not received in one stage of the multi-stage macroblock level pipeline. .   The method of claim 1, further comprising accessing a memory unit after the step of decoding the first video stream. Accessing the plurality of video streams;
Identifying a second video standard used for a video stream of the plurality of video streams;
Interleaving portions of the plurality of video streams;
Determining a plurality of subsets of hardware decoding blocks of the plurality of hardware decoding blocks;
The method of claim 1, further comprising: decoding the plurality of video streams using the plurality of subsets of hardware decoding blocks.   The method of claim 6, wherein the plurality of video streams includes at least one digital still image stream and a digital movie stream.   The method of claim 7, wherein the portion of the plurality of video streams is a frame of the digital still image stream and the digital movie stream.   The method of claim 6, wherein the plurality of video streams includes a plurality of digital movie streams.   The method of claim 9, wherein the portion of the plurality of video streams is a macroblock of the plurality of digital movie streams.

Images