JP2007529178A - Video decoder with extensible compression and having a buffer for storing and retrieving reference frame data - Google Patents

Abstract

  The present invention relates to a video decoder having means for compressing reference frame data (6) using an extensible compression method. The video decoder further comprises buffer means (8) for temporarily storing at least the vertical aperture (range) of the motion vector and a horizontal row (slice) of macroblocks in the video line per reference frame. . Also, the video decoder decompresses the reference frame data so that the motion compensation means (10) of the decoder can reconstruct a vector prediction image and a macroblock using the decompressed reference frame data. Means (7). The invention also relates to a method to be implemented by such a video decoder.

Description

  The application of the invention relates to the field of video decoders, in particular to video decoders with a simplified memory access profile.

  In the field of digital video, most common image coding types are I-pictures (internally coded pictures) that are coded without reference to any other picture and are often referred to as reference frames or anchor frames; P-picture (predicted coded picture) that is coded using motion compensated prediction from a past I-reference picture or P-reference picture and may be considered a reference or anchor frame; and previous (backward) ) A B-picture (bidirectional predictive coded picture) coded using an I-picture or P-picture followed by a (forward) I-picture or P-picture. These image types are sometimes referred to as I, P or B frames.

  A compression standard called MPEG (Moving Picture Experts Group) compression is a set of methods for compression and decompression of full motion video images using the frame compression technique described above. MPEG compression can use both motion compensation and discrete cosine transform (DCT) processes, in particular, and can result in very high compression ratios. To better understand this compression standard, Barry G. Haskell, Atul Puri and Arun N. See “Digital Video: An Introduction to MPEG-2” by Netravli (issued by Chapman & Hall in 1997).

  Currently, most video decoders, such as MPEG-2 decoders, use external memory to form video frames from P-pictures and B-pictures with vector-controlled prediction to previously stored reference frames. This external memory is mostly DRAM based because they represent the mainstream market for stand-alone memory devices. DRAM-based memory provides a burst access mode for high bandwidth performance. This means that by giving only a single read or write command, many consecutive data words (bursts) are transferred to or from the memory. In order to take advantage of the available data bandwidth, read and write accesses must be burst oriented. DRAM-based memories tend to have efficient memory transfers only in large sized bursts.

  The first drawback is that vector control prediction requires random position block based access to one or more reference frames in memory. The efficiency in such access to and from DRAM based memory is quite low. The second drawback is video content dependent dynamics in the memory access bandwidth required to reconstruct the vector prediction frame.

  Many digital systems use MPEG-2 as a compression standard, but there are market differences between so-called main level systems and high level systems. In both systems, not only the encoder implementation but also the decoder implementation is quite different. The difference in processing speed and memory requirements is 5-6 times. Another fast coming market difference is between systems that can perform single high-level decoding and double high-level decoding (on-chip). For dual high-level MPEG-2 decoding, one or more prior art MPEG-2 decoders require significant system resources such as memory bandwidth, especially to external memory, and a memory footprint in reference frame storage. .

  Due to improvements in mainstream CMOS performance, high decoding speeds do not yield decoding blocks that are six times larger in high level systems. Unfortunately, memory requirements scale linearly in both access bandwidth and capacity, thereby significantly impacting the decoder architecture. In particular, the difference in access bandwidth represents a different approach in the case of external memory. This is complicated even when the external memory has to be shared with other clients such as CPUs, scalers, graphics accelerators, image component processors. Memory resources shared with other clients is a common situation when the MPEG decoder is part of a system-on-chip that uses integrated external memory.

  The already known patent publication US 6 088 391 relates to a memory system in a B frame of pixel data. In this case, each B frame includes a plurality of sections, and each of the plurality of sections includes pixel data corresponding to an upper end field and a lower end field of the frame. The memory system includes a memory organized into a plurality of segments for storing pixel data. In this case, the number of segments is equal to the number of frame sections + 2 further sections. However, each segment is half the size of the frame section. The memory system also includes a segmentation device that receives the pixel data and separates the pixel data according to the top and bottom fields of each frame. The segmentation device tracks the segments to determine the two available segments of the memory and stores the pixel data from the top field in one of the available segments for each section of each frame. At the same time, the pixel data from the bottom field is stored in another available segment of memory. A segment pointer table is preferably included to track segments of memory for interlaced display. The decoder system includes a memory and a segmentation device, includes a reconstruction unit for receiving and decoding the video data into pixel data, and includes a display circuit for retrieving the pixel data from the segment. It is out. A method for storing and retrieving pixel data includes separating the pixel data into fields and storing them in each segment. After half of the frame has been stored, the data is retrieved by a display device for interlaced display.

  The disadvantages of the aforementioned decoder system and method according to US Pat. No. 6,088,391 are that only a partial reduction in memory size requirements is possible, not reducing memory bandwidth requirements, and not simplifying memory access profiles. In other words, the dynamics in the required memory access bandwidth is not reduced.

  Accordingly, a video decoder and associated implementation implemented thereby that can simplify the memory access profile, reduce dynamics in memory access bandwidth, and achieve further reduction in memory size requirements and memory access bandwidth A way to do it is necessary.

  In view of the above, an object of the present invention is an improved video decoder having an integrated memory buffer combined with data compression / decompression, which can achieve a simple access profile to external memory regardless of video content Another object of the present invention is to provide a video decoder capable of obtaining a low and deterministic memory access bandwidth to an external memory.

  This object is achieved in accordance with the characterizing part of claim 1.

  Means for compressing the reference frame data using an extensible compression method, and at least a vertical opening (range) of the motion vector and a horizontal row (slice) of macroblocks in a video line per reference frame; Buffer means for storing the data and decompressing the reference frame data so that the means for motion compensation (MC) reconstructs the vector prediction image and macroblock using the decompressed reference frame data By providing means for enabling both the size of the reference frame to be stored and the memory access bandwidth requirements are reduced.

  A further object of the present invention is a method for reducing memory access bandwidth in a video decoder having an integrated memory buffer in combination with data compression / decompression while simplifying the memory access profile, Regardless, it is to provide a method that can achieve a simple access profile to external memory and obtain a low and deterministic memory access bandwidth to external memory.

  This object is achieved according to the characterizing part of claim 18.

  Variable length decoding (VLD) of the compressed video data; and inverse scanning, inverse quantization, and inverse discrete cosine transform (IDCT) decoding of the inner coded image, inner coded macroblock, and inner coded data information A step of performing motion compensation for decoding a vector prediction image and a macroblock, a decoded inner coded macroblock, decoded inner coded data information, and a motion compensated vector prediction macroblock Combining with reference frame data or output frame data, compressing said reference frame data using an extensible compression method, and at least in the vertical aperture (range) of the motion vector and the video line per reference frame Macroblock Temporarily storing a sequence (slice) in the buffer means; and decompressing the reference frame data so that the means for motion compensation (MC) uses the decompressed reference frame data to perform vector prediction By providing the steps of allowing the image and macroblock to be reconstructed and outputting the decoded image data, both the size of the reference frame to be stored and the memory access bandwidth requirement are reduced. .

  Preferred embodiments are described in the dependent claims.

  In the drawings, like reference characters designate like elements throughout the several views.

  Other objects and features of the present invention will become apparent from the following detailed description considered in conjunction with the accompanying drawings. However, it should be understood that the drawings are merely for illustrative purposes and are not intended to be limiting of the invention, and reference should be made to the appended claims in connection with the present invention. Also, the drawings are not necessarily drawn to scale, and unless otherwise indicated, they are merely intended to conceptually illustrate the structures and procedures described herein. Needless to say.

  In order to simplify the external memory access profile and to eliminate the dynamics in the external memory access bandwidth required by the video decoder, it is proposed in the present invention to integrate the memory. This integrated memory is used as a buffer 8, which is accessed in a first-in first-out (FIFO) mode when video data is transferred from the external memory 9 into the buffer and is predicted by a motion vector in a video decoder, for example. It is accessed on a block basis by the pre-fetch unit of the device constituting the frame. The function of the buffer 8 is to hide the dynamics in the complex (vector controlled) memory access profile and memory access bandwidth from the external memory 9.

  In practice, the buffer performs one FIFO per reference frame. Therefore, in the case of an MPEG-2 decoder, the buffer 8 includes a maximum of two FIFOs. The preferred granularity of the FIFO element of buffer 8 in FIFO mode is one slice, which is one row (row) of macroblocks. Suppose a slice extends over the entire horizontal range of the image. This assumption is not intended to be limiting. Actually, transfer of one slice (that is, one FIFO element) requires some efficient burst access from the external memory 9. A preferred further optimization is that one FIFO element expressed in bytes is exactly equal to the number of bytes obtained by an integer number of burst accesses from the external memory 9.

  FIG. 1 shows the initialization and update method for such a FIFO in the case of an embodiment of ATSC high level MPEG-2 decoding (vertical range of +/− 128, which is +/− 8 slices). Yes. In the figure, the reference frame buffer (FIFO) is indicated by 8, the reference pointer is indicated by 11, the vertical range motion vector is indicated by 12, and the external memory is indicated by 9. Assume that the decoder begins to decode the vector prediction picture. The macroblocks are entered in a sequential order starting from the top left, scanning from left to right, thereby moving from top to bottom and ending at the bottom right corner. The first state is a state where the FIFO 8 is about half of the whole (see FIG. 1, (left side)), which is the top of the reference frame that covers half the vertical opening of the motion vector 12 and one slice. . Since all possible vector reference image data is exactly in the FIFO 8, the first (first) input slice (Slice 1) of the vector prediction image can be completely processed. When the first (first) macroblock of the second slice (Slice 2) of the vector prediction image has to be decoded, the next slice of the reference frame must be transferred from the external memory 9 to the FIFO 8 (FIG. 1). Center and right). Since the FIFO 8 is about half of the total, the FIFO element is still not dropped, or old data is dropped. This process continues until the vertical offset from the top of the input slice exceeds half of the vertical aperture. From this point onwards, the decoded vector prediction picture never references the first slice in the FIFO 8, which causes this first slice to be dropped. When the next slice of the vector prediction image is decoded, the second slice in the FIFO 8 is dropped. Hereinafter, the same applies as shown in FIG. This continues until the last slice of the reference frame is in FIFA8.

  An advantageous approach for the current reference frame run-out state in FIFO 8 already has video data of the next required reference frame at a predetermined position in FIFO 8 at the start of decoding of the next vector prediction image. It is that. This is the first slice of the next required reference frame when the current slice of the vector prediction image has been decoded and the next slice should still be decoded as shown in FIG. This can be done by loading into the FIFO 8. When the last slice of the predicted image is decoded, the state is the same as in FIG. 1, but the first portion of the next required reference frame is in the FIFO buffer 8.

  In the case of MPEG-1 and MPEG-2, the buffer size (bits) is based on the assumption that the video in the buffer 8 has been decompressed (vertical range of motion vectors over the entire horizontal size of the image + macroblock (1 horizontal row) x maximum number of pixels per line x maximum number of reference frames x number of bytes per pixel x number of bits per byte.

  A single high-level MPEG-2 decoder for ATSC has a vertical range of 256 motion vectors, requires 16 lines for one macroblock row, and up to 1920 per line With a maximum of two reference frames, 1.5 bytes per pixel, and clearly 8 bits per byte. Therefore, if data compression is not applied in a single high level MPEG-2 decoder, a buffer memory of about 13 Mbits must be integrated. This 13 Mbit memory can be integrated in one block together with a high-speed MPEG decoding pipe. Such a block can cope with main level decoding at 50/60 Hz without the need for external memory 9. In the case of high-level decoding, the buffer 8 is used for vector control prediction, which is the operation with the most access. The lack of storage capacity must be added externally, but only requires a very simple minimum bandwidth interface. In any case, the decoder output must be fed to the output via the external display memory 13, during which the output can be mixed with graphics and other video streams. . In some main level systems, the display memory 13 can be omitted entirely.

  Due to the simple access profile to the external memory 9, it is proposed in the present invention to add a block-based memory compression algorithm for compression to and decompression from the external memory 9. Any block-based memory compression algorithm can be added. However, it is preferred that the compression algorithm be extensible, such as the one described in WO0117268 A1, the teachings of which are incorporated herein by reference.

  A video decoder according to the first embodiment is schematically shown in FIG. The decoder is preferably an MPEG decoder. However, it should be noted that the present invention is not limited to MPEG and may be used for any particular video standard or configuration. The video decoder according to the invention is based on a prior art video decoder. The compressed video data is recovered from the compressed data memory 1 and entropy decoded by a variable length decoder (VLD) 2 that converts the data into discrete cosine transform (DCT) data. An inverse scanner (IS) 3, an inverse quantizer (IQ) 4, and an inverse discrete cosine transform (IDCT) 5 process the inner coded delta information to convert the data into macroblocks of pixel data. A macroblock (MB) is a basic coding unit for the MPEG standard. The macroblock consists of one 16 pixel × 16 line portion of luminance component (Y) or four 8 pixel × 8 line blocks and several spatially corresponding 8 × 8 blocks of chrominance components Cr, Cb. It consists of. The number of blocks of chrominance values depends on which particular format is used. The vector prediction frame is reconstructed by block-based fetching (retrieving) from the external prediction memory 9 by the motion compensator 10 and addition of delta information if present, or by an internally coded macroblock.

  Such prior art MPEG-2 decoders require a maximum theoretical transfer rate from external memory which is 200% of the video transfer rate. For example, a high resolution video having a 1920 × 1080 interlace format at 60 Hz has a net transfer rate (no blanking) of about 62.2 Mpixel / s, which is about 93.3 Mbytes / s (YUV 4: 2: 0 format). Thus, in this case, the prior art high level MPEG-2 decoder theoretically requires a maximum memory access bandwidth of 187 Mbyte / s. However, system-on-chip must use worse case numbers due to complex memory access profiles, and SDRAM is only efficient in large packets.

  In the present invention, as shown in FIG. 5, a video data compressor 6, a video data decompressor 7, and a buffer 8 are added to the prior art decoder. The compressor 6 compresses the reference frame data using an extensible compression method. In this case, the compressed reference frame data is later stored in the external memory means 9, thereby forming a compressed reference frame memory. Thereafter, the compressed reference frame data is recovered from the external memory 9, and at least the vertical opening (range) of the motion vector and the horizontal row (slice) of macroblocks in the video line per reference frame are stored in the external memory. 9 and temporarily stored in a buffer 8 arranged between the means 10 for motion compensation. The reference frame data is decompressed by the decompressor 7, so that the means 10 for motion compensation (MC) reconstructs (restores) the vector prediction image and the macroblock using the decompressed reference frame data. be able to.

The size of the buffer 8 may be equal to (2 × 128 + 16) × 1920 × 1.5 × 2 × 8 = 12.53376 × 10 ⁶ bits for decoding in one example, and the scaler 14 is incorporated. Is preferably further added with 16 × 1920 × 1.5 × 8≈0.4 × 10 ⁶ bits for line-to-line conversion buffering, that is, about 13 Mbit in total. The access profile to the external memory 9 is very simple. However, the size of the reference frame was reduced by the compression ratio and memory access bandwidth, as shown in FIG. In FIG. 6, the decoding block is indicated by 15 and the MB format converter is indicated by 16. Note that in FIG. 6, only half of the buffer 8 is used to store one necessary reference frame. FIG. 7 shows a preferred embodiment in which the reference frame for the P-picture is compressed two times less than the reference frame for the B-picture. The preferred expandable compression method of FIG. 7 has the property that 2N: 1 ratio of compressed data can be easily obtained by taking about half of the most significant data of the compressed data of N: 1 ratio. . One skilled in the art can map the top and bottom compressed data into memory, thereby providing a simple access profile and efficient memory access to the external memory 9. Note that the double and double hierarchy levels can be expanded to other than double and multiple hierarchy levels.

  In the second embodiment according to FIG. 5, the size of the buffer 8 can be further reduced if the data compression / decompression is applied in a slightly different manner than that described above. Data compression is applied on the decoded reference frame before transfer to the external memory 9. However, data decompression is applied to data retrieved from buffer 8, and thus data that includes a compressed reference frame. This compressed data has been loaded from the external memory 9 into the buffer 8. The data compression method preferably has reasonable compression factors, low implementation costs, very high quality, reliability for iterative coding / decoding, and easy pixel access requirements. Reasonable data compression ratios at acceptable implementation costs and sufficiently high subjective image quality are 2: 1 and 4: 1. According to those skilled in the art, a 2: 1 compression ratio is considered a lossless compression and a 4: 1 compression ratio is considered very high image quality. In the MPEG-2 standard, a large number of P-pictures can be subsequently coded, whereby specific macroblocks are repeatedly compressed and decompressed by the codec. To prevent the decoder from deviating from the local reconstruction loop applied at the encoder, accurate quantization must be performed. In order to allow real-time operation of the motion compensated mechanism, easy access to the pixels in the compressed region must be possible.

  Another advantage provided by the present invention is that when using buffer 8 in combination with compression and decompression, reference frames in P-pictures and B-pictures can be given at different compression ratios. P-pictures are fundamentally more robust than B-pictures, and therefore B-pictures, because of the continuous prediction of P-pictures, and hence the risk of accumulating errors due to compression. Also requires a reference frame that is not compressed. For example, the buffer size required in a high level MPEG-2 decoder is that one 2: 1 compressed reference frame is used to reconstruct a P-picture and two 4: 1 to reconstruct a B-picture. If a compressed reference frame is used, it is reduced from about 13 Mbit to about 3 Mbit. The advantage of using extensible compression is that only a 2: 1 compressed reference frame needs to be stored in memory. The extensible compression method makes it very easy to obtain the required 4: 1 compressed reference frame directly from the 2: 1 compressed reference frame, and this feature is known to those skilled in the art. For example, a 2: 1 compressed reference frame can be divided into two half planes. The first plane containing the most significant data represents a 4: 1 compression ratio and the second plane containing the least significant data in combination with the first plane represents a 2: 1 compressed reference frame. As will be apparent to those skilled in the art, more hierarchical levels can be introduced, or other factors greater than 2 can be implemented.

  As described above, the decoder according to the present invention has a relatively low memory access bandwidth and an easy access profile to an external memory, and has a double high level MPEG-2, a single high level MPEG-2, and at least a dual main level MPEG-2. Can be decoded. An embodiment using a reference frame that is less compressed in a vector prediction image (eg P-image) used recursively than in a vector prediction image (eg B-image) used non-recursively (8) It can also be used without accompanying. The advantage is that the memory access width can be reduced and the integrated buffer (8) is not required. However, the drawback is that the memory access profile to the external memory is not simplified.

  FIG. 5 schematically shows a video decoder according to the second embodiment of the present invention. This second embodiment is preferable when the buffer size must be reduced as compared with the first embodiment described above. Frame prediction is buffered. The buffer 8 contains compressed video data that is decompressed by the decompressor 7 prior to construction of the predicted frame. In one embodiment, the amount of buffer memory to decode is equal to (2 × 128 + 16) × 1920 × 1.5 × 2 × 8 / C (bits). Here, C is a compression ratio. For example, the size of the buffer 8 can be limited to 3.3 Mbit instead of about 12.6 Mbit when a compression ratio of 4: 1 is applied for the reference frame. The general concept is illustrated in FIG.

  Another advantage of this second embodiment is that since only one reference frame is required, the reference frame in the P-picture can be compressed with a smaller compression ratio than the reference frame in the B-picture. With the same amount of memory, the reference frame in the P-picture can have up to 2 times lower compression than the reference frame in the B-picture.

  A preferred general concept of the second embodiment is shown in FIG. For example, a compression ratio of 4: 1 for both reference frames in the B-picture and a minimum of 2: 1 compression for the reference frames in the P-picture. Note that when extensible compression is applied, it is possible to store a reference frame that is compressed only at a 2: 1 ratio. The advantage is that continuous predicted P-pictures are processed with little compression and thus with few defects. B-pictures are discontinuously predicted and can therefore be greatly compromised, and thus may have a large compression ratio. The disadvantage is that the memory footprint is twice as large in the compressed reference frame.

  For B-pictures and P-pictures, a similar calculation yields a 2.1 Mbit buffer memory with 6: 1 and 3: 1 compression ratios, respectively. Figures 10 and 11 show different implementation options in the same basic concept.

  The present invention also provides a method for simplifying a memory access profile and reducing dynamics in the memory access bandwidth to a reference frame memory of a video decoder, wherein the compressed video data is subjected to variable length decoding ( VLD), inversely scanning, inverse quantization, and inverse discrete cosine transform (IDCT) decoding of the inner coded image, inner coded macroblock, and inner coded data information, and vector predictive image and macroblock Performing motion compensation to decode, combining decoded inner coded macroblock, decoded inner coded data information, and motion compensated vector prediction macroblock into reference frame data or output frame data When Compressing the reference frame data using an extensible compression method, and at least a vertical opening (range) of the motion vector and a horizontal row (slice) of macroblocks in a video line per reference frame as buffer means Temporarily storing and decompressing the reference frame data so that the means for motion compensation (MC) can reconstruct a vector prediction image and a macroblock using the decompressed reference frame data And a method of outputting decoded image data.

  In one embodiment, the method comprises the steps of storing the compressed reference frame data in an external memory means, recovering the compressed reference frame data from the external memory means, and the recovered reference A step of decompressing frame data; a step of temporarily storing the decompressed reference frame data in the buffer means; and a step of reconstructing a vector prediction image and a macroblock using the decompressed reference frame data And further.

  In another embodiment, the method includes storing the compressed reference frame data in an external memory means, retrieving the compressed reference frame data from the external memory means, and the compressed reference frame. Temporarily storing frame data in the buffer means, decompressing the temporarily stored reference frame data, and reconstructing a vector prediction image and a macroblock using the decompressed reference frame data. Further comprising the steps of configuring.

  In another embodiment, the method is used as a reference frame by compressing one reference frame at a first compression rate and a second compression rate, and a reference frame compressed at the first compression rate. And reconstructing a vector prediction image that is not used as a reference frame by the reference frame compressed at the second compression rate.

  In yet another embodiment, the method includes compressing one reference frame at a first compression rate and a second compression rate, and a P-picture by the reference frame compressed at the first compression rate. And reconstructing a B-picture with a reference frame compressed using the second compression ratio.

  In still another embodiment of the method, the first compression rate is less than or equal to the second compression rate.

  In yet another embodiment of the method, the first compression rate is half of the second compression rate.

  In yet another embodiment of the method, the first compression ratio is 2: 1 and the second compression ratio is 4: 1.

  In yet another embodiment of the method, the first compression ratio is 3: 1 and the second compression ratio is 6: 1.

  In yet another embodiment of the method, the first compression ratio is 4: 1 and the second compression ratio is 8: 1.

  In yet another embodiment, the method includes data for a reference frame compressed using the second compression rate, and data for the same reference frame compressed using the first compression rate. And directly storing only the data for the reference frame at the first compression rate in the external memory means.

  In yet another embodiment, the method temporarily and hierarchically stores the compressed reference frame data for the reference frame compressed using the first compression ratio in the external memory means. The stored first sub-image includes top-level data representing the same reference frame compressed using the second compression rate greater than the first compression rate, and the second sub-image Further includes the step of including the least significant data so that both sub-images together represent data for a reference frame compressed using the first compression ratio.

  In yet another embodiment of the method, the second compression rate is twice the first compression rate.

  In yet another embodiment, the method temporarily and hierarchically stores the compressed reference frame data for the reference frame compressed using the first compression ratio in the external memory means. The stored first sub-image includes top-level data representing the same reference frame compressed using the second compression rate greater than the first compression rate, and the second sub-image Further includes the step of including the least significant data so that both sub-images together represent data for a reference frame compressed using the first compression ratio.

  In yet another embodiment, the method includes the buffer means being an integrated memory buffer.

  As mentioned above, although the novel characteristic used as the basis of this invention applied to preferable embodiment of this invention was illustrated and demonstrated, it was pointed out by those skilled in the art, without departing from the thought of this invention. It goes without saying that various omissions, substitutions, and changes can be made in the forms and details of these and the operations thereof. For example, it goes without saying that all combinations of elements and / or methods that achieve the same result by performing substantially the same function in substantially the same way fall within the scope of the invention. In addition, the structures and / or elements and / or method steps illustrated and / or described in connection with any disclosed form or embodiment of the invention are generally considered as design choices, and any It will be appreciated that other disclosed, described, or suggested forms or embodiments may be incorporated. Accordingly, the invention is limited only as described by the claims appended hereto.

An initialization and update method in the FIFO is shown. 2 further illustrates the initialization and update method in the FIFO according to FIG. 3 further illustrates the initialization and update method in the FIFO according to FIGS. 1 is a schematic diagram of a video decoder according to a first embodiment of the present invention. Fig. 3 discloses a schematic diagram of a video decoder according to a second embodiment of the present invention. It shows how the size of the reference frame is reduced by the compression rate and the memory access bandwidth. Fig. 4 shows a preferred embodiment in which the reference frame for the P-picture is compressed twice as much as the reference frame for the B-picture. 3 shows a general concept of the second embodiment. 2 shows a preferred general concept of the second embodiment. Fig. 8 shows a first other implementation option of the second embodiment. Fig. 8 shows a second other implementation option of the second embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Compressed data memory 2 Variable length decoder 3 Inverse scanner 4 Inverse quantizer 5 Inverse discrete cosine transform 6 Video data compressor 7 Video data decompressor 8 Buffer 9 External memory 10 Means for motion compensation

Claims (54)

Means for variable length decoding of compressed video data;
-Means for inner coded image, inner coded macroblock, inverse scanning of inner coded delta information, inverse quantization, inverse discrete cosine transform decoding;
-Means for motion compensation for decoding vector prediction images and macroblocks;
Means for combining the decoded inner coded macroblock, the decoded inner coded data information, and the motion compensated vector prediction macroblock into reference frame data or output frame data;
A video decoder comprising:
Means for compressing the reference frame data using an extensible compression method;
Buffer means for temporarily storing at least a vertical opening of the motion vector and a row of macroblocks in a video line per reference frame;
Means for decompressing the reference frame data so that the means for motion compensation can reconstruct vector prediction images and macroblocks using the decompressed reference frame data;
Means for outputting decoded image data;
A video decoder, further comprising: Means for storing the compressed reference frame data in external memory means;
Means for recovering the compressed reference frame data from the external memory means;
Further comprising
The means for decompressing the reference frame data is arranged to decompress the recovered reference frame data;
-Further comprising means for temporarily storing the decompressed reference frame data in the buffer means;
The means for motion compensation is arranged to reconstruct vector predicted images and macroblocks using the decompressed reference frame data;
The video decoder according to claim 1, wherein: Means for storing the compressed reference frame data in external memory means;
Means for recovering the compressed reference frame data from the external memory means;
Means for temporarily storing the compressed reference frame data in the buffer means;
Further comprising
The means for decompressing reference frame data is arranged to decompress the temporarily stored reference frame data;
The means for motion compensation is arranged to reconstruct vector predicted images and macroblocks using the decompressed reference frame data;
The video decoder according to claim 1, wherein:   The means for retrieving the reference frame data from the external memory is arranged to retrieve the data by repeated burst accesses of a predetermined number of consecutive data words, and the buffer means is a first-in first-out FIFO mode 4. The arrangement according to claim 2, characterized in that the preferred granularity of FIFO elements in the buffer is one row of macroblocks extending over the entire horizontal range of the image. Video decoder.   5. The size of one FIFO element, expressed in bytes, is arranged exactly equal to the number of bytes obtained by an integer burst access number for retrieving data from the external memory means. Video decoder.   The means for compressing reference frame data is arranged to compress one reference frame at a first compression rate and a second compression rate, and the decoder compresses at the first compression rate. A P-image is reconstructed with a reference frame that has been converted, and a B-image is reconstructed with a reference frame that has been compressed using the second compression ratio. The video decoder according to 2 or 3.   The video decoder according to claim 6, wherein the first compression rate is equal to or lower than the second compression rate.   The video decoder according to claim 7, wherein the first compression rate is half of the second compression rate.   The video decoder according to claim 8, wherein the first compression ratio is 2: 1 and the second compression ratio is 4: 1.   9. The video decoder according to claim 8, wherein the first compression ratio is 3: 1 and the second compression ratio is 6: 1.   The video decoder according to claim 8, wherein the first compression ratio is 4: 1 and the second compression ratio is 8: 1.   The decoder arranges to obtain data for a reference frame compressed using the second compression rate directly from data for the same reference frame compressed using the first compression rate. The means for temporarily storing the compressed reference frame data in the external memory means is arranged to store only the data for the reference frame at the first compression rate. The video decoder according to claim 6, wherein   The means for storing the compressed reference frame data in the external memory means is arranged to hierarchically store data for the reference frame that is compressed using the first compression rate. Whereby the stored first sub-image includes top-level data representing the same reference frame compressed using the second compression rate greater than the first compression rate and a second 13. The sub-image according to claim 12, wherein the sub-image includes least significant data so that both sub-images together represent data for a reference frame compressed using the first compression rate. Video decoder.   14. The video decoder according to claim 13, wherein the second compression rate is twice the first compression rate.   Said buffer means is at least (vertical range of motion vectors, spanning the entire horizontal size of the image + one row of macroblocks) x maximum number of pixels per line x maximum number of reference frames x number of bytes per pixel x bytes The video decoder according to claim 1, wherein the video decoder has a bit capacity of the number of bits per unit.   In MPEG-1 video or MPEG-2 video, the buffer means is at least (vertical range of motion vectors + macro that covers the entire horizontal size of an image) per reference frame of a minimum of one reference frame and a maximum of two reference frames. The horizontal capacity of a block) × the maximum number of pixels per line × the maximum number of reference frames × the number of bytes per pixel × the number of bits per byte. The video decoder described.   17. A video decoder according to claim 1, wherein the buffer means is an integrated memory buffer. A method for simplifying a memory access profile and reducing dynamics in memory access bandwidth to a reference frame memory of a video decoder, comprising:
-Variable length decoding compressed video data;
-Decoding inner coded image, inner coded macroblock, inner coded data information by inverse scanning, inverse quantization, inverse discrete cosine transform decoding;
Performing motion compensation for decoding vector predicted images and macroblocks;
Combining the decoded inner coded macroblock, the decoded inner coded data information, and the motion compensated vector prediction macroblock into reference frame data or output frame data;
Compressing the reference frame data using an extensible compression method;
Temporarily storing at least the vertical aperture of the motion vector and the horizontal row of macroblocks in the video line per reference frame in the buffer means;
-Decompressing the reference frame data so that the means for motion compensation can reconstruct a vector prediction image and macroblocks using the decompressed reference frame data;
Outputting the decoded image data;
A method comprising: -Storing the compressed reference frame data in an external memory means;
Retrieving the compressed reference frame data from the external memory means;
-Decompressing the recovered reference frame data;
Temporarily storing the decompressed reference frame data in the buffer means;
Reconstructing a vector prediction image and a macroblock using the decompressed reference frame data;
The method of claim 18, further comprising: -Storing the compressed reference frame data in an external memory means;
Retrieving the compressed reference frame data from the external memory means;
Temporarily storing the compressed reference frame data in the buffer means;
-Decompressing the temporarily stored reference frame data;
Reconstructing a vector prediction image and a macroblock using the decompressed reference frame data;
The method of claim 18, further comprising: Retrieving the reference frame data from the external memory by repeated burst accesses of a predetermined number of consecutive data words;
Accessing the buffer means in a first-in first-out FIFO mode, wherein the preferred granularity of the FIFO elements in the buffer is one row of macroblocks extending over the entire horizontal range of the image;
The method according to claim 19 or 20, further comprising:   -Further comprising setting the size of one FIFO element, expressed in bytes, exactly equal to the number of bytes obtained by an integer number of burst accesses for retrieving data from said external memory means Item 22. The method according to Item 21. -Compressing one reference frame at a first compression rate and a second compression rate;
-A vector prediction image used as a reference frame by the reference frame compressed at the first compression rate and not used as a reference frame by the reference frame compressed at the second compression rate Reconfiguring the
The method according to claim 19 or 20, further comprising: -Compressing one reference frame at a first compression rate and a second compression rate;
Reconstructing a P-image with a reference frame compressed at the first compression rate and reconstructing a B-image with a reference frame compressed using the second compression rate;
24. The method according to claim 19, 20 or 23, further comprising:   25. A method according to claim 23 or 24, wherein the first compression rate is less than or equal to the second compression rate.   25. A method according to claim 23 or 24, characterized in that the first compression rate is half of the second compression rate.   27. The method of claim 26, wherein the first compression ratio is 2: 1 and the second compression ratio is 4: 1.   27. The method of claim 26, wherein the first compression ratio is 3: 1 and the second compression ratio is 6: 1.   27. The method of claim 26, wherein the first compression ratio is 4: 1 and the second compression ratio is 8: 1. Obtaining data for a reference frame compressed using the second compression rate directly from data for the same reference frame compressed using the first compression rate;
-Temporarily storing in the external memory means only data for the reference frame at the first compression rate;
26. The method according to any one of claims 23 to 25, further comprising:   Storing the compressed reference frame data for the reference frame compressed using the first compression ratio temporarily and hierarchically in the external memory means, thereby The image includes top data representing the same reference frame compressed using the second compression ratio greater than the first compression ratio, and the second sub-image includes the bottom data, thereby The method of claim 30, further comprising causing both sub-images to represent data for a reference frame that is both compressed using the first compression ratio.   32. The method of claim 31, wherein the second compression rate is twice the first compression rate.   -Storing the compressed reference frame data for the reference frame to be compressed using the first compression rate temporarily and hierarchically in the external memory means; The sub-image includes top data representing the same reference frame compressed using the second compression ratio greater than the first compression ratio, and the second sub-image includes the bottom data, 33. The method of any one of claims 30 to 32, further comprising: causing both sub-images to represent data for a reference frame that is both compressed using the first compression ratio. The method according to item.   34. A method according to any one of claims 18 to 33, characterized in that the buffer means is an integrated memory buffer. Means for variable length decoding of compressed video data;
-Means for inner coded image, inner coded macroblock, inverse scanning of inner coded data information, inverse quantization, inverse discrete cosine transform decoding;
-Means for motion compensation decoding vector predictive pictures and macroblocks;
Means for combining the decoded inner coded macroblock, the decoded inner coded data information, and the motion compensated vector prediction macroblock into reference frame data or output frame data;
A video decoder comprising:
-Further comprising means for compressing the reference frame data using an extensible compression method;
The means for compressing reference frame data is adapted to compress one reference frame at a first compression rate and a second compression rate;
Means for decompressing the reference frame data so that said means for motion compensation can reconstruct vector prediction images and macroblocks using said decompressed reference frame data;
Means for outputting decoded image data;
Further comprising
The decoder reconstructs a P-picture with reference frames compressed at the first compression rate, and reconstructs a B-picture with reference frames compressed using the second compression rate; It has become,
A video decoder.   36. The video decoder of claim 35, wherein the first compression rate is less than or equal to the second compression rate.   37. The video decoder of claim 36, wherein the first compression rate is half of the second compression rate.   38. The video decoder of claim 37, wherein the first compression ratio is 2: 1 and the second compression ratio is 4: 1.   38. The video decoder of claim 37, wherein the first compression ratio is 3: 1 and the second compression ratio is 6: 1.   38. The video decoder of claim 37, wherein the first compression ratio is 4: 1 and the second compression ratio is 8: 1.   The decoder arranges to obtain data for a reference frame compressed using the second compression rate directly from data for the same reference frame compressed using the first compression rate. The means for temporarily storing the compressed reference frame data in the external memory means is arranged to store only the data for the reference frame at the first compression rate. 36. Video decoder according to claim 35, characterized in that   The means for storing the compressed reference frame data in the external memory means is arranged to hierarchically store data for the reference frame that is compressed using the first compression rate. Whereby the stored first sub-image includes top-level data representing the same reference frame compressed using the second compression rate greater than the first compression rate and a second 42. The method of claim 41, wherein the sub-image includes least significant data so that both sub-images together represent data for a reference frame compressed using the first compression ratio. Video decoder.   43. The video decoder of claim 42, wherein the second compression rate is twice the first compression rate. A method for reducing memory access bandwidth to a reference frame memory of a video decoder, comprising:
-Variable length decoding compressed video data;
-Decoding inner coded image, inner coded macroblock, inner coded data information by inverse scanning, inverse quantization, inverse discrete cosine transform decoding;
Performing motion compensation for decoding vector predicted images and macroblocks;
Combining the decoded inner coded macroblock, the decoded inner coded data information, and the motion compensated vector prediction macroblock into reference frame data or output frame data;
Compressing the reference frame data using an extensible compression method;
-Compressing one reference frame at a first compression rate and a second compression rate;
-Decompressing the reference frame data so that the means for motion compensation can reconstruct a vector prediction image and macroblocks using the decompressed reference frame data;
-A vector prediction image used as a reference frame by the reference frame compressed at the first compression rate and not used as a reference frame by the reference frame compressed at the second compression rate Reconfiguring the
Outputting the decoded image data;
A method comprising the steps of: -Compressing one reference frame at a first compression rate and a second compression rate;
Reconstructing a P-image with a reference frame compressed at the first compression rate and reconstructing a B-image with a reference frame compressed using the second compression rate;
45. The method of claim 44, further comprising:   46. A method according to claim 44 or 45, wherein the first compression rate is less than or equal to the second compression rate.   The method of claim 46, wherein the first compression ratio is half of the second compression ratio.   48. The method of claim 47, wherein the first compression ratio is 2: 1 and the second compression ratio is 4: 1.   48. The method of claim 47, wherein the first compression ratio is 3: 1 and the second compression ratio is 6: 1.   48. The method of claim 47, wherein the first compression ratio is 4: 1 and the second compression ratio is 8: 1. Obtaining data for a reference frame compressed using the second compression rate directly from data for the same reference frame compressed using the first compression rate;
-Temporarily storing in the external memory means only data for the reference frame at the first compression rate;
47. A method according to any one of claims 44 to 46, further comprising:   Storing the compressed reference frame data for the reference frame compressed using the first compression ratio temporarily and hierarchically in the external memory means, thereby The image includes top data representing the same reference frame compressed using the second compression ratio greater than the first compression ratio, and the second sub-image includes the bottom data, thereby 52. The method of claim 51, further comprising causing both sub-images to represent data for a reference frame that is both compressed using the first compression ratio.   53. The method of claim 52, wherein the second compression rate is twice the first compression rate.   -Storing the compressed reference frame data for the reference frame to be compressed using the first compression rate temporarily and hierarchically in the external memory means; The sub-image includes top data representing the same reference frame compressed using the second compression ratio greater than the first compression ratio, and the second sub-image includes the bottom data, 54. The method of any one of claims 51 to 53, further comprising: causing both sub-images to represent data for a reference frame that is both compressed using the first compression ratio. The method according to item.

Images