KR100556341B1 - Vedeo decoder system having reduced memory bandwidth - Google Patents

Abstract

The present invention relates to a video decoder system for multimedia signal processing, and more particularly, to provide a video decoder system having a reduced memory bandwidth capable of supporting an H.264 HD video decoder by embedding a separate dual bus and a memory controller. do.

In the present invention, one bus performs a read / write operation with an external memory for inter mode prediction using a memory controller belonging to the bus, and the other bus uses another memory controller belonging to the bus. Make other blocks read / write with external memory. In addition, the memory controller belonging to the inter-mode prediction is composed of a 96-bit data bus, and the memory controller for reading / writing an external block with a 32-bit data bus is configured.

Memory Bandwidth, External Memory, Dual Bus, Dual Memory Controller, Inter Mode

Description

Video decoder system with reduced memory bandwidth {Vedeo decoder system having reduced memory bandwidth}

1 is an overall block diagram of a video decoder system according to the present invention.

2A and 2B illustrate a write path / read path when decoding one frame.

FIG. 3A illustrates a 4x4 block word structure of a reference frame memory, and FIG. 3B illustrates a 4x4 block word structure of a frame memory.

4 illustrates a read / write path when FMO is activated / deactivated in the DF block.

FIG. 5 is a diagram for explaining conventional quarter sample luma interpolation in inter mode prediction. FIG.

Explanation of symbols on the main parts of the drawings

100: video decoder 200a, 200b: external memory

300a, 300b: dual bus 400a, 400b: dual memory controller

The present invention relates to a video decoder system for multimedia signal processing, and more particularly, to provide a video decoder system having a reduced memory bandwidth capable of supporting an H.264 HD video decoder by embedding a separate dual bus and a memory controller. It is.

When video or sound is converted to digital data, the amount of data is so large that it takes up a lot of storage space if not compressed, which is inefficient. Therefore, a compression technique for reducing information of digital data is required, and the compression technique can improve storage utilization and transmission function through a network. For example, image compression methods include GIF and JPEG formats, and video and sound compression methods include MPEG, H.263, and H.264.

Among them, H.264 is a new video compression coding standard developed by the Joint Video Team (JVT) jointly formed by the ITU and the International Organization for Standardization / ISO (IEC). 14496 10 Advanced Video Coding 'is a video compression standard that considers error robustness and network-friendly method in consideration of rapidly changing wireless environment and Internet environment.

In the H.264 video decoder system, coding or decoding of image data is performed in units of macro blocks (MB), and in the case of a memory, an external memory for storing an input video stream and frames for motion compensation is present. .

The input video stream is divided into a series of pictures or groups of frames called a group of pictures (GOP). Each GOP includes a number of pictures, and each picture is divided into a number of slices. Each slice contains a number of macroblocks, each macroblock having four 8x8 brightness blocks and two 8x8 chrominance blocks. And two kinds of pictures may appear in one GOP, one is an intra mode picture (or I-picture) and the other is a predictive motion compensated picture (P-picture). In an I-picture, all macroblocks are in intra mode and decoded without considering motion compensation. The P-picture is compressed using one previous frame, and each macroblock can be decoded in an INTER mode using motion compensation.

In implementing the existing H.264 video decoder system, when the profile is used as a single bus in the main profile as well as the baseline, the memory bandwidth is enormously required. That is, in order to perform the video decoding process, access to an external memory including a frame memory and a reference frame memory, which stores the entire frame, is required. As the current or previous frame memory is frequently accessed during the video restoration process, as in the prior art, If a single system bus is used, the resulting system bus is also very inefficient.

External frame memory access in an H.264 video decoder system includes (1) memory access for variable-length decoding, and (2) multiple criteria for inter-mode prediction for each block size (16x16, 8x8, 8x16, 16x8, 4x4). Frame memory access, (3) memory access of neighboring macroblocks for intra mode prediction, and (4) memory access for deblocking filtering. In particular, among such memory accesses, the memory bandwidth when performing inter mode prediction is larger than the sum of the memory bandwidths of the remaining blocks.

FIG. 5 is a diagram illustrating quarter sample luma interpolation in inter mode prediction. FIG.

Referring to FIG. 5, luma prediction in inter mode prediction is performed by applying a six-tap filter (1, -5,20,20, -5,1) to perform quarter-sample luma interpolation, for example, G, H. If the motion vector is located between M, N pixels, then A, B, c, d, e, f, g, h, I, j, k, m, n, p, q, r C, D, E, F, I, J, K, L, M, N, P, Q, R, S, T, U pixels are required. As a result, it can be predicted that the amount of data to be read from the external reference frame is significant. The H.264 decoder may have a motion vector in units of 4 × 4 blocks in inter mode prediction, and in the worst case, access to an external frame memory may be increased by performing quarter sample luma interpolation in units of 4 × 4 blocks.

To support high resolution (HD), suppose the encoder decodes one frame into 4x4 blocks with motion vectors at 30 frames per second (where the decoder has 32-bit access) to perform luma sample interpolation from the reference picture. The following memory bandwidth is required. At this time, the data transfer is accessed using the burst mode of the SDRAM.

Luma sample interpolation of a 4x4 block requires 3 words * 9 lines (the description of 3 words, 9 lines will be described later). If you define 10 cycles to read lines one by one in burst mode, it takes 90 cycles. Since one macroblock exists in 16 as 4x4 blocks, it becomes 90 cycles * 16. Since HD has 120 * 67.5 macroblocks, it is calculated as 90 cycles * 16 * 120 * 67.5. In addition, if decoding is performed at 30 frames per second, a memory bandwidth of 90 cycles * 16 * 120 * 67.5 cycles * 30 (about 300 MHz) is required. Therefore, it can be seen that a high performance chip having a memory bandwidth of at least about 500 MHz is required to support the H.264 HD class decoder.

As described above, the existing H.264 video decoder system requires a large amount of data access, and as a result, memory bandwidth requirements (ie, data storage and retrieval speed) are pointed out as a big problem.

In order to solve this problem, the first method is an extension of the data bus (for example, 32 bits to 64 bits). When the memory bandwidth is calculated using the data bus extension method, the luma sample interpolation of the 4x4 block requires 2 words * 9 lines. If you define 8 cycles to read one line in burst mode (burst 2 access), 72 cycles are required. One macroblock is 72 cycles * 16 because there are 16 4x4 blocks. HD requires 120 x 67.5 macros, so 72 x 16 * 120 * 67.5 cycles are required. In addition, if decoding is performed at 30 frames per second, it takes 72 * 16 * 120 * 67.5 cycles * 30 (about 250 MHz). A large amount of memory bandwidth is still required, and even if the data bus is extended to 128 bits, no reduction in memory bandwidth can be expected.

The second is to simply split the data bus into two using two memory controllers. That is, the decoding read / write path for inter mode prediction and the read / write path with external memory for other blocks are separated. However, this method can also show almost similar performance as the first method.

Therefore, a video decoder system capable of supporting an H.264 HD video decoder even at a low memory bandwidth is required.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a separate bus and memory controller that controls access to an external memory during inter-mode prediction, thereby minimizing the load on each block.

Accordingly, a video decoder system capable of supporting an H.264 HD video decoder even at a low memory bandwidth is provided.

A video decoder system according to the present invention comprises: a video decoder comprising a variable length decoding block, an inverse discrete cosine transform / quantization block, an inter block, an intra block, a deblocking filter block; An external memory for storing decoded data using the video decoder or outputting stored data for decoding, the external memory comprising a frame memory and a reference frame memory; And a first bus, a first memory controller, a second bus, and a second memory controller for controlling access to the external memory, wherein, when decoding one frame, the inter block is connected to the first bus and the first memory controller. Inter-mode prediction is performed by reading data from the reference frame memory, and subblocks except for the inter-block read data from the frame memory through the second bus and a second memory controller to perform corresponding operations. It is done.

Hereinafter, the present invention will be described with reference to the accompanying drawings, FIGS. 1 to 3.

1 is an overall block diagram of a video decoder system according to the present invention, which includes a video decoder 100, external memories 200a and 200b, dual buses 300a and 300b connecting respective components, and dual for external memory control. Memory controllers 400a and 400b.

The video decoder 100 performs restoration of the input video stream, and includes a variable length decoding (VLD) block 102 and an inverse discrete cosine transform / quantization (IDCT / Q) block. 104, an intra block 106, an inter block 108, and a deblocking filter (DF) block 110.

The VLD block 102 is responsible for bit stream parsing input by context-adaptive variable length decoding (CAVLD) coding, and the IDCT / Q block 104 is a frequency in pixels. Time domain transform and inverse quantization on the inputted quantized pixel data, the intra block 106 performs decoding on the current pixel through correlation with neighboring macroblocks in the intra frame picture, and 108 is to decode the inter frame by comparing with the previous image in the inter-frame picture, DF block 110 is designed to reduce the blocking phenomenon between macroblocks that may occur when the video stream is reconstructed do.

The external memories 200a and 200b store data decoded using the video decoder 100 or output data stored for decoding, and are composed of a reference frame memory 200a and a frame memory 200b.

The dual buses 300a and 300b and the dual memory controllers 400a and 400b control access to the external memories 200a and 200b.

In decoding one frame, the inter block 108 reads data from the reference frame memory 200a through the bus 300a and the memory controller 400a to perform inter mode prediction, and the inter frame block 108 The subblocks (VLD block, IDCT / Q block, etc.) except for read data from the frame memory 200b through the bus 300b and the memory controller 400b to perform corresponding operations.

In summary, the present invention provides the dual buses 300a and 300b and the dual memory controllers 400a and 400b so that one bus 300a uses the memory controller 400a to provide an external memory 200a, for inter mode prediction. The read / write operation with the 200b is performed, and the other bus 300b uses the memory controller 400b to allow other blocks to read / write with the external memories 200a and 200b. In addition, the memory controller 400a belonging to the inter mode prediction is composed of a 96-bit data bus, and the memory controller 400b for reading / writing an external memory with a 32-bit data bus is configured. .

For inter mode prediction, if the data bus for the reference frame memory 200a is configured to 96 bits, access time is reduced to one word while three words and two words are required for 32-bit and 64-bit storage, respectively. . Also, if the memory controller supports "window access" to read 9 lines at a time, the following memory bandwidth is required. Here, "window access" during the operation of the memory controller means that the memory controller performs a window operation instead of a burst operation. When one macroblock is not composed of consecutive column addresses in the frame memory map, one macroblock is selected. For reading, non-contiguous column addresses can be defined as operating using SDRAM multi-bank, RAS, and CAS timings as contiguous addresses.

To calculate the memory bandwidth in this case, luma sample interpolation of 4x4 blocks would require 1 word * 9 lines. If you access in windowed mode instead of burst mode, you can read 9 lines in about 18 cycles. One macroblock is 18 cycles * 16 since there are 16 4x4 blocks. HD requires 120 * 67.5 macros, which requires 18 * 16 * 120 * 67.5 cycles. Also, if decoding is performed at 30 frames per second, it takes 18 * 16 * 120 * 67.5 cycles * 30 (about 70MHz).

It can be seen that the memory bandwidth is significantly lower than the conventional methods such as bus expansion and bus simple separation. As a result, H.264 video decoders can operate below 100MHz to support high-definition (HD). In the above calculation, chroma sample interpolation may operate using a memory controller that performs luma sample interpolation or a memory controller belonging to a system bus, and performs operations according to various application requirements in system configuration.

Basic read / write path

2A and 2B illustrate a read path for determining a frame memory or a reference frame memory according to a write path and each sub-block (VLD, Intra, Inter, etc.) and performing a read operation when decoding one frame.

First, referring to FIG. 2A, when the write path is decoded one frame, the final operation block is the DF block 110. The DF block 110 includes an engine (not shown) that performs deblocking filtering, and a DMA and bus 300b belonging to a direct memory access (DMA) (not shown) for storing data in an external memory, that is, the bus 300a. It consists of DMA belonging to). At this time, the DMA operation is as follows.

When the operation of the DF block 110 ends, the DMA belonging to the bus 300a stores data in the reference frame memory 200a through the memory controller 400a. In addition, the DMA belonging to the bus 300b stores data in the frame memory 200b using the memory controller 300b.

The read path uses a frame memory or a reference frame memory according to the operation of each subblock. In FIG. 2B, sub-blocks such as the VLD block 102 and the intra block 106 read data from the frame memory 200b using the bus 300b to perform operations corresponding to the respective blocks (step 1). The inter block 108 reads data from the reference frame memory 200a using the bus 300a to perform inter mode prediction (step 2). In this case, when performing inter mode prediction, the chroma prediction may use the frame memory 200b in consideration of memory bandwidth. After all, luma prediction always uses reference frame memory 200a, and chroma prediction uses reference frame memory 200a or frame memory 200b depending on the application.

Read / write path at the end of decoding one macroblock

If the flag of the DF block 110 is activated after the inter mode prediction or the intra mode prediction, the video decoder 100 stores the data in the external reference frame memory 200a after the deblocking filtering. If the flag of the DF block 110 is not activated, data is stored in the external frame memory 200b immediately after inter or intra mode prediction. That is, reference frame data for inter mode prediction is stored in the reference frame memory 200a, and predicted results obtained by performing inter or intra mode prediction are stored in the frame memory 200b.

First, for inter mode prediction, the reference frame reference is read from the reference frame memory 200a using the memory controller 400a via the bus 300a as shown in step 2 of FIG. 2B. Second, the predicted result after inter or intra prediction is stored in the frame memory 200b using the memory controller 400b through the bus 300b as shown in step 1 of FIG. 2A. Here, the current decoded macroblock is stored in the frame memory 200b through the memory controller 400b and simultaneously in the reference frame memory 200a through the memory controller 400a. ).

When decoding a macroblock, when the macroblock is in the inter mode, the data stored in the reference frame memory 200a is read using the memory controller 400a through the bus 300a, and conversely, the macroblock is in the intra mode. In this case, the data stored in the frame memory 200b is read using the memory controller 400b. Therefore, since the memory bandwidth of the memory controller 400a does not affect the memory bandwidth of the memory controller 400b when performing inter mode prediction, the high resolution HD may be sufficiently supported even at the low frequency mentioned above.

Mapping of Reference Frame Memory

3A illustrates a 4x4 block word structure of the reference frame memory 200a belonging to the memory controller 400a, and FIG. 3B illustrates a 4x4 block word structure of the frame memory 200b belonging to the memory controller 400b.

In order to perform luma sample interpolation of a 4x4 block, if the motion vector is located between 0, 1, 4, and 5 pixels as shown in FIG. 4A, the luma sample interpolation requires l4, l0, 0, 1, Pixels of 2 and 3 are needed in one word (as already mentioned, because we use a 6-tap filter). If the motion vector is located between 3, 7, r0, and r1 pixels, the horizontal pixels required for interpolation are 1, 2, 3, r0, r4, r8 pixels. Therefore, the horizontal axis stores l4, l0 adjacent to the left, r0, r4, r8 pixels adjacent to the right, and 0, 1, 2, 3 pixels decoded in the current 4x4 block as one word in the reference frame memory. If you save 32 bits, 3 words are read, but if you save them in one word as above, you can get 1/3 of the memory controller.

Next, consider the portrait orientation. If the motion vector is located between 0, 1, 4, and 5 pixels, the pixels needed to perform luma sample interpolation on the vertical axis are pixels u5, u0 of the adjacent block above and 0, 4, 8, which are pixels decoded in the current 4x4 block. 12 pixels are required. If the motion vector is located between 12, 13, d0, and d4 pixels, 4, 8, 12, d0, d4, and d8 pixels are required among the pixels for which luma sample interpolation is required in the vertical direction. Therefore, d0, d4, d8 adjacent to the bottom of the vertical axis Luma sample interpolation is performed using 4, 8, and 12 pixels decoded in the current 4x4 block. As a result, 9 lines are required on the vertical axis to perform luma sample interpolation for a 4x4 block.

Read / write path when FMO is enabled / disabled in DF block

One thing to note is that the H.264 video decoder supports flexible macroblock ordering (FMO) as well as raster scan. If the flag of the DF block 110 is activated during H.264 video decoding, no problem occurs. If the flag of the DF block is deactivated, the following sequence should be followed. Here, the flag is extracted by using a distribution of inverse quantization coefficients of dequantized image data and a motion vector indicating a difference between a previous frame and a current frame, and information indicating whether loop filtering of the decoded image is necessary.

As described above, the inter mode prediction or the intra mode prediction is performed and then stored in the frame memory 200b through the memory controller 400b. When one frame decoding is finished, the DF block 110 reads data from the frame memory 200b using the memory controller 400b in a raster scan method even if the flag is inactivated (step 1 of FIG. 4).

In addition, when one macroblock is read and then the second macroblock is read, in combination with the previously read first macroblock, a word is formed into a structure as shown in FIG. 3 by using the memory controller 400a. Save to the first macroblock position of 200a) (step 2 of Fig. 4).

Subsequently, the third macroblock is read from the frame memory 200b through the memory controller 400b and combined with the first macroblock and the second macroblock previously obtained as shown in FIG. 3, and then the reference is performed using the memory controller 400b. The second macroblock of the frame memory 200a is stored. In order to construct a memory map like the reference frame memory of FIG. 3, a memory capable of storing two macroblocks must be present therein. As described above, by configuring a single word structure in 96 bits in the reference frame memory, memory access time can be reduced when performing inter mode prediction.

As described above, according to the present invention, by providing a separate bus and memory controller that controls access to external memory during inter-mode prediction, the load on each block is minimized, and H.264 HD video decoder at less than 100 MHz It has the advantage that it can provide a video decoder system that supports.

The video decoder system according to the present invention can decode DVD-level videos without difficulty even over the Internet, and can be expected to be useful for content services such as HD video on demand (VOD), a receiver for digital broadcasting, and a real-time video call. It can be applied to various applications.

Those skilled in the art will appreciate that various changes and modifications can be made without departing from the spirit of the present invention. Therefore, the technical scope of the present invention should not be limited to the contents described in the embodiments, but should be defined by the claims.

Claims (7)

A video decoder comprising a variable length decoding block, an inverse discrete cosine transform / quantization block, an inter block, an intra block, a deblocking filter block; An external memory for storing decoded data using the video decoder or outputting stored data for decoding, the external memory comprising a frame memory and a reference frame memory; A first bus, a first memory controller, a second bus, and a second memory controller controlling access to the external memory; In decoding one frame, the inter block performs inter-mode prediction by reading data from the reference frame memory through the first bus and the first memory controller, and subblocks except for the inter block include the second bus and the second bus. 2. The video decoder system of claim 1, wherein the video decoder system reads data from the frame memory and performs corresponding operations. The method of claim 1, wherein the first memory controller controls access to the reference frame memory using a 96-bit data bus, and the second memory controller controls access to the frame memory using a 32-bit data bus. Video decoder system. The video decoder system of claim 1, wherein the predicted result value in the inter block or intra block is stored in the frame memory through the second bus and the second memory controller. The method of claim 1, wherein when performing luma sample interpolation in inter frame prediction, one block is formed by using two pixels of the left block and five pixels of the right block in the horizontal direction when constructing one block, and configuring the vertical direction. And a second line of the upper block and three lines of the lower block. The video decoder system of claim 1, wherein, in performing inter-frame prediction, when performing chroma sample interpolation, either a frame memory or a reference frame memory can be used. The video decoder system of claim 1, wherein in the deblocking filter block, one macroblock is configured by using a memory inside the deblocking filter block when storing decoded data in a reference frame memory when FMO is activated. The method of claim 1, wherein the first and second memory controllers access one macroblock with consecutive column addresses using multi-bank, RAS, and CAS timings. Video decoder system.

Images