KR101188453B1 - Apparatus and Method for memory access of video decoder - Google Patents

Abstract

Disclosed are a memory access apparatus and a method of a video decoder. The memory access device checks the size and the motion vector of each partition in the macroblock to perform motion compensation, and sets a checker and window for determining whether to use a window including one or more partitions according to the size of the partition, and sets the window. And a setting unit for setting the burst mode according to the size of and a access unit for obtaining the reference image data from the external memory in units of windows.

Description

Apparatus and Method for memory access of video decoder

The present invention relates to a video decoder, and more particularly, to a memory access apparatus and method of a video decoder.

When an image or sound is converted into digital data, the amount of digital data converted is quite large. If the digital data is not compressed, the digital data takes up a lot of storage space. Therefore, there is a need for a compression technique for reducing the amount of digital data, and by utilizing the compression technique, storage space utilization and transmission over a network can be improved. For example, image compression methods include GIF, JPEG, and video and audio compression methods such as MPEG, H.263, and H.264. Among these, H.264 technology is a new video compression coding standard developed by the Joint Video Team (JVT) jointly formed by the ITU and the International Organization for Standardization / ISO (IEC). Also called video coding, it is a video compression standard considering error robustness and network-friendly method in consideration of rapidly changing wireless environment and Internet environment.

In the H.264 video system, coding or decoding of image data is performed in units of macro blocks (MB), and there is an external memory for storing an input video stream and frames for motion compensation. The input video stream is divided into a series of pictures or groups of frames called a group of pictures (GOP). Each GOP includes a number of pictures, and each picture is divided into a number of slices. Each slice contains a number of macro blocks, each macro block having four 8x8 luminance blocks and two 8x8 chrominance blocks. In addition, two kinds of pictures may appear in one GOP, one is an intra mode picture (or I-picture) and the other is a predictive motion compensated picture (P-picture). In an I-picture, all macroblocks are in intra mode and decoded without considering motion compensation. The P-picture is compressed using one previous frame, and each macroblock can be decoded in an INTER mode using motion compensation.

In the video decoder, the memory access required in the motion compensation part and the display part occupies most of them. In the case of H.264, which has recently been proposed as a new international standard for video encoding, the motion estimation and compensation method is much more complicated than in the prior art, so the amount of memory access required for motion compensation is much higher than in the related art. Therefore, there is a need for an efficient memory access method in a video decoder.

The present invention minimizes the frequency of memory access required for motion compensation in a video decoder.

According to one aspect of the present invention, a memory access apparatus of a video decoder for motion compensation is disclosed.

The memory access device according to an embodiment of the present invention determines the size and motion vector of each partition in the macroblock to perform the motion compensation, and determines whether to use a window including one or more partitions according to the size of the partition. And a setting unit for setting the window, setting a burst mode according to the size of the window, and an access unit for obtaining reference image data from an external memory in units of the window.

Here, the reference image data is pixel data including at least one of luminance data and color difference data.

The interpolation unit may further include an interpolation unit performing interpolation on the reference image data when the motion vector represents a half pixel or a quarter pixel instead of an integer pixel.

Here, the setting unit sets the leftmost of the horizontal motion vectors as the starting point in the horizontal direction, the horizontal length corresponds to the number of horizontal pixel data to be read in one macro block, and the vertical length corresponds to a block belonging to the set horizontal length range. Set the window to include it.

Here, when the window is not used, the access unit obtains the reference image data in units of partitions.

According to another aspect of the invention, a method is disclosed in which a memory access device of a video decoder accesses a memory for motion compensation.

According to an embodiment of the present invention, a memory access method includes identifying a size and a motion vector of each partition in a macroblock to perform motion compensation, and determining whether to use a window including one or more partitions according to the size of the partition. And setting a window, setting a burst mode according to the size of the window, and acquiring reference image data from an external memory in units of the window.

Here, the reference image data is pixel data including at least one of luminance data and color difference data.

Here, if the motion vector represents a half pixel or a quarter pixel instead of an integer pixel, the method further includes interpolating the reference image data.

Here, the window sets the leftmost of the horizontal motion vectors as the starting point in the horizontal direction, the horizontal length corresponds to the number of horizontal pixel data to be read in one macro block, and the vertical length corresponds to a block within the set horizontal length range. It is set to include.

The method may further include obtaining the reference image data in units of partitions when the window is not used.

The present invention can increase memory access efficiency by minimizing the frequency of memory access required for motion compensation in a video decoder.

In addition, the present invention can minimize the amount of power consumed when the memory access by reducing the number of memory access.

1 illustrates various prediction modes of H.264.
2 is a configuration diagram schematically illustrating a configuration of a video decoder.
3 is a configuration diagram schematically illustrating a configuration of a memory access apparatus of a video decoder.
4 is a diagram illustrating a case where pixel data is separately read from an external memory.
5 illustrates window settings.
6 and 7 are diagrams for explaining the interpolation method.
8 is a diagram illustrating a structure of a memory access apparatus of a video decoder.
9 illustrates a memory map.
10 is a flowchart illustrating a method of accessing an external memory by a memory access device of a video decoder.

The present invention may be variously modified and have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail with reference to the accompanying drawings. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In addition, numerals used in the description of the present invention are merely an identifier for distinguishing one component from another.

Also, in this specification, when an element is referred to as being "connected" or "connected" with another element, the element may be directly connected or directly connected to the other element, It should be understood that, unless an opposite description is present, it may be connected or connected via another element in the middle.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, the same reference numerals will be used for the same means regardless of the reference numerals in order to facilitate the overall understanding.

1 illustrates various prediction modes of H.264.

Referring to FIG. 1, a macro block partition may include one 16 × 16 sample, two 16 × 8 samples, two 8 × 16 samples, or four 8 × 8 samples in one macro block. When the 8x8 mode is selected, each of the four 8x8 macroblock partitions in the macroblock may be divided into four types again. Each divided area is called a macroblock subpartition. As shown in FIG. 1, the macro block subpartition may include one 8 × 8 sample, two 8 × 4 samples, two 4 × 8 samples, or four 4 × 4 samples in the macro block partition.

These partitions and subpartitions may be composed of various combinations within each macro block. In addition, a separate motion vector is required for each partition or subpartition. In general, large partition sizes are appropriate for homogeneous areas of the frame, and small partition sizes are suitable for detailed areas.

2 is a configuration diagram schematically illustrating a configuration of a video decoder.

2, the video decoder includes a variable length decoder 10, an inverse quantizer 20, an inverse transformer 30, a predictor 40, a filter 50, and an external memory 60.

The variable length decoder 10 receives the compressed bitstream and variably decodes the bitstream.

The inverse quantizer 20 inversely quantizes the variable-length decoded bitstream, and the inverse transformer 30 inversely transforms the inverse quantized bitstream.

The prediction unit 40 performs intra prediction or inter prediction according to whether an image compression method is intra prediction or inter prediction.

That is, the predictor 40 includes an intra predictor 41 and a motion compensator 43. The intra predictor 41 performs intra prediction, and the motion compensator 43 performs motion compensation by reading a reference image stored in the external memory 60 when the image compression method is inter prediction. Here, the reference image is stored in the external memory 60 because of its large capacity, and the memory access device 100 of the motion compensator 43 obtains the reference image in units of partitions or windows. The memory access device 100 sets a window including one or more partitions according to the size of the partition of the macro block of the reference image, and reads data from the external memory 60 in an amount corresponding to the set window. The memory access device 100 will be described in detail with reference to FIG. 3.

The filter 50 may be a deblocking filter (DF) that reduces blocking between macroblocks that may occur when the bitstream is restored. The deblocking filter may include an engine (not shown) that performs deblocking filtering and a direct memory access (DMA) (not shown) for storing data in an external memory.

3 is a configuration diagram schematically illustrating a configuration of a memory access apparatus of a video decoder.

Referring to FIG. 3, the memory access device 100 includes a confirmation unit 110, a setting unit 120, an access unit 130, an internal memory 140, and an interpolation unit 150.

The identification unit 110 confirms the size and motion vector of each partition in the macro block. The identification unit 110 determines whether to set a window according to the size of the partition. For example, the checker 110 may determine that the window is used when the partition size is 8x16, 8x8, 8x4, 4x8, and 4x4, and the window is not used when the size of the partition is 16x16 or 16x8.

4 is a diagram illustrating a case where pixel data is separately read from an external memory. Referring to FIG. 4, when the video decoder separately reads pixel data from the external memory 60, a phenomenon in which the pixel data is read, such as sram0 and sram1, may occur. Here, two pixels on the left side and an upper side, and three pixels on the right side and the lower side, are regions necessary for interpolation. In addition, since the memory has an inherent latency, when the video decoder reads pixel data from the external memory 60 individually, the cycle time is large. As a result, the amount of data to be fetched at a time is small and the number of accesses is very large, resulting in low memory access efficiency. Thus, in order to improve access efficiency, the present specification proposes a memory access method using a window.

When it is determined by the checker 110 to use the window, the setting unit 120 sets the window and the burst mode. That is, the setting unit 120 may set the starting point, the size (the length and the length of the window) of the window using the motion vector, and set the burst mode corresponding to the length of the window. Here, the burst mode (continuous recording / reproducing method) is a method of improving the use efficiency of the bus by reading a large amount of data at once from an external buffer through direct memory access (DMA). For example, as shown in FIG. 5, a window may be set.

5 is a diagram illustrating window setting.

Referring to FIG. 5, assuming that all motion vectors in a macro block are expressed in units of 4 × 4 regardless of partitions or sub-partitions, the setting unit 120 may set the leftmost of the horizontal motion vectors as a horizontal starting point. At this time, if the motion vector of the block 0 represents a half pixel or a quarter pixel instead of an integer pixel, the setting unit 120 may move the starting point two pixels to the left and the bottom. This is to obtain pixel data around the block for interpolation.

The width of the window may be 32 pixels corresponding to the amount of data that can be read once in burst 8. Burst 4 can read 16 pixel data at a time, but when 4x4 partitions are adjacent, the number of horizontal pixel data that should be read in one macro block is 2 + 16 + 3 = 21, so burst 4 is not enough. Burst 16, on the other hand, is sufficient but inefficient because it reads more pixel data than necessary.

The vertical length of the window may be set to a starting point and a size so as to include a block belonging to a range of the set horizontal length.

If the set window does not include all 4 × 4 blocks in the macro block, the setting unit 120 sets the window again as shown in FIG. 5. That is, the setting unit 120 may reset the window based on the leftmost of the blocks not included.

The access unit 130 obtains reference image data from the external memory 60 in units of windows or partitions according to whether a window is used, and stores the reference image data obtained from the external memory 60 in the internal memory 140.

The interpolator 150 performs interpolation on the reference image data stored in the internal memory 140 when the motion vector indicates a half pixel or a quarter pixel instead of an integer pixel.

6 and 7 are diagrams for explaining the interpolation method.

Referring to FIG. 6, when interpolation is required, that is, when a motion vector represents a half pixel or quarter pixel instead of an integer pixel, pixel data around a selected 4 × 4 block is required. In other words, two pixel data are required on the left side and the upper side, and three pixel data are required on the right side and the lower side. Accordingly, the total pixel data for the interpolation operation requires (2 + x + 3) Ｘ (2 + y + 3) assuming that the block size is xＸy. However, when the motion vector represents an integer pixel, the pixel data around the block is not necessary since the interpolation operation does not need to be performed.

Referring to FIG. 7, FIG. 7 illustrates the concept of interpolation. As shown in FIG. 7, when the motion vector is an integer (1, 1), interpolation is not performed. When the motion vector represents a non-integer half pixel or quarter pixel (0.75, 0.5), interpolation is performed. do. Pixel data shown in blue in FIG. 7 does not actually exist, and is obtained by interpolating surrounding pixel data according to a motion vector.

The pixel data includes luminance data or color difference data. The memory access apparatus 100 may perform the above-described operations on the luminance data and the color difference data, respectively.

For example, the color difference data may be composed of Cb and Cr data. When the Cb and Cr data are interleaved, efficient use of the window may be possible. Table 1 below shows a memory map of the color difference data.

Cb _0,0 Cr _0.0 Cb _1,0 Cr _1.0 Cb _2.0 Cr _2.0 ... Cb _0,1 Cr _0.1 Cb _1,1 Cr _1.1 Cb _2,1 Cr _2.1 ... Cb _0,2 Cr _0.2 Cb _1,2 Cr _1.2 Cb _2.2 Cr _2.2 ... ... ... ... ... ... ... ...

8 is a diagram illustrating a structure of a memory access apparatus of a video decoder.

Referring to FIG. 8, the memory access device may include a frame memory access controller (FMAC), a full-pel register (FREG), a full-pel register (FBU), a full-pel buffer (FBU), a memory buffer for septic storage, and a HQBUF (Half / Quarter). -pel Buffer, half-pixel and quarter-pixel storage memory buffers) and interpolator.

FMAC analyzes the motion vector from the bitstream, reads the refinery data corresponding to the motion vector from the SDRAM through the SDC (SDRAM controller), temporarily stores it in the FREG, and distributes the read data to the FBUF. Hereinafter, although the data width of the SDRAM is assumed to be 32 bits, it is applicable to SDRAM having various data widths.

SDRAM is accessed only in burst 8, and FREG is a temporary register that stores the data read in burst 8. The FBUF consists of 16 dedicated buffers corresponding to each of the 16 subblocks. If the motion vector corresponds to half pixel or quarter pixel, data from the FBUF is input to the interpolator, and this operation is sequentially applied to 16 sub blocks for each 4 * 4 sub block.

While writing the data of the FREG to the FBUF, an overwrite problem of the data of the FREG is caused. This is because the interval between two burst accesses is 11 cycles less than 16 cycles. In order to solve this problem, an FBUF having a memory map as shown in FIG. 9 may be used. That is, if the FBUF is divided into two SRAMs, the write is completed in eight cycles, and the overwrite problem can be solved.

The interpolator is a 6-tap FIR filter. The design focuses on the smallest circuit size and memory size to minimize the number of cycles required for calculation. The design methods of the interpolator are largely 1-D, 2-D, and separated 1-D (1-D filter with vertical / horizontal separation). The 1-D interpolator is the simplest method but takes the long time to calculate. The 2-D interpolator has the shortest calculation time, but has the disadvantage of increasing the circuit size. Separated 1-D interpolators are efficient in terms of circuit size and computation time, and recent studies are based on this method. At the heart of the interpolator is a data structure in memory that stores the septics to be supplied to the filter.

Most separated 1-D interpolators accept data from external memory. Independent verification of the interpolator is possible, but in practice, commercial codec chips use SDRAM or DDR SDRAM as external memory, and SRAM as internal memory for temporary storage of pixel data.

However, to maximize the performance of the interpolator, a limited data width SRAM is not used, and a data width of 9 bytes (= 72 bits) is used. This is because the data width is the most efficient in data transmission between FREG and RBUF. Accordingly, the number of cycles for the separated 1-D interpolator to interpolate one macroblock is 144 cycles.

10 is a flowchart illustrating a method of accessing an external memory by a memory access device of a video decoder.

In operation S1010, the memory access device 100 checks the size and the motion vector of each partition in the macroblock.

In operation S1020, the memory access device 100 determines whether to use a window according to the size of a partition. For example, the checker 110 may determine that the window is used when the partition size is 8x16, 8x8, 8x4, 4x8, and 4x4, and the window is not used when the size of the partition is 16x16 or 16x8.

In operation S1030, when it is determined that the window is used, the memory access device 100 sets the window and the burst mode. That is, the memory access device 100 may set the starting point, the size (the length and the length of the window) of the window using the motion vector, and set the burst mode corresponding to the length of the window.

If the window is not used, step S1060 is entered.

In operation S1060, the memory access device 100 determines to use a partition, and checks the location and size of each partition. In this case, the memory access device 100 may set a burst mode corresponding to the size of a partition.

In operation S1040, the memory access device 100 reads and stores the reference image data from the external memory 60 in units of windows or partitions according to whether a window is used.

In operation S1050, when the motion vector indicates a half pixel or a quarter pixel instead of an integer pixel, the memory access device 100 performs interpolation on the reference image data stored in the internal memory 140.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

10: variable length decoder
20: Inverse quantization department
30: inverse transform unit
40: prediction unit
50: filter
60: external memory

Claims (10)

In the memory access device of the video decoder for motion compensation,
A checking unit to check a size and a motion vector of each partition in the macroblock to perform the motion compensation, and determine whether to use a window including one or more partitions according to the size of the partition;
A setting unit which sets the window and sets a burst mode according to the size of the window; And
An access unit for acquiring reference image data from an external memory in units of windows,
The setting unit sets the leftmost of the horizontal motion vectors as the starting point in the horizontal direction, and the horizontal length corresponds to the number of horizontal pixel data to be read in one macro block, and the vertical length includes a block belonging to the set horizontal length range. And setting the window by setting the window so as to enable the memory access device.
The method of claim 1,
And the reference image data is pixel data including at least one of luminance data and chrominance data.
The method of claim 1,
And an interpolation unit for performing interpolation on the reference image data when the motion vector indicates a half pixel or a quarter pixel instead of an integer pixel.
delete The method of claim 1,
And the access unit acquires the reference image data in units of partitions when the window is not used.
In the memory access device of the video decoder to access the memory for motion compensation,
Identifying a size and a motion vector of each partition in the macroblock to perform the motion compensation;
Determining whether to use a window including one or more partitions according to the size of the partition;
Setting the window and setting a burst mode according to the size of the window; And
Acquiring reference image data from an external memory on a window basis;
The window sets the leftmost of the horizontal motion vectors as the starting point in the horizontal direction, the horizontal length corresponds to the number of horizontal pixel data to be read in one macro block, and the vertical length includes a block belonging to the set horizontal length range. Memory access method characterized in that it is set to.
The method according to claim 6,
And the reference image data is pixel data including at least one of luminance data and chrominance data.
The method according to claim 6,
And performing interpolation on the reference image data when the motion vector represents a half pixel or a quarter pixel rather than an integer pixel.
delete The method according to claim 6,
If the window is not used, acquiring the reference image data in units of partitions.

Images