KR101031493B1 - Interpolation architecture of motion compensation unit in decoders based on H.264 video coding standard - Google Patents

Abstract

The present invention relates to a motion compensator for a decoder based on the H.264 standard and an interpolation calculation method thereof, comprising: a VLD including a parser for classifying input data, an Exp-Golomb for parameter extraction, and a CAVLD for coefficient extraction; Among the H.264 decoders comprising an ITQ for quantization and coefficient transformation, a motion compensator for inter-slice motion interpolation and an IP for prediction within a slice, and a DF for block elimination filtering, the motion compensator is a VLD. A predictor for obtaining a motion vector using a parameter received from a; a pixel data handler connected to a part of obtaining the motion vector, calculating an address of a reference memory and reading data; and an interpolation operation connected to the pixel data handler. It consists of an interpolator for reconstructing the image data, the interpolator is provided with a buffer memory of two luminance and chrominance, the interpolation operation is operated when the data is filled in any one memory, the internal memory to store the prediction pixel . Accordingly, the present invention reduces the number of reference memory accesses in the motion compensator structure of the H.264 decoder, improves the performance of the motion compensator and decoder by reducing the number of interpolation operation cycles, and reduces the number of external reference memory accesses that consume a lot of power. As a result, power consumption is also reduced.

 H.264 decoder, motion compensation, interpolation operation, reference memory

Description

Interpolation architecture of motion compensation unit in decoders based on H.264 video coding standard}

The present invention relates to a motion compensator for a decoder based on the H.264 standard and an interpolation calculation method thereof. More particularly, the present invention relates to a hardware implementation of a semiconductor based on H.264 of ITU-T, an international video compression standard. In particular, the present invention relates to an interpolation calculator structure of a motion compensator for a decoder based on the H.264 standard in the field of digital system design for designing a decoder for reproducing a compressed image according to an image compression standard.

Among the video compression standards, H.264 (or MPEG-4 part-10 / AVC) jointly developed by ITU-T and ISO / IEC is in the spotlight as the next generation video compression technology with high compression ratio and high quality. It has been adopted as a standard video compression codec in H, DTV, Blue ray, and HD-DVD, and is expected to be further expanded in the future.

The H.264 is a VLD (Variable Length Decoder) consisting of a Parser 111 for classifying input data, an Exp-Golomb 112 for parameter extraction, and a CAVLD 113 for coefficient extraction. 110); Integer Transform and Quantization (ITQ) 123 for quantization and coefficient transformation, Motion Compensation (MC) 121 for inter-slice motion interpolation, and Intra Prediction (IP) 122 for prediction within slices. A prediction unit 120 configured; And a deblocking filter (DF) 130 for block removal filtering.

Decoder uses complex algorithms to achieve high compression ratio and high picture quality. Decoding using dedicated hardware is generally used, and even in this case, hardware has a pipelined structure. The pipeline structure generally includes a Parser 111 for input data classification, an Exp-Golomb 112 for parameter extraction, and a second step of CAVLD 113 for coefficient extraction, quantization and A reconstructor (ITQ-MCIP) 120 consisting of an ITQ 123 for coefficient transformation, a motion compensator (MC) 121 for inter-slice motion interpolation, and an IP 122 for prediction within a slice. A third step and a fourth step of the DF 130 for block elimination filtering.

In particular, since the motion compensator (MC) 121 is an operator that requires the most computation cycles in most cases, power consumption and a large area are required. Therefore, designing the motion compensator effectively affects the performance of the entire decoder. Crazy

The motion compensator (MC) 121 can largely divide motion vectors into subdivisions of calculation, reference memory access, and interpolation value calculation. Reference memory access and interpolation value calculation affect performance. Its size is very large and often uses external SDRAM. The more and more reference memory accesses, the more power consumption and computational cycles.

On the other hand, the calculation of interpolation values varies in complexity depending on the operation mode, which requires the most computation cycles. Therefore, a method for reducing the number of operators as much as possible and reducing them is required, and also has a close relationship with the reference memory approach. Do not wait for the data to be effective.

In order to suppress access to the reference memory, once retrieved data should be reused as much as possible. To this end, a large buffer or cache memory is used. Using a large buffer can bring a lot of data at once, which increases the reusability of the data, but if it cannot be reused, memory access to fill the buffer can degrade performance and the area of the buffer itself increases. In addition, the use of cache memory can increase the possibility of data recycling with a small memory size, but there is a problem of complicated control.

In order to solve the above problems, in the present invention, in order to reduce the number of interpolation operation cycles in the motion compensator, a finite impulse response (FIR) filter, which is a main operator, is arranged in a pipelined structure of a 2 × 2 structure, and data is stored in the filter. Using the two-dimensional shift register file structure to store the reference pixel data for supplying the data efficiently, and increasing the size of the two-dimensional shift register file increases the possibility of reusing the data in the reference memory. It is possible to increase the number of reference memory accesses, which can reduce the number of operation cycles and reference memory accesses and the power consumption of the motion-compensated motion compensator in the H.264 decoder. Motion compensator and its interpolation method To have a purpose.

A configuration for achieving the object includes a variable length decoder (VLD) including a parser for classifying input data, an Exp-Golomb for parameter extraction, and a CAVLD for coefficient extraction; A prediction unit including an Integer Transform and Quantization (ITQ) for quantization and coefficient transformation, a Motion Compensation (MC) for inter-slice motion interpolation, and an Intra Prediction (IP) for prediction in a slice; And a motion compensator (MC) configured of a deblocking filter (DF) for block elimination filtering, wherein the motion compensator (MC) is connected to a predictor for obtaining a motion vector using a parameter received from a VLD, and a part for obtaining the motion vector. And an interpolator for calculating an address of a reference memory and reading data, and an interpolator connected to the pixel data handler to restore image data through an interpolation operation. When the data is filled in any one memory, an interpolation operator operates to configure the internal memory in which the prediction pixel is stored.

As another feature of the present invention, a two-dimensional shift register file for luminance data interpolation operation and chrominance data interpolation operation of the interpolation operator is provided, and the luminance data interpolation operation includes a pair of filter operators (PE) in the two-dimensional shift register file. -L1, PE-L2), consisting of a four-stage pipeline of registers and clips, and output to the final one register, the chrominance data interpolation operation is a pair of filter operators (PE-C1, PE-C2) consists of a four-stage pipeline of registers and clips, which are output to one final register and process two pixel data in one cycle.

As another feature of the present invention, the two-dimensional shift register file for luminance data calculation is composed of 13 x 9 and 9 x 9 bytes in size, and the row data rotation of 9 bytes is possible in the horizontal direction, and in the vertical direction. Is capable of rotating a 13-byte or 9-byte column data, an external input is made in rows 1, 6, and 9, and an output shift register is provided for supplying data to a filter operator in the horizontal and vertical directions, respectively. The two-dimensional shift register file for the chrominance data is provided in the size of 5 x 3 bytes, two-byte row data rotation in the horizontal direction, 5 bytes of column data rotation in the vertical direction, An external input is made in three rows, and an output shift for supplying data to a filter operator in the horizontal and vertical directions, respectively. A register is provided.

As another feature of the present invention, the output data of the two-dimensional shift register file is input to filter operators PE-L1 and PE-L2, and the filter operators PE-L1 and PE-L2 are an adder and a subtractor. And a shifter, the output data of the two-dimensional shift register file is input to filter operators PE-C1 and PE-C2, and the filter operators PE-C1 and PE-C2 are configured as an adder and a shifter. .

In still another aspect of the present invention, a motion compensation operation is performed on two 4 × 4 luminance subblocks and 2 × 2 chrominance subblocks having the same motion vector by retrieving luminance image data and chrominance image data from a reference memory. Only 4 x 9 byte data is read for the x 8 luminance subblock, and motion compensation is performed to recycle image data of the reference memory.

As described above, the present invention improves the performance of the motion compensator and the decoder by reducing the number of reference memory accesses in the motion compensator structure of the H.264 decoder, and reduces the number of interpolation operation cycles, and increases the number of external reference memory accesses with high power consumption. As a result, the power consumption is reduced.

2 is a diagram illustrating an internal structure of a motion compensator of an H.264 decoder according to the present invention, and FIG. 3 is a diagram illustrating an operation of a luminance interpolator of an H.264 decoder according to the present invention. 5 is a structural diagram illustrating an operation of the luminance filter operator PE-L1 according to the present invention, and FIG. 6 is an operation of the luminance filter operator PE-L2 according to the present invention. 7 is a structural diagram illustrating an operation of a color difference interpolator of an H.264 decoder according to the present invention, and FIG. 8 is a structural diagram illustrating an operation of a two-dimensional shift register file for color difference data according to the present invention. 9 is a structural diagram illustrating an operation of the color difference filter calculator PE-C1 according to the present invention, and FIG. 10 is a structural diagram illustrating an operation of the color difference filter calculator PE-C2 according to the present invention.

Hereinafter, the components will be described with reference to the drawings.

2 is an internal structure diagram of a motion compensator of the H.264 decoder according to the present invention, and includes a parser 111 for classifying input data, an Exp-Golomb 112 for parameter extraction, and a CAVLD 113 for coefficient extraction. A Variable Length Decoder (VLD) 110; Integer Transform and Quantization (ITQ) 123 for quantization and coefficient transformation, Motion Compensation (MC) 121 for inter-slice motion interpolation, and Intra Prediction (IP) 122 for prediction within slices. A prediction unit 120 configured; And the motion compensator (MC) 121 in detail in the operator structure of the general H.264 decoder of FIG. 1, which is composed of blocks of a deblocking filter (DF) 130 for block elimination filtering.

The motion compensator (MC) 121 is connected to a predictor 100 for obtaining a motion vector using a parameter received from a VLD and a predictor 100 for obtaining the motion vector, and calculates an address of the reference memory 600. A pixel data handler 200 for reading data and an interpolator 300 connected to the pixel data handler 200 for restoring image data through interpolation operations. Two buffer memories 400 are provided, and when data is filled in one of the memories, the interpolation operator 500 operates, and the interpolators of luminance and chrominance operate simultaneously, and the internal memory 700 stores prediction pixels. It is composed.

Two-dimensional shift register files 510 and 510a are provided for the luminance data interpolation operation and the chrominance data interpolation operation of the interpolation operator 500. The structure of the luminance interpolator for the luminance data interpolation operation is shown in FIG. The dimensional shift register file 510 includes a pair of filter operators (PE-L1) 520, a filter operator (PE-L2) 530, a four-stage pipeline of a register 540, and a clip 550. , And is output to the last one register 560.

In addition, the color difference data interpolation operation is performed by performing a pair of filter operators (PE-C1) 520a, filter operators (PE-C2) 530a, and a register 540a in the two-dimensional shift register file 510a. And a four-stage pipeline of the clip 550a, which are output to a single register 560a to process two pixel data of the luminance data interpolation operation and the chrominance data interpolation operation in one cycle. .

The two-dimensional shift register file 510 for interpolating the luminance data is configured as one of 13 x 9 or 9 x 9 bytes in size as shown in FIG. 4, and the row data can be rotated by 9 bytes in the horizontal direction 511. In the vertical direction 512, 13-byte or 9-byte column data rotation is possible, and external inputs are made in rows 1, 6, and 9, and data is provided to the filter operator in the horizontal and vertical directions 511 and 512, respectively. Output shift registers 513 and 514 are provided to supply.

9 rows of 13 or 9 bytes of data are connected to the horizontal direction 511 register, and 13 bytes or 9 bytes of data of 13 or 9 columns are connected to the vertical direction 512 register through a shifter. In any case, the output shift registers 513 and 514, which receive the data, input six bytes of data four times into the filter operator to calculate four 1 / 2-pel values. Operates simultaneously with a latency difference of one cycle, producing two results in one cycle.

Output data of the 2D shift register file 510 is input to a filter operator (PE-L1) 520 and a filter operator (PE-L2) 530, as shown in FIGS. 3, 5, and 6. The filter operators PE-L1 and PE-L2 520 and 530 include an adder, a subtractor and a shifter.

There are two filter operators including a 6-tap FIR filter function and a rounding correction function, PE-L1 of FIG. 5 and PE-L2 of FIG. 6, and there are differences in a method of generating an output value of out_filter and h1 value. Here, A to F are luminance pixel data of integer positions for calculating the 1 / 2-pel value, and both are configured to perform a filter operation without a multiplier, and the size of the adder and the subtractor to minimize the error due to rounding. Adjust the and position, and clip operator is provided to calculate the 1 / 4-pel value.

As shown in FIG. 8, two two-dimensional shift register files 510a for chrominance data interpolation of FIG. 7 are provided with a size of 5 x 3 bytes, and 3 bytes of row data can be rotated in the horizontal direction 511a. 5 bytes of column data rotation is possible in the vertical direction 512a, and an external input is made in three rows, and an output shift for supplying data to the filter calculator in the horizontal and vertical directions 511a and 512a, respectively. Registers 513a and 514a are provided.

Output data of the 2D shift register file 510a is input to a filter operator (PE-C1) 520a and a filter operator (PE-C2) 530a as shown in FIGS. 7, 9, and 10. The filter calculators PE-C1 and PE-C2 520a and 530a may include an adder and a shifter, and the filter calculators PE-C1 and PE-C2 520a and 530a may have a rounding error correction function. There is only a difference, and the inputs A to D are color difference pixel data at integer positions.

The chrominance interpolator for the chrominance data interpolation has a structure similar to that of the luminance interpolator of FIG. 3, but there are some differences in the structure and operation according to the difference of the calculation methods, and the luminance is PE in the 2D shift register file 510. One prediction pixel is computed through -L1, PE-L2, and the clip 550 operator, but the chrominance interpolator must pass twice the path performed by the clip 550a operator in the two-dimensional shift register file 510a. Prediction pixels are calculated.

The motion compensator of the present invention has a structure for reconstructing an image of a 4 × 4 subblock based on luminance and requires 9 × 9 bytes of reference image data to reconstruct an image of a 4 × 4 subblock. Nevertheless, in the present invention, one of the register files of the luminance interpolator is 13 x 9 bytes to suppress reference memory access and reduce the number of operation cycles through the reuse of reference image data.

After processing the first 4 x 4 subblock, if the motion vector of the second (right) 4 x 4 subblock is the same as the first, importing reference image data of 13 x 9 byte size causes the two 4 x 4 subblocks to It can process the motion compensation operation for. Also, when processing the lower subblock of the first 4x4 subblock, if the motion vector is the same, only 4x9 bytes of reference data are loaded and the rest of the existing data is recycled. Only when reference data of 13 x 9 bytes cannot be processed, the reference data previously imported into the 9 x 9 byte register file is used. In this case, since the reference data can be imported in advance, the total number of calculation cycles does not change.As a result of analyzing the block size distribution for a commonly used standard video, a 4 x 4 subblock is 8 to 35% and a 4 x 8 sub Since the blocks are 12-15%, assuming that the average number of 4x4 subblocks is 20% and 4x8 subblocks are 13%, the number of reference memory accesses can be reduced by 70%, resulting in a very high data recycling rate.

Therefore, in the present invention, the data is recycled by a simple control method using the 2D shift register, and the 2D shift register is structurally connected to the interpolation operator so that parallel operation is performed in a pipelined manner without waiting for input data. Reduce computation cycles

While the invention has been shown and described with respect to particular embodiments, it will be apparent to those skilled in the art that various modifications and variations can be made without departing from the spirit and scope of the invention as set forth in the appended claims. Anyone can grow up easily.

1 is a block diagram of an operation unit of a typical H.264 decoder.

2 is a diagram illustrating an internal structure of a motion compensator of an H.264 decoder according to the present invention.

3 is a structural diagram showing an operation of a luminance interpolator of an H.264 decoder according to the present invention;

4 is a structural diagram illustrating an operation of a two-dimensional shift register file for luminance data according to the present invention.

5 is a structural diagram illustrating an operation of the luminance filter operator PE-L1 according to the present invention;

6 is a structural diagram for explaining the operation of the luminance filter operator PE-L2 according to the present invention;

7 is a structural diagram showing an operation of a chrominance interpolator of an H.264 decoder according to the present invention;

8 is a structural diagram illustrating an operation of a two-dimensional shift register file for color difference data according to the present invention.

9 is a structural diagram illustrating an operation of a color difference filter calculator PE-C1 according to the present invention;

10 is a structural diagram illustrating an operation of a color difference filter calculator PE-C2 according to the present invention.

Description of the Related Art [0002]

110: VLD 111: Parser

112: Exp-Golomb 113: CAVLD

120: restorer (ITQ-MCIP) 121: motion compensator (MC)

122: IP 123: ITQ

130: DF 131: DF calculator

100: part of obtaining a motion vector 200: pixel data handler

300: interpolator 400: buffer memory

500: interpolation operator 600: reference memory

700: internal memory

Claims (7)

In a motion compensator for a decoder based on the H.264 standard, A predictor 100 for obtaining a motion vector using a parameter received from a variable length decoder (VLD); A pixel data handler (200) connected to the predictor (100) for calculating an address of the reference memory (600) and reading data; Two buffer memories 400 connected to the pixel data handler 200 and storing luminance data and color difference data, and the luminance data or color difference data is inputted to the buffer memory 400. Or an interpolator 300 configured to interpolate the color difference data to restore image data; And A motion compensator for a decoder based on the H.264 standard, which is connected to the interpolator (300) and comprises an internal memory (700) for storing image data reconstructed by the interpolation operator (500). The method of claim 1, The interpolation operator 500 includes a luminance data interpolation operator and a color difference data interpolation operator, and processes two pixel data of the luminance data interpolation operation and the color difference data interpolation operation in one cycle. Each interpolation operator 500, Two-dimensional shift register files 510 and 510a; A pair of filter operators connected to the two-dimensional shift register files 510 and 510a and configured by a first filter operator PE-L1 520 and 520a and a second filter operator PE-L2 530 and 530a; A four-stage pipeline connected to the pair of filter operators, each of which comprises a register 540 and 540a and a clip 550 and 550a; And A motion compensator for a decoder based on the H.264 standard, comprising a register (560,560a) connected to the four-stage pipeline. 3. The method of claim 2, The two-dimensional shift register file 510 of the luminance data interpolation operator is It consists of either 13 x 9 and 9 x 9 bytes or two 9 x 9 or 13 x 9 bytes, with the horizontal direction 511 capable of row data rotation and the vertical direction 512 capable of column data rotation. In addition, external inputs are made in rows 1, 6, and 9, and output shift registers 513 and 514 are provided in the horizontal direction 511 or the vertical direction 512, respectively, and a pair of filter calculators of the luminance data interpolation calculator are provided. Feeds data to (PE-L1, PE-L2) (520, 530), The two-dimensional shift register file 510a of the color difference data interpolation operator is It is provided with two pieces of 5 x 3 bytes, the horizontal direction 511a is capable of row data rotation, the vertical direction 512a is capable of rotation of column data, and the external input is made in three rows, and the horizontal direction ( 511a or output shift registers 513a and 514a in the vertical direction 512a, respectively. A motion compensator for a decoder based on the H.264 standard, characterized in that it is supplied. 3. The method of claim 2, Output data of the two-dimensional shift register file 510 of the luminance data interpolation calculator is input to a pair of filter operators PE-L1 and PE-L2 520 and 530 of the luminance data interpolation calculator, and the chrominance data interpolation calculator The output data of the two-dimensional shift register file 510a is input to a pair of filter operators PE-C1 and PE-C2 520a and 530a of the color difference data interpolation operator. And each filter operator (520, 520a, 530, 530a) comprises an adder, a subtractor and a shifter. The method of claim 3, The interpolator 300 loads 13 x 9 byte luminance image data and 5 x 3 byte chroma image data from the reference memory, and a 13 x 9 two-dimensional shift register and a 5 x 3 two-dimensional shift register. The data is supplied to the interpolation operator 500 to perform motion compensation operations on two 4 x 4 luminance subblocks and 2 x 2 chrominance subblocks having the same motion vector, and for the 4 x 8 luminance subblocks. In addition, only 4 x 9 byte data is read and motion compensation operations are performed to recycle the image data in the reference memory, and another 9 x 9 two-dimensional shift register and a 5 x 3 two-dimensional shift register have the following 4 x 4 Based on the H.264 standard, minimizing the delay time by reading and storing pixel data required when the motion vector of the luminance subblock and the 2 × 2 chrominance subblock is different. Motion compensator for the decoder. In the interpolation calculation method of a motion compensator for a decoder based on the H.264 standard, A first step of obtaining a motion vector by using a parameter received from a variable length decoder (VLD) in a predictor; A second step of calculating an address of a reference memory and reading luminance data or chrominance data using a value of the motion vector obtained from a pixel data handler; Reconstructing image data by interpolating luminance data or color difference data inputted from an interpolator connected to the pixel data handler; And And a fourth step of storing image data reconstructed by the interpolation operation. In the third step, row data rotation of the input luminance data or color difference data is performed in a horizontal direction, and column data rotation is performed in a vertical direction to time delay the luminance data or color difference data in the horizontal and vertical directions, respectively. An interpolation calculation method of a motion compensator for a decoder based on the H.264 standard, characterized in that to supply a minimum. The method of claim 6, In the third step, the interpolator retrieves 13 x 9 byte luminance image data and 5 x 3 byte chroma image data from the reference memory, and uses two 4 x 4 luminance subblocks having the same motion vector and 2 x. A motion compensation operation is performed on the two color difference subblocks, and additionally, only 4x9 byte data is read for the 4x8 luminance subblock and motion compensation operation is performed to recycle image data in the reference memory. Interpolation method of motion compensator for decoder based on .264 standard.

Images