KR20070050335A - An estimation method for cbp and a block mode decision method using estimated cbp on a moving picture encoding system - Google Patents

Abstract

The present invention is a method of predicting a coded block pattern (CBP) with inter-block average and variance in a video encoder for Internet Protocol TV (IP-TV) and a method of determining a block mode using the same.

In the present invention, since all modes of a block can be determined only by the first 16x16 block operation in video encoding, the amount of computation is drastically reduced and thus the computation time can be drastically reduced because no additional operation for mode determination is required.

Coded Block Pattern (CBP), Average, Variance, Block Mode Determination, H.264

Description

{An estimation method for CBP and a block mode decision method using estimated CBP on a moving picture encoding system}

1 is a block diagram of a video encoder for IP-TV (H.264 and MPEG-4 Video Compression: Video Coding for Next-Generation Multimedia ', Iain E. G. Richardson, Wiley, 2003) applied to the present invention.

2 is a diagram illustrating a large block division mode used in an H.264 video codec, and is a diagram for explaining an H.264 block mode applied to the present invention.

3 is a DC component of an average or DCT which is a portion having the highest value where the frequency / distribution value (X) and the DCT output value (Y) intersect through comparison between the one-dimensional DCT output value and the Gaussian probability density function in the present invention. Is a graph showing the similarity of DCT and probability density function for CBP prediction.

4 is a flowchart illustrating a mode determination method using CBP prediction according to an actual implementation of the present invention.

FIG. 5 is a diagram illustrating a method of determining a block mode by predicted CBP according to the present invention. FIG. 5 is a diagram illustrating an example of CBP distribution according to H.264 large block partitioning mode when CBP is predicted in a large block.

FIG. 6 is a conceptual diagram of clustering for determining a block mode by predicted CBP according to the present invention. When CBP is predicted on a large block as shown in FIG. 5, a diagram illustrating a split mode of a large block is determined by the present invention. to be.

The present invention relates to a method of predicting a coded block pattern (CBP) in a video encoder and a mode of a block using the same, and more particularly, to an average and variance of CPBs in a video encoder for IP-TV. The present invention relates to a CBP prediction method performed and a block mode determination method for avoiding image degradation while minimizing H.264 encoding operations using the same.

Digital video data used in video conferencing, high-definition television, video-on-demand (VOD) receivers, personal computers that support Moving Picture Experts Group (MPEG) video, game consoles, terrestrial digital broadcast receivers, digital satellite broadcast receivers, and cable television (CATV) Compression is compressed by an efficient compression method rather than being used as it increases the amount of data in the process of digitizing the analog signal and the characteristics of the image itself.

The compression of digital image data uses three methods. The method of reducing temporal redundancy, the method of reducing spatial redundancy, and the method using the statistical characteristics of the generated code are mainly used. Among them, a representative method of reducing temporal redundancy is motion estimation and compensation, which is used in most video compression standards such as MPEG and H.263.

The motion estimation and compensation method that finds the most similar part from a previous or subsequent reference screen for a specific part of the current screen, and transmits only the difference component of the two parts, the more precisely the motion vector is found, the less the difference component to be transmitted. Can be reduced more effectively, but significant estimation time and computation is required to find the most similar part of the previous or subsequent screen. Therefore, efforts have been made to reduce the motion estimation time, which takes the most time when encoding a video.

On the other hand, the motion estimation method includes a pixel-by-pixel basis and a block-by-block basis. Among them, the block-based estimation method is the most popular algorithm. to be.

The block unit estimation method divides an image into blocks of a predetermined size and finds a block that best matches a block of the current image in a search region of a previous image. The difference between the found block and the current video block is called a motion vector, which is encoded and processed. Various matching functions can be used to calculate the matching between two blocks. The most commonly used function is sum of absolute difference (SAD), which is a sum of absolute values of the difference between pixels between two blocks.

In the case of the H.264 codec, the search is performed through the Cost function based on Rate Distortion Optimization (RDO) instead of the conventional SAD-oriented search method. The Cost function used in H.264 searches using the Rate-Distortion Cost, which is the sum of the number of encoded coefficients multiplied by the Lagrangian multiplier. In this case, the number of encoded coefficients is determined by substituting a value proportional to the quantization coefficient value, and the search is performed by determining the Cost value by multiplying the fixed Lagrangian multiplier value.

In addition, in the H.264 video encoding, in order to simultaneously obtain compression efficiency and high image quality, 8 different blocks are all formed, unlike encoding of 16 to 16 large blocks or 8 to 8 large blocks in the existing video encoding. It is configured to have a mode and to select a mode having a minimum value among each block.

However, in order to determine eight different blocking modes, each encoding operation as well as a refinement search and a subpixel search must be performed independently for each mode, which consumes much computation and computation time compared to the existing video encoding algorithm. . Therefore, the video encoder for IP-TV can be actually implemented only when the operation for determining the block mode is minimized to reduce the operation time and avoid the degradation of the image quality.

As a prior art, US Patent Publication 2003-633273 (February 3, 2005), US Patent 6,628,845 (September 30, 2003), Korea Patent Publication 01-75406 (April 29, 2004), Korea Patent Publication 02-69129 (2000) 10. 16) and Korean Patent Laid-Open Publication No. 03-42350 (June 27, 2003) and the like, but all of them predict the PBP code mode by predicting the coded block pattern (CBP) values as in the present invention. Blocks are not taught the skills to determine.

Accordingly, an aspect of the present invention is to solve the above-described problems, and an object of the present invention is to provide a coded block pattern (CBP) prediction method in a video encoder for IP-TV. .

Another technical problem to be solved by the present invention is to determine a block mode through a coded block pattern (CBP) prediction method in a video encoder for IP-TV, thereby reducing image quality while minimizing H.264 coding operations. It is an object of the present invention to provide a method of determining a block mode to avoid.

In order to achieve the above object, the present invention derives a CBP prediction method by defining a coded block pattern (CBP) in an H.264 video encoder, and performs block determination in eight modes through the predicted CBP. .

According to a preferred embodiment of the present invention, the CBP prediction method of the present invention comprises: determining a current macroblock and a corresponding reference macroblock by performing motion estimation in units of 16 × 16 pixel macroblocks; Calculating an average and a variance between individual 4x4 blocks in the current macroblock and corresponding individual 4x4 blocks in the reference macroconvex; And predicting CBP values of 4 × 4 blocks in the current macroblock using the calculated mean and variance.

Preferably in the step of calculating the mean and variance,

The average is the following equation (Equation 2)

Is calculated using

The variance is the following equation (Equation 3)

Is calculated using

는 단위 4x4 블록에서 시간 t-k, i 번째 열과 j 번째 행에서의 화소 값이며

는

를 통해 예측된 4x4 블록의 i 번째 열과 j 번째 행에서의 화소 값임을 특징으로 한다.In the above equations,

Is the pixel value in the time tk, the i th column and the j th row in the unit 4x4 block

Is

It is characterized in that the pixel value in the i th column and j th row of the 4x4 block predicted through.

Preferably the step of predicting the CBP values,

Next Equation (Equation 4)

Calculating a predicted CBP value using

는

이면 1, 그렇지 않으면 0의 값을 갖는 단위 계단 함수이며

는 CBP 예측을 위해 도입된 파라미터로서 다음 수학식(수학식 6)과 같이 적응적으로 갱신되며,In the above equation

Is

Is a unit step function with a value of 1, otherwise 0

Is a parameter introduced for CBP prediction, and is adaptively updated as shown in Equation 6 below.

은 적응 계수로서 0.1에서 0.25 사이의 임의 값이 선택되며 CBP는 블록 모드가 결정된 후 DCT 및 양자화를 거친후 구해지는 실제 CBP 값이며 함수

는

로서 계수

는 4.6 이상의 임의의 값이다.here

Is an adaptation coefficient and a random value between 0.1 and 0.25 is chosen, and CBP is the actual CBP value obtained after DCT and quantization after the block mode is determined.

Is

Coefficient as

Is any value greater than or equal to 4.6.

According to another aspect of the present invention, in a method for determining a block mode for dynamic compensation using CBP values predicted using the above-described CBP prediction method, based on the prediction CBP values of the current macroblock, the current macro A first step of determining among the block modes configurable by the 4x4 blocks constituting the block, a block mode that does not require additional motion estimation and corresponding 4x4 blocks; Performing a further motion estimation on 4x4 blocks corresponding to a block mode in which additional motion estimation is needed; A third step of predicting CBP values for 4 × 4 blocks for which additional motion estimation has been performed in a second step; And among the block modes configurable by the 4x4 blocks constituting the current macroblock, based on the prediction CBP values of the current macroblock including the prediction CBP values obtained by the third step, a block mode that does not require additional motion estimation. And a fourth step of determining corresponding 4x4 blocks.

Preferably, the block modes configurable by the 4x4 blocks are 16x16 block mode, 16x8 block mode, 8x16 block mode, 8x8 block mode, 8x4 block mode and 4x8 block mode, and the first step is 16x16 block mode, 16x8 block mode. If the number of 1s among the predicted CBP values of 4x4 blocks corresponding to each of the block mode, 8x16 block mode, 8x8 block mode, 8x4 block mode, and 4x8 block mode is less than or equal to a predetermined number, the block mode does not require additional motion estimation. Determining the block mode.

Hereinafter, a code block pattern prediction method for a video encoder and a block mode determination method using the same according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram of a video encoder (H.264 and MPEG-4 Video Compression: Video Coding for Next-Generation Multimedia ', Iain EG Richardson, Wiley, 2003) applied to the present invention. The middle motion estimator is composed of a refiner estimator for estimating a motion vector of a refinery unit and a subpixel estimator for estimating an optimal half pixel and a motion pixel of a quarter pixel based on the found motion vector of the refinery.

In FIG. 1, the current image 100 is denoted as Fn (Current), and the images 102 reconstructed by MC (motion compensator) in the encoder one unit time ago are denoted as F'n-1 (Referenced). The reconstructed image (the same image as the decoded image in the decoder) 112 for the current image is denoted as F'n, where n is a time index.

In FIG. 1, a block denoted by ME is a motion estimator 103 and a block denoted by MC is a motion compensation 107. Also, the block labeled Choose INTRA Prediction is a prediction mode determiner 105 for INTRA (I picture) Prediction (Intra prediction). In the case of H.264, there are four prediction modes in the case of 16x16 luminance image and 8x8 color image. In the case of 4x4, there are nine prediction modes. One of the prediction modes should be selected and sent to INTRA Prediction. The next block marked as an INTRA Prediction is an intra predictor 108 that generates an INTRA prediction value using the INTRA prediction mode predicted in the Choose INTRA Prediction 105. The switch 113 is connected to the motion compensator 107 when in inter mode and to the intra predictor 108 when in intra mode.

In addition, block 116 denoted by T is a part for performing DCT transformation and based on integer 4x4 prediction in H.264. Block 118 denoted by Q is a part of quantizing the DCT transformed value with the quantizer. Block 120, denoted as Reorder, is a zigzag scan to compress the values from the quantizer 118 into variable length coding (VLC) or CABAC.

Entropy encoder 122 is a part for compressing data by Huffman encoding or CABAC.

Q ⁻¹ is an inverse quantizer 119 that performs the opposite function of quantizer 118 described above. T- ¹ is a portion for performing inverse DCT on the data passed through the inverse quantizer 119 to the inverse DCT converter 117.

Through this, image restoration (motion compensation) is performed by adding data having the same quantization error as the prediction data.

The filter 110 is a filter for the first reconstructed image, and removes the deterioration of image quality by passing the image through which the quantization error or block phenomenon appears. The image passing through the filter 110 is the reconstructed image and is the same as the image decoded at the decoder, which is stored as F'n 112 and referred to as F'n-1 when encoding the next image.

The signal Dn input to the DCT converter 116 is obtained as a difference frame as a difference picture, that is, a difference between the original picture and the predicted picture in the subtractor 114.

The output uF'n of the adder 115 is obtained as the sum of the data Dn 'passed through the inverse quantizer 119 and the inverse converter 117 to the unfiltered reconstructed image, that is, the difference image.

The present invention relates to the motion estimator 103 and the motion compensator 107 in FIG. 1, wherein the motion estimator 103 is divided into a refiner estimator for performing a refinement search and a subpixel estimator for performing a subpixel search. After estimation, subpixel estimation is performed.

Hereinafter, an adaptive CBP prediction method and a block determination method of an H.264 encoder using the adaptively predicted CBP will be described.

First, the difference between the existing technology and the present invention will be described to help understanding of the present invention.

Conventionally, 4x4 DCT and quantization are performed to derive the CBP, and the resultant CBP is used to determine the skip mode in the P frame or the direct mode in the B frame. For the remaining modes shown in FIG. 2, a refinement and a subpixel estimation are performed, and a rate distortion calculation is performed to select a mode having a minimum rate distortion value.

2 is merely a block mode of H.264 vs. block. The existing block mode selection method performs motion prediction (ME) for 16x16 blocks and again performs 2 times for 16x8 block mode, 2 times for 8x16 block mode, 4 times for 8x8 block mode and 4 times for 8x4 block mode. Eight times, eight times for 4x8 block mode and 16 times for 4x4 block mode, 41 MEs are performed in total to select the block mode with the smallest SAD or MSE. It is a major factor.

On the other hand, in the present invention, after the first 16x16 block mode refinement and subpixel motion estimation are finished, the average value and the variance of the blocks determined by the estimation are calculated without performing a separate DCT operation. The block mode is determined using the set CBP value. As a result, the block mode is determined through the combination of the 4x4 blocks in which CBP is represented by 1 instead of the refiner, the subpixel, and the rate distortion operation for each mode.

Let us look at the principle of the present invention in detail through the equation. In general, CBP is determined as follows.

First, let X∈R ^{4X4 be} the DCT result of the motion compensated 4x4 pixel data. In this case, the CBP is determined through a quantization process such that Equation 1 is an element of X∈R ^4X4 .

는 양자화된 X∈R^4X4의 i 번째 열 요소, j 번째 행 요소 이며

는 양자화 되기 전 X∈R^4X4의 i 번째 열 요소, j 번째 행 요소 이며

는 H.264의 양자화 계수이다.

는 QP, i, j의 함수로서 양자화 계수에 12을 더한 것을 6으로 나눈 나머지와 i, j의 요소에 따라 결정되는 양자화 함수이며 f는 양자화 레벨 오프셋 값이다. In Equation 1, i and j are subscripts representing the i th column element and the j th row element of X∈R ^4X4 .

Is the i th column element, the j th row element of quantized X∈R ^4X4

Is the i th column element, the j th row element of X∈R ^4X4 before quantization

Is the quantization coefficient of H.264.

Is a function of QP, i, and j, which is a quantization function determined according to the remainder of quantization coefficient plus 12 divided by 6 and the elements of i and j, and f is a quantization level offset value.

In Equation 1, if any one of the quantization coefficients QP is not 0, the CBP value of the 4x4 block is 1, otherwise it is 0.

CBP prediction considers the result of the DCT operation as a result of the two-dimensional probability density function and predicts the CBP as an average and variance as shown in FIG. 3.

That is, as shown in Fig. 3, the portion having the highest value where the comparison between the one-dimensional DCT output value and the Gaussian probability density function X and Y intersects is the average or DC component of the DCT.

In this case, the average value is the same as the DC component of the 4x4 DCT operation. For the other AC components except the DC component, the CBP is predicted to be 1 if the variance exceeds an adaptively determined threshold value, and 0 otherwise. .

The average is calculated as in Equation 2 and the variance is calculated as in Equation 3.

는 단위 4x4 블록에서 시간 t-k, i 번째 열과 j 번째 행에서의 화소 값이며

는

를 통해 예측된 4x4 블록의 i 번째 열과 j 번째 행에서의 화소 값이다. CBP 예측은 수학식 4와 같이 수행되며 예측된 CBP인 ECBP가 0이 아니면 1의 값을 가진다. In Equations 2 and 3

Is the pixel value in the time tk, the i th column and the j th row in the unit 4x4 block

Is

Pixel values in the i th column and the j th row of the 4x4 block predicted through. The CBP prediction is performed as shown in Equation 4, and the predicted CBP has a value of 1 if the ECBP is not 0.

는

이면 1, 그렇지 않으면 0의 값을 갖는 단위 계단 함수이며

는 CBP 예측을 위해 도입된 파라미터로서 수학식 5와 같이 적응적으로 갱신된다.In equation (5)

Is

Is a unit step function with a value of 1, otherwise 0

Is adaptively updated as shown in Equation 5 as a parameter introduced for CBP prediction.

은 적응 계수로서 0.1에서 0.25 사이의 임의 값이 선택되며 CBP는 블록 모드가 결정된 후 DCT 및 양자화를 거친후 구해지는 실제 CBP 값이며 함수

는

로서 계수

는 4.6 이상의 임의의 값이다.In equation (5)

Is an adaptation coefficient and a random value between 0.1 and 0.25 is chosen, and CBP is the actual CBP value obtained after DCT and quantization after the block mode is determined.

Is

Coefficient as

Is any value greater than or equal to 4.6.

The CBP prediction method described above with reference to FIG. 4 will be described in detail. In FIG. 4, after performing initialization for large block encoding in a video encoder (FIG. 1) (400), a motion vector (MV) of a 16 × 16 block is predicted (410), and a CBP is searched through a 16 × 16 refiner / subpixel search. Prediction 420. In step 420, if the CBP prediction is performed on the predicted block after the 16x16 search and the derivative of the motion vector is 0 (dMV = 0), and all 16 CBPs are 0, it is determined as a skip / direct mode (425).

In step 430, if the skip / direct check result, that is, the predicted CBP is all 0, the process proceeds to step 490 to record the large block related parameter in the H.264 sequence.

If the result of the skip / direct check in step 430, that is, the predicted CBP is not all zeros, proceeds to step 440 to predict the I block, and if yes, to step 470. In operation 440, the prediction is performed using a Kalman filter (445).

Next, in step 450, the RD_cost evaluation is performed. The mode is then determined in step 460. If ECBP = 0, no further operation is required. In this case, when the mode determination CPB is 1 through mode clustering using the predicted CBP, a search is performed in the corresponding block (465). In step 470, DCT Q / IQ IDCT and DC Hadamard are obtained after the step 460 or in step 440. Here, the actual CBP is obtained (475). In step 480, the parameters for CBP prediction are updated, and in step 490, the large block related parameters are recorded in the H.264 sequence.

On the other hand, the CBP prediction as shown in Equation 5 performs the mode decision through the steps shown in FIG. 2 and updates the parameter through the actual CBP obtained through the DCT and the quantization process. Since the block mode may be shown as shown in FIG. 5 through the predicted CBP, when the clustering algorithm as shown in FIG. 6 is introduced, the block mode is determined without any additional operation and the predicted CBP is determined in the case of the block having the predicted CBP of 0. An additional search process is performed on only one block to determine a block mode with higher compression efficiency.

In FIG. 5, the upper left block mode pattern 510 shows the CBP obtained in the MC process after the end of the ME process for 16 × 16 blocks. Since the CBP has a value of 1 or 0 in 4x4 block units, the CBP may have a value such as the top left corner in a 16x16 large block. CBP equals 0 means that all data values in the DCT-quantization process of the 4x4 block are all zero. Therefore, when CBP is 0, it is not necessary to perform motion estimation (ME) for an additional next block mode. The remaining figures show the CBP distribution for each large block mode. If CBP is less than 1 or 2 in each large block mode, there is no need to perform additional block mode ME. The upper second block mode pattern 520 in FIG. 5 is the CBP distribution for the 16 × 8 block mode. Compared to the 16x16 block mode, 16x8 has only 1 CBP of the top 16x8 block and 0 the rest. Therefore, for the top 16x8, additional mode-specific ME may be unnecessary. The upper right block mode pattern 530 is for 8 × 16. In the case of the left and right 8x16 blocks, 4 and 3 CBPs are 1 respectively, so 8x16 mode cannot be selected.

In FIG. 5, the lower left block mode pattern 540 is a pattern for an 8 × 8 block mode. The upper two 8x8 blocks do not need additional ME because the CBP value is 1, but the lower two require 3 and 2 respectively so that additional ME is required. Bottom second pattern 550 is for 8x4 block mode. In block mode smaller than 8x8, if you do not perform additional ME only when CBP is zero, there are only two blocks that do not require additional ME. The third block mode pattern 560 relates to a 4x8 block. Three 4x8 blocks do not require additional ME. The fourth block mode pattern 570 relates to a 4x4 block. Nine of 16 4x4 blocks with zero CBP do not need motion estimation.

Referring to FIG. 6 based on this information, the upper 16x8 block 610 and the upper 4x8 block 630 at the lower right do not need to perform additional ME. This is illustrated in the second picture 600. Therefore, the lower left 8x8 block 640 and the lower right 4x8 block 650 must perform additional ME. As a result of the additional ME, assume that the CBP of the 8x8 block 640 has all changed from 1 to 0, and assume that the 4x8 block 650 still has two 1s even though the additional ME has performed the additional ME. Then, the configuration of each mode can be combined again. The left 8x16 mode 660 can be selected since only one CBP is 1. Then, the opposite 8x16 block is also selectable because only one CBP is 1 for the upper 8x8 block 670, and the upper 8x4 block 680 for the lower 8x8 is selected because both CBPs are 0 and the lower 8x4 block 690 In the case of C, although CBP is all 1, no additional ME is performed in this state.

As described above, a method of predicting a coded block pattern (CBP) by inter-block average and variance in a video encoder for IP-TV according to the present invention and a method of determining a mode of a block using the same are used in video encoding. Since all the modes of the block can be determined by the first 16x16 block operation alone, there is no need for additional operation to determine the mode, which greatly reduces the amount of computation and thus greatly reduces the computation time.

Claims (5)

Performing motion estimation in units of 16 × 16 pixel size macroblocks to determine the current macroblock and the corresponding reference macroblock; Calculating an average and a variance between individual 4x4 blocks in the current macroblock and corresponding individual 4x4 blocks in the reference macroconvex; And Predicting CBP values of 4x4 blocks in the current macroblock using the calculated mean and variance. The method of claim 1, wherein in calculating the mean and variance: The average is the following equation (Equation 2)

Is calculated using The variance is the following equation (Equation 3)

Is calculated using In the above equations,

Is the pixel value in the time tk, the i th column and the j th row in the unit 4x4 block

Is

The CBP prediction method which is the pixel value in the i th column and the j th row of the 4x4 block predicted through. The method of claim 2, wherein predicting the CBP values comprises: Next Equation (Equation 4)

Calculating a predicted CBP value using In the above equation

Is

Is a unit step function with a value of 1, otherwise 0

Is a parameter introduced for CBP prediction, and is adaptively updated as shown in Equation 6 below.

here

Is an adaptation coefficient and a random value between 0.1 and 0.25 is chosen, and CBP is the actual CBP value obtained after DCT and quantization after the block mode is determined.

Is

Coefficient as

Is an arbitrary value of 4.6 or greater. A method for determining a block mode for dynamic compensation using CBP values predicted using the CBP prediction method according to claim 1, Determining, based on the predicted CBP values of the current macroblock, among the block modes configurable by the 4x4 blocks constituting the current macroblock, a block mode that does not require additional motion estimation and corresponding 4x4 blocks; Performing a further motion estimation on 4x4 blocks corresponding to a block mode in which additional motion estimation is needed; A third step of predicting CBP values for 4 × 4 blocks for which additional motion estimation has been performed in a second step; And Among the block modes configurable by the 4x4 blocks constituting the current macroblock, based on the prediction CBP values of the current macroblock including the prediction CBP values obtained by the third step, a block mode that does not require additional motion estimation and And a fourth step of determining corresponding 4x4 blocks. The method according to claim 4, wherein the block modes configurable by the 4x4 blocks are 16x16 block mode, 16x8 block mode, 8x16 block mode, 8x8 block mode, 8x4 block mode, and 4x8 block mode. The first step may be performed if the number of 1's is less than or equal to a predetermined number of 4x4 blocks corresponding to each of 16x16 block mode, 16x8 block mode, 8x16 block mode, 8x8 block mode, 8x4 block mode, and 4x8 block mode. Determining a block mode as a block mode that does not require additional motion estimation.

Images