CN100401789C - Quick selection of prediction modes in H.264/AVC frame - Google Patents

Abstract

The present invention relates to a rapid selecting method for an intraframe predicting mode of an H. 264/AVC real-time encoder. In the H. 264/AVC encoder, infratrame encoding uses an intraframe predicting method. The correlativity among a chrominance block, a luminance block, the inner part of the luminance block and the selection of the predicating mode of the luminance block is used. Different judging bases (RDO or SAD) are determined according to the rate distortion optimization (RDO) criterion to rapidly select the intraframe predicating mode. Compared with the full searching method, the method provided by the present invention has the advantages that the signal-to-noise ratio and the code rate changes are small, and the encoding speed is increased by more than 6 times on average.

Description

Fast selection method of H.264/AVC intra-frame prediction mode

Technical Field

The invention relates to a video coding method for information processing, in particular to a quick selection method for an H.264/AVC intra-frame prediction mode.

Background

Since ITU-T promulgated h.261, a block-based hybrid coding approach has been adopted and developed as a framework. The establishment of international standards H.261, H.263, MPEG-1, MPEG-2, MPEG-4, etc. has greatly facilitated the application of multimedia technology. However, with the increase of new services and the coming popularization of high definition tv, and the lower transmission code rate of the existing transmission media such as Cable Modem, Xdsl and UMTS, the coding efficiency needs to be improved urgently; there is also a need for video coding standards that can accommodate existing or future networks. These problems provide a tangible basis for the generation of the new video coding standard H.264/AVC.

These video coding standards employ a hybrid coding framework, i.e., the partitioning of an input video image into macroblocks, which are predicted, transformed, quantized, and entropy encoded. Prediction includes intra-prediction, which may remove spatial redundancy, and inter-prediction, which may remove temporal redundancy.

H.264/AVC still follows this block-based hybrid coding scheme, but contains many new features: VCL layer and NAL layer separation, intra estimation of spatial domain, 1/4 pixel precision motion estimation, adaptive block size motion compensation, multi-reference frame motion compensation prediction, concept of generic B-frames, low complexity 16-bit integer transform and quantization, in-loop deblocking filter, and efficient entropy coding. These new features enable on the one hand a significant increase in coding efficiency; on the other hand, the method is network-friendly and can be effectively used in various networks and application environments.

H.264/AVC adopts intra-frame prediction to remove spatial redundancy in order to improve the coding efficiency of intra-frame images. This intra prediction technique still uses macroblocks as processing units, i.e. a frame image is usually divided into 16 × 16 image blocks for encoding. Whereas the basic processing block unit is reduced from 8 x 8 to 4x4 with respect to other standards, i.e. the transform employs a 4x4 transform matrix. The luma macroblock thus supports two intra prediction block sizes: 4 × 4 and 16 × 16, while chroma only supports 8 × 8. And each current block which is subjected to intra-frame prediction predicts the pixel value of the current block by using the adjacent pixel value of the adjacent block, and then calculates a residual error, transforms and entropy codes. The intra prediction of each 4x4 block in the h.264/AVC standard comprises 9 prediction modes (see fig. 1). Whereas luma 16 × 16 and chroma 8 × 8 support only 4 prediction modes, respectively: vertical (V), horizontal (H), direct current (D) and flat modes (P), abbreviated V, H, D and P modes, respectively.

Meanwhile, the technology is also applied to interframe coding, namely after the current macro block is subjected to interframe prediction, the current macro block is subjected to intraframe prediction, rate distortion optimization is taken as a final judgment basis, and a mode with small rate distortion is determined as a final mode. Therefore, the coding efficiency of the interframe coding macro block can be improved, and the fault-tolerant capability can be improved.

For example, a current macroblock M in a frame (I, P, or B) is intra-frame predictive encoded, the pixel value of the current block is predicted by using the adjacent pixel values of the adjacent block and the current block, then the predicted value is subtracted from the pixel value of the current block to obtain a residual matrix, and the residual matrix is transformed, quantized, and entropy encoded to obtain an encoded code stream of the macroblock.

But the prediction modes using the pixel values of the neighboring blocks are different, thus generating a plurality of prediction modes (see fig. 1). In determining the best of these modes, a metric is required, and currently, rate-distortion optimization (RDO-rate distortion optimization) is used, which is defined as follows:

J_MODE(S_k，I_k|Q，λ_MODE)＝D_REC(S_k，I_k|Q)+λ_MODER_REC(S_k，I_kequation (1)

In the formula D_RECRepresenting the total distortion after encoding, R_RECRepresenting the overall code rate. Lambda [ alpha ]_MODELagrange parameter > 0, by determining the coding option I_kThe sum of distortion and code rate after coding is minimized.

The present invention adopts a simplified decision criterion sad (sum of Absolute difference) in the mode decision of chroma macro block and luma 16 × 16 block, which is defined as follows:

<math><mrow> <mi>SAD</mi> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>0</mn> </mrow> <mrow> <mi>N</mi> <mo>-</mo> <mn>1</mn> </mrow> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>=</mo> <mn>0</mn> </mrow> <mrow> <mi>N</mi> <mo>-</mo> <mn>1</mn> </mrow> </munderover> <mi>abs</mi> <mrow> <mo>(</mo> <mi>f</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>-</mo> <msup> <mi>f</mi> <mo>&prime;</mo> </msup> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> </mrow></math> formula (2)

Where f (i, j) is the original signal, f' (i, j) is the predicted signal after intra prediction, and N is equal to 8 (chroma block) or 16 (luma block).

The H.264/AVC check model provides a full search method for determining the intra-frame prediction mode of the current macroblock, the intra-frame prediction mode of the chroma macroblock is taken as an outer loop, two block modes of the brightness macroblock are taken as inner loops, namely, all the modes of the brightness macroblock I4 and I16 are compared respectively corresponding to the intra-frame prediction mode of each chroma macroblock. Therefore, the method can scan all modes, so that the coding quality is highest and the computation amount is also largest. The total number of modes searched is 4 × (16 × 9+4), and there are 592 modes, which means that 592 different J's are calculated to obtain the best mode for a macroblock_MODE. This makes it impossible for H.264/AVC to implement coding under existing hardware conditions.

Disclosure of Invention

Compared with the full search method, the method can improve the encoding speed by more than 6 times on average while the signal-to-noise ratio and the code rate change are small.

In order to achieve the purpose, the invention adopts the following technical scheme:

a fast selection method of H.264/AVC real-time encoder intra-frame prediction mode, in H.264/AVC encoder, intra-frame coding adopts intra-frame prediction method, characterized in that using the correlation between chroma block and luminance block intra-frame prediction mode of current macroblock, the correlation between mode selection of 4x4 luminance block I4 and 16x16 luminance block I16 and the correlation inside the luminance block prediction mode, using simplified rate distortion optimization RDO as decision basis, fast selecting intra-frame prediction mode.

The quick selection steps are as follows:

a) the mode selection of the chroma block is determined to be independent, i.e. the chroma macroblock mode is consistent whether the luma macroblock selects I16 or I4, so that the chroma prediction mode is independently decided according to SAD first to determine the best prediction mode of chroma.

b) After the optimal prediction mode of the chroma is determined, the following basis is provided for the judgment of the brightness fast mode: for luminance I16, when the chroma block selects the dc D mode, the luminance can select the dc D, horizontal H, vertical V, flat P mode;

when the chroma block selects a horizontal H mode, the luminance can select a direct current D, horizontal H and flat P mode; when the chroma block selects a vertical V mode, the luminance can select a direct current D mode, a vertical V mode and a flat P mode; when the chroma block selects a flat P mode, the luminance can select a direct current D and flat P mode; for the luminance block I4, the mode selected by the luminance block I8 is the mode that most 4 × 4 blocks under the luminance block I4 may select;

c) for the macroblock, the selection probability of I4 is much greater than that of I16, so we first make the mode decision of 4 × 4 block under I4; and b, judging the possible mode of the current block according to the modes of the adjacent blocks, calculating the RDO in the two modes respectively with the possible mode determined in the step b, comparing the RDO with the threshold T0, determining the current best mode of 4 multiplied by 4 if the RDO is less than T0, and otherwise determining the RDO with a smaller value as the possible best mode.

d) Comparing the RDO of the modes adjacent to the prediction direction of the possible best mode obtained in the step c, and further determining the possible best mode; if it is vertical V, horizontal H or direct current D, comparing the modes which are not compared in the three modes; if it is other mode, then all other modes not compared are compared;

e) after finishing the mode decision of 16 4 × 4 blocks of the current macroblock, calculating the macroblock texture complexity, measured by MSE, and comparing with the threshold T1, if greater than T1, terminating the decision in advance, and the result obtained above is the macroblock best mode; if less than T1, making a mode decision for luma 16x16 blocks;

f) determining the possible mode of the current macro block I16 according to the relation between the best mode of the chroma block and the brightness block in the step b; counting the modes of 16 subblocks under the condition of I4 in the step e to obtain two modes with the maximum probability; and comprehensively considering the possible modes, and finally determining the possible mode of I16. Determining the optimal prediction mode of I16 according to SAD;

g) the threshold T1 of the adaptation determining step e is used to determine whether to prematurely terminate the I16 decision and the T0 of the adaptation determining step c is used to determine whether to prematurely terminate the I4 decision.

The specific steps of obtaining the threshold value by the self-adaptive method are as follows:

the threshold T0 is obtained adaptively by the following formula:

T0＝α×((2^q_bus-qp_const)/quant_coef[qp_rem][0][0])²

where α is a correction factor, whose value ranges from (0, 1), QP _ rem ═ QP% 6,

QP _ bits +15, QP _ const [ (1 < q _ bits)/6, quant _ coef is the quantization matrix, QP is the quantization parameter;

the self-adaptive method for calculating the threshold value T1 comprises the following steps:

initializing T1 ═ 1 < 20;

the first macroblock: and sequentially carrying out I4 and I16 mode decisions to obtain MSE (minimum of square error) of the two modes, and assigning the average value to T1 as the threshold value of the next macro block.

Other macro blocks: if only the I4 mode is decided, T1 does not change. If I4 and I16 mode decisions are made in sequence, the mean of the MSEs of the two is assigned to T1.

The invention comprehensively utilizes the method to select the intra-frame prediction mode. Experimental results show that the coding speed can be increased by multiple times, and the code rate and the PSNR of the coding can be ensured to be slightly changed compared with the full search method.

Drawings

Fig. 1 is a block diagram of information processing video coding of one embodiment of the present invention.

FIG. 2 is a process flow diagram of one embodiment of the invention

FIG. 3 is a 4x4 block intra prediction directional diagram

FIG. 4 is a neighbor block location diagram of a current block

FIG. 5 is a graph of rate distortion change for I16 and I4

Detailed Description

A preferred embodiment of the fast selection method of intra prediction mode of H.264/AVC real-time encoder of the present invention is described below with reference to the accompanying drawings:

on a check model Jm72 encoder, specifically, a PC with an environment of 2.4GHz CPU is used as I frame sequence, the encoding sequence is IIII, CABAC is adopted for entropy encoding, quantization parameters are 14 and 28 respectively, an RDO threshold T0 of brightness I4 and a threshold T1 of macroblock complexity can be obtained by self-adaptive calculation, and the following method is adopted: in our experimental process, it was found that the full search method provided by the verification model neglects the correlation between the mode selections. In practice, there is a strong correlation between the mode selection of the 4 × 4 luminance block (I4) and the 16 × 16 luminance block (I16), and the intra prediction mode of the luminance block of a macroblock. If these correlations can be exploited, the best mode for intra prediction can be determined quickly and efficiently.

(1) Referring to fig. 1 and 2, first, it can be observed that the chroma macroblock mode is consistent whether the luma macroblock selects I16 or I4, and thus it can be determined that the mode selection of the chroma block is independent throughout the mode selection process, so the best prediction mode, in which the chroma prediction mode can decide the chroma block independently before the luma decision, can provide the basis for the luma macroblock mode decision. Although the mode selection cannot be completely consistent for luminance I16 and chrominance I8, there is a link: when the chroma block selects the D mode, the luma may select D, H, V, P mode; when the chroma block selects the H mode, the luma may select D, H, P mode; when the chroma block selects the V mode, the luma may select D, V, P mode; when the chroma block selects P mode, the luma can select D, P mode; for luma I4 and chroma I8, I8 selects the most likely mode of most 4 × 4 blocks under luma I4. There is a preference between the selection of modes within the luminance block, with V, H, D modes having preference over other modes. In the case of I4, the most preferred mode for the current block can be decided by the modes of the neighboring blocks (fig. 4). For the preferred mode, the mode adjacent to the preferred mode in the direction has a large selection possibility.

In practice, the selection probability of I4 is much greater than that of I16, so I4 is preferentially detected in the decision process. In the case of I4, of the determined 16 best modes of 4 × 4 blocks, the two modes with the highest probability are likely to coincide with the best mode of I16, so that the chroma modes can be jointly examined to finally determine the I16 best mode.

The h.264/AVC encoder adopts RDO as a mode decision criterion, but the structure is complex, and especially, it takes time to write a code stream, so that different decision bases can be adopted according to specific modes: firstly, chroma mode selection does not depend on mode selection of brightness, and coding bits of the mode only comprise a chroma mode and a chroma residual error, so that rate distortion differences among different modes can be reflected by adopting SAD; ② 4x4 block coded bits at luminance I4 includes entropy coding of block mode and residual. Since the 4 × 4 block size is small, the difference between the number of bits for block mode coding and the number of bits for residual coding between different modes can cause the rate distortion to vary greatly, so the rate distortion is selected as the mode decision basis of I4. And the distribution of the rate distortion of the 4x4 blocks is more concentrated, a threshold T0 can be defined in the distribution range, if the rate distortion is less than T0 in a certain mode, the best mode can be considered to be reached; ③ I16 encoded bits mainly include macroblock mode and residual entropy coding, so SAD can reflect rate distortion change, and is used as the basis for mode selection.

Since the RDO curves of I16 and I4 are different with the macroblock texture complexity (fig. 5), the macroblock texture complexity should be considered in the mode selection, and the decision between the two should be determined by using a certain value of the macroblock texture complexity as the threshold T1, i.e. in the texture flat area (less than T1), the reconstructed image generated by I16 can better match the original image, I16 probability is selected to be greater than I4, I16 rate distortion on the detail content gradually increases with the gradual increase of the texture complexity, and when the rate distortion is greater than I4 when the texture complexity is greater than or equal to T1, I4 is more suitable for the macroblock coding mode selection. There are many ways to measure texture complexity, such as arithmetic mean, geometric mean, median, variance, and mean square error. The invention selects the mean square error to measure the texture complexity of the macro block.

The invention fully utilizes the correlation, organically combines the correlation together, and determines the judgment basis of different modes according to the RDO criterion to obtain the rapid selection method of the intra-frame prediction mode, the flow is shown as figure 2, and the specific steps are implemented as follows:

firstly, intra-frame prediction is carried out on a chroma block of a current macro block, and an optimal mode in a chroma macro block frame is determined by taking SAD as a judgment basis.

Secondly, I4 intra prediction is performed on the luminance block of the current macroblock, firstly, the possible mode is determined according to the mode of the adjacent block (figure 2) of the current 4 multiplied by 4 block, and the mode is compared with the best mode of the chrominance block to determine the most possible block mode. Computing J of the most probable pattern_MODEComparing with the threshold value T0, if it is less than T0, it can be determined as the best mode, and go to step c. Otherwise, jumping to the step III.

Calculating J for another mode (excluded mode above)_MODEJ calculated in step 2_MODEComparing to obtain the minimum J_MODEA pattern of values. If it is less than T0, the mode is confirmed to be the best mode, and step (c) is skipped to when compared with T0.

Selecting the minimum J_MODEAs candidate patterns, J of two patterns adjacent to the prediction direction thereof are calculated_MODEComparing J of the three modes_MODEIf J of the neighbor mode_MODEAll values are greater than J of the candidate pattern_MODESkipping step (c). Otherwise go to step (v).

Fifthly, if is the minimum J_MODEIs V, H or D, calculates J which the three modes have not calculated in the above process_MODEComparison to obtain the minimum J_MODEJumping to step (c); if the mode is other mode, jumping to step (c).

Sixthly, calculating other residual modes to obtain the minimum J_MODEMode, jump to step.

Seventhly the minimum J_MODEThe mode is the best mode for the 4 × 4 block.

And (7) circulating the steps from (c) to obtain 16 optimal modes of 4 multiplied by 4 blocks. The macroblock complexity is calculated and if greater than a threshold T1 (fig. 3), the macroblockI4, end decision. Otherwise, calculate J for the entire macroblock in case of I4_MODEAnd C, jumping to the step ninthly.

Ninthly, determining the possible mode of the current macro block I16 according to the best mode of the chroma block. Counting the patterns of 16 subblocks under the condition of I4, and obtaining two patterns with the highest probability, wherein if the patterns belong to V, H or D, the patterns are possible patterns of I16; if other, P mode becomes a possible mode of I16.

R comparing the above determined I16 possible patterns to obtain the best I16 pattern, calculating its J_MODE. Macroblock minimum J in case of and I4_MODEAnd comparing to obtain the best block size and the corresponding best prediction mode.

It can be found from the above method that the chroma modes to be detected in intra-frame predictive coding of the present invention are 4 in general, and each 4 × 4 block of the luminance I4 has at least 1 or at most 9. If terminated early, the luminance I16 need not be detected, otherwise the luminance I16 detects a minimum of 2 or a maximum of 4 modes, in which case almost all modes are covered. But the number of modes detected overall is greatly reduced.

The inter-frame coding of H.264/AVC also uses the intra-frame prediction coding technology, and the method is also applicable, but only slightly modified: in the I4 intra prediction of step 2, when the neighboring block mode of the 4 × 4 block is the intra coding mode, it is not necessary to consider direct use; if the neighboring block is inter mode, its intra prediction mode may be defaulted to direct current mode (D).

(2) Adaptive method for calculating threshold T0

H.264/AVC uses a DCT-like transform. Formula (3) is 4 × 4DCT integer transform, E is a residual matrix after prediction of the 4 × 4 luminance block, X is a DCT-like integer transform matrix, and I is a coefficient matrix after integer transform and before quantization.

I＝XEX^TFormula (3)

Wherein: $X=\left[\begin{array}{cccc}1& 1& 1& 1\\ 2& 1& -1& -2\\ 1& -1& -1& 1\\ 1& -2& 2& -1\end{array}\right]$

let DC be the DC coefficient after the change, s be the original pixel value of the current 4 × 4 block, and s' be the pixel value of the reconstructed block after prediction, as shown in equation (4). If DC approaches 0, other AC coefficients AC also approach 0, and the entire transformed matrix tends to 0 matrix, so that the subsequent process can be omitted.

<math><mrow> <mi>DC</mi> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>x</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>y</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <mi>E</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>x</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>y</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <mrow> <mo>(</mo> <mi>s</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <msup> <mi>s</mi> <mo>&prime;</mo> </msup> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> </mrow></math> Formula (4)

In h.264/AVC, if the absolute value of the quantized value of a transform coefficient is less than or equal to 1 after quantization, the residual matrix may be regarded as a 0 matrix. I.e. the following inequality is satisfied:

|DC|＜(2^q_bits-qp_const)/quant_coef[qp_rem][0][0]formula (5)

Wherein: QP _ per QP/6 QP _ rem QP% 6 QP _ bits QP _ per +15 QP _ const (1 < q _ bits)/6, QP is the quantization parameter.

For each 4x4 block, equation (6) gives the calculation of SAD as follows:

<math><mrow> <mi>SAD</mi> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>x</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>y</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <mo>|</mo> <mi>s</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <msup> <mi>s</mi> <mo>&prime;</mo> </msup> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>|</mo> </mrow></math> formula (6)

It is clear that the following inequality holds:

<math><mrow> <mo>|</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>x</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>y</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <mrow> <mo>(</mo> <mi>s</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <msup> <mi>s</mi> <mo>&prime;</mo> </msup> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>|</mo> <mo>&le;</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>x</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>y</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <mo>|</mo> <mi>s</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <msup> <mi>s</mi> <mo>&prime;</mo> </msup> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>|</mo> </mrow></math>

namely, the absolute value of DC is less than or equal to SAD.

Then the determination of the 0 matrix is satisfied if the following is true.

SAD＜(2^q_bits-qp_const)/quant_coef[qp_rem][0][0]Formula (7)

The threshold T0 is determined by the RDO of the 4x4 block, as shown in equation 8, mode represents the selected prediction mode, and R includes the mode type and entropy coding of the residual.

J_mode(mode)＝SSD+λ_modeR (mode, E) equation (8)

Wherein, <math><mrow> <mi>SSD</mi> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>x</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>y</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <msup> <mrow> <mo>|</mo> <mi>s</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <mi>p</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>|</mo> </mrow> <mn>2</mn> </msup> <mo>,</mo> </mrow></math> p[x，y]the predicted residual error is obtained by adding the residual error after transformation, quantization, inverse quantization and inverse transformation with the value of the reference block, so that the following inequality holds:

|s(x，y)-p(x，y)|＜|s(x，y)-s′(x，y)|

thus, it is possible to provide <math><mrow> <mi>SSD</mi> <mo>&le;</mo> <mrow> <mo>(</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>x</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>y</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <msup> <mrow> <mo>|</mo> <mi>s</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <msup> <mi>s</mi> <mo>&prime;</mo> </msup> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>|</mo> </mrow> <mn>2</mn> </msup> <mo>)</mo> </mrow> <mo>&lt;</mo> <msup> <mrow> <mo>(</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>x</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>y</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>3</mn> </munderover> <mo>|</mo> <mi>s</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <msup> <mi>s</mi> <mo>&prime;</mo> </msup> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>|</mo> <mo>)</mo> </mrow> <mn>2</mn> </msup> </mrow></math>

Therefore SSD < SAD²I.e. by

SSD＜((2^q_bits-qp_const)/quant_coef[qp_rem][0][0])₂Formula (9)

At this point, the block residual quantized coefficient will tend to 0, thus resulting in T0.

T0＝((2^q_bits-qp_const)/quant_coef[qp_rem][0][0])²Formula (10)

In practice, there may be a difference, and therefore a correction factor α is set to 0.5:

T0＝α×((2^q_bits-qp_const)/quant_coef[qp_rem][0][0])²formula (11)

(3) Adaptive method for calculating threshold T1

In practice, the greater the T1, the more likely it is to search for I16, and therefore the more patterns searched; the smaller the T1, the less likely it is to search for I16, and the fewer patterns to search. The invention determines the value of the current T1 through the previous macro block of the current macro block, and the specific method is as follows:

initialization: t1 ═ 1 < 20

The first macroblock: and sequentially carrying out I4 and I16 mode decisions to obtain MSEs of the two modes, and giving the average value to T1 as the threshold value of the next macro block.

Other macro blocks: if only the I4 mode is decided, T1 does not change. If I4 and I16 mode decisions are made in sequence, the mean of the MSEs of the two is assigned to T1.

Three CIF sequences, Mobile, Bus and Foreman, with different texture complexity were tested. Experimental results show that the fast intra-frame prediction mode selection method can improve the I-frame coding speed by about 6 times compared with the original full search method of H.264/AVC. While the speed is improved, the average signal-to-noise ratio is changed by about 0.1db, and the code rate is slightly changed.

The second embodiment of the present invention is: aiming at P frame coding, the coding sequence is I PPP, the motion estimation search range is +/-16, 5 reference frames, the motion estimation algorithm is a full search algorithm, the entropy coding adopts CABAC, and the quantization parameters are respectively 14 and 28. The adaptive method described above is also applied to the threshold T0 for RDO of luminance I44 × 4 blocks and the threshold T1 for macroblock complexity. The three CIF sequences, Mobile, Bus and Foreman, were still tested. Experimental results show that the rapid intra-frame prediction mode selection method can improve the coding speed by about 25% compared with the original H.264/AVC inter-frame full search method under the inter-frame condition. While the speed is improved, the average signal-to-noise ratio changes by about 0.02db, and the code rate hardly changes.

Claims (1)

1. A H.264/AVC real-time encoder intra-frame prediction mode fast selection method, in H.264/AVC encoder, intra-frame coding adopts the intra-frame prediction method, characterized by that utilize the correlation between chroma block and luminance block intra-frame prediction mode of the present macroblock, the correlation between mode selection of 4x4 luminance block I4 and 16x16 luminance block I16 and the correlation within the prediction mode of the luminance block, regard simplified rate distortion optimization RDO as the decision basis, carry on the intra-frame prediction mode selection fast; the quick selection steps are as follows:

a. determining that the mode selection of the chroma block has independence, namely, the chroma macroblock mode is consistent no matter the brightness macroblock selects I16 or I4, so that the chroma prediction mode is independently judged according to SAD, and the optimal prediction mode of chroma is determined;

b. after the optimal prediction mode of the chroma is determined, the following basis is provided for the judgment of the brightness fast mode: for luma I16, when the chroma block selects direct D mode, luma can select direct D, horizontal H, vertical V, flat P mode;

when the chroma block selects the H mode, the luminance can select the direct D, horizontal H and flat P modes; when the chroma block selects a vertical V mode, the luminance can select a direct current D mode, a vertical V mode and a flat P mode; when the chroma block selects a flat P mode, the luminance can select a direct current D and flat P mode; for the luminance block I4, the mode selected by the luminance block I8 is the mode that most 4x4 blocks under the luminance block I4 may select;

c. for the macroblock, the selection probability of I4 is much greater than that of I16, so we first make the mode decision of 4x4 blocks under I4; judging the possible mode of the current block according to the modes of the adjacent blocks, calculating the RDO under the two modes respectively with the possible mode determined by the step b, comparing the RDO with a threshold T0, if the RDO is less than T0, determining the current mode as the best mode of 4x4, otherwise, determining the mode with the smaller value as the possible best mode;

d. and d, comparing the RDOs of the modes adjacent to the prediction direction of the possible best mode obtained in the step c, and further determining the possible best mode. If it is vertical V, horizontal H or direct current D, comparing the modes which are not compared in the three modes; if it is other mode, then all other modes not compared are compared;

e. after finishing the mode decision of 16 4x4 blocks of the current macroblock, calculating the macroblock texture complexity, measured by MSE, and comparing with a threshold value T1, if the MSE is greater than T1, terminating the decision in advance, and the obtained result is the macroblock best mode; if less than T1, making a mode decision for the luma 16x16 block;

f. determining the possible mode of the current macro block I16 according to the relation between the best mode of the chroma block and the brightness block in the step b; counting the modes of 16 subblocks under the condition of I4 in the step e to obtain two modes with the maximum probability; comprehensively considering the possible modes, and finally determining the possible mode of I16; determining the optimal prediction mode of I16 according to SAD;

g. the threshold value T1 of the adaptive determination step e is used to determine whether to terminate the I16 decision and the adaptive determination early

The T0 in step c is used for judging whether to terminate the I4 in advance;

the specific steps of obtaining the threshold value by the self-adaptive method are as follows:

the threshold T0 is obtained adaptively by the following formula:

T0＝α×((2^q-bits-qp_const)/quant_coef[qp_rem][0][0])²

wherein α is a correction factor, and its value range is (0, 1), QP _ rem is QP% 6, QP _ bits is QP _ per +15, QP _ const is (1 < q _ bits)/6, quant _ coef is a quantization matrix, and QP is a quantization parameter;

the self-adaptive method for calculating the threshold value T1 comprises the following steps:

initializing T1 ═ 1 < 20;

the first macroblock: and sequentially carrying out I4 and I16 mode decisions to obtain MSE (minimum of square error) of the two modes, and assigning the average value to T1 as the threshold value of the next macro block.

Other macro blocks: if only the I4 mode is decided, T1 does not change. If I4 and I16 mode decisions are made in sequence, the mean of the MSEs of the two is assigned to T1.

Images