CN1694535A - Complexity hierarchical mode selection method - Google Patents

Abstract

This invention discloses a mode selection method with hierarchical complexity H264 is a video encode standard of a new generation, which absorbs the advantage of the current video encode standard and puts forward many new methods to increase encode efficiency and image quality greatly, yet it increases the complexity of computation. This invention optimizes the modes in H264 and screens the candidate modes so as to reduce the complexity when selecting the mode and improving the aim for distributing hardware resources to get the optimized peak S/N ratio and code rate.

Description

A kind of mode selecting method of complexity hierarchical

Technical field

The present invention relates to the field of video applications in the multimedia, relate in particular to a kind of mode selecting method of complexity hierarchical.

Background technology

H.264 be the present up-to-date video encoding standard that ITU-T and MPEG group joint specialist group JVT formulates, this coding standard can obtain very high code efficiency, especially be significantly improved than MPEG-4 at low bit rate connection, be fit to very much the needs of low broadband, high network quality Video Applications.But, H.264, adopted the algorithm of many high computation complexities in order to improve code efficiency, cause final amount of calculation very big, therefore very high to the requirement of software and hardware, also strengthened the difficulty of real-time application simultaneously.

In H.264, model selection and estimation have accounted for for about 80% scramble time.Therefore be necessary deep research is done in model selection and two processes of estimation, and and reduce computation complexity possibly.In the identifying code of standard, there are 10 patterns available, i.e. SKIP, 16 * 16,16 * 8,8 * 16,8 * 8,8 * 4,4 * 8,4 * 4, intra4 * 4 and intra16 * 16.H.264 when carrying out model selection, first-selection adopts all patterns to carry out estimation to current block, uses rate distortion (Rate-distortion) to optimize the relatively size of code check under every kind of pattern then, and the pattern that code check is little will be as final pattern.This process is the process of a full search, and it has searched for all candidate pattern blindly, thereby has caused great computation complexity.Though can access optimum effect by this method, in a lot of hardware devices, can not support this huge computing capability requirement.Therefore, under as far as possible little mass loss prerequisite, filtering some candidates' pattern, is the key to H.264 being optimized.

Along with the development of cable network and wireless network, the new real-time video communication technology has obtained popularizing and has used, video monitoring for example, video TV, video conference etc.But different application is also different to the requirement of quality, resolution and the frame per second of video.Along with the development of embedded mobile communication equipment, realize that on embedded platform real-time video communication has become possibility.Each opposite sex at embedded platform, proposed a kind of mode selecting method of complexity hierarchical, thereby reduced the computation complexity that the model selection process is brought, through the optimization in model selection, thereby reached the reduction computation complexity, and then obtained optimized PSNR and code check.

Summary of the invention

The object of the present invention is to provide a kind of mode selecting method of complexity hierarchical, the situation that has the various computing ability at existing hardware, when H.264 doing model selection, need be optimized, reach and under the condition of limited hardware resource, greatly reduce computation complexity, the effect of the optimum of finally getting.

The technical solution used in the present invention is as follows:

1. the mode selecting method of a complexity hierarchical is summed up as follows:

(1) present frame is divided into 16 * 16 macro block, then a macro block of the present frame macro block corresponding with former frame is carried out difference, obtain absolute error and; Current macro is divided into 48 * 8 sub-piece, so obtain 48 * 8 absolute error and, and record absolute error and maximum 8 * 8; Absolute error and computing formula as follows:

${\mathrm{SAD}}_{16\×16}=\underset{i=0}{\overset{15}{\mathrm{\Σ}}}\underset{j=0}{\overset{15}{\mathrm{\Σ}}}|f(i,j)-g(i,j)|,$ ${\mathrm{SAD}}_{8\×8}=\underset{i=0}{\overset{7}{\mathrm{\Σ}}}\underset{j=0}{\overset{7}{\mathrm{\Σ}}}|f(i,j)-g(i,j)|$

Wherein

SAD be absolute error and,

F is the primitive definition of present frame,

G is the primitive definition of former frame;

(2) absolute error and maximum 8 * 8 are carried out estimation with rhombus or horizontal vertical search method, obtain motion vector MV, this MV can be used as good future position; In H.264, for the P frame, 7 kinds of patterns 16 * 16,16 * 8,8 * 16,8 * 8 are arranged, 8 * 4,4 * 8,4 * 4 wherein also comprise pattern intra4 * 4 and intra16 * 16 of SKIP and I type, totally 10 candidate pattern; This predicted value can be used as SKIP so, and 16 * 16,16 * 8, the future position of 8 * 16 patterns; Wherein the ratio maximum that accounts in whole pattern is chosen of 16 * 16 patterns so the efficient of this predicted value is very high, can reduce many time complexities;

(3) set a threshold values T1, write down 4 SAD in the macro block with N _{8 * 8}The number of＜T1;

If N==4, SKIP, 16 * 16 and intra16 * 16 patterns as candidate pattern;

If N==3, SKIP, 16 * 16,16 * 8,8 * 16 and intra16 * 16 as candidate pattern;

If N==2, SKIP, 16 * 16,16 * 8,8 * 16, intra4 * 4 and intra16 * 16 are as candidate pattern;

If N==1, SKIP, 16 * 16,16 * 8,8 * 16,8 * 8, intra4 * 4 and intra16 * 16 are as candidate pattern;

During (4) for N==0,, can obtain 4 new SAD like this motion vector of the MV that obtains as current macro _{8 * 8}According to twice 4 SAD in front and back _{8 * 8}Difference DELTA SAD _{8 * 8}The characteristic that changes judges that this employing 16 * 8 still is 8 * 16 patterns; Wherein need use a threshold values T2 when judging 16 * 8 or 8 * 16 patterns, choosing according to the experiment statistics data of T2 obtained; At last all candidate pattern are finally judged which pattern of selection with rate-distortion optimization.

2. in the step (1) present frame is carried out after macro-block level cuts apart, becoming 48 * 8 sub-piece when the pre-treatment macroblock partition, obtain 48 * 8 absolute error and, and record absolute error and maximum 8 * 8; Wherein absolute error and maximum 8 * 8 are used for the generation forecast value.

3. in the step (2) absolute error and maximum 8 * 8 are carried out with rhombus or horizontal vertical search method

Estimation obtains motion vector MV, and this vector will be as the predicted value of higher level's piece.

4. set a threshold values T1 in the step (3), write down 4 SAD in the macro block with N _{8 * 8}The number of＜T1; The size of this threshold values is set in advance, and it is to judge whether one 8 * 8 be the boundary of static block.

5. step (3) is filtered some candidates' pattern according to the situation of N; Because 10 patterns are arranged in H.264, comprise SKIP, 16 * 16,16 * 8,8 * 16,8 * 8,8 * 4,4 * 8,4 * 4, intra4 * 4 and intra16 * 16; If each pattern all searched for will greatly increase computation complexity, so can filter out a part of pattern according to the situation of N.

In the step (4) because the choosing and 4 SAD of front and back secondary of pattern 16 * 8,8 * 16 _{8 * 8}Variation substantial connection is arranged, therefore can pass through SAD _{8 * 8}Situation of change judge which kind of pattern of selecting.

7. judging that threshold values of 16 * 8,8 * 16 o'clock needs of preference pattern is used as judgment value in the step (4), the size of this value is decided according to a large amount of experiment test results.

In the step (4) to last selected candidate pattern, adopt rate-distortion optimization finally to judge which pattern of selection.

The useful effect that the present invention has is: can carry out model selection to the vision signal of various forms, especially under hardware computing capability condition of limited, more can embody the superiority of this invention, if under the very strong situation of hardware computing capability, the present invention does not influence the final optimization pass result who obtains after the estimation, because all done model selection for each macro block in each frame this moment.On the contrary, if under hardware computing capability condition of limited, the present invention has demonstrated fully its inherent superiority, because the method that the present invention proposes makes originally searching under the situation of 10 candidate block, scope is on average narrowed down to 6 patterns.Some impossible candidate pattern Direct Filtration are fallen, so just can greatly be reduced time complexity.Certainly during model selection, the present invention also fully takes into account the limit of the computing capability of hardware, thus adopt the whole bag of tricks to handle, thus reach optimized effect.

Description of drawings

Fig. 1 is 7 kinds of pattern diagram of existing P frame;

Fig. 2 is existing diamond search method schematic diagram;

Fig. 3 is existing horizontal vertical searching method schematic diagram;

Fig. 4 is the variation schematic diagram of twice absolute error of the present invention and difference;

Fig. 5 is definite method schematic diagram of threshold values of the present invention.

Embodiment

The mode selecting method of piece in video coding that the present invention proposes mainly is useful in and passes through on the computing capability limited hardware to filter some candidate pattern, carries out model selection as far as possible in few candidate pattern.Thereby under the prerequisite that keeps picture quality, obtain optimum sign indicating number, also reduced computation complexity simultaneously.Mainly carry out as follows:

1. the division of macro block and the absolute error and the SAD that obtain 48 * 8 sub-pieces _{8 * 8}

Present frame is carried out the division of macro block, be divided into 16 * 16 macro block, to carrying out 8 * 8 cut apart again, generate 48 * 8 sub-piece then when the pre-treatment macro block.Again this macro block of present frame macro block corresponding with former frame carried out difference, obtain 4 absolute errors and SAD _{8 * 8}, see Fig. 4 (a).

2. to SAD _{8 * 8}That maximum sub-piece carries out estimation.Estimation adopts two-mode method, according to SAD _{8 * 8}Size choose a pattern automatically.Work as SAD _{8 * 8}Greater than a prior specified value, then adopt the diamond search method; Otherwise adopt the horizontal vertical search method.Finally obtain motion vector (MV).

3. adopt different searching methods

At the different absolute error of 8 * 8 sub-pieces and, adopts different searching methods, this is because can be avoided searching for unnecessary like this and the search deficiency.

1) adopts the diamond search method under absolute error and big situation, to use, thereby can find optimum point fast.Divide three kinds of situations, Fig. 2 a is x or y direction translation search, and Fig. 2 b is oblique translation search, and Fig. 2 c is to center search, respectively as shown in Figure 2.

2) adopt the horizontal vertical searching method can reduce the search point, and reach comparatively ideal effect.As shown in Figure 3: establish a starting point, at first seek absolute error and minimum point on its horizontal direction, seek absolute error and minimum point again on the vertical direction of horizontal optimum point, the result of search is final optimum point.Therefore search procedure is as follows: search center is No. 1 point, compare on its left side and No. 2 points, draw the absolute error of No. 1 point and littler, compare with No. 3 points in the right of 1 again, draw No. 1 point absolute error and minimum, therefore No. 1 point is the optimum point on the horizontal direction, in like manner, continuation is searched on the vertical direction of No. 1 point, and final result is that No. 5 points are optimum points.

4. threshold values T1's determines

Set a threshold values T1, be used for judging whether current 8 * 8 belong to static block.Through experiment test, obtain judging that one 8 * 8 sub-piece is that static threshold values T1 is 150-300.Consider that threshold values is big more, the error of generation is big more, therefore gets T1=150.

5. write down 4 SAD in the macro block with N _{8 * 8}Number less than T1.According to different N values can Direct Filtration candidate pattern once.Be that YITIAOGEN is tested the experimental result that the result comes out factually below:

1) if N==4, so only SKIP, 16 * 16 and intra16 * 16 patterns as candidate pattern;

2) if N==3, SKIP, 16 * 16,16 * 8,8 * 16 and intra16 * 16 as candidate pattern;

3) if N==2, SKIP, 16 * 16,16 * 8,8 * 16, intra4 * 4 and intra16 * 16 are as candidate pattern;

4) if N==1, SKIP, 16 * 16,16 * 8,8 * 16,8 * 8, intra4 * 4 and intra16 * 16 are as candidate pattern;

6.16 twice sad value variation characteristic in * 8 and 8 * 16 front and back

For the N==0 situation, can not effectively filter certain module, therefore adopting other a certain pattern of the direct selection of method is final pattern, but mainly is to look for SKIP, 16 * 16,16 * 8 and 8 * 16 pattern.The motion vector of the MV that obtains previously, can obtain 4 new absolute errors and SAD like this as current block _{8 * 8}, see Fig. 4 (b).The SAD that front and back are obtained for twice _{8 * 8}Carry out difference and obtain Δ SAD _{8 * 8}, see Fig. 4 (c).Be formulated as:

ΔSAD _i＝newSAD _i-oldSAD _i

Wherein

Δ SAD _iBe the absolute error and the difference of i 8 * 8 sub-pieces,

NewSAD _iBe i 8 * 8 sub-pieces a back absolute error and,

OldSAD _iBe i 8 * 8 sub-pieces a preceding absolute error and.

Can judge which kind of pattern of selection by following rule:

7. directly selecting some patterns is candidate pattern

1) judgement of threshold values T2

For the selection of T2, Ce Shi result chooses by experiment.Wish to find one when code check minimum and the picture quality threshold values preferably the time.By last experiment test, T2=150 is optimum, sees Fig. 5.For different quantization steps, T2=150 can be used as optimum threshold values equally.

2) for the N=0 situation, at first to judge whether and Δ SAD that two 8 * 8 sub-pieces are only arranged greater than T2.If the judgement below then continuing, the judgement below then not carrying out, and to all to be thought of as candidate pattern to 10 kinds of all patterns.

If a) Δ SAD ₀＞=0, Δ SAD ₁＞=0 and Δ SAD ₀+ Δ SAD ₁If＞T2 or Δ SAD ₂＞=0, Δ SAD ₃＞=0 and Δ SAD ₂+ Δ SAD ₃＞T2 then selects 16 * 8 as final pattern.

B) if Δ SAD ₀＞=0, Δ SAD ₁＞=0 and Δ SAD ₀+ Δ SAD ₁If＜T2 or Δ SAD ₂＞=0, Δ SAD ₃＞=0 and Δ SAD ₂+ Δ SAD ₃＜T2 chooses SKIP, and 16 * 16 as candidate pattern.

C) if Δ SAD ₀＞=0, Δ SAD ₂＞=0 and Δ SAD ₀+ Δ SAD ₂If＞T2 or Δ SAD ₁＞=0, Δ SAD ₃＞=0 and Δ SAD ₁+ Δ SAD ₃|＞T2, select 8 * 16 as final pattern.

D) if Δ SAD ₀＞=0, Δ SAD ₂＞=0 and Δ SAD ₀+ Δ SAD ₂If＜T2 or Δ SAD ₁＞=0, Δ SAD ₃＞=0 and Δ SAD ₁+ Δ SAD ₃＜T2 chooses SKIP, and 16 * 16 as candidate pattern.

8. the judgement of final optimal pattern

Top selected pattern is obtained final optimization model with the Rate-Distortion optimization method.

The present invention improves on the basis of identifying code H.264.Following technology during H.264 this encoder has adopted:

1) reference frame number: 1;

2) frame type: I frame, P frame;

3) Rate-Distortion optimizes (RDO);

4) adopt the CAVAL entropy coding;

5) whole pixel motion estimation range: 16;

6) motion estimation mode: 10 kinds of patterns;

7) whole pixel search.

The result of the test of the inventive method shows: debug on PC, use akyio, coastguard, foreman, mobile, video sequence such as news are tested, form is CIF (352 * 288), resultant image quality on average descends 0.067, and code check rises 3.43%, and has obtained good speed-up ratio 32.8.

Claims (8)

1. the mode selecting method of a complexity hierarchical is characterized in that:

(1) present frame is divided into 16 * 16 macro block, then a macro block of the present frame macro block corresponding with former frame is carried out difference, obtain absolute error and; Current macro is divided into 48 * 8 sub-piece, so obtain 48 * 8 absolute error and, and record absolute error and maximum 8 * 8; Absolute error and computing formula as follows:

${\mathrm{SAD}}_{16\×16}=\underset{i=0}{\overset{15}{\mathrm{\Σ}}}\underset{j=0}{\overset{15}{\mathrm{\Σ}}}|f(i,j)-g(i,j)|,{\mathrm{SAD}}_{8\×8}=\underset{i=0}{\overset{7}{\mathrm{\Σ}}}\underset{j=0}{\overset{7}{\mathrm{\Σ}}}|f(i,j)-g(i,j)|$

Wherein

SAD be absolute error and,

F is the primitive definition of present frame,

G is the primitive definition of former frame;

(2) absolute error and maximum 8 * 8 are carried out estimation with rhombus or horizontal vertical search method, obtain motion vector MV, this MV can be used as good future position; In H.264, for the P frame, 7 kinds of patterns 16 * 16,16 * 8,8 * 16,8 * 8 are arranged, 8 * 4,4 * 8,4 * 4 wherein also comprise pattern intra4 * 4 and intra16 * 16 of SKIP and I type, totally 10 candidate pattern; This predicted value can be used as SKIP so, and 16 * 16,16 * 8, the future position of 8 * 16 patterns; Wherein the ratio maximum that accounts in whole pattern is chosen of 16 * 16 patterns so the efficient of this predicted value is very high, can reduce many time complexities;

(3) set a threshold values T1, write down 4 SAD in the macro block with N _{8 * 8}The number of＜T1; If N==4, SKIP, 16 * 16 and intra16 * 16 patterns as candidate pattern; If N==3, SKIP, 16 * 16,16 * 8,8 * 16 and intra16 * 16 as candidate pattern; If N==2, SKIP, 16 * 16,16 * 8,8 * 16, intra4 * 4 and intra16 * 16 are as candidate pattern; If N==1, SKIP, 16 * 16,16 * 8,8 * 16,8 * 8, intra4 * 4 and intra16 * 16 are as candidate pattern;

During (4) for N==0,, can obtain 4 new SAD like this motion vector of the MV that obtains as current macro _{8 * 8}According to twice 4 SAD in front and back _{8 * 8}Difference DELTA SAD _{8 * 8}The characteristic that changes judges that this employing 16 * 8 still is 8 * 16 patterns; Wherein need use a threshold values T2 when judging 16 * 8 or 8 * 16 patterns, choosing according to the experiment statistics data of T2 obtained; At last all candidate pattern are finally judged which pattern of selection with rate-distortion optimization.

2. the mode selecting method of a kind of complexity hierarchical according to claim 1, it is characterized in that: in the step (1) present frame is carried out after macro-block level cuts apart, becoming 48 * 8 sub-piece when the pre-treatment macroblock partition, obtain 48 * 8 absolute error and, and record absolute error and maximum 8 * 8; Wherein absolute error and maximum 8 * 8 are used for the generation forecast value.

3. the mode selecting method of a kind of complexity hierarchical according to claim 1, it is characterized in that: in the step (2) absolute error and maximum 8 * 8 are carried out estimation with rhombus or horizontal vertical search method, obtain motion vector MV, this vector will be as the predicted value of higher level's piece.

4. the mode selecting method of a kind of complexity hierarchical according to claim 1 is characterized in that: set a threshold values T1 in the step (3), write down 4 SAD in the macro block with N _{8 * 8}The number of＜T1; The size of this threshold values is set in advance, and it is to judge whether one 8 * 8 be the boundary of static block.

5. the mode selecting method of a kind of complexity hierarchical according to claim 1 is characterized in that the situation of step (3) according to N, filters some candidates' pattern; Because 10 patterns are arranged in H.264, comprise SKIP, 16 * 16,16 * 8,8 * 16,8 * 8,8 * 4,4 * 8,4 * 4, intra4 * 4 and intra16 * 16; If each pattern all searched for will greatly increase computation complexity, so can filter out a part of pattern according to the situation of N.

6. the mode selecting method of a kind of complexity hierarchical according to claim 1 is characterized in that: choosing and 4 SAD of front and back secondary owing to pattern 16 * 8,8 * 16 in the step (4) _{8 * 8}Variation substantial connection is arranged, therefore can pass through SAD _{8 * 8}Situation of change judge which kind of pattern of selecting.

7. the mode selecting method of a kind of complexity hierarchical according to claim 1, it is characterized in that: judging preference pattern 16 * 8 in the step (4), threshold values of 8 * 16 o'clock needs is used as judgment value, and the size of this value is decided according to a large amount of experiment test results.

8. the mode selecting method of a kind of complexity hierarchical according to claim 1 is characterized in that: to last selected candidate pattern, adopt rate-distortion optimization finally to judge which pattern of selection in the step (4).

Images