CN103517069A - HEVC intra-frame prediction quick mode selection method based on texture analysis - Google Patents

Abstract

The invention discloses an HEVC intra-frame prediction quick mode selection method based on texture analysis. According to the method, before intra-frame prediction is carried out on an encoding tree unit, the main texture direction and the texture complexity of each 4*4 unit in the encoding tree unit are determined according to the sum of absolute values of gradients in the horizontal direction, the vertical direction, the lower left direction and the lower right direction, and division of the current encoding tree unit is determined according to the principle that a larger encoding unit is adopted in a texture smooth area and a smaller encoding unit is adopted in a texture complex area. During prediction, according to the main texture direction of the prediction unit, a plurality of prediction modes with least probability are eliminated, and rough mode selection and rate-distortion optimization mode selection are carried out according to an HEVC encoding standard. According to the HEVC intra-frame prediction quick mode selection method based on texture analysis, on the premise that encoding quality is guaranteed, the encoding speed can be obviously improved.

Description

A kind of HEVC infra-frame prediction fast schema selection method based on texture analysis

Technical field

The invention belongs to multimedia coding field, be specifically related to a kind of infra-frame prediction fast schema selection method for HEVC video encoding standard, is a kind of method of carrying out coding unit division and prediction direction fast mode decision according to analyzing image texture.

Background technology

Along with video encoding standard HEVC(High Efficiency Video Coding of new generation) issue, the code efficiency of video further promotes.Than before video encoding standard H.264/AVC, the in the situation that of same-code quality, can save nearly half code check, yet this take and sacrifices encoder complexity as cost.How, not changing code flow structure and keeping, under the prerequisite of original code check, farthest reducing algorithm complex, significantly improving coding rate, is the key point of HEVC large-scale promotion and application.

Intraframe coding can provide reference information for interframe encode, and can stop the error drift being caused by error code, and therefore in Video coding, intraframe coding is absolutely necessary.Affect HEVC intraframe coding consuming time mainly contain two parts: the division of coding unit and the selection of prediction direction.

Quick division for coding unit mainly contains two kinds of methods, the one, the basis rate distortion costs of coding unit utilizes the method for the statistical learnings such as Bayes to determine the division of coding unit, yet the learning outcome of off-line can not well be common to all video sequences, and online mode of learning can be introduced extra computation complexity again; The 2nd, according to the picture material complexity of present encoding unit, determine the division of coding unit, the key of this method is computed image complexity and selects suitable threshold value.

Fast mode decision for prediction direction mainly contains three kinds of methods, the one, according to thering is stronger correlation between adjacent area or consecutive frame, utilize the prediction direction Candidate Set of the information reduction present encoding unit of coding unit around, yet in practical application, often occur that scene is switched or the video of texture-rich, in this case temporal correlation a little less than, cause coding efficiency obviously to decline; The 2nd, utilize correlation higher between the grain direction of best prediction direction and coding unit, reduce the quantity of candidate's prediction direction; The 3rd, by a kind of mode of low complex degree, estimate the code check of each predictive mode and the computation complexity of distortion reduction rate distortion costs, this method has retained all prediction direction, and do not increase extra computation complexity, but because the randomness of natural image video sequence is stronger, be difficult to a rate distortion estimation model that is applicable to all video sequences of structure, than second method, the coding efficiency of this method declines obvious.

In video encoding standard of new generation, HEVC has introduced new code tree unit (Coding Tree Unit, CTU) concept, for substituting the macro block concept of existing video encoding standard, also has in addition the concepts such as coding unit, predicting unit, converter unit.Code tree unit is the quad-tree structure of a recurrence, and the degree of depth can from 0 to 3, and the size of coding unit can from 64 * 64 to 8 * 8.For minimum coding unit, the intraframe coding of HEVC can be divided into again 4 * 4 predicting unit, and the size of predicting unit can from 64 * 64 to 4 * 4.And for infra-frame prediction, HEVC provides 35 kinds of predictive modes to each predicting unit.Than existing video encoding standard H.264/AVC, the optimization problem that coding unit is divided and prediction direction is selected is more complicated.The present invention describes according to the image complexity of texture analysis structure present encoding tree unit, and determines fast accordingly the division of coding unit; Utilize correlation stronger between grain direction that texture analysis obtains and prediction direction, reduce the quantity of predictive mode.

Summary of the invention

In order to overcome the above-mentioned defect of prior art, the present invention proposes a kind of HEVC infra-frame prediction fast schema selection method based on texture analysis, the method is determined the division of code tree unit according to the complexity of video content, and according to the grain direction of image, reduces the Candidate Set of prediction direction.Experimental results show that this method can be good at adapting to the Real Time Compression application of high clear video image, has improved coding rate greatly.

For achieving the above object, the technical solution used in the present invention is:

A HEVC infra-frame prediction fast schema selection method for texture analysis, the method step is as follows:

Step (1), to present encoding tree unit, calculate image complexity and the main grain direction of each possible coding unit;

Step (2), to present encoding tree unit, according to Z scanning sequency from top to bottom repeated execution of steps (3) to step (6), until scanned present encoding tree unit;

Step (3), if the image complexity of present encoding unit is less than or equal to threshold value, is judged that this coding unit is as smooth unit, otherwise is judged to be complex unit;

Step (4), for level and smooth coding unit, utilize step (5) to calculate the rate distortion costs of present encoding unit;

Step (5), for each predicting unit in present encoding unit, retain the main grain direction in this unit 8 prediction direction and planar prediction and DC predictive mode around, according to HEVC video encoding standard, carry out that coarse mode is selected and rate-distortion optimization selection;

Step (6), for complicated coding unit, according to step (3) to four subelements of step (6) successively recursive scanning, if four subelements are all judged to be smooth unit, utilize step (5) to calculate the rate distortion costs of present encoding unit, and judge that according to rate distortion costs minimum principle whether present encoding unit is divided, and upgrades threshold value simultaneously.

Described step (1) specifically comprises the steps:

Step (11), to each 4 * 4 the unit in present encoding tree unit, respectively along absolute value and the SAG of level, vertical, lower-left, bottom right four direction compute gradient compute gradient;

Step (12), using minimal gradient absolute value and gradient direction as the main grain direction of this unit, described main grain direction refers to the grain direction of the most of pixel in this unit;

Step (13), the absolute value that SAG on main grain direction and its orthogonal direction is differed from are defined as the Texture complication of this unit, and utilize the product of quantization step and this cell size as initial threshold.

Described step (5) specifically comprises the steps:

Step (51), define candidate's predictive mode array;

Step (52), according to step (1), obtain the main grain direction of current predicting unit, and using main grain direction and 8 prediction direction around thereof as candidate's prediction direction;

Step (53), candidate's prediction direction is mapped to the angle predictive mode in HEVC coding standard, and adds the angle predictive mode of selection and planar and DC pattern to candidate's predictive mode array;

Step (54), to each predictive mode in candidate's predictive mode array, according to HEVC coding standard, carry out coarse mode selection and R-D optimized mode selection.

Described step (6) specifically comprises the steps:

Step (61), according to step (3) to four subelements of step (6) successively recursive scanning, if four subelements are all judged to be smooth unit, utilize step (5) to calculate the rate distortion costs Cost of present encoding unit _coarse, otherwise the subelement of next level of continuation recursive scanning;

Step (62) is if the rate distortion costs Cost of present encoding unit _coarsethe rate distortion costs and the Cost that are less than or equal to four subelements _fine, threshold value is tuned up, otherwise threshold value is turned down.

The update strategy that in described step (62), threshold value tunes up is:

TH′＝TH+α×(Cost _fine-Cost _coarse)/2

In formula, TH is the threshold value before upgrading, and TH ' is the threshold value after upgrading, and α is the multiplication factor of the linear model between Texture complication and rate distortion costs.

The update strategy that in described step (62), threshold value is turned down is:

TH′＝TH-a

In formula, TH is the threshold value before upgrading, and TH ' is the threshold value after upgrading, and a is constant.

The present invention's advantage is compared with prior art:

1, the present invention is from the angle of texture analysis, the selection of the division of code tree unit and prediction direction is taken into consideration, utilize gradient can either extract Texture complication and can extract the direction of texture again, guaranteeing, under the prerequisite of coding quality, can to save the more scramble time.

2, the present invention upgrades in time and determines the threshold value whether coding unit is divided according to final rate distortion costs, can make full use of the strong correlation of adjacent area in code tree unit, to weaken threshold value, chooses the impact on coding quality.

Accompanying drawing explanation

Fig. 1 is the HEVC infra-frame prediction fast schema selection method flow chart based on texture analysis;

Fig. 2 is the code tree dividing elements structure chart in HEVC video encoding standard;

Fig. 3 is direction gradient computational methods schematic diagrames;

Fig. 4 is that the coding unit based on gradient is divided detail flowchart fast.

Embodiment

Below in conjunction with the drawings and specific embodiments, the present invention is further illustrated.

In order to meet the Real Time Compression application demand of high clear video image, improve the coding rate of HEVC video encoder, the present invention proposes a kind of HEVC infra-frame prediction fast schema selection method based on texture analysis.The method mainly comprises two parts: the coding unit based on Texture complication divides fast and the predictive mode based on grain direction is selected fast, the quick division of coding unit relates to step (1) in summary of the invention to step (4) and step (6), and the quick selection of predictive mode relates to the step (5) in summary of the invention.First utilize gradient operator to calculate the complexity of each subelement in present encoding tree unit, determine a thinner coding unit division, according to this, divide and determine a thicker coding unit division again, finally according to the rate distortion costs of follow-up infra-frame prediction, determine final coding unit division; In selecting the process of optimum prediction direction, exclude with main grain direction and differ predictive mode far away, to save the time of infra-frame prediction.Launch below to illustrate.

1. the coding unit based on Texture complication is divided fast

The video encoding standard HEVC of a new generation has introduced code tree unit (Coding Tree Unit, CTU) concept, and a frame video is comprised of the code tree unit of a plurality of non-overlapping copies.Under default situations, the size of code tree unit is 64 * 64.Code tree unit is the recursive structure of a quaternary tree form, and it can be divided into the coding unit of many levels flexibly, as shown in Figure 1.The size of coding unit can from 64 * 64 to 8 * 8.In the reference encoder device of HEVC standard, the method that utilance aberration optimizing is selected to each possible division all once predict, convert, the process such as quantification, entropy coding, inverse quantization, inverse transformation, reconstruction, than H.264/AVC, encoder complexity significantly promotes, and has seriously restricted the practical application of HEVC.

Statistics is found by experiment, and the size of coding unit and the Texture complication of picture material are closely related.The simple region of texture adopts larger coding unit, and the region of texture complexity adopts less coding unit.The metric form of Texture complication has multiple, as variance, gradient, entropy etc.Gradient not only comprises the Texture complication information of image, has also comprised the grain direction information of image, can assist to determine fast the predictive mode of predicting unit.Based on this, the present invention proposes a kind of coding unit fast separating method based on gradient.

Before introducing the quick execution mode of dividing of coding unit, the definition of the computational methods of the gradient of using in given first the present invention and main grain direction, Texture complication.Gradient in the present invention refers to the directional derivative that pixel brightness value obtains along directions such as level, vertical, lower-left, bottom rights, as shown in Figure 3.Computing formula is as shown in the formula of (1)-(4).

$\left\{\begin{array}{c}{g}_h(x,y)=p(x,y)-p(x+2,y)\\ g_h(x+1,y)=p(x+1,y)-p(x+3,y)\\ g_h(x,y+1)=p(x,y+2)-p(x+2,y+2)\\ g_h(x+1,y+1)=p(x+1,y+2)-p(x+3,y+2)\end{array}\right.---\left(1\right)$

$\left\{\begin{array}{c}{g}_v(x,y)=p(x,y)-p(x,y+2)\\ g_v(x+1,y)=p(x+2,y)-p(x+2,y+2)\\ g_v(x,y+1)=p(x+1,y+1)-p(x+1,y+3)\\ g_v(x+1,y+1)=p(x+3,y+1)-p(x+3,y+3)\end{array}\right.---\left(2\right)$

$\left\{\begin{array}{c}{g}_{\mathrm{ld}}(x,y)=\sqrt{2}[p(x,y)-p(x+2,y+2)]\\ g_{\mathrm{ld}}(x+1,y)=\sqrt{2}[p(x+1,y)-p(x+3,y+2)]\\ g_{\mathrm{ld}}(x,y+1)=\sqrt{2}[p(x,y+1)-p(x+2,y+3)]\\ g_{\mathrm{ld}}(x+1,y+1)=\sqrt{2}[p(x+1,y+1)-p(x+3,y+3)]\end{array}\right.---\left(3\right)$

$\left\{\begin{array}{c}{g}_{\mathrm{rd}}(x,y)=\sqrt{2}[p(x+2,y)-p(x,y+2)]\\ g_{\mathrm{rd}}(x+1,y)=\sqrt{2}[p(x+3,y)-p(x+1,y+2)]\\ g_{\mathrm{rd}}(x,y+1)=\sqrt{2}[p(x+2,y+1)-p(x,y+3)]\\ g_{\mathrm{rd}}(x+1,y+1)=\sqrt{2}[p(x+3,y+1)-p(x+1,y+3)]\end{array}\right.---\left(4\right)$

G in formula _h(), g _v(), g _ld(), g _rd() represents respectively the gradient filter in level, vertical, lower-left, lower right, and p () represents the luminance component pixel value in present encoding tree unit, and (x, y) represents the coordinate of each 4 * 4 unit top left corner pixel.In the present invention, utilize SAG to represent along the gradient absolute value of a direction and, computing formula is as shown in (5) formula.

SAG _dir＝∑|g _dir(x,y)|,dir∈{h,v,ld,rd} （5）

Using minimal gradient absolute value and direction as main grain direction, the gradient absolute value of main grain direction and be denoted as SAG _min, the gradient absolute value of its orthogonal direction and be denoted as SAG _org.Texture complication complexity is defined as SAG _orgwith SAG _minbetween difference, that is:

complexity＝SAG _org-SAG _min （6）

The detailed process of dividing as shown in Figure 4.To each code tree unit, be first divided into the unit of N * N 4 * 4, to each unit Grad in calculated level, vertical, lower-left, lower right respectively.Then Texture complication and the division to each coding unit of calculating of code tree unit from 64 * 64 to 8 * 8 recurrence, finally determines the division of whole code tree unit.Recursive procedure is described below:

(1), to present encoding unit, (5) formula of utilization is calculated respectively gradient absolute value and the SAG on four direction _h, SAG _v, SAG _ld, SAG _rd, and obtain main grain direction and the Texture complication complexity of present encoding unit.

(2) if the Texture complication complexity of present encoding unit is less than threshold value TH, directly present encoding unit is set to smooth region, and calculation rate distortion cost, no longer divides, and returns to last layer recurrence.

(3) otherwise, check whether present encoding unit can be divided, if cannot divide, calculation rate distortion cost return to last layer recurrence.

(4) otherwise, by present encoding dividing elements, be four subelements, enter successively lower one deck recurrence.

(5) check whether four subelements are all confirmed as smooth region, and smooth region calculates the rate distortion costs of present encoding unit in this way, otherwise go to step (8).

(6) if the rate distortion costs of present encoding unit be less than or equal to four subelements rate distortion costs and, present encoding unit is not divided, and threshold value TH is tuned up simultaneously.

(7) if the rate distortion costs of present encoding unit be greater than four subelements rate distortion costs and, present encoding dividing elements, simultaneously that threshold value TH fine setting is little.

(8) detect whether there is not coding unit, if having, return to last layer recurrence, otherwise exit recurrence.

In general, the distortion that the corresponding encoder of existing video encoding standard produces is mainly caused by quantification, if the difference between neighbor is less than quantization step, illustrates that this difference has not just existed after quantizing.Therefore can set by quantization step the threshold value of Texture complication, yet quantization step in video encoding standard is for transform domain.The present invention, according to the law of conservation of energy between spatial domain and transform domain, is defined as the size of present encoding unit and the product of quantization step, i.e. TH=CUsize * QPstep by the initial threshold in recursive procedure.

The update strategy that threshold value TH tunes up is: TH '=TH+ α * (Cost _fine-Cost _coarse)/2, the α is here the multiplication factor of the linear model between Texture complication and rate distortion costs.

Threshold value TH finely tunes little update strategy: TH ' is TH '=TH-a, and a is here constant.

So just can be according to the division of Texture complication and rate distortion costs acquisition code tree unit.

2. the predictive mode based on grain direction is selected fast

Than H.264/AVC, video encoding standard HEVC of new generation has introduced the concept of predicting unit (Prediction Unit, PU).For infra-frame prediction, predicting unit has two kinds of dividing mode.For minimum coding unit, predicting unit can be identical with coding unit, also can on coding unit basis, divide by further four forks; And for remaining coding unit, predicting unit can only be identical with coding unit.HEVC also provides more meticulous prediction direction simultaneously, and each predicting unit has 35 kinds of predictive modes, has greatly increased encoder complexity.

Improve code efficiency be at all guaranteeing, under the prerequisite of coding quality, to reduce candidate's predictive mode as far as possible.Generally, final predictive mode and the grain direction of selecting is close.Table 1 has provided the optimum prediction mode probability close with grain direction in various predicting unit.As can be seen from the table, the probability that optimum prediction mode and grain direction are close can reach more than 80%.In addition, in the quick division of first step coding unit of the present invention, obtained the main grain direction of each coding unit.Therefore, can utilize this intermediate object program to reduce the quantity of candidate's predictive mode as far as possible, thereby improve code efficiency.It should be noted that, for all predicting unit, the planar that HEVC provides and DC predictive mode all retain, and the present invention only considers to reduce candidate's prediction direction of angle prediction.

The probability that each predicting unit of table 1 optimum prediction mode under different quantization steps is close with grain direction

Quantization step	64×64	32×32	16×16	8×8	4×4
						22	100%	79.63%	80.76%	84.32%	87.45%
27	86.36%	78.80%	83.64%	87.67%	89.24%
						32	80.43%	82.21%	89.47%	88.88%	89.86%
37	78.67%	86.53%	89.83%	89.20%	89.06%
						On average	86.37%	81.79%	85.93%	87.52%	88.90%

The probability close with grain direction due to optimum prediction mode is higher, and the present invention utilizes main grain direction several prediction direction around to carry out angle prediction.Embodiment is as follows:

(1) define candidate's predictive mode array that a size is 11, for storing candidate's predictive mode of screening, comprising planar pattern, DC pattern, main grain direction and main grain direction some prediction direction around.In specific embodiment of the invention, in main grain direction both sides, 4 prediction direction have respectively been got.

(2) obtain the corresponding prediction mode number mode of main grain direction of current predicting unit _texture, and utilize formula (7) to calculate angle estimation range [mode _left, mode _right].

$\left\{\begin{array}{c}{\mathrm{mode}}_{\mathrm{left}}={\mathrm{mode}}_{\mathrm{texture}}-4\\ {\mathrm{mode}}_{\mathrm{right}}={\mathrm{mode}}_{\mathrm{texture}}+4\end{array}\right.---\left(7\right)$

(3) utilize formula (8) to obtain the predictive mode in main grain direction left side.

$\left\{\begin{array}{cc}\mathrm{mode}\mathrm{Idx}=\mathrm{mode}+33& \mathrm{mode}<2\\ \mathrm{mode}\mathrm{Idx}=\mathrm{mode}& \mathrm{mode}\&GreaterEqual;2\end{array}\right.,\mathrm{mode}\&Element;[{\mathrm{mode}}_{\mathrm{left}},{\mathrm{mode}}_{\mathrm{texture}}-1]---\left(8\right)$

(4) utilize formula (9) to obtain the predictive mode on main grain direction right side.

$\left\{\begin{array}{cc}\mathrm{mode}\mathrm{Idx}=\mathrm{mode}-33& \mathrm{mode}>34\\ \mathrm{mode}\mathrm{Idx}=\mathrm{mode}& \mathrm{mode}\&le;34\end{array}\right.,\mathrm{mode}\&Element;[{\mathrm{mode}}_{\mathrm{texture}}+1,{\mathrm{mode}}_{\mathrm{right}}]---\left(9\right)$

(5) to candidate's predictive mode set of screening, { modeIdx} stores in candidate's predictive mode array.According to coarse mode selection and rate-distortion optimization in HEVC standard, select therefrom to choose best predictive mode.

Disclosed is above only specific embodiments of the invention.According to technological thought provided by the invention, those skilled in the art can think and variation, all should fall within the scope of protection of the present invention.

Claims (6)

1. the HEVC infra-frame prediction fast schema selection method based on texture analysis, is characterized in that: the method step is as follows:

Step (1), to present encoding tree unit, calculate image complexity and the main grain direction of each possible coding unit;

Step (2), to present encoding tree unit, according to Z scanning sequency from top to bottom repeated execution of steps (3) to step (6), until scanned present encoding tree unit;

Step (3), if the image complexity of present encoding unit is less than or equal to threshold value, is judged that this coding unit is as smooth unit, otherwise is judged to be complex unit;

Step (4), for level and smooth coding unit, utilize step (5) to calculate the rate distortion costs of present encoding unit;

Step (5), for each predicting unit in present encoding unit, only retain the main grain direction in this unit 8 prediction direction and planar prediction and DC predictive mode around, according to HEVC video encoding standard, carry out that coarse mode is selected and rate-distortion optimization selection;

Step (6), for complicated coding unit, according to step (3) to four subelements of step (6) successively recursive scanning, if four subelements are all judged to be smooth unit, utilize step (5) to calculate the rate distortion costs of present encoding unit, and judge that according to rate distortion costs minimum principle whether present encoding unit is divided, and upgrades threshold value simultaneously.

2. the HEVC infra-frame prediction fast schema selection method based on texture analysis as claimed in claim 1, is characterized in that: described step (1) specifically comprises the steps:

Step (11), to each 4 * 4 the unit in present encoding tree unit, respectively along absolute value and the SAG of level, vertical, lower-left, bottom right four direction compute gradient compute gradient;

Step (12), using minimal gradient absolute value and gradient direction as the main grain direction of this unit, described main grain direction refers to the grain direction of the most of pixel in this unit;

Step (13), the absolute value that SAG on main grain direction and its orthogonal direction is differed from are defined as the Texture complication of this unit, and utilize the product of quantization step and this cell size as initial threshold.

3. the HEVC infra-frame prediction fast schema selection method based on texture analysis as claimed in claim 1, is characterized in that: described step (5) specifically comprises the steps:

Step (51), define candidate's predictive mode array;

Step (52), according to step (1), obtain the main grain direction of current predicting unit, and using main grain direction and 8 prediction direction around thereof as candidate's prediction direction;

Step (53), candidate's prediction direction is mapped to the angle predictive mode in HEVC coding standard, and adds the angle predictive mode of selection and planar and DC pattern to candidate's predictive mode array;

Step (54), to each predictive mode in candidate's predictive mode array, according to HEVC coding standard, carry out coarse mode selection and R-D optimized mode selection.

4. the HEVC infra-frame prediction fast schema selection method based on texture analysis as claimed in claim 1, is characterized in that: described step (6) specifically comprises the steps:

Step (61), according to step (3) to four subelements of step (6) successively recursive scanning, if four subelements are all judged to be smooth unit, utilize step (5) to calculate the rate distortion costs Cost of present encoding unit _coarse, otherwise the subelement of next level of continuation recursive scanning;

Step (62) is if the rate distortion costs Cost of present encoding unit _coarsethe rate distortion costs and the Cost that are less than or equal to four subelements _fine, threshold value is tuned up, otherwise threshold value is turned down.

5. the HEVC infra-frame prediction fast schema selection method based on texture analysis as claimed in claim 4, is characterized in that: the update strategy that in described step (62), threshold value tunes up is:

TH′＝TH+α×(Cost _fine-Cost _coarse)/2

In formula, TH is the threshold value before upgrading, and TH ' is the threshold value after upgrading, and α is the multiplication factor of the linear model between Texture complication and rate distortion costs.

6. the HEVC infra-frame prediction fast schema selection method based on texture analysis as claimed in claim 4, is characterized in that: the update strategy that in described step (62), threshold value is turned down is:

TH′＝TH-a

In formula, TH is the threshold value before upgrading, and TH ' is the threshold value after upgrading, and a is constant.

Images