CN102752596B - Rate distortion optimization method - Google Patents

Abstract

The invention relates to a rate distortion optimization method, which comprises the following steps: A. performing rate distortion calculation by using a first calculation index to obtain a first group of distortion/code rate values; B. performing rate distortion calculation by using a second calculation index to obtain a second group of distortion/code rate values; C. acquiring a distortion/code rate value corresponding to the minimum coding cost in a first group of distortion/code rate values by adopting a first rate-distortion optimization mode; D. obtaining a plurality of candidate code rate values and corresponding coding costs according to the distortion/code rate value of the minimum coding cost; E. obtaining a second group of distortion/code rate value compensation costs corresponding to the candidate code rate values by adopting a second rate distortion optimization mode; F. and determining the minimum value of the comprehensive cost according to the coding cost and the compensation cost, and further acquiring the optimal coding mode. The rate distortion optimization method of the invention enables the rate distortion optimization result to be optimal by integrating two rate distortion optimization indexes.

Description

A kind of Rate-distortion optimization method

Technical field

The present invention relates to image/video codec domain, more particularly, relate to a kind of Rate-distortion optimization method.

Background technology

With international standard H264 for reference, when basic coding unit is encoded, different coding modes can be selected.The selection of coding mode comprises infra-frame prediction mode or inter prediction way choice, also partitioning scheme (the routine INTRA-4x4 to basic coding unit can be comprised, INTRA-8x8, INTRA-16x16, SKIP, DIRECT, INTER-16x16, INTER-16x8, INTER-8x16, INTER-8x8, INTER-8x8 can be further divided into INTER-8x8, INTER-8x4, INTER-4x8, INTER-4x4) selection, can also comprise prediction block position (routine Intra_4x4_Vertical, Intra_4x4_Horizontal, Intra_4x4_Diagonal_Down_Left, Intra_4x4_Diagonal_Down_Right, Intra_4x4_Vertical_Right, Intra_4x4_Horizontal_Down, Intra_4x4_Vertical_Left, Intra_4x4_Horizontal_Up, Intra_4x4_DC) selection.The determination of coding mode is that percent of pass aberration optimizing realizes, and wherein rate-distortion optimization is the process of minimizing to following cost function J,

J(s,c,mode|QP)=D(s,c,mode|QP)+λ _modeR(s,c,mode|QP) （1）

Wherein D is distortion value, and R is code check value, s and c represents former figure respectively and build the corresponding basic coding unit of image again by encoding and decoding process, and mode represents the coding mode selected of basic coding unit, and QP is quantization parameter, λ _modebe used to the LaGrange parameter of compromise distortion value and code check value.

Rate-distortion optimization is under the condition that quantization parameter QP is fixed, determines the mode that above-mentioned cost function J can be made minimum.In H264 standard, λ _modedetermine namely have by quantization parameter QP

λ _mode=0.85×2 ^(QP-12)/3 （2）

Above-mentioned rate-distortion optimization is at appointment λ _modewhen determine optimum coding mode mode.But, above-mentioned λ _modedetermination mode just with MSE(mean squared error, mean square error) statistical approximation result when measuring distortion.And MSE correctly can express distortion, can often occur on the contrary feeling inconsistent situation with the distortion of human eye.Such as the distorted image that some MSE values are close, the distortion sensation of human eye may have a great difference.So measure the method for distortion with MSE and perfect not, the result that therefore can cause above-mentioned rate-distortion optimization also and non-optimal.

Therefore, be necessary to provide a kind of Rate-distortion optimization method, to solve the problem existing for prior art.

Summary of the invention

The technical problem to be solved in the present invention is, for the expression distortion that Rate-distortion optimization method of the prior art can not be correct, cause the defect of the result badly of rate-distortion optimization, there is provided a kind of by comprehensive two kinds of rate-distortion optimization indexs, make the Rate-distortion optimization method of rate-distortion optimization result the best.

The technical solution adopted for the present invention to solve the technical problems is: the present invention relates to a kind of Rate-distortion optimization method, it comprises step:

A, adopt the first parameter to carry out rate distortion calculating, obtain first group of distortion/code check value;

B, adopt the second parameter to carry out rate distortion calculating, obtain second group of distortion/code check value;

C, adopt the first rate-distortion optimization mode, obtain the distortion/code check value of corresponding minimum code cost in described first group of distortion/code check value;

D, distortion/code check value according to described minimum code cost, obtain multiple candidate code check value and corresponding Coding cost;

E, adopt the second rate-distortion optimization mode, obtain the compensation cost of the second group distortion/code check value corresponding with described candidate code check value;

F, according to described Coding cost and described compensation cost, determine the minimum value of integrate-cost, and then obtain forced coding pattern.

In Rate-distortion optimization method of the present invention, described first parameter is the squared difference and the index that correspond to mean square error, is specially:

$\mathrm{SSD}=\underset{(x,y)\∈A}{\mathrm{\Σ}}{(s(x,y)-c(x,y))}^2,$

Wherein SSD is squared difference and index, s represents the basic coding unit of former figure, c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and x, y represent the location of pixels in described basic coding unit, and A represents the pixel coverage of described basic coding unit.

In Rate-distortion optimization method of the present invention, described second parameter is structure similarity index, is specially:

SSIM(s,c)=L(s,c)·G(s.c)·H(s,c)，

$L(s,c)=\frac{2{\mathrm{\μ}}_s{\mathrm{\μ}}_c+C_1}{{\mathrm{\μ}}_s^2+{\mathrm{\μ}}_c^2+C_1},$

$G(s,c)=\frac{2{\mathrm{\σ}}_s{\mathrm{\σ}}_c+C_2}{{\mathrm{\σ}}_s^2+{\mathrm{\σ}}_c^2+C_2},$

$H(s,c)=\frac{{\mathrm{\σ}}_{\mathrm{sc}}+C_3}{{\mathrm{\σ}}_s{\mathrm{\σ}}_c+C_3},$

Wherein SSIM is structure similarity index, and s represents the basic coding unit of former figure, and c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and L represents brightness similarity, and G represents contrast similarity, and H represents structural similarity; μ _sfor the mean value of pixel in s, μ _cfor the mean value of pixel in c, for the variance yields of pixel in s, for the variance yields of pixel in c, σ _scfor the covariance value of respective pixel in pixel in s and c, C ₁, C ₂, C ₃for constant.

In Rate-distortion optimization method of the present invention, described step C is specially: obtain Coding cost J corresponding in described first group of distortion/code check value by following formula _i ¹, then according to described Coding cost J _i ¹determine the distortion/code check value of corresponding minimum code cost,

J _i ¹(s,c,mode|QP)=D _i ¹(s,c,mode|QP)+λ ₁R _i(s,c,mode|QP)，

Wherein J _i ¹for described Coding cost, D _i ¹for described first group of distortion value, R _ifor described code check value, s represents the basic coding unit of former figure, and c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and mode represents the coding mode selected of described basic coding unit, and QP is quantization parameter, λ ₁be used to the LaGrange parameter of compromise described first group of distortion value and described code check value, i is positive integer; Wherein

λ ₁=0.85×2 ^(QP-12)/3。

In Rate-distortion optimization method of the present invention, in described step D, by the code check value of described minimum code cost, and at least two the code check values adjacent with the code check value of described minimum code cost are as described candidate code check value.

In Rate-distortion optimization method of the present invention, described step e is specially: the compensation cost being obtained the second group distortion/code check value corresponding with described candidate code check value by following formula,

$J_i^2=D_i^2+k_i\·R_i,$

Wherein for described second group of distortion value, R _ifor described code check value, for described compensation cost, k _ithe curve formed for described second group of distortion/code check value is at the slope of respective points, and i is positive integer.

In Rate-distortion optimization method of the present invention, described step F is specially: obtain described integrate-cost J by following formula _i, and according to described integrate-cost J _iminimum value determine described forced coding pattern,

$J_i=w_1\·J_i^1+w_2\·J_i^2,$

Wherein for described Coding cost, for described compensation cost, w ₁, w ₂for weight coefficient, i is positive integer.

In Rate-distortion optimization method of the present invention, during as estimated the motion vector of inter prediction, described step C is specially: obtain Coding cost J corresponding in described first group of distortion/code check value by following formula _i ¹, then according to described Coding cost J _i ¹determine the distortion/code check value of corresponding minimum code cost,

J _i ¹(s,c,mv|QP)=D _i ¹(s,c,mv|QP)+λ _motionR _i(s,c,mv|QP)，

Wherein for described Coding cost, D _i ¹for described first group of distortion value, R _ifor described code check value, s represents the basic coding unit of former figure, and c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and mv represents motion vector during inter prediction, and QP is quantization parameter, λ _motionbe LaGrange parameter when asking motion vector, i is positive integer; Wherein

λ _motion=0.85 × 2 ^(QP-12)/3or ${\mathrm{\λ}}_{\mathrm{motion}}=\sqrt{0.85\×2^{(\mathrm{QP}-12)/3}}.$

In Rate-distortion optimization method of the present invention, described Rate-distortion optimization method also comprises step: according to the minimum value of described integrate-cost, obtain best LaGrange parameter, according to described best LaGrange parameter, obtain the LaGrange parameter when motion vector of inter prediction is estimated.

In Rate-distortion optimization method of the present invention, obtain described best LaGrange parameter according to following formula,

λ ^* _mode=(J-D)/R，

Wherein J is minimum integrate-cost, and D is the distortion value of described forced coding pattern, and R is the code check value of described forced coding pattern, λ ^* _modefor described best LaGrange parameter,

LaGrange parameter when the described motion vector to inter prediction is estimated is obtained by following formula,

λ ^* _motion=λ ^* _modeor ${{\mathrm{\λ}}^{*}}_{\mathrm{motion}}=\sqrt{{{\mathrm{\λ}}^{*}}_{\mathrm{mode}}},$

Wherein λ ^* _motionfor LaGrange parameter when the described motion vector to inter prediction is estimated.

Implement Rate-distortion optimization method of the present invention, there is following beneficial effect: by comprehensive two kinds of rate-distortion optimization indexs, make rate-distortion optimization result best.The Rate-distortion optimization method avoiding prior art can not be correct expression distortion, cause the technical problem of the result badly of rate-distortion optimization.

Accompanying drawing explanation

Below in conjunction with drawings and Examples, the invention will be further described, in accompanying drawing:

Fig. 1 is the flow chart of the first preferred embodiment of Rate-distortion optimization method of the present invention;

Fig. 2 is the idiographic flow block diagram of the first preferred embodiment of Rate-distortion optimization method of the present invention;

Fig. 3 is the schematic diagram obtaining candidate's code check value in Rate-distortion optimization method of the present invention.

Embodiment

In order to make object of the present invention, technical scheme and advantage clearly understand, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.

Please refer to Fig. 1, Fig. 1 is the flow chart of the first preferred embodiment of Rate-distortion optimization method of the present invention.This Rate-distortion optimization method starts from:

Step 101, adopts the first parameter to carry out rate distortion calculating, obtains first group of distortion/code check value;

Step 102, adopts the second parameter to carry out rate distortion calculating, obtains second group of distortion/code check value;

Step 103, adopts the first rate-distortion optimization mode, obtains the distortion/code check value of corresponding minimum code cost in first group of distortion/code check value,

Step 104, according to the distortion/code check value of minimum code cost, obtains multiple candidate code check value and corresponding Coding cost;

Step 105, adopts the second rate-distortion optimization mode, obtains the compensation cost of the second group distortion/code check value corresponding with candidate code check value;

Step 106, according to described Coding cost and described compensation cost, determines the minimum value of integrate-cost, and then obtains forced coding pattern.

Please refer to Fig. 2, Fig. 2 is the idiographic flow block diagram of the first preferred embodiment of Rate-distortion optimization method of the present invention.Below by the specific implementation process of Fig. 2 detailed description the first Rate-distortion optimization method of the present invention.

First, before step 101, input coding elementary cell is to code device, then all coding modes of this coding elementary cell are selected one by one, for each coding mode, complete corresponding Code And Decode process, obtain encode elementary cell corresponding build image again, the processing method of Code And Decode can be carried out according to various video encoding standard.Such as, coded treatment can perform following steps successively: determine prediction block, residual computations, dct transform (discrete cosine transform, Discrete Cosine Transform), quantification, entropy code.Decoding process is then the reverse operating to above steps.The concrete grammar of Rate-distortion optimization method of the present invention to this Code And Decode process does not limit.

Come step 101 subsequently, in a step 101, adopt the first parameter to the basic coding unit of former figure and build the corresponding basic coding unit of image again by encoding and decoding process, carrying out rate distortion calculating, obtain first group of distortion/code check value.In the present embodiment, the first parameter is corresponding MSE(mean square error, mean squared error) SSD(squared difference and, sum of squared difference) index.This rate distortion calculates and is specially:

$\mathrm{SSD}=\underset{(x,y)\∈A}{\mathrm{\Σ}}{(s(x,y)-c(x,y))}^2,$

Wherein SSD is squared difference and index, s represents the basic coding unit of former figure, c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and x, y represent the position of each pixel in described basic coding unit, and A represents the pixel coverage of basic coding unit.

By the calculating of above formula, s and c corresponding according to each coding mode, obtains corresponding code check value R _iwith the distortion value D expressed by SSD index _i ¹, first group of distortion/code check value can be obtained like this, be designated as (D _i ¹, R _i), wherein i is positive integer, and the maximum of i is to should the number of coding mode of basic coding unit.

Come step 102 subsequently, in a step 102, adopt the second parameter to the basic coding unit of former figure and build the corresponding basic coding unit of image again by encoding and decoding process, carrying out rate distortion calculating, obtain second group of distortion/code check value.In the present embodiment, the second parameter is SSIM(structural similarity, structural similarity) index.This rate distortion calculates and is specially:

SSIM(s,c)=L(s,c)·G(s.c)·H(s,c)，

$L(s,c)=\frac{2{\mathrm{\μ}}_s{\mathrm{\μ}}_c+C_1}{{\mathrm{\μ}}_s^2+{\mathrm{\μ}}_c^2+C_1},$

$G(s,c)=\frac{2{\mathrm{\σ}}_s{\mathrm{\σ}}_c+C_2}{{\mathrm{\σ}}_s^2+{\mathrm{\σ}}_c^2+C_2},$

$H(s,c)=\frac{{\mathrm{\σ}}_{\mathrm{sc}}+C_3}{{\mathrm{\σ}}_s{\mathrm{\σ}}_c+C_3},$

Wherein SSIM is structure similarity index, and s represents the basic coding unit of former figure, and c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and L represents brightness similarity, and G represents contrast similarity, and H represents structural similarity; μ _sfor the mean value of pixel in s, μ _cfor the mean value of pixel in c, for the variance yields of pixel in s, for the variance yields of pixel in c, σ _scfor the covariance value of respective pixel in pixel in s and c, C ₁, C ₂, C ₃for constant, C ₁, C ₂, C ₃value can obtain by experiment.

By the calculating of above formula, s and c corresponding according to each coding mode, can obtain corresponding code check value R _iwith the distortion value of expressing by SSIM index second group of distortion/code check value can be obtained like this, be designated as wherein i is positive integer, and the maximum of i is to should the number of coding mode of basic coding unit.

Because employing SSD index calculate and SSIM index calculate are the different calculation methods of distortion value, and to code check value R _ithere is no impact, so first group of distortion/code check value (D _i ¹, R _i) and second group of distortion/code check value in code check value R _iidentical.

Come step 103 subsequently, in step 103, adopt the first rate-distortion optimization mode, obtain the distortion/code check value of corresponding minimum code cost in first group of distortion/code check value.Be specially, obtain Coding cost J corresponding in first group of distortion/code check value by following formula _i ¹, then according to Coding cost J _i ¹determine the distortion/code check value of corresponding minimum code cost,

J _i ¹(s,c,mode|QP)=D _i ¹(s,c,mode|QP)+λ ₁R _i(s,c,mode|QP)，

Wherein J _i ¹for Coding cost, D _i ¹be first group of distortion value, R _ifor code check value, s represents the basic coding unit of former figure, and c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and mode represents the coding mode selected of described basic coding unit, and QP is quantization parameter, λ ₁be used to the LaGrange parameter (i.e. first method parameter) of compromise described first group of distortion value and described code check value, i is positive integer; Wherein

λ ₁=0.85×2 ^(QP-12)/3。

Come step 104 subsequently, at step 104, according to the distortion/code check value of minimum code cost, obtain multiple candidate code check value and corresponding Coding cost.Specifically as shown in Figure 3, Fig. 3 is the schematic diagram obtaining candidate's code check value in Rate-distortion optimization method of the present invention.In Fig. 3, abscissa is code check value R _i, ordinate is distortion value D _i, comprising first group of distortion/code check value (D _i ¹, R _i) matched curve and second group of distortion/code check value matched curve.According to first group of distortion/code check value (D _i ¹, R _i) the code check value of minimum code cost that obtains, and at least two the code check values adjacent with the code check value of this minimum code cost are as candidate code check value.As in Fig. 3 coding cost minimum, then by code check value R ₃, and with code check value R ₃adjacent code check value R ₂with code check value R ₄alternatively code check value, candidate's code check value R ₂corresponding Coding cost is J ₂ ¹, candidate's code check value R ₃corresponding Coding cost is J ₃ ¹, candidate's code check value R ₄corresponding Coding cost is J ₄ ¹.Certainly also can select more candidate's code check value here, as will with code check value R ₃4 adjacent code check value R ₁, code check value R ₂, code check value R ₄and code check value R ₅all alternatively code check value.On the other hand, if R ₃the left side or the right when corresponding to the adjacent code check value of coding mode, then only select R ₃with corresponding monolateral adjacent code check value alternatively code check value.

Come step 105 subsequently, in step 105, adopt the second rate-distortion optimization mode, obtain the compensation cost of the second group distortion/code check value corresponding with candidate code check value.

For the data point in Fig. 3, code check value sorts from small to large and is followed successively by R ₁, R ₂, R ₃, R ₄and R ₅, the curve that such second group of distortion/code check value is formed is at candidate's code check value R ₂corresponding point (D ₂ ², R ₂) slope be:

$k_2=-\frac{D_3^2-D_1^2}{R_3-R_1};$

The curve that second group of distortion/code check value is formed is at candidate's code check value R ₃corresponding point (D ₃ ², R ₃) slope be:

$k_3=-\frac{D_4^2-D_2^2}{R_4-R_2};$

The curve that second group of distortion/code check value is formed is at candidate's code check value R ₄corresponding point (D ₄ ², R ₄) slope be:

$k_4=-\frac{D_5^2-D_3^2}{R_5-R_3};$

The compensation cost of the second group distortion/code check value corresponding with candidate code check value is obtained again by following formula,

$J_i^2=D_i^2+k_i\·R_i,$

Wherein be second group of distortion value, R _ifor code check value, for compensating cost, k _ithe curve formed for described second group of distortion/code check value is at the slope (above try to achieve) of respective points, and i is positive integer.Candidate's code check value R can be calculated respectively like this ₂compensation cost candidate's code check value R ₃compensation cost and candidate's code check value R ₄compensation cost in this step, the curve that second group of distortion/code check value is formed also can obtain by other modes at the slope of respective points, as first asked for the matched curve of R ~ D, recycles this matched curve slope calculations.Therefore the curve that second group of distortion/code check value is formed does not limit the scope of the invention at the concrete obtain manner of the slope of respective points.

Come step 106 subsequently, according to Coding cost J _i ¹and compensation cost determine integrate-cost J _iminimum value, and then obtain forced coding pattern.

Integrate-cost J is obtained by following formula _i:

$J_i=w_1\·J_i^1+w_2\·J_i^2,$

Wherein for Coding cost, for compensating cost, w ₁, w ₂for weight coefficient, can get fixed value or be set by the user, i is positive integer.Last according to integrate-cost J _iminimum value determination forced coding pattern export.

As the second preferred embodiment of Rate-distortion optimization method of the present invention, rate distortion when Rate-distortion optimization method of the present invention also can be estimated the motion vector of inter prediction is optimized.Be with the difference of the first preferred embodiment,

Coding cost J corresponding in first group of distortion/code check value is obtained by following formula _i ¹, then according to Coding cost J _i ¹determine the distortion/code check value of corresponding minimum code cost,

J _i ¹(s,c,mv|PQ)=D _i ¹(s,c,mv|PQ)+λ _motionR _i(s,c,mv|PQ)，

Wherein for Coding cost, D _i ¹be first group of distortion value, R _ifor code check value, s represents the basic coding unit of former figure, and c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and mv represents motion vector during inter prediction, and QP is quantization parameter, λ _motionbe the LaGrange parameter (first method parameter) when asking motion vector, i is positive integer; Wherein

λ _motion=0.85 × 2 ^(QP-12)/3or ${\mathrm{\λ}}_{\mathrm{motion}}=\sqrt{0.85\×2^{(\mathrm{QP}-12/3)}}.$

Other steps and the first preferred embodiment same or similar, specifically refer to the first preferred embodiment of the present invention.

As the 3rd preferred embodiment of Rate-distortion optimization method of the present invention, Rate-distortion optimization method of the present invention also can according to the minimum value of the integrate-cost in the first preferred embodiment, obtain LaGrange parameter when estimating the motion vector of inter prediction, rate distortion during to estimate the motion vector of inter prediction is better optimized.

First according to the minimum value of the integrate-cost in the first preferred embodiment, obtain best LaGrange parameter, be specially:

λ ^* _mode=(J-D)/R，

Wherein J is minimum integrate-cost, and D is the distortion value of forced coding pattern, and R is the code check value of forced coding pattern, λ ^* _modefor best LaGrange parameter.

Then according to best LaGrange parameter, obtain the LaGrange parameter when motion vector of inter prediction is estimated, be specially

λ ^* _motion=λ ^* _modeor ${{\mathrm{\λ}}^{*}}_{\mathrm{motion}}=\sqrt{{{\mathrm{\λ}}^{*}}_{\mathrm{mode}}},$

Wherein λ ^* _motionlaGrange parameter during for estimating the motion vector of inter prediction.

In sum, Rate-distortion optimization method of the present invention, by comprehensive two kinds of rate-distortion optimization indexs, makes rate-distortion optimization result best.Employ the combination of SSD index and SSIM index in an embodiment of the present invention, complex optimum can certainly be carried out by other index, such as: SSD can use SAD(absolute difference sum, Sum of Absolute Differences) or SATD(Hadamard conversion after again absolute value summation, Sum of Absolute Transformed Differences) etc. replacement, SSIM can use the multiple dimensioned SSIM of MS-SSIM(, Multi-Scale SSIM) or the replacement such as VIF (visual information fidelity, Visual Information Fidelity); Rate-distortion optimization method of the present invention can look for forced coding pattern from all coding modes of coding elementary cell simultaneously, also can look for forced coding pattern from a part of coding mode of specifying; Rate-distortion optimization can realize according to the type (i.e. I frame, P frame or B frame) of frame coding is independent separately in addition; Rate-distortion optimization method of the present invention avoids the expression distortion that the Rate-distortion optimization method of prior art can not be correct well, causes the technical problem of the result badly of rate-distortion optimization.

The foregoing is only embodiments of the invention; not thereby the scope of the claims of the present invention is limited; every equivalent structure transformation utilizing specification of the present invention and accompanying drawing content to do, or be directly or indirectly used in other relevant technical fields, be all in like manner included in scope of patent protection of the present invention.

Claims (7)

1. a Rate-distortion optimization method, is characterized in that, comprises step:

A, adopt the first parameter to carry out rate distortion calculating, obtain first group of distortion/code check value;

B, adopt the second parameter to carry out rate distortion calculating, obtain second group of distortion/code check value;

C, adopt the first rate-distortion optimization mode, obtain the distortion/code check value of corresponding minimum code cost in described first group of distortion/code check value;

D, distortion/code check value according to described minimum code cost, obtain the multiple candidate code check value in described first group of distortion/code check value and corresponding Coding cost;

E, adopt the second rate-distortion optimization mode, obtain the compensation cost of the second group distortion/code check value corresponding with described candidate code check value;

F, according to described Coding cost and described compensation cost, determine the minimum value of integrate-cost, and then obtain forced coding pattern;

Wherein, in described step D, by the code check value of described minimum code cost, and at least two the code check values adjacent with the code check value of described minimum code cost are as described candidate code check value;

Described step e is specially: the compensation cost being obtained the second group distortion/code check value corresponding with described candidate code check value by following formula,

$J_i^2=D_i^2+k_i\·R_i,$

Wherein for described second group of distortion value, R _ifor described code check value, for described compensation cost, k _ithe curve formed for described second group of distortion/code check value is at the slope of respective points, and i is positive integer;

Described step F is specially: obtain described integrate-cost J by following formula _i, and according to described integrate-cost J _iminimum value determine described forced coding pattern,

$J_i=w_1\·J_i^1+w_2\·J_i^2,$

Wherein for described Coding cost, for described compensation cost, w ₁, w ₂for weight coefficient, i is positive integer.

2. Rate-distortion optimization method according to claim 1, is characterized in that, described first parameter is the squared difference and the index that correspond to mean square error, is specially:

$\mathrm{SSD}=\underset{(x,y)\∈A}{\mathrm{\Σ}}{(s(x,y)-c(x,y))}^2,$

Wherein SSD is squared difference and index, s represents the basic coding unit of former figure, c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and x, y represent the location of pixels in described basic coding unit, and A represents the pixel coverage of described basic coding unit.

3. Rate-distortion optimization method according to claim 1, is characterized in that, described second parameter is structure similarity index, is specially:

SSIM(s,c)＝L(s,c)·G(s.c)·H(s,c)，

$L(s,c)=\frac{2{\mathrm{\μ}}_s{\mathrm{\μ}}_c+C_1}{{\mathrm{\μ}}_s^2+{\mathrm{\μ}}_c^2+C_1},$

$G(s,c)=\frac{2{\mathrm{\σ}}_s{\mathrm{\σ}}_c+C_2}{{\mathrm{\σ}}_s^2+{\mathrm{\σ}}_c^2+C_2},$

$H(s,c)=\frac{{\mathrm{\σ}}_{\mathrm{sc}}+C_3}{{\mathrm{\σ}}_s{\mathrm{\σ}}_c+C_3},$

Wherein SSIM is structure similarity index, and s represents the basic coding unit of former figure, and c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and L represents brightness similarity, and G represents contrast similarity, and H represents structural similarity; μ _sfor the mean value of pixel in s, μ _cfor the mean value of pixel in c, for the variance yields of pixel in s, for the variance yields of pixel in c, σ _scfor the covariance value of respective pixel in pixel in s and c, C ₁, C ₂, C ₃for constant.

4. Rate-distortion optimization method according to claim 1, is characterized in that, described step C is specially: obtain Coding cost J corresponding in described first group of distortion/code check value by following formula _i ¹, then according to described Coding cost J _i ¹determine the distortion/code check value of corresponding minimum code cost,

J _i ¹(s,c,mod e|QP)＝D _i ¹(s,c,mod e|QP)+λ ₁R _i(s,c,mod e|QP)，

Wherein J _i ¹for described Coding cost, D _i ¹for described first group of distortion value, R _ifor described code check value, s represents the basic coding unit of former figure, and c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and mode represents the coding mode selected of described basic coding unit, and QP is quantization parameter, λ ₁be used to the LaGrange parameter of compromise described first group of distortion value and described code check value, i is positive integer; Wherein

λ ₁＝0.85×2 ^(QP-12)/3。

5. Rate-distortion optimization method according to claim 1, is characterized in that, during as estimated the motion vector of inter prediction, described step C is specially: obtain Coding cost J corresponding in described first group of distortion/code check value by following formula _i ¹, then according to described Coding cost J _i ¹determine the distortion/code check value of corresponding minimum code cost,

J _i ¹(s,c,mv|QP)＝D _i ¹(s,c,mv|QP)+λ _motion R _i(s,c,mv|QP)，

Wherein for described Coding cost, D _i ¹for described first group of distortion value, R _ifor described code check value, s represents the basic coding unit of former figure, and c represents and builds the corresponding basic coding unit of image again by encoding and decoding process, and mv represents motion vector during inter prediction, and QP is quantization parameter, λ _motionbe LaGrange parameter when asking motion vector, i is positive integer; Wherein

λ _motion=0.85 × 2 ^(QP-12)/3or ${\mathrm{\λ}}_{\mathrm{motion}}=\sqrt{0.85\×2^{(\mathrm{QP}-12)/3}}.$

6. Rate-distortion optimization method according to claim 5, it is characterized in that, described Rate-distortion optimization method also comprises step: according to the minimum value of described integrate-cost, obtain best LaGrange parameter, according to described best LaGrange parameter, obtain the LaGrange parameter when motion vector of inter prediction is estimated.

7. Rate-distortion optimization method according to claim 6, is characterized in that, obtains described best LaGrange parameter according to following formula,

λ ^* _{mod e}＝(J-D)/R，

Wherein J is minimum integrate-cost, and D is the distortion value of described forced coding pattern, and R is the code check value of described forced coding pattern, λ ^* _{mod e}for described best LaGrange parameter,

LaGrange parameter when the described motion vector to inter prediction is estimated is obtained by following formula,

λ ^* _motion=λ ^* _{mod e}or ${{\mathrm{\λ}}^{*}}_{\mathrm{motion}}=\sqrt{{{\mathrm{\λ}}^{*}}_{\mathrm{mode}}},$

Wherein λ ^* _motionfor LaGrange parameter when the described motion vector to inter prediction is estimated.