CN102088605B - Rapid interlayer prediction selection method for scalable video coding - Google Patents

Abstract

The present invention discloses a rapid interlayer prediction selection method for scalable video coding, wherein the method comprises the steps of: predicting an interlayer motion correlativity and a spatial motion correlativity of macroblock-level by using macroblock mode information, predicting an interlayer motion correlativity and a spatial motion correlativity of subblock-level by using a distance between a motion vector and a prediction vector, and rapidly selecting an interlayer motion prediction according to the motion correlativity of macroblock-level and subblock-level; selecting the interlayer motion prediction through the above mentioned steps under the condition of using interlayer residual error prediction to obtain a motion searching result; then estimating a macroblock distortion and the number of coding bits under the condition of not using the interlayer residual error prediction and using the interlayer residual error prediction by using the motion searching result, and obtaining a ratio of two rate distortion costs under the two conditions; and performing a rapid selection according to the correlativity between the rate distortion cost ratio and an interlayer residual error prediction. The rapid interlayer prediction selection method for scalable video coding effectively reduces the complexity of interlayer prediction selection on the premise of basically having no damage to the image quality.

Description

The quick inter-layer prediction system of selection of scalable video

Technical field

The invention belongs to video coding (compression) field, be specifically related to the quick inter-layer prediction system of selection of a kind of scalable video.

Background technology

Fast development along with network technology and video coding technique; People are vigorous to the increasing demand of obtaining video information through network, and how under the network environment of isomery, simultaneously various terminal uses to be provided personalized service is the new challenge that networked Video Applications faces.(ScalableVideo Coding SVC) has received extensive concern as the strong instrument that solves this difficult problem to scalable video, is the research focus of field of video encoding.Scalable video provides the video flowing with frame per second, resolution and quality scalable, and video server end or receiving terminal can extract video flowing flexibly according to factors such as the screen size of the network bandwidth, terminal equipment, disposal abilities.

Scalable video standard (Scalable Video Coding; SVC) as the scalable extension of standard H.264, the pattern of its support comprises: Intra4 * 4, Intra16 * 16, IntraBL, Inter16 * 16, Inter16 * 8, Inter8 * 16, Inter8 * 8, Inter8 * 4, Inter4 * 8, Inter4 * 4, SKIP, Direct, BLSKIP etc.Wherein, IntraBL and BLSKIP are emerging patterns among the SVC, and all the other patterns all are H.264 existing in the standard.

SVC is made up of a basic layer (Base Layer) and a plurality of enhancement layer (Enhancement Layer), and basic layer provides the most basic video quality, and enhancement layer perhaps improves basic layer from time/spatial resolution qualitatively.In order to reduce the information redundancy between basic layer and the enhancement layer, SVC has adopted the inter-layer prediction coding techniques, utilizes residual error data, movable information and the data reconstruction of basic layer macro block (Macroblock) that the corresponding informance of enhancement layer macro block is predicted.The inter-layer prediction coding techniques comprises inter-layer residue prediction (Inter-Layer Residual Prediction; ILRP), inter-layer motion prediction (Inter-Layer Motion Prediction; ILMP) and interlayer Intra prediction (Inter-Layer IntraPrediction, ILIP) three kinds.

The inter-layer prediction coding techniques has also greatly aggravated computation complexity when improving code efficiency.In encryption algorithm, model selection (Mode Decision) and motion search (Motion Search) are the parts of consumption calculations resource.Inter-layer residue prediction doubles the complexity of model selection, and inter-layer motion prediction has also increased the complexity of motion search.At present, realize, lay particular emphasis on mostly and reduce pattern (Mode) number of participating in calculating, ignored the complexity of inter-layer prediction coding itself to the low complex degree of SVC.

Summary of the invention

The invention provides the quick inter-layer prediction system of selection of a kind of scalable video, do not losing under the condition of picture quality basically, effectively reduce inter-layer prediction and select complexity.

The quick inter-layer prediction system of selection of a kind of scalable video may further comprise the steps:

(1) utilize macro block mode information to come the interlayer motion relevance and the spatial movement correlation of predicted macroblock level: to calculate the rate distortion costs of BLSKIP pattern and Direct pattern, respectively as the interlayer motion relevance of macro-block level and the tolerance of spatial movement correlation;

(2) utilize distance between motion vector and the predictive vector to come the interlayer motion relevance and the spatial movement correlation of predictor piece level:

At first do not use the motion search of inter-layer motion prediction, obtain motion vector; Then, calculate described motion vector and use the distance between the predictive vector under the inter-layer motion prediction condition, be first distance, and calculate described motion vector and do not use the distance between the predictive vector under the inter-layer motion prediction condition, be second distance; At last, calculate the difference of first distance and second distance, with the interlayer motion relevance and the spatial movement correlation of measuring sub-piece level;

(3) select inter-layer motion prediction fast according to the motion relevance of macro-block level and sub-piece level:

If the interlayer motion relevance all is better than the spatial movement correlation on macro-block level and sub-piece level, then select to use inter-layer motion prediction, proceed motion search; Otherwise, select not use inter-layer motion prediction, finish this motion search;

(4) under the situation of using inter-layer residue prediction, adopt step (the 1)-method of (3) to carry out the selection of inter-layer motion prediction, obtain the motion search result; Then, utilize the motion search result to estimate not use inter-layer residue prediction and macro block distortion and the coded-bit number used under two kinds of situation of inter-layer residue prediction, and obtain two kinds of rate distortion costs estimated values under the situation; Obtain two kinds of rate distortion costs ratios under the situation at last;

(5) select fast according to the correlation between described rate distortion costs ratio and the inter-layer residue prediction.

The quick inter-layer prediction system of selection of scalable video of the present invention; Utilize the movable information and the pattern information of macro block to come to select fast inter-layer motion prediction; Utilize the rate distortion costs of estimating to select fast inter-layer residue prediction; Basically do not losing under the condition of picture quality, effectively reducing inter-layer prediction and select complexity, improving inter-layer prediction and selected speed.The quick inter-layer prediction system of selection of scalable video of the present invention can also realize that (like fast schema selection method) combines flexibly, further accelerates coding rate with existing SVC low complex degree.

Description of drawings

Fig. 1 is the flow chart of the quick inter-layer prediction system of selection of scalable video of the present invention.

Embodiment

Specify the present invention below in conjunction with embodiment and accompanying drawing, but the present invention is not limited to this.

As shown in Figure 1, the quick inter-layer prediction system of selection of a kind of scalable video may further comprise the steps:

(1) macro-block level motion relevance prediction

Utilize macro block mode information to come the interlayer motion relevance and the spatial movement correlation of predicted macroblock level: the rate distortion costs J that calculates BLSKIP pattern and Direct pattern _BLSKIPAnd J _Direct, respectively as the interlayer motion relevance of macro-block level and the tolerance of spatial movement correlation.

Inter-layer motion prediction selects to have influenced encoding block predictive vector (Predict Motion Vector; PMV) account form: do not use under the inter-layer motion prediction situation; (Motion Vector MV) gets intermediate value and obtains that (note is made pmv predictive vector by the motion vector of adjacent block on the space _BL); Use under the inter-layer motion prediction situation, the MV convergent-divergent that predictive vector is gone up the correspondence position piece by basic layer obtains that (note is made pmv _EL).So inter-layer motion prediction is selected to can be regarded as the selection of sub-piece between spatial movement correlation and interlayer motion relevance,, then select not use ILMP if the spatial movement correlation is stronger; If the interlayer motion relevance is stronger, then select to use ILMP.

In macro-block coding pattern, Direct pattern and BLSKIP pattern do not need the motion search process, and its motion vector information obtains according to the motion vector information conversion of space adjacent macroblocks or the corresponding macro block of basic layer.According to the account form of motion vector, Direct pattern and BLSKIP pattern are used for measuring the spatial movement correlation and the interlayer motion relevance of macro-block level respectively: its rate distortion costs is more little, and corresponding motion relevance is also just strong more.Consider that Direct pattern and BLSKIP pattern are to use two kinds of higher patterns of frequency, preferentially calculating these two kinds of patterns can not increase algorithm complex.

(2) sub-piece level motion relevance prediction

Can know the distance dependent between bit number that motion vector encoder consumed and motion vector and the predictive vector by Golomb entropy coding mode: this distance is more little, and number of coded bits is few more, and corresponding rate distortion costs is also little; This distance is big more, and number of coded bits is many more, and corresponding rate distortion costs is also big.In inter-layer motion prediction was selected, rate distortion costs was more little, and the possibility of selection is high more, and its corresponding motion relevance is also just strong more.

This step utilizes the distance between motion vector and the predictive vector to come the interlayer motion relevance and the spatial movement correlation of predictor piece level, and detailed process is:

At first do not use the motion search of inter-layer motion prediction, obtain motion vector mv _NoILMPThen, calculate described motion vector and use the distance between the predictive vector under the inter-layer motion prediction condition, be first distance, and calculate described motion vector and do not use the distance between the predictive vector under the inter-layer motion prediction condition, be second distance; At last, calculate difference (the Δ MVD of first distance and second distance _e), shown in (1):

Δ MVD _e=MVD (mv _NoILMP, pmv _BL)-MVD (mv _NoILMP, pmv _EL) (1) wherein, mv _NoILMPBe not use the motion vector that obtains under the inter-layer motion prediction condition, pmv _BL, pmv _ELBe to use respectively and do not use the predictive vector under the inter-layer motion prediction condition, MVD () to be used for the compute vectors distance, computational methods are:

MVD(v ₁，v ₂)＝|v _1x-v _2x|+|v _1y-v _2y| (2)

Wherein, v ₁, v ₂Be vector, v _1x, v _2x, v _1y, v _2yRepresent its x, y component respectively.

According to difference (the Δ MVD of first distance with second distance _e) can measure the interlayer motion relevance and the spatial movement correlation of sub-piece level: if Δ MVD _e＜T shows that the interlayer motion relevance is stronger; Otherwise, show that the spatial movement correlation is stronger.Wherein, T is a threshold value.

(3) inter-layer motion prediction is selected fast

Motion relevance result according to macro-block level and sub-piece level accomplishes quick selection inter-layer motion prediction:

If satisfy formula (3), promptly the interlayer motion relevance all is better than the spatial movement correlation on macro-block level and sub-piece level, then selects to use inter-layer motion prediction, proceeds motion search; Otherwise, select not use inter-layer motion prediction, finish this motion search.

J _BLSKIP＜J _Direct?&?&?ΔMVD _e＜T (3)

Wherein, J is the rate distortion costs value.

(4) the inter-layer residue prediction rate distortion costs is estimated

Under the situation of using inter-layer residue prediction, adopt the method in step (1)-(3), carry out inter-layer motion prediction and select, obtain the motion search result;

Then, utilize the motion search result, estimate not using macro block distortion and coded-bit number under two kinds of situation of inter-layer residue prediction and use inter-layer residue prediction, and obtain two kinds of rate distortion costs estimated values under the situation according to formula (4)-(6):

$\hat{J}=\hat{D}+\mathrm{\λ}\×\hat{R}---\left(4\right)$

$\hat{D}=\underset{x=0}{\overset{15}{\mathrm{\Σ}}}\underset{y=0}{\overset{15}{\mathrm{\Σ}}}{\left(\mathrm{Diff}(x,y)\right)}^2---\left(5\right)$

$\hat{R}={\log }_2({\mathrm{\σ}}_Y+{\mathrm{\σ}}_U+{\mathrm{\σ}}_V)---\left(6\right)$

Wherein, Be the rate distortion costs estimated value,

Be macro block distortion estimated value,

Be the number of coded bits estimated value, λ is a Lagrangian, and (x is to carry out the residual block that motion compensation obtains, σ according to the motion search result y) to Diff _YBe the standard variance of residual block luminance component, σ _U, σ _VIt is the standard variance on two chromatic components of residual block.In order to save amount of calculation, residual block calculates by following method:

Diff _NoILRP＝S-P _ILRP (7)

Diff _ILRP＝S-P _ILRP-Res _base (8)

Wherein, Diff _NoILRPFor not using the residual block of inter-layer residue prediction, Diff _ILRPFor using the residual block of inter-layer residue prediction, S is an original macro, P _ILRPBe the prediction reference piece that obtains according to the motion search result, Res _BaseIt is residual error data through the basic layer corresponding blocks of interpolation processing.

At last, calculating the rate distortion costs ratio (η) that does not use under two kinds of situation of inter-layer residue prediction and use inter-layer residue prediction is:

$\mathrm{\η}=\frac{{\hat{J}}_{\mathrm{NoILRP}}}{{\hat{J}}_{\mathrm{ILRP}}}---\left(9\right)$

Wherein, for not use the rate distortion costs estimated value under the inter-layer residue prediction,

is for using the rate distortion costs estimated value under the inter-layer residue prediction.

(5) inter-layer residue prediction is selected fast

Correlation according between described rate distortion costs ratio and the inter-layer residue prediction is selected fast:

If satisfy η＞T _Up, select to use inter-layer residue prediction, rebuild calculating; If satisfy η＜T _Low, select not use inter-layer residue prediction, adopt the method in step (1)-(3) to carry out motion search again, and accomplish to rebuild and calculate; Otherwise, recomputate rate distortion costs, and, adopt the method in step (1)-(3) to carry out motion search again not using the situation of inter-layer residue prediction, select according to the rate distortion costs size then, and accomplish to rebuild and calculate.Wherein, T _Up, T _LowBe threshold value, satisfy T _Up＞T _Low

Claims (1)

1. quick inter-layer prediction system of selection of scalable video may further comprise the steps:

(1) rate distortion costs of calculating BLSKIP pattern and Direct pattern is respectively as the interlayer motion relevance of macro-block level and the tolerance of spatial movement correlation;

(2) at first do not use the motion search of inter-layer motion prediction, obtain motion vector; Then, calculate described motion vector and use the distance between the predictive vector under the inter-layer motion prediction condition, be first distance, and calculate described motion vector and do not use the distance between the predictive vector under the inter-layer motion prediction condition, be second distance; At last, calculate the difference of first distance and second distance, with the interlayer motion relevance and the spatial movement correlation of measuring sub-piece level, if the difference of first distance and second distance less than threshold value, then the interlayer motion relevance is stronger; Otherwise then spatial movement correlation is stronger;

(3) the motion relevance result according to macro-block level and sub-piece level accomplishes quick selection inter-layer motion prediction: if the interlayer motion relevance all is better than the spatial movement correlation on macro-block level and sub-piece level; Be the rate distortion costs value of the distortion cost value of BLSKIP pattern rate less than the Direct pattern; And the difference of first distance and second distance is less than above-mentioned threshold value; Then select to use inter-layer motion prediction, proceed motion search; Otherwise, select not use inter-layer motion prediction, finish this motion search;

(4) under the situation of using inter-layer residue prediction, adopt step (the 1)-method of (3) to carry out the selection of inter-layer motion prediction, obtain the motion search result; Then, utilize the motion search result to estimate not use inter-layer residue prediction and macro block distortion and the coded-bit number used under two kinds of situation of inter-layer residue prediction, and obtain two kinds of rate distortion costs estimated values under the situation; Obtain two kinds of rate distortion costs ratios under the situation at last, the computational methods of the rate distortion costs estimated value under two kinds of situation are following:

$\hat{J}=\hat{D}+\mathrm{\λ}\×\hat{R}---\left(1\right)$

$\hat{D}=\underset{x=0}{\overset{15}{\mathrm{\Σ}}}\underset{y=0}{\overset{15}{\mathrm{\Σ}}}{\left(\mathrm{Diff}(x,y)\right)}^2---\left(2\right)$

$\hat{R}={\log }_2({\mathrm{\σ}}_Y+{\mathrm{\σ}}_U+{\mathrm{\σ}}_V)---\left(3\right)$

Wherein, Be the rate distortion costs estimated value,

Be macro block distortion estimated value,

Be the number of coded bits estimated value, λ is a Lagrangian, and (x is to carry out the residual block that motion compensation obtains, σ according to the motion search result y) to Diff _YBe the standard variance of residual block luminance component, σ _U, σ _VIt is the standard variance on two chromatic components of residual block;

Do not use inter-layer residue prediction and the residual block Diff that uses under two kinds of situation of inter-layer residue prediction _NoILRPAnd Diff _ILRPCalculate by following method:

Diff _NoILRP＝S-P _ILRP (4)

Diff _ILRP＝S-P _ILRP-Res _base (5)

Wherein, S is an original macro, P _ILRPBe the prediction reference piece that obtains according to the motion search result, Res _BaseIt is residual error data through the basic layer corresponding blocks of interpolation processing;

(5) select fast according to the correlation between described rate distortion costs ratio and the inter-layer residue prediction; If rate distortion costs ratio greater than first threshold value, is selected to use inter-layer residue prediction, rebuild calculating; If rate distortion costs ratio less than second threshold value, selects not use inter-layer residue prediction, adopt the method in step (1)-(3) to carry out motion search again, and accomplish to rebuild and calculate; Otherwise, recomputate rate distortion costs, and to not using the situation of inter-layer residue prediction; Adopt the method in step (1)-(3) to carry out motion search again; Select according to the rate distortion costs size then, and accomplish to rebuild and calculate, wherein first threshold value is greater than second threshold value.

Images