CN101917615A - Enhancement type bi-directional motion vector predicting method in mixed video coding framework - Google Patents

Abstract

The invention discloses an enhancement type bi-directional motion vector predicting method aiming at interframe predictive coding in a mixed video coding framework, comprising the following steps of: (1) solving an alternative set containing a plurality of bi-directional motion vector prediction values through an prediction operator by comprehensively utilizing the time domain correlation and the space domain correlation of motion information for a current coding module; and (2) selecting an optimal motion vector prediction value from the alternative set by adopting a self-adaptation motion information prediction selection method to carry out subsequent coding computation and decoding computation as the motion vector prediction of the current coding module. The invention enhances the prediction accuracy of a final motion vector by comprehensively utilizing the correlated characteristics of the motion information on time domains and space domains and can independently complete on coding and decoding ends without detuning by utilizing a self-adaptation selection technology based on a prediction coding block SAD standard, thereby enhancing the compressive property of the bi-directional interframe predictive coding and being widely used for the field of mixed video communication.

Description

Enhancement type bi-directional motion vector predicting method in a kind of mixed video coding framework

Technical field

The present invention relates to technology of video compressing encoding, the enhancement type bi-directional motion vector predicting method in especially a kind of mixed video coding framework.

Background technology

In mixed video coding framework, an important characteristic adopts block-based motion compensation/forecasting mechanism to realize inter prediction encoding exactly.Often exist very strong correlation between the time domain of image and the spatial domain adjacent block movable information, the motion vector that utilizes encoding block is a important technology in the interframe encode to the motion-vector prediction mechanism of present encoding piece.The accurate motion vectors prediction not only can reduce final prediction residual, and can reduce and be used in the code stream and transmit the relevant side information of motion, so the compression efficiency that will improve interframe encode just should improve the accuracy of motion-vector prediction as much as possible.Simultaneously, because bi-directional predicted mechanism has the characteristic of better removal frame-to-frame correlation than single directional prediction mechanism, therefore, in the research of current video coding, more and more obtain paying attention to.Bi-directional predicted on the scramble time, can be used to from forward direction and back to the coded message of a plurality of reference frames, can utilize the coded message of adjacent block on the space, mode is very flexible, available information is very abundant.

At present, up-to-date video coding international standard is H.264/AVC (the Advanced Video Coding) that is formulated by joint video team (JVT, Joint VideoTeam).In H.264/AVC, will adopt bi-directional predicted encoding block to be referred to as the B piece.The motion-vector prediction technology of H.264/AVC middle B piece has mainly been utilized the relativity of time domain and the spatial correlation of movable information.In this standard, the motion-vector prediction of B piece roughly can be divided into two classes: direct prediction and conventional prediction, for different macro-block coding patterns, can adopt different Forecasting Methodologies, and is as shown in table 1.Below two kinds of Forecasting Methodologies are described respectively:

The method of motion vector prediction of B piece among the table 1H.264/AVC

(1) conventional Forecasting Methodology:

When the coding mode of B piece was the also non-Direct pattern of non-Skip pattern, its all motion-vector prediction all adopted conventional method.Suppose that E is current macro or macroblock partition, A, B, C, D correspond respectively to E left side, upside, upper right side and upper left side piece or cut apart.If there be a plurality of cutting apart in the left side of E, then make the A that is of top side wherein; If there be a plurality of cutting apart in the upside of E, then make the B that is of the leftmost side wherein.When the C piece does not exist, with the information substitution C piece of D piece.

Make the motion vector of A, B, C piece be respectively MV _A, MV _BAnd MV _C, then the mode of asking for of the motion vector predictor MVP of E is:

If current the first half that is divided into 16 * 8 patterns, then MVP=MV _BIf current the latter half that is divided into 16 * 8 patterns, then MVP=MV _A

If the current left-half that is divided into 8 * 16 patterns, then MVP=MV _AIf the current right half part that is divided into 8 * 16 patterns, then MVP=MV _B

If do not belong to 16 * 8 and 8 * 16 patterns current cutting apart, then

MVP＝median(MV _A，MV _B，MV _C) (1)

As seen Chang Gui method of motion vector prediction has only utilized the spatial domain correlation of movable information.

(2) direct Forecasting Methodology

When the coding mode of B piece was Skip or Direct, its motion-vector prediction all adopted the direct method to infer and draws.H.264/AVC support two kinds of different direct methods, be time domain direct and spatial domain direct, in encoding-decoding process, what indicate current use by the direct_spatial_mv_pred_flag syntactic element in the Slice header is the direct method in time domain or spatial domain.

When utilizing the time domain motion prediction, suppose that usually moving object on the frame is at the uniform velocity motion closing on.Based on such hypothesis, at first in reference formation List1, search out the correspondence position piece (co-located block) of present encoding piece in the first reference frame, and take out the motion vector of correspondence position piece at the first reference frame of reference formation List0

As shown in Figure 2, suppose present frame and be TD with reference to the time gap between the first reference frame of formation List0 _B, be TD with reference to the first reference frame of formation List0 with reference to the time gap between the first reference frame of formation List1 _D, the forward and backward motion vector predictor of present encoding piece then

With

Calculating be calculated as follows respectively:

${\stackrel{\&RightArrow;}{\mathrm{MV}}}_{\mathrm{<figure-callout\; id="0"\; label="L"\; filenames="HSA00000127079800031.png"\; state="\{\{state\}\}">L</figure-callout>}0}=\frac{{\mathrm{TD}}_B}{{\mathrm{TD}}_D}\&times;\stackrel{\&RightArrow;}{\mathrm{MV}}---\left(2\right)$

${\stackrel{\&RightArrow;}{\mathrm{MV}}}_{L1}=\frac{{\mathrm{TD}}_B-{\mathrm{TD}}_D}{{\mathrm{TD}}_D}\&times;\stackrel{\&RightArrow;}{\mathrm{MV}}---\left(3\right)$

Then with before described conventional Forecasting Methodology is very similar for the Forecasting Methodology of B piece spatial domain direct motion vector H.264/AVC, all adopts the intermediate value of the closing on motion vector motion-vector prediction as the present encoding piece.

By above-mentioned analysis as can be seen H.264/AVC the direct method of motion vector prediction of B piece utilized the time-domain of movable information or the correlation of spatial domain, but the application time-domain that can not be simultaneously comprehensive and the correlation of spatial domain.

Summary of the invention

The objective of the invention is to overcome the deficiencies in the prior art, the enhancement type bi-directional motion vector predicting method in a kind of mixed video coding framework that improves bi-directional predicted accuracy and final encoding compression efficient is provided.

The present invention solves its technical problem and takes following technical scheme to realize:

Enhancement type bi-directional motion vector predicting method in a kind of mixed video coding framework comprises the steps:

(1) for the present encoding piece, relativity of time domain, the spatial correlation of comprehensive utilization movable information are asked for an alternative set that comprises a plurality of bi-directional motion vector predicted values by predictive operator;

(2) when alternative set ask for finish after, adopt adaptive motion information prediction system of selection from alternative set, to choose optimum motion vector predictor, carry out follow-up coding as the motion-vector prediction of present encoding piece and calculate and conciliate yardage and calculate.

And described predictive operator comprises time domain prediction operator, spatial domain prediction operator H.264/AVC and Space Time territory associated prediction operator H.264/AVC.

And described selected set comprises time-domain predicted value (MVP _T), spatial domain predicted value (MVP _S) and Space Time territory associated prediction value (MVP _T-S).

And, the described method of choosing optimum motion vector predictor is: use based on the absolute error between the bi-directional predictive coding piece with as interpretational criteria, corresponding to least absolute error and one group of bi-directional motion vector be chosen to be optimum motion vector predictor.

And, the acquiring method of described Space Time territory associated prediction operator is: at first utilize the spatial domain prediction method to seek out a predictive coding piece in the first reference frame of reference frame lists, then the existing motion vector information of this predictive coding piece itself is obtained by the time domain prediction method.

And, when choosing optimum motion vector predictor, also by expansion be used to calculate absolute error and front and back reduce a probability that hour occurs erroneous judgement when the encoding block size to predictive coding piece area.

Advantage of the present invention and good effect are:

The present invention fully utilizes the correlation properties of movable information on time domain and spatial domain, fully excavate and utilized the correlation of motion vector on time domain and frequency domain, improved the prediction accuracy of final motion vector, utilize adaptively selected technology simultaneously based on predictive coding piece SAD criterion, can independently there be finishing of off resonance at the Code And Decode end, therefore need not in code stream, to transmit corresponding side information, improve the compression performance of bidirectional interframe predictive coding, can be widely used in the mixed video communications field.

Description of drawings

Fig. 1 is the flow chart of enhancement type bi-directional motion vector prediction of the present invention;

Fig. 2 is the schematic diagram of time domain motion-vector prediction operator of the present invention;

Fig. 3 is the schematic diagram of spatial domain of the present invention motion-vector prediction operator;

Fig. 4 is the schematic diagram of Space Time of the present invention territory Union Movement vector prediction operator;

Fig. 5 is of the present invention based on the acquiring method schematic diagram of front and back to the optimal motion vector predictor of predictive coding piece SAD criterion;

Fig. 6 is used to calculate the Method for Area schematic diagram of the predictive coding piece of SAD for expansion of the present invention;

Fig. 7 for the rate distortion curve ratio that " Mobile " sequence of CIF size adopted present embodiment method and standard employing method emulation H.264/AVC and obtain than schematic diagram;

Fig. 8 for the rate distortion curve ratio that " Crew " sequence of 720p size adopted present embodiment method and standard employing method emulation H.264/AVC and obtain than schematic diagram.

Embodiment

Below in conjunction with example the present invention is described further.

Enhancement type bi-directional motion vector predicting method in a kind of mixed video coding framework, design in order to improve performance bi-directional predicted in the interframe encode, it is on the basis of the movable information correlation that makes full use of time domain, spatial domain, the optimization technique of the optimal motion vector predictor of combining adaptive, improve the accuracy of motion prediction, and then promoted the encoding compression performance of total system.For this reason, when the bi-directional predicted B piece of one of encoding and decoding, adopt enhancement type bi-directional motion vector predicting method of the present invention to realize, its step is as follows:

Be used for all alternating binary coding patterns of mixed video coding framework inter prediction encoding, comprise the steps:

(1) for the present encoding piece, relativity of time domain, the spatial correlation of comprehensive utilization movable information are asked for an alternative set that comprises a plurality of bi-directional motion vector predicted values by a series of predictive operators;

(2) alternative set ask for finish after, adopt adaptive motion information prediction system of selection from alternative set, to choose optimum motion vector predictor, carry out follow-up coding as the motion-vector prediction of present encoding piece and calculate and conciliate yardage and calculate.

Can realize bi-directional motion vector prediction in the mixed video coding framework by above-mentioned steps.

For making purpose of the present invention, technical scheme and advantage clearer, below with reference to the accompanying drawing embodiment that develops simultaneously, the present invention is described in more detail.

The handling process of enhancement type bi-directional motion vector predicting method of the present invention, as shown in Figure 1, as can be seen from the figure the present invention comprises that mainly the MVP trim set generates processing procedure and MVP self adaptation preferred process process is formed.

(1) MVP trim set generation processing procedure is as follows:

Trim set comprises following three kinds of different predicted values at the present encoding block motion vector: time-domain predicted value MVP _T, spatial domain predicted value MVP _SWith Space Time territory associated prediction value MVP _T-S, MVP wherein _T, MVP _SConsistent with time domain direct motion-vector prediction and the spatial domain direct motion-vector prediction stipulated in the standard H.264/AVC.The acquiring method of three kinds of predicted values is respectively: time-domain predicted value MVP _TAsk for as shown in Figure 2, to the existing movable information of reference frame, and factor such as binding time distance is tried to achieve the motion vector predictor of current B piece before and after utilizing; Spatial domain predicted value MVP _SAsk for as shown in Figure 3, suppose that E is current macro or macroblock partition, A, B, C, D correspond respectively to E left side, upside, upper right side and upper left side piece or cut apart, the movable information of A, B, C, D correspondence can be calculated the motion vector predictor of current B piece by the method for calculating intermediate value; Space Time territory associated prediction value MVP _T-SAsk for shown in Figure 4, for reference frame lists LIST_X (X gets 0 or 1), at first by spatial domain predicted value MVP _SFind corresponding predictive coding piece in the 0th frame of LIST_X, according to the time continuity hypothesis of object of which movement in the video, the movable information of the predictive coding piece in the 0th frame itself also has correlation with the present encoding piece.So MVP _T-SValue be taken as this motion vector (time interval of supposing interframe all is 1) in the 1st frame of predictive coding piece in the 0th frame.

After finishing above-mentioned motion predict value calculating, constitute a set for an available motion vector predictor of B piece, can select MVP for the forward prediction value _T_ L0, MVP _S_ L0 or MVP _T-S_ L0, and can select MVP for the back forecast value _T_ L1, MVP _S_ L1 or MVP _T-S_ L1.

(2) MVP self adaptation preferred process process is as follows:

Before and after present embodiment has adopted based on the comparison to the absolute error of predictive coding interblock with (Sum ofAbsolute Difference, SAD) criterion is asked for optimum MVP, as shown in Figure 5.For the present encoding piece, the predictive coding piece that finds by the forward motion vector prediction is B _Fw, the predictive coding piece that the backward motion vector prediction is found is B _Bw, the size of supposing the present encoding piece is M * N, then under the present case before and after to the absolute error and the SAD of predictive coding interblock _{Fw_bw}Be calculated as follows shown in the formula:

${\mathrm{SAD}}_{\mathrm{fw}\_\mathrm{bw}}=\underset{i}{\overset{M}{\mathrm{\&Sigma;}}}\underset{j}{\overset{N}{\mathrm{\&Sigma;}}}|B_{\mathrm{fw}}(i,j)-B_{\mathrm{bw}}(i,j)|---\left(4\right)$

Because front and back all exist very strong content relevance between predictive coding piece and present encoding piece, so SAD _{Fw_bw}Can reflect to a great extent current motion vector choose whether accurate, SAD _{Fw_bw}The result of more little then account for motion vector prediction is accurate more, and therefore, optimum motion-vector prediction can be by comparing SAD _{Fw_bw}Minimum value find the solution out, suppose that sagittal optimal motion vector prediction is respectively MVP _Opt_ L0 and MVP _Opt_ L1, its computing formula is:

$({\mathrm{MVP}}_{\mathrm{opt}}\_\mathrm{<figure-callout\; id="0"\; label="L"\; filenames="HSA00000127079800031.png"\; state="\{\{state\}\}">L</figure-callout>}0,{\mathrm{MVP}}_{\mathrm{opt}}\_L1)=\underset{(\mathrm{MVP}\_\mathrm{<figure-callout\; id="0"\; label="L"\; filenames="HSA00000127079800031.png"\; state="\{\{state\}\}">L</figure-callout>}0,\mathrm{MVP}\_L1)}{\arg \min }\left\{{\mathrm{SAD}}_{\mathrm{fw}\_\mathrm{bw}}(\mathrm{MVP}\_\mathrm{<figure-callout\; id="0"\; label="L"\; filenames="HSA00000127079800031.png"\; state="\{\{state\}\}">L</figure-callout>}0,\mathrm{MVP}\_L1)\right\},$

MVP_L0={MVP wherein _T_ L0, MVP _S_ L0, MVP _T-S_ L0}, (5)

MVP_L1＝{MVP _T_L1，MVP _S_L1，MVP _T-S_L1}

Because the value of MVP_L0 and MVP_L0 at most all is 3 kinds (when time domains, spatial domain, when Space Time territory predicted value is all inequality), so the calculating of SAD at most can be above 9 times.

As seen the advantage that adopts this method is to guarantee that the encoding and decoding end can independently seek out identical optimal result, and does not need to increase the relevant side information in the code stream.

As previously mentioned, SAD _{Fw_bw}Calculating based on before and after belong to the hypothesis of same moving object to predictive coding piece and present encoding piece, but in actual test process, find sometimes picture material distribute comparatively complicated in this hypothesis can go wrong, thereby cause the erroneous judgement of optimal motion vector prediction.This situation is when present encoding piece small-sized, and the possibility of appearance can increase.In order to address this problem, the predictive coding piece SAD calculation criterion of extend sizes has been proposed, promptly by expansion be used to calculate absolute error and front and back reduce a probability that hour occurs erroneous judgement when the encoding block size to predictive coding piece area.As shown in Figure 6, suppose that the pairing predictive coding piece of present encoding piece is B, its size is M * N, is that middle mind-set expands all around with B, and predictive coding piece exB then can be expanded.If the expansion size at the upper and lower, left and right four direction is H ₀, H ₁, W ₀, W ₁Individual pixel is (W through the expansion predictive coding piece exB size after expanding then ₀+ M+W ₁) * (H ₀+ N+H ₁), corresponding front and back are to the absolute error and the SAD of predictive coding interblock _{Fw_bw}Calculation expression should corresponding being rewritten as:

${\mathrm{SAD}}_{\mathrm{fw}\_\mathrm{bw}}=\underset{i}{\overset{{\mathrm{<figure-callout\; id="0"\; label="W"\; filenames="HSA00000127079800031.png"\; state="\{\{state\}\}">W</figure-callout>}}_0+M+W_1}{\mathrm{\&Sigma;}}}\underset{\mathrm{<figure-callout\; id="0"\; label="j\; H"\; filenames="HSA00000127079800031.png"\; state="\{\{state\}\}">j</figure-callout>}}{\overset{H_{}}{\mathrm{\&Sigma;}}}|{\mathrm{exB}}_{\mathrm{fw}}(i,j)-{\mathrm{exB}}_{\mathrm{bw}}(i,j)|---\left(6\right)$

By adopting with the predictive coding piece is the extension blocks at center, contained more useful information when make judging the motion-vector prediction accuracy, can improve effectively based on the motion-vector prediction optimal selection algorithm of SAD " inefficacy " problem in the complicated image zone.At present, the spreading parameter H that uses among the present invention ₀, H ₁, W ₀, W ₁Value all is 8 pixels.

More than be the implementation method of enhancement type bi-directional motion vector predicting method of the present invention.In order to analyze the performance of the specific embodiment of the invention, on code tester platform KTA2.6, realized the technology among the present invention, and the technology of the present invention and the performance of standard code method have H.264/AVC been compared.Below be emulation explanation and correlated results.

Be used for enhancement type bi-directional motion vector predicting method of the present invention in order comprehensively to test, two kinds of coding structures have been selected in the experiment for use, be that IBBP result and length are 7 level B frame structure (IbBbBbBbBbP, wherein " B " expression can be used for the B frame of frame for referencial use, and " b " expression is not used in the B frame of frame for referencial use).All codings all adopt High Profile, and choose four groups of different quantization parameters and encode (QP _{I_slice}, QP _{P_slice}, QP _{B_slice})={ (22,23,24), (27,28,29), (32,33,34), (37,38,39) }.The sequence information that uses in the experiment is as shown in table 2.

The cycle tests that uses in table 2 experiment

Make macro block mode B_Skip in the experiment, B_Direct_16x16, B_Bi_16x16, B_LX_Bi_16x8 (X desirable 0 or 1), B_LX_Bi_8x16, B_Bi_LX_16x8, B_Bi_LX_8x16, B_Bi_Bi_16x8, B_Bi_Bi_8x16, and sub-partition mode B_Direct_8x8 and B_Bi_8x8 under the B_8x8 pattern open the enhancement type bi-directional motion vector predicting method described in the present invention, and the result of (method H.264/AVC) compares when not opening.The result that corresponding code stream is saved is by table 3 mistake! Do not find Reference source.Provide.

Table 3 the present invention and H.264/AVC standard technique result comparison

Fig. 7 and Fig. 8 have provided respectively and have adopted rate distortion curve ratio that enhancement type bi-directional motion vector predicting method of the present invention and standard employing method emulation H.264/AVC obtains than schematic diagram.Can reach a conclusion, the EBMVP technology is bigger in the QP value, and when promptly hanging down code check, performance improvement is more obvious.Because the shared proportion of movable information strengthens in the code stream of low code check, so the advantage that EBMVP raising motion-vector prediction precision is embodied is just more obvious.

Except the improvement of algorithm to the encoding compression performance, another problem that needs to consider is exactly the influence of technology of the present invention to the encoding and decoding complexity.According to before description as can be known:

The MVP trim set all is to utilize the motion vector of existing reference frame and by simple predictive operator generation, therefore can increase the complexity of encoding and decoding end hardly;

Cut apart for each bi-directional predicted piece or piece, the calculating of front and back SAD between the predictive coding piece is carried out at most 9 times.But analyze as can be known, this opposite extreme situations has only when the result of time domain, spatial domain and Space Time territory associated prediction is different and just can take place, and has only the very complicated part of motion conditions just might occur in image.And for motions most in the image simply, zone stably, the calculating of front and back SAD between the predictive coding piece all can be less than 9 times.Therefore with respect to whole codec, its influence to complexity is very little.

Go up conclusion and obtained checking yet in the encoding and decoding test of reality, the EMS memory occupation of codec and running time all almost do not increase.

It is emphasized that; embodiment of the present invention is illustrative; rather than it is determinate; therefore the present invention is not limited to the embodiment described in the embodiment; every other execution modes that drawn by those skilled in the art's technical scheme according to the present invention belong to the scope of protection of the invention equally.

Claims (6)

1. the enhancement type bi-directional motion vector predicting method in the mixed video coding framework is characterized in that: comprise the steps:

(1) for the present encoding piece, relativity of time domain, the spatial correlation of comprehensive utilization movable information are asked for an alternative set that comprises a plurality of bi-directional motion vector predicted values by predictive operator;

(2) when alternative set ask for finish after, adopt adaptive motion information prediction system of selection from alternative set, to choose optimum motion vector predictor, carry out follow-up coding as the motion-vector prediction of present encoding piece and calculate and conciliate yardage and calculate.

2. the enhancement type bi-directional motion vector predicting method in a kind of mixed video coding framework according to claim 1 is characterized in that: described predictive operator comprises time domain prediction operator, spatial domain prediction operator H.264/AVC and Space Time territory associated prediction operator H.264/AVC.

3. the enhancement type bi-directional motion vector predicting method in a kind of mixed video coding framework according to claim 1 is characterized in that: described selected set comprises time-domain predicted value (MVP _T), spatial domain predicted value (MVP _S) and Space Time territory associated prediction value (MVP _T-S).

4. the enhancement type bi-directional motion vector predicting method in a kind of mixed video coding framework according to claim 1, it is characterized in that: the described method of choosing optimum motion vector predictor is: use based on the absolute error between the bi-directional predictive coding piece with as interpretational criteria, corresponding to least absolute error and one group of bi-directional motion vector be chosen to be optimum motion vector predictor.

5. the enhancement type bi-directional motion vector predicting method in a kind of mixed video coding framework according to claim 2, it is characterized in that: the acquiring method of described Space Time territory associated prediction operator is: at first utilize the spatial domain prediction method to seek out a predictive coding piece in the first reference frame of reference frame lists, then the existing motion vector information of this predictive coding piece itself is obtained by the time domain prediction method.

6. the enhancement type bi-directional motion vector predicting method in a kind of mixed video coding framework according to claim 4, it is characterized in that: when choosing optimum motion vector predictor, also by expansion be used to calculate absolute error and front and back reduce a probability that hour occurs erroneous judgement when the encoding block size to predictive coding piece area.

Images