CN101185340A - Motion-compensated fine-grained scalable video coding method and system with drift control - Google Patents

Abstract

An adaptively formed reference block is used to encode a block in the current frame in the enhancement layer, in particular, the reference block reconstructs the reference block of the frame from the base layer, the reference block in the enhancement layer reference frame and the base layer The reconstructed prediction residual blocks are formed together. Also, the reference block used for encoding is adjusted depending on the transform coefficients of the residual layer reconstructed at the base layer. Also, the actual reference signal used for coding is a weighted average of the reference signal from the reconstructed frame in the base layer, the reference signal from the enhancement layer reference frame and the base layer reconstructed prediction residual together.

Description

Motion compensated fine granularity scalable video coding method and system with drift control

Technical field

The present invention relates to field of video encoding, relate more specifically to scalable video.

Background technology

In video coding, thus can be by coming that based on other frame of video frame of video is predicted that the time redundancy that will exist between the frame of video minimizes.These other frames are called reference frame.Can predict with the different mode times of implementation:

The used reference frame of the reference frame that-decoder uses and encoder is identical.This is a most frequently used method in the non-scalable video encoder of routine.In normal running, between the used reference frame of reference frame that encoder is used and decoder any not matching can not be arranged.

-encoder uses disabled reference frame for decoder.An example is that encoder uses primitive frame to substitute reconstruction frames as the reference frame.

Compare with the frame that uses in the encoder, decoder uses the reference frame of only partly being rebuild.If the bit stream of same number of frames not complete decoding or the reference frame of himself is that part is rebuild, then this frame is that part is rebuild.

When according to second and the 3rd method time of implementation prediction, may exist between employed reference frame of encoder and the employed reference frame of decoder not match.If do not match in the decoder-side accumulation, then the quality of reconstruction video can be affected.

Not matching in time prediction is called drift between encoder and the decoder.Many video coding systems are designed to nothing drift (drift-free), and this is because the error of accumulation can cause illusion occurring in the video of rebuilding.Sometimes, in order more effectively to obtain certain video coding feature such as the SNR scalability, drift is avoided all the time fully.

The signal to noise ratio (snr) extensible video stream has such character: the video than the low quality level can be rebuild from partial bit stream.Fine granularity scalability (FGS) is one type a SNR scalability, can block scalable stream arbitrarily in this SNR scalability.How Fig. 1 explanation produces the stream of FGS character in MPEG-4.At first, basic layer (base layer) is encoded in non-scalable bitstream.Then, the FGS layer is encoded on basic layer.MPEG-4 FGS does not adopt correlation any time in the FGS layer.As shown in Figure 2, when do not predict service time in FGS layer coding, the FGS layer was predicted from basic layer reconstruction frames.These means have maximum bit stream flexibility, and this is because block the decoding that the FGS stream of a frame can't influence other frame, but this coding efficiency does not have competitiveness.

Expectation is introduced another predictive loop to improve code efficiency in FGS layer coding.But, because the FGS layer of any frame can be by partial decoding of h, so can accumulate and cause drift by the caused error of the difference between the reference frame that uses in decoder and encoder.Fig. 3 has illustrated this point.

The technology of leaking prediction (leaky prediction) and be seeking balance between the coding efficiency that has been used in the SNR enhancement layer coding and the drift control (for example, " A robust fine granularity scalability using trellis-based predictiveleak " referring to people such as Huang, IEEE Transaction on Circuits and Systems for VideoTechnology, the 372-385 page or leaf, the 12nd volume, the 6th edition, in June, 2002).For the FGS layer to the n frame is encoded, utilize basic layer reconstruction frames (x _b ⁿ) and enhancement layer reference frame (r _e ^N-1) linear combination form actual reference frame.If enhancement layer reference frame part in decoder is rebuild, then leak Forecasting Methodology and will limit propagation by the used reference frame (r of encoder _e ^{N-1, F}) and the used reference frame (r of decoder _e ^{N-1, D}) between the caused error that do not match, this is because error (ε _e ^N-1) when having new reference signal to form, will be attenuated at every turn.

$r_a^{n,D}=\mathrm{\α}\·x_b^n+(1-\mathrm{\α})\·r_e^{n-1,D}=\mathrm{\α}\·x_b^n+(1-\mathrm{\α})\·r_e^{n-1,E}-(1-\mathrm{\α})\·{\mathrm{\ϵ}}_e^{n-1}=r_a^{n,E}-(1-\mathrm{\α})\·{\mathrm{\ϵ}}_e^{n-1}$

(1)

Wherein, r _a ^{N, D}And r _a ^{N, E}Be respectively the actual reference frame that uses in the FGS layer coding in decoder and encoder, and α is that the value that provided by 0＜α≤1 is so that obtain the decay of error signal.

The third technology is to come the DCT coefficient in the piece that will encode in enhancement layer is classified (referring to Comer " Conditionalreplacement for improved coding efficiency in fine-grain scalable videocoding " according to the value of corresponding quantization parameter in the basic layer, International Conference on Image Processing, the 2nd volume, the 57-60 page or leaf, 2002).Each coefficient judges it is that basic layer or enhancement layer are used for predicting.If the quantization parameter in the basic layer is zero, then will use the DCT coefficient that calculates from the enhancement layer reference frame to predict corresponding DCT coefficient the enhancement layer.If this quantization parameter non-zero in the basic layer then will use the DCT coefficient that calculates from the reference block from basic layer reconstruction reference frame to predict corresponding DCT coefficient the enhancement layer.

FGS code device developed and be included in associating telescopic video model 1.0 (Joint Scalable Video Model, JSVM) in, this model is that MPEG/VCEG is at the employed official of the standardization activity of AVC scalable extension model.But JSVM1.0 FGS code device is not to design for the management drift.The FGS layer that is positioned at the anchor frame (Anchor Frame) of GOP (image sets) boundary is encoded in the mode that is similar to MPEG-4 FGS, wherein in MPEG-4 FGS and the unused time redundancy.Use for some, the length of GOP can shorten to 1 frame.In this case, JSVM1.0 FGS code device is unusual poor efficiency, and this is because each frame is all encoded as the anchor frame.

Fig. 4 has provided the predicted path in the 3 layers of extensible video stream.Between two discrete (discrete) layers, insert the FGS layer.Last layer scattering enhancement layer can be spatial enhancement layer or coarse SNR fgs layer.This upper strata enhancement layer uses basic layer texture prediction pattern or residual prediction mode to encode based on the FGS layer usually.In this basic layer texture prediction pattern, the reference that the piece in rebuilding the FGS layer is encoded to the piece in the discrete enhancement layer in upper strata with work.In residual prediction mode, the prediction that the residual error of rebuilding from basic layer and FGS layer is encoded to the prediction residual the enhancement layer with work.Even middle FGS layer is only partly rebuild, also still can carry out the decoding of upper strata enhancement layer.But, because the decoding of FGS layer segment, so the upper strata enhancement layer has drifting problem.

Summary of the invention

The invention provides fine granulation SNR telescopic video encoding and decoding, its in the FGS layer service time redundancy to improve coding efficiency, control drift simultaneously.More specifically, the present invention focuses on the reference block that uses in the predictive coding in the FGS layer and should how to form, and controls required signaling of this process and mechanism.

The present invention has improved the FGS code efficiency, especially the code efficiency under low deferred constraint condition.The present invention is effectively in the control drift, and therefore can correspondingly design fine granular scalability (FGS) the code device with better performance.

According to the present invention, when in enhancement layer, the piece in the present frame being encoded, the reference block that uses self adaptation to form.Especially, this reference block is come together to form with the prediction residue block that basic layer is rebuild by reference block in the basic layer reconstruction frames and the reference block in the enhancement layer reference frame.And the reference block that is used for encoding depends at the coefficient of basic layer coding to be adjusted.And the actual reference signal that is used for encoding is to come from the reference signal of basic layer reconstruction frames and come from the weighted average of the reference signal of enhancement layer reference frame with basic layer reconstructed predicted residual error.

Therefore, a first aspect of the present invention provides the method that is used for the motion compensation scalable video, and wherein this method comprises the formation reference block and adjusts reference block.This method comprises that further the right to choose repeated factor makes this reference block form the weighted average of this basic layer reference block and this enhancement layer reference block.

A second aspect of the present invention provides software application product, this software application product have storage medium with program code stored so that carry out method of the present invention.

A third aspect of the present invention provides the electronic module that is used in the use of motion compensated video coding.This electronic module comprises and is used for the adjustment piece that the method according to this invention forms the formation piece of this reference block and is used to adjust reference block.

A fourth aspect of the present invention provides the electronic equipment such as portable terminal, and this electronic equipment has one of decoder module and coding module or the two, and wherein this decoder module and this coding module have the module that is used for the motion compensated video coding.This electronic module comprises and is used for the adjustment piece that the method according to this invention forms the formation piece of this reference block and is used to adjust reference block.

After reading the description that 5-11 makes in conjunction with the accompanying drawings, it is obvious that the present invention will become.

Description of drawings

Fig. 1 has illustrated the fine granular scalability (MPEG-4) that does not have time prediction in the FGS layer.

Fig. 2 has illustrated when do not predict service time in FGS layer coding basic layer and the FGS layer employed reference block of encoding.

Fig. 3 has illustrated the fine granular scalability with time prediction.

Fig. 4 shows the use of FGS information when the enhancement layer of prediction upper strata.

Fig. 5 has illustrated the generation according to the reference block of the FGS of having layer of the present invention time prediction and drift control.

Fig. 6 has illustrated that the adaptive reference piece according to the basic layer of dependence of the present invention (base-layer dependent) forms.

Fig. 7 has illustrated according to the present invention by carry out interpolation on the difference reference frame and has handled and form reference block.

Fig. 8 has illustrated the differential reference block adjustment according to the basic layer of dependence of the present invention.

Fig. 9 has illustrated the FGS encoder that relies on the reference block generation type of basic layer according to utilization of the present invention.

Figure 10 has illustrated the FGS decoder that relies on the reference block formation utilization of basic layer according to utilization of the present invention.

Figure 11 has illustrated the electronic equipment that has in scalable encoder and the salable decoder at least one according to of the present invention.

Embodiment

As when the typical predictive coding in the non-scalable single-layer video codec, for the piece X of M in the FGS layer * N pixel size ⁿEncode, used reference block R _a ⁿCoefficient Q based on coding in basic layer _b ⁿ, from coming from the reference block X of basic layer reconstruction frames _b ⁿWith the reference block R that comes from the enhancement layer reference frame _e ^N-1Form R adaptively _a ⁿFig. 5 has provided the relation between these pieces.Here, piece is the rectangular area in the frame.Relevant block big or small the same in the size of piece and the coefficient domain in the spatial domain.

According to the present invention, identical primitive frame is encoded in enhancement layer and basic layer in FGS code device, but encodes with the different quality level.Basic layer is total to the piece that position (collocated) piece is meant coding in basic layer, and wherein said corresponding to the identical original block of just handling in enhancement layer.

Hereinafter, Q _b ⁿBe the piece of the quantization parameter of coding in basic layer, wherein said corresponding to the identical original block of just encoding in enhancement layer.In the present invention, has only independent coefficient Q _b ⁿ(whether u is that zero information is important v).

If $Q_b^n=0$ , promptly all coefficient Q _b ⁿ(u, v) (0≤u＜M, 0≤v＜N) are 0, then reference block R _a ⁿAs X _b ⁿAnd R _e ^N-1Weighted average be calculated as follows:

$R_a^n=\mathrm{\α}\·X_b^n+(1-\mathrm{\α})\·R_e^{n-1}$ If $Q_b^n=0$ ?(2)

Wherein α is a weight factor.

Otherwise, to X _b ⁿAnd R _e ^N-1Carry out conversion respectively to obtain conversion coefficient $F_{X_b^n}=f\left(X_b^n\right),$ $F_{R_e^{n-1}}=f\left(R_e^{n-1}\right).$ Coefficient block F _Ran(u, v) (0≤u＜M, 0≤v＜N) form based on basic series of strata numerical value:

$F_{R_a^n}(u,v)=\mathrm{\β}\·F_{X_b^n}(u,v)+(1-\mathrm{\β})\·F_{R_e^{n-1}}(u,v)$ If $Q_b^n(u,v)=0$ (3)

$F_{R_a^n}(u,v)=F_{X_b^n}(u,v)$ If $Q_b^n(u,v)\≠0$ (4)

Wherein β is a weight factor.

Then, by to F _RanCarry out following inverse transformation and obtain actual reference block:

$R_a^n=g\left(F_{R_a^n}\right)$

The ownership repeated factor is always in [0,1] scope.Being formed among Fig. 6 of adaptive reference piece that relies on basic layer illustrates.

In an embodiment of the invention, weight factor α is set to 0, and weight factor β is set to 1.In this case, if the piece of encoding in the FGS layer has some nonzero coefficients in basic layer, then basic layer reconstructed block with selected as actual reference block, perhaps, if the piece that is encoded does not have any nonzero coefficient in basic layer, then the enhancement layer reference block with selected as actual reference block.This is a kind of simple designs.The decision that should to come from basic layer reconstruction frames still be the enhancement layer reference frame is only made in piece level (block level) about the data of reference block, and without any need for additional transformation or weighted average operation.

In another embodiment of the present invention, the value of weight factor α is not limited, and the value of weight factor β depend on consider the frequency of coefficient.

In another execution mode, weight factor α is not limited, and the value of weight factor β depends on the FGS code period that current coefficient is encoded therein.

Hereinafter, such as x _b ⁿAnd r _a ⁿAnd so on the small letter variable be used for general the discussion.Such as X _b ⁿAnd R _a ⁿAnd so on the capitalization variable be used for representing the signal of spatial domain, and their corresponding conversion coefficient is expressed as F _XbnAnd F _RanDeng.

The invention provides many algorithms that are used for being created in the optimal reference signal that FGS layer coding use.Utilize these algorithms, time prediction merges to improve coding efficiency with FGS layer coding effectively, simultaneously control drift effectively.

As above discuss, in order to introduce time prediction and control drift in the FGS layer, actual reference signal is from the reference signal of the reconstruction frames in the basic layer with from the weighted average of the reference signal of reinforced layer reference frame:

$r_a^n=\mathrm{\α}\·x_b^n+(1-\mathrm{\α})\·r_e^{n-1}---\left(5\right)$

Basic layer reconstruction signal x _b ⁿItself be from basic layer reference signal r _b ^N-1With basic layer reconstructed predicted residual error p _b ⁿCalculate:

$x_b^n=r_b^{n-1}+p_b^n---\left(6\right)$

Actual reference signal can be constructed according to following:

$r_a^n=\mathrm{\α}\·r_b^{n-1}+\mathrm{\α}\·p_b^n+(1-\mathrm{\α})\·r_e^{n-1}---\left(7\right)$

According to the present invention, this relation is by being basic layer reconstructed predicted residual error p _b ⁿIntroduce independently zoom factor α _pAnd be summarised as:

$r_a^n=\mathrm{\α}\·r_b^{n-1}+{\mathrm{\α}}_p\·p_b^n+(1-\mathrm{\α})\·r_e^{n-1}---\left(8\right)$

This independent zoom factor α _pHas the value from 0 to 1.When this zoom factor equaled 1, basic layer reconstructed predicted residual error was not scaled.

Therefore, the algorithm that is used to produce actual reference block can be summarized as follows:

If all coefficient Q _b ⁿ(u v) equals 0, then $X_b^n=R_b^{n-1}$ 。Actual reference block R _a ⁿBe calculated as R _b ^N-1And R _e ^N-1Weighted average:

$R_a^n=\mathrm{\α}\·R_b^{n-1}+(1-\mathrm{\α})\·R_e^{n-1}$ If $Q_b^n=0$ (9)

Otherwise, respectively to R _b ^N-1And R _e ^N-1Carry out conversion to obtain conversion coefficient $F_{R_b^{n-1}}=f\left(R_b^{n-1}\right),$ ${\text{F}}_{R_e^{n-1}}=f\left(R_e^{n-1}\right)$ 。Coefficient block F _Rrn(u, v), 0≤u＜M, 0≤v＜N forms with such coefficient, and each of this coefficient all is from the coefficient of basic layer reference frame with from the weighted average of the coefficient of enhancement layer reference frame.Weight factor depends on basic series of strata numerical value.

$F_{R_a^n}(u,v)=\mathrm{\β}\·F_{R_b^{n-1}}(u,v)+(1-\mathrm{\β})\·F_{R_e^{n-1}}(u,v)$ If $Q_b^n(u,v)=0$

(10)

$F_{R_a^n}(u,v)=\mathrm{\γ}\·F_{R_b^{n-1}}(u,v)+{\mathrm{\γ}}_p\·F_{P_b^n}(u,v)+(1-\mathrm{\γ})\·F_{R_e^{n-1}}(u,v)$ If $Q_b^n(u,v)\≠0$

(11)

Then, by to F _RanCarry out inverse transformation and obtain actual reference block:

$R_a^n=g\left(F_{R_a^n}\right)$

Equation 9,10 and 11 can reorganize as follows:

$R_a^n=R_b^{n-1}+(1-\mathrm{\α})\·R_d^{n-1}$ If $Q_b^n=0$ (12)

$F_{R_a}(u,v)=F_{R_b^{n-1}}(u,v)+(1-\mathrm{\β})\·F_{R_d^{n-1}}(u,v)$ If $Q_b^n(u,v)=0$ (13)

$F_{R_a^n}(u,v)=F_{R_b^{n-1}}(u,v)+{\mathrm{\γ}}_p\·F_{P_b^n}(u,v)+(1-\mathrm{\γ})\·F_{R_d^{n-1}}(u,v)$ If $Q_b^n(u,v)\≠0---\left(14\right)$

In equation 12,13 and 14, R _d ^N-1Be differential reference block, $R_d^{n-1}=R_e^{n-1}-R_b^{n-1}.$

Because conversion is linear, so the difference between the conversion coefficient can be calculated by differential reference block is carried out conversion.

$F_{{\text{R}}_d^{n-1}}=F_{R_e^{n-1}}-F_{R_b^{n-1}}=f\left(R_e^{n-1}\right)-f\left(R_b^{n-1}\right)=f(R_e^{n-1}-R_b^{n-1})=f\left(R_d^{n-1}\right)---\left(15\right)$

Three equations can be merged into a unified equation.

$R_a^n=R_b^{n-1}+{R_d^{n-1}}^{\′}+{\mathrm{\γ}}_p\·P_b^n---\left(16\right)$

Function R _d ^N-1' be defined as:

If all coefficients in the base layer block are 0, then differential reference block is carried out convergent-divergent by a zoom factor (1-α):

${R_d^{n-1}}^{\′}=(1-\mathrm{\α})\·R_d^{n-1}$ If $Q_b^n=0$ (17)

Otherwise, to R _d ^N-1Carry out conversion to obtain conversion coefficient $F_{R_d^{n-1}}=f\left(R_d^{n-1}\right)$ 。Whether based on basic layer coefficients is 0 to come each coefficient is carried out convergent-divergent.

${F_{{\text{R}}_d^{n-1}}}^{\′}(u,v)=(1-\mathrm{\β})\·F_{R_d^{n-1}}(u,v)$ If $Q_b^n(u,v)=0$ (18)

${F_{{\text{R}}_d^{n-1}}}^{\′}(u,v)=(1-\mathrm{\γ})\·F_{R_d^{n-1}}(u,v)$ If $Q_b^n(u,v)\≠0$ (19)

To F _Rdn-1' carry out inverse transformation to obtain ${R_d^{n-1}}^{\′}=g\left({F_{R_d^{n-1}}}^{\′}\right).$

Utilize this means, can produce R by the difference reference frame is carried out motion compensation _d ^N-1, wherein calculate this difference reference frame by from the enhancement layer reference frame, deducting basic layer reference frame.Illustrated among Fig. 7 by on the difference reference frame, carrying out interpolation and handled and form reference block, and the differential reference block adjustment that relies on basic layer has been described among Fig. 8.An example of interpolation filter is the filter that is used for bilinear interpolation.By using the difference reference frame, except the complexity that reduces interpolation, only need a direct transform.

In the foregoing description, if basic layer reconstructed predicted residual error is 0, then basic layer is rebuild (x _b ⁿ) and basic layer reference signal (r _b ^N-1) identical.For some realization, application can select following equation to replace equation 12,13 and 14 to realize so that simplify.

$R_a^n=X_b^n+(1-\mathrm{\α})\·R_d^{n-1}$ If $Q_b^n=0$ (20)

$F_{R_a^n}(u,v)=F_{X_b^n}(u,v)+(1-\mathrm{\β})\·F_{R_d^{n-1}}(u,v)$ If $Q_b^n(u,v)=0$ (21)

$F_{R_a^n}(u,v)=F_{X_b^n}(u,v)+({\mathrm{\γ}}_p-1){\·F}_{P_b^n}(u,v)+(1-\mathrm{\γ})\·F_{R_d^{n-1}}(u,v)$ If $Q_b^n(u,v)\≠0$

(22)

Even basic layer is rebuild the additional operations of carrying out such as loop filtering, also can user's formula 20,21 and 22, although because to " R _b ^N-1+ P _b ⁿ" additional operations and make X _b ⁿAlways do not equal R _b ^N-1+ P _b ⁿ

According to the present invention, all be that 0 piece is further classified to its basic layer coefficients, and different weight factors is used for different classes of piece.

A kind of sorting technique is to depend on whether a piece has any such contiguous block and come piece is classified, and wherein this contiguous block has the basic layer coefficients of non-zero.A kind of mode of carrying out classification like this is to use the coding context index (coding context index) that basic layer coded block flag is encoded of being used for as definition in H.264.In H.264, if the coded block flag of left contiguous block and top contiguous block is 0, the context index of then encoding is 0.If have only the coded block flag of left contiguous block non-vanishing, the context index of then encoding is 1.If have only the coded block flag of top contiguous block non-vanishing, the context index of then encoding is 2.Otherwise the coding context index is 3.

Another kind of sorting technique be to use clear and definite signaling indicate reference block be only from basic layer reconstruction frames, only from use the basic layer reconstruction frames that mode obtains described in the present invention and strengthen between the reference frame weighted average or only from enhancement layer.This signaling can be carried out in macro block (MB) level, and only carries out at those MB that do not have any nonzero coefficient in basic layer.

Map function needs, and this is that then different weight factors is used for the different coefficients of piece because if the piece in the basic layer has any nonzero coefficient.If the equal weight factor is used for all coefficients of piece, then map function is not necessary.

According to the present invention, the quantity of nonzero coefficient is counted in base layer block.If the quantity of nonzero coefficient is greater than or equal to predetermined quantity Tc, then all coefficients use single weight factor in this piece.The value of weight factor can depend on the quantity of nonzero coefficient in the basic layer.This number of threshold values Tc determines whether whole should be used the equal weight factor.Tc is positioned at from 1 scope to block size.For example, for 4 * 4,16 coefficients are arranged in the piece, Tc is in from 1 to 16 scope.

A kind of special circumstances are Tc=1, that is: the coefficient of all in the piece uses the equal weight factor all the time.In this case, do not need additional transformation.But the value of weight factor can depend on the quantity of nonzero coefficient in the base layer block.

Weight factor

The weight factor of different frame or different sheets can change, and perhaps for a certain amount of frame or sheet, weight factor can be fixed.Weight factor β can depend on the quantity of nonzero coefficient in the basic layer.

Here be that between the quantity of nonzero coefficient in weight factor β, γ and the basic layer one concerns example.In this example, γ is constant for sheet.When having only a nonzero coefficient in the basic layer, β equals β ₁, β ₁≤ γ.When the quantity of nonzero coefficient in the basic layer is n and n during less than Tc, user's formula β=β ₁+ (γ-β ₁) (n-1)/T _c-1 calculates β.If n is equal to, or greater than Tc, then β equals γ.

The coding of a plurality of FGS layers

Under the situation when having discrete basic layer and a plurality of FGS layer on discrete basic layer, user's basic layer of can selecting to disperse uses as " basic layer " and FGS layer is topmost used to realize above mentioned algorithm as " enhancement layer ".This is known as double loop structure.

The user also can use the multiring code structure, and is as follows:

-the first coding loop is normal discrete basic layer coding loop.

-the second coding loop is used for using at this open algorithm of describing encodes to a FGS layer." basic layer " is that discrete basic layer and " enhancement layer " are FGS layers.

-in the 3rd coding loop, " basic layer " is that a FGS layer and " enhancement layer " they are the 2nd FGS layers, or the like.

In case " basic layer " is the FGS layer, " basic layer " coefficient of then considering is to be lower than coefficient in the layer of this layer at this FGS layer and other.If the coefficient at the same position place in any one deck of these layers is non-vanishing, then think Q _b ⁿ(u, v) non-zero.Using other coding structure is quite simple with this algorithm application in FGS code device.

In many loop structures, exist different modes to calculate actual reference signal, this reference signal is used in the coding of the 2nd FGS layer or higher FGS layer.Owing to must distinguish the FGS layer, formula (16) needs to change slightly.Basic layer still uses subscript " b ", but a FGS enhancement layer that is located immediately on the basic layer top uses subscript " e1 ", and the 2nd FGS enhancement layer uses subscript " e2 " etc.R _D1 ^N-1' be to be used for a FGS enhancement layer is encoded and the difference reference signal of the adjustment of calculating.Except target changed down, equation (23) was equivalent to equation (16).

$R_{a_1}^n=R_b^{n-1}+R_{d_1}^{n-1}+{\mathrm{\γ}}_{p_b}\·P_b^n={R_{e_1}^{n-1}}^{\′}+{\mathrm{\γ}}_{p_b}\·P_b^n---\left(23\right)$

For the 2nd FGS enhancement layer, actual reference signal can be as calculating in the equation (24).R _D2 ^N-1' be to calculate from the difference reference frame, the difference between the reference frame of this difference reference frame reference frame that is the 2nd FGS enhancement layer and first enhancement layer wherein.As can be seen, rebuild the residual error item except many one, this equation does not have a great difference with (23).

$R_{a_2}^n=R_{e_1}^{n-1}+{R_{d_2}^{n-1}}^{\′}+{\mathrm{\γ}}_{p_b}\·P_b^n+{\mathrm{\γ}}_{p_{e_t}}\·P_{e_1}^n---\left(24\right)$

Exist three kinds of distinct methods to calculate R _E1 ^N-1First method is that the reference frame of a FGS enhancement layer is carried out motion compensation.In the present invention, can use and be used to calculate R _E1 ^N-1One of two other methods (method A and method B).For method A, with R _E1 ^N-1Be set to equal R _b ^N-1+ R _D1 ^N-1For method B, with R _E1 ^N-1Be set to equal R _b ^N-1+ R _D1 ^N-1'.

In the present invention, double loop or many loops FGS coding selection can be selected and be signaled in bit stream by encoder.When for frame, when bit stream was changed into multiring code from double loop FGS coding, this frame can use two reference frames, promptly basic layer reference frame and the highest enhancing reference frame.But all layers of this frame will be rebuild fully, and next frame has and is used for many loops FGS all required reference frames of encoding like this.If bit stream is changed into double loop from many loops, then present frame will be encoded in many loops, but only have basic layer and the highest FGS layer to be rebuild fully, and this is because no longer need the frame in any intermediate layer for next frame.

Utilize the present invention, use motion compensation to calculate new fallout predictor and the FGS layer is encoded being used for.Need to rebuild required enhancement layer reference frame like this.But,, then under some constraints, can avoid the FGS layer is all rebuild if decoder is wanted the layer on these FGS layers is decoded.For example, suppose to exist two the FGS layers (F1, F2) on discrete basic layer (L0), the L0 top, and on FGS layer F2 top, have discrete enhancement layer (L3).If layer L3 will be in basic layer L0 as MB between (inter-MB) macro block of reconstruction fully of encoding as fallout predictor, the substitute is it and only use reconstruction residual error in the prediction, then when decoder wanted to rebuild layer L3, its need were decoded to the residual information of layer F1 and F2 and are not needed motion compensation.

The general introduction of FGS code device

Fig. 9 and Figure 10 are the block diagrams of FGS encoder of the present invention, and wherein the information of reference block depends on basic layer.In these block diagrams, a FGS layer only is shown.But, should be appreciated that it is very simple expanding to the structure with a plurality of FGS layers from a FGS layer.

As can finding out from block diagram, FGS code device is the double loop video code device with additional " reference block formation module ".

Figure 11 has described the typical mobile device according to embodiment of the present invention.Mobile device 1 shown in Figure 11 can be used in cellular data and voice communication.Should be noted that the present invention is not limited to this specific implementations, this execution mode is represented one of various different execution modes.Mobile device 1 comprises (master) microprocessor or microcontroller 100, and the assembly that is associated with the operation of microprocessor controlling mobile equipment.These assemblies comprise display controller 130, nonvolatile memory 140, the volatile memory 150 such as random access storage device (RAM), audio frequency I/O (I/O) interface 160 that is connected with microphone 161, loud speaker 162 and/or headphone 163, the keypad controller 170 that is connected with keypad 175 or keyboard, any auxiliary I/O (I/O) interface 200 and the short-range communication interface 180 that is connected with display apparatus module 135.Such equipment also is included in the miscellaneous equipment subsystem shown in 190 place's generality usually.

Mobile device 1 can communicate and/or can similarly communicate via data network via speech network, any public land mobile network (PLMN), especially GSM (global system for mobile communications) or UMTS (Universal Mobile Telecommunications System) such as for example digital cellular network form.Typically, via air interface is the cellular communication interface subsystem and cooperating proceeds to the voice and/or the data communication of base station (BS) or Node B (not shown) for further assembly (above seeing), and wherein base station (BS) or Node B are Radio Access Network (RAN) parts of cellular network foundation structure.

The cellular communication interface subsystem of schematic representation comprises in Figure 11: cellular interface 110, digital signal processor (DSP) 120, receiver (RX) 121, transmitter (TX) 122 and one or more local oscillator (LO) 123, and this cellular communication interface subsystem makes it possible to communicate with one or more public land mobile network (PLMN).Digital signal processor (DSP) 120 is to transmitter (TX) 122 transmission signals of communication 124 and from receiver (RX) 121 receiving communication signals 125.Except process communication signals, digital signal processor 120 also provides receiver control signal 126 and transmitter control signal 127.For example, except the signal that will launch and the signal that will receive divide other modulation and demodulation, can control gain level on the signal of communication that is applied in receiver (RX) 121 and the transmitter (TX) 122 adaptively by the automatic gaining controling algorithm of realization in digital signal processor (DSP) 120.Other transmitter control algolithm also can realize in digital signal processor (DSP) 120 so that provide more Advanced Control for transmitter receiver 121/122.

If the communication that mobile device 1 is undertaken by PLMN appears in single-frequency or the at interval nearer frequency set, then single local oscillator (LO) 123 can use with transmitter (TX) 122 and receiver (RX) 121 cooperations.Alternately, if the emission of the voice/data communications frequency different with receiving use then can use a plurality of local oscillators to generate a plurality of correspondent frequency.

Although the mobile device of describing among Figure 11 1 with as or antenna 129 with classification antenna system (not shown) use, mobile device 1 can signal receives and the single antenna construction of emission be used with being used for.The information that includes voice messaging and data message is sent to cellular interface 110 or this interface transmission certainly via the data link between digital signal processor (DSP) 120 and the cellular interface 110.The detailed design of cellular interface 110 will be depended on the wireless network that mobile device 1 is intended to operate therein such as frequency band, assembly selection, power level etc.

After may relating to any required network registry of registering desired subscriber identification module (SIM) 210 in cellular network or activation and finishing, mobile device 1 can be with after wireless network sends and receives the signal of communication that comprises voice signal and data-signal.Antenna 129 is routed to receiver 121 from the signal that wireless network receives, and this receiver 121 provides such as signal amplification, down-conversion, filtering, channel and selected and simulate operation to digital conversion.Simulating to digital conversion of received signal allow to use digital signal processor (DSP) 120 to carry out more complicated communication function, such as digital demodulation and decoding.Utilize similar mode to handle the signal that will be emitted to network, this mode comprises modulation and the coding that is for example undertaken by digital signal processor (DSP) 120, and subsequently signal is provided to transmitter 122 and be emitted to wireless network to carry out numeral to analog converting, up-conversion, filtering, amplification and via antenna 129.

Also can be designated as the function of microprocessor/microcontroller (μ C) 110 management mobile devices 1 of equipment platform microprocessor.The operating system software 149 that processor 110 uses preferably is stored in the permanent memory such as nonvolatile storage 140, and this permanent memory for example can be embodied as flash memory, battery backup RAM, any other nonvolatile memory technology or its any combination.Except the low layer function and (figure) basic user interface function operations system 149 of controlling mobile equipment 10, nonvolatile storage 140 also comprises a plurality of high layer software application programs or module, such as voice communication software application 142, data communication software application 141, manager module (not shown), perhaps the software module (not shown) of any other type.These modules are carried out by processor 100 and provide high-level interface between the user of mobile device 1 and mobile devices 1.This interface typically comprises the graphic assembly that provided by the display 135 by display controller 130 control and by the keypad 175 that is connected to processor 100 via keypad controller 170, auxiliary I/O (I/O) interface 200, and/or the I/O assembly that is provided of short reach (sr) communication interface 180.Auxiliary I/O interface 200 especially comprises USB (USB) interface, serial line interface, MMC (multimedia card) interface and relevant interface technology/standard, and any other standardized or proprietary data communication bus technology, and short-range communication interface radio frequency (RF) low-power interface especially comprises WLAN (WLAN (wireless local area network)) and Bluetooth Communication Technology or IRDA (infrared data access) interface.Here the RF low-power interface technology that relates to especially should be understood that to comprise any IEEE 801.xx standard technique, can be from the description of institute of Electrical and Electronic Engineers's acquisition to it.And each of auxiliary I/O interface 200 and short-range communication interface 180 can be represented one or more interface of one or more input/output interface technology of support and communication interface technique respectively.Operating system, specific device software applications or module or their part can be loaded on the volatile memory 150 (realizing to operate quickly typically) such as random access storage device temporarily on the basis of DRAM (directly random access storage device) technology.And, the signal of communication that will receive for good and all write be arranged in nonvolatile memory 140 or be arranged in any preferably removably connect via auxiliary I/O interface be used to store the file system of massage storage of data before, also the signal of communication of this reception can be stored in the volatile memory 150 temporarily.Should be appreciated that assembly representative described above is in this typical components with the specific conventional mobile device 1 of cellular form.The present invention is not limited to these concrete assemblies, and their realization as described herein only is used for explanation and for integrality.

The exemplary software application module of mobile device 1 is a personal information manager application, and this application provides the PDA that typically comprises contact manager, calendar, task manager etc. functional.Such personal information manager is carried out by processor 100, and it can visit the assembly of mobile device 1, and can be mutual with other software application module.For example, with the mutual permission management of telephone call of voice communication software application, voice mail etc., and make it possible to manage SMS (flexible message service), MMS (multimedia service), E-mail communication and other data transmission alternately with data communication software is used.Nonvolatile memory 140 preferably provide file system so that on equipment permanent storage especially comprise calendar, contact person's etc. data item.For example the ability of carrying out data communication via cellular interface, short-range communication interface or auxiliary I/O interface and network has realized uploading, downloading and synchronously via such network.

Application module 141 to 149 representatives are configured to software application or the functions of the equipments by processor 100 execution.In the known mobile device of the overwhelming majority, the integrated operation of single processor management and controlling mobile equipment and all functions of the equipments and software application.Such notion is suitable for for current mobile device.The functional realization of enhanced multimedia for example comprises: reproduce video stream application, processing digital images and by digital camera functionality characteristic capture video sequences integrated or that removably connect.This realization can also comprise game (gaming) application with advanced figure and necessary computing capability.A kind of mode of the processing capability requirement of having carried out in the past is by carrying out the problem that powerful and general processor cores solves ever-increasing computing capability.It is to carry out two or more independent processor kernels that another kind provides the means of computing capability, and this is a known method in the art.Those skilled in the art can directly understand the benefit with several independent processor kernels.Yet, general processor is designed to finish various different task and need not distinct task preselected carried out specialization, and multiprocessor is arranged and can be comprised that one or more general processor and one or more are suitable for handling the application specific processor of preplanned mission collection.Yet in an equipment, especially the realization of several processors needs assembly is carried out complete and complicated design more traditionally in such as the mobile device of equipment 1.

Hereinafter, the present invention will provide such notion: it allows additional processor cores simply to be integrated in the existing treatment facility realization, and this realization makes it possible to omit complete and complicated design again.This creationary notion can frame of reference chip (SoC) design.The notion of System on Chip/SoC (SoC) is that several (perhaps whole) assemblies at least with treatment facility are integrated in the integrated chip of single height.Such System on Chip/SoC can all be included in numeral, simulation, mixed signal and common radio-frequency enabled on the chip.Typical treatment facility comprises many integrated circuits of carrying out different task.These integrated circuits can especially comprise microprocessor, memory, universal asynchronous receiver-transmitter (UART), serial port, direct memory visit (DMA) controller etc.Universal asynchronous receiver-transmitter (UART) is changed between the parallel position of data and serial bit.The nearest improvement of semiconductor technology makes the integrated circuit of ultra-large integrated (VLSI) can allow the phenomenal growth of complexity, becomes possibility on the one chip thereby numerous assemblies of system are integrated in.Referring to Figure 11, its one or more assembly, for example controller 130 and 170, memory assembly 150 and 140, and one or more interface 200,180 and 110 can be in processor 100 be integrated in the single chip of final formation System on Chip/SoC (SoC).

In addition, equipment 1 is equipped with and is used for inventive operation according to the present invention is carried out scalable coding 105 and scalable decoding 106 to video data module.Utilize CPU 100, described module 105 and 106 can be used separately.But equipment 1 is suitable for carrying out respectively video data encoding or decoding.Described video data can receive by the communication module of equipment, and perhaps these data also can be stored in any storage device of expecting in the equipment 1.

Although at one or more execution mode of the present invention it is described, those skilled in the art are to be understood that aforementioned and various other variations, the omission on its form and the details and are offset and all can make without departing from the present invention.

Claims (41)

1. method that is used for the motion compensation scalable video comprises:

Come together to form reference block based on basic layer reference block, enhancement layer reference block with basic layer reconstructed predicted residual error piece, wherein this reference block is used at the fine granularity scalable layer piece in the present frame being encoded, should be used as the reference of in basic layer, frame being rebuild by basic layer reference block, and the reference frame of this enhancement layer reference block from this fine granularity scalable layer forms; And

At least adjust this reference block based on the conversion coefficient of this basic layer reconstructed predicted residual error piece.

2. method according to claim 1, wherein when this conversion coefficient of basic layer reconstructed predicted residual error piece was zero entirely, described adjustment comprised:

The right to choose repeated factor makes this reference block be formed the weighted average of this basic layer reference block and this enhancement layer reference block.

3. method according to claim 1, wherein when this conversion coefficient of basic layer reconstructed predicted residual error piece comprised one or more nonzero coefficient, described formation comprised:

Substantially layer reference block is transformed to basic layer conversion coefficient;

This enhancement layer reference block is transformed to the enhancement layer conversion coefficient;

Calculate the conversion coefficient that is used for this reference block based on this basic layer conversion coefficient and this enhancement layer conversion coefficient; And

Change this reference block conversion coefficient, to obtain this reference block.

4. method according to claim 3, wherein said calculating comprises:

For each reference block conversion coefficient is selected first weight factor and second weight factor, make:

If it is zero being somebody's turn to do the common bit map coefficient of basic layer reconstructed predicted residual error piece, then this reference block conversion coefficient is formed this basic layer conversion coefficient and the weighted average that should be total to position enhancement layer conversion coefficient altogether at least based on this first weight factor; And

If common bit map coefficient non-zero that should basic layer reconstructed predicted residual error piece, then this reference block conversion coefficient is formed this position basic layer conversion coefficient and the weighted average of position enhancement layer conversion coefficient altogether altogether at least based on this second weight factor.

5. method according to claim 4 wherein based on the frequency of this coefficient, is determined in this first weight factor and second weight factor at least one separately for each of this conversion coefficient, and wherein this frequency is represented by the position in the transform block.

6. method according to claim 4 is wherein determined in this first weight factor and second weight factor at least one based on the fine granular scalability cycle, and wherein this current coefficient is encoded in the cycle in this fine granular scalability.

7. method according to claim 2, wherein whether this weight factor has one or more such adjacent block based on this piece at least and determines, and this adjacent block has the non-zero transform coefficient of this basic layer reconstructed predicted residual error piece.

8. method according to claim 7 is wherein determined this weight factor based on the coding context index that is used for the coded block flag of this basic layer reconstructed predicted residual error piece is encoded at least.

9. method according to claim 2 wherein forms this reference block according to one of three kinds of modes at the piece in the macro block:

A) only form from this basic layer reference block;

B) only form from this enhancement layer reference block; And

C) form the weighted average of this basic layer reference block and this enhancement layer reference block, wherein coming signalisation at the macro-block level service marking is the mode that piece in the macro block forms this reference block.

10. method according to claim 4 further comprises:

Substantially the quantity and the predetermined value of the non-zero transform coefficient of layer reconstructed predicted residual error piece compare with this; And

If described quantity is greater than or equal to this predetermined value, this first weight factor then is set equals this second weight factor.

11. method according to claim 10, wherein when this first weight factor and second weight factor were set to equate, their value was calculated based on the number of the non-zero transform coefficient in this basic layer reconstructed predicted residual error piece.

12. method according to claim 1 is wherein used the reference block of this formation and is somebody's turn to do the reference signal that the convergent-divergent pattern sum conduct of layer reconstructed predicted residual error piece substantially is used to encode.

13. method according to claim 12 wherein is that this zoom factor of 1 is used to calculate this convergent-divergent pattern of layer reconstructed predicted residual error piece substantially.

14. method according to claim 1, wherein this fine granularity scalable layer comprises a plurality of fine granularity scalable layers, wherein said a plurality of fine granularity scalable layer comprises top layer, and wherein discrete basic layer is used to obtain this basic layer reference block, and this top layer is used to obtain this enhancement layer reference block.

15. method according to claim 1, wherein this fine granularity scalable layer comprises a plurality of fine granularity scalable layers, wherein said a plurality of scalable layer comprises top layer, and wherein be used to obtain this enhancement layer reference block when anterior layer, and the layer that is located immediately at below the anterior layer is used to obtain this basic layer reference block.

16. method according to claim 1, wherein said adjustment comprises:

Differential reference block is calculated as the enhancement layer reference block and is somebody's turn to do the difference between the layer reference block substantially;

At least adjust this differential reference block based on the conversion coefficient of this basic layer reconstructed predicted residual error piece; And

Obtain differential reference block and this reference block that is somebody's turn to do basic layer reference block sum as this adjustment.

17. method according to claim 16 wherein when this conversion coefficient of basic layer reconstructed predicted residual error piece is zero entirely, comprises at the described adjustment of differential reference block:

Selection is applied to the feasible weighted average that this reference block is formed this basic layer reference block and this enhancement layer reference block of weight factor of this differential reference block.

18. method according to claim 16 wherein when this conversion coefficient of basic layer reconstructed predicted residual error piece comprises one or more nonzero coefficient, comprises at the described adjustment of differential reference block:

Differential reference block is transformed to conversion coefficient;

Adjust conversion coefficient; And

Change this conversion coefficient, to obtain the differential reference block of adjustment.

19. an electronic module that uses in the motion compensation scalable video comprises:

Form module, be used for forming reference block based on basic layer reference block, enhancement layer reference block and basic layer reconstructed predicted residual error piece, wherein this reference block is used for using in the fine granularity scalable layer is encoded to the piece in the present frame, should be used as the reference of in basic layer, frame being rebuild by basic layer reference block, and the reference frame of this enhancement layer reference block from this fine granularity scalable layer forms; And

Adjusting module is used for adjusting this reference block based on the conversion coefficient of this basic layer reconstructed predicted residual error piece at least.

20. electronic module according to claim 19, wherein when this conversion coefficient of basic layer reconstructed predicted residual error piece was zero entirely, described adjusting module is suitable for the right to choose repeated factor made this reference block be formed the weighted average of this basic layer reference block and this enhancement layer reference block.

21. electronic module according to claim 19, wherein when this conversion coefficient of basic layer reconstructed predicted residual error piece comprised one or more nonzero coefficient, described formation module comprised:

Conversion module is used for this basic layer reference block is transformed into basic layer conversion coefficient and this enhancement layer reference block is transformed into the enhancement layer conversion coefficient;

Computing module is used for calculating the conversion coefficient that is used for this reference block based on this basic layer conversion coefficient and this enhancement layer conversion coefficient; And

Inverse transform module is used to change this reference block conversion coefficient, to obtain this reference block.

22. electronic module according to claim 21, wherein said computing module are suitable for selecting first weight factor and second weight factor for each reference block conversion coefficient, make:

If it is zero being somebody's turn to do the common bit map coefficient of basic layer reconstructed predicted residual error piece, then this reference block conversion coefficient is formed this basic layer conversion coefficient and the weighted average that should be total to position enhancement layer conversion coefficient altogether at least based on this first weight factor; And

If common bit map coefficient non-zero that should basic layer reconstructed predicted residual error piece, then this reference block conversion coefficient is formed this position basic layer conversion coefficient and the weighted average of position enhancement layer conversion coefficient altogether altogether at least based on this second weight factor.

23. electronic module according to claim 20, wherein whether this weight factor has one or more such adjacent block based on this piece at least and determines, this adjacent block has one or more non-zero transform coefficients of this basic layer reconstructed predicted residual error piece.

24. electronic module according to claim 20, wherein said computing module further is suitable for the quantity and the predetermined value of non-zero transform coefficient in this basic layer reconstructed predicted residual error piece are compared, if make that described quantity is greater than or equal to this predetermined value, then this first weight factor is set to equal this second weight factor.

25. electronic module according to claim 19, wherein this adjusting module is suitable for:

Differential reference block is calculated as the enhancement layer reference block and is somebody's turn to do the difference between the layer reference block substantially;

At least adjust this differential reference block based on the conversion coefficient of this basic layer reconstructed predicted residual error piece; And

Obtain differential reference block and this reference block that is somebody's turn to do basic layer reference block sum as this adjustment.

26. a software application product comprises having the storage medium that is used for the software application used at the motion compensation scalable video, described software application comprises:

Be used for forming the program code of reference block based on basic layer reference block, enhancement layer reference block and basic layer reconstructed predicted residual error piece, wherein this reference block is used for using in the fine granularity scalable layer is encoded to the piece in the present frame, should be used as the reference of in basic layer, frame being rebuild by basic layer reference block, and the reference frame of this enhancement layer reference block from this fine granularity scalable layer forms; And

Be used for adjusting based on the conversion coefficient of this basic layer reconstructed predicted residual error piece at least the program code of this reference block.

27. software application product according to claim 26, wherein said software application further comprises:

The program code that is used for the right to choose repeated factor makes that this reference block is formed the weighted average of this basic layer reference block and this enhancement layer reference block when this conversion coefficient of basic layer reconstructed predicted residual error piece is zero entirely.

28. software application product according to claim 26, this program code that wherein is used to form this reference block comprises:

Be used for to be somebody's turn to do the code that basic layer reference block is transformed to basic layer conversion coefficient and this enhancement layer reference block is transformed to the enhancement layer conversion coefficient;

Be used for calculating the code of the conversion coefficient that is used for this reference block based on this basic layer conversion coefficient and this enhancement layer conversion coefficient; And

Be used for when this conversion coefficient of this basic layer reconstructed predicted residual error piece comprises one or more nonzero coefficient, changing this reference block conversion coefficient to obtain the code of reference block.

29. software program product according to claim 28, wherein, at each reference block conversion coefficient, this reference block conversion coefficient is formed common position basic layer conversion coefficient and is somebody's turn to do the weighted average of position enhancement layer conversion coefficient altogether, if common bit map coefficient that should basic layer reconstructed predicted residual error piece is zero then carries out based on first weight factor, if bit map coefficient non-zero altogether that should basic layer reconstructed predicted residual error piece then carry out based on second weight factor.

30. software application product according to claim 29, wherein said software application further comprises:

Be used for to be somebody's turn to do the quantity of basic layer reconstructed predicted residual error piece non-zero transform coefficient and the program code that predetermined value compares, if make that described number is greater than or equal to this predetermined value, this first weight factor then is set equals this second weight factor, wherein this first and second weight factor is calculated based on the quantity of the non-zero transform coefficient of this basic layer reconstructed predicted residual error piece.

31. software application product according to claim 26, wherein this fine granularity scalable layer comprises a plurality of fine granularity scalable layers, wherein said a plurality of fine granularity scalable layer comprises top layer, and wherein discrete basic layer is used to obtain this basic layer reference block, and this top layer is used to obtain this enhancement layer reference block.

32. software application product according to claim 26, wherein this fine granularity scalable layer comprises a plurality of fine granularity scalable layers, wherein said a plurality of fine granularity scalable layer comprises top layer, and wherein be used to obtain this enhancement layer reference block when anterior layer, and the layer that is located immediately at below the anterior layer is used to obtain this basic layer reference block.

33. software application product according to claim 26, this program code that wherein is used to adjust this reference block comprises the code that is used to carry out following operation:

Differential reference block is calculated as the enhancement layer reference block and is somebody's turn to do the difference between the layer reference block substantially;

At least adjust this differential reference block based on the conversion coefficient of this basic layer reconstructed predicted residual error piece; And

Obtain differential reference block and this reference block that is somebody's turn to do basic layer reference block sum as this adjustment.

34. an electronic equipment comprises:

Communication module is used for setting up communication link has video data with transmission bit stream with another electronic equipment; And

The video data processing module is used for using at the motion compensation scalable video of this video data, and this processing module comprises:

Form module, be used for forming reference block based on basic layer reference block, enhancement layer reference block and basic layer reconstructed predicted residual error piece, wherein this reference block is used for using in the fine granularity scalable layer is encoded to the piece in the present frame, should be used as the reference of in this basic layer, frame being rebuild by basic layer reference block, and the reference frame of this enhancement layer reference block from this fine granularity scalable layer forms; And

Adjusting module is used for adjusting this reference block based on the conversion coefficient of this basic layer reconstructed predicted residual error piece at least.

35. electronic equipment according to claim 34, wherein when this conversion coefficient of basic layer reconstructed predicted residual error piece was zero entirely, described adjusting module is suitable for the right to choose repeated factor made this reference block be formed the weighted average of this basic layer reference block and this enhancement layer reference block.

36. electronic equipment according to claim 34, wherein when this conversion coefficient of basic layer reconstructed predicted residual error piece comprised one or more nonzero coefficient, described formation module comprised:

Conversion module is used for this basic layer reference block is transformed to basic layer conversion coefficient and this enhancement layer reference block is transformed to the enhancement layer conversion coefficient;

Computing module is used for calculating the conversion coefficient that is used for this reference block based on this basic layer conversion coefficient and this enhancement layer conversion coefficient; And

Inverse transform module is used to change this reference block conversion coefficient, to obtain this reference block.

37. electronic equipment according to claim 36, wherein said computing module are suitable for selecting first weight factor and second weight factor for each reference block conversion coefficient, make:

If it is zero being somebody's turn to do the common bit map coefficient of basic layer reconstructed predicted residual error piece, then this reference block conversion coefficient is formed this basic layer conversion coefficient and the weighted average that should be total to position enhancement layer conversion coefficient altogether at least based on this first weight factor; And

If common bit map coefficient non-zero that should basic layer reconstructed predicted residual error piece, then this reference block conversion coefficient is formed this position basic layer conversion coefficient and the weighted average of position enhancement layer conversion coefficient altogether altogether at least based on this second weight factor.

38. according to the described electronic equipment of claim 37, wherein said computing module further is suitable for the quantity and the predetermined value of non-zero transform coefficient in this basic layer reconstructed predicted residual error piece are compared, if make that described quantity is greater than or equal to this predetermined value, this first weight factor then be set equal this second weight factor.

39. electronic equipment according to claim 34, wherein this processing module comprises the video decode module and wherein this formation module and this adjusting module are the parts of this video decode module.

40. electronic equipment according to claim 34, wherein this processing module comprises video encoding module and wherein this formation module and this adjusting module are the parts of this video encoding module.

41. electronic equipment according to claim 34 comprises portable terminal.

Images