CN102204256A - Image prediction method and system - Google Patents

Abstract

A method for Computing a predicted frame from a first and a second reference frames, said method comprising for each block of pixels in the predicted frame the acts of defining a first block of pixels in the first reference frame collocated with a third block of pixels which is the block of pixels in the predicted frame; defining a second block of pixels corresponding, in the second reference frame, to the first block of pixels along the motion vector of said first block from said first to second reference frames; Computing a first set of coefficients allowing the transformation of the pixels of the first block into pixels of the second block; Computing pixels of the third block using the first set of coefficients and pixels from a fourth block collocated in the first reference frame with the second block of pixels.

Description

The image prediction method and system

Technical field

The present invention relates generally to image processing, and more specifically, relate to image prediction.

Background technology

Prediction is that a kind of statistical estimate is handled, and wherein estimates one or more stochastic variables according to the observation of other stochastic variables.When the variable that will estimate is associated with " in the future " and observable variable and " past " when being associated in some sense, this is called as prediction.One of the simplest, the most general Predicting Technique is linear prediction.Linear prediction for example is to predict a vector according to another vector.The most common purposes of prediction is to estimate stationary random process (that is sample of the immovable random process of its joint probability distribution (random stochastic process) when being shifted, according to the observation of several samples formerly in time or space.The Another Application of prediction is when being observed " formerly " when block of pixels is estimated block of pixels, being applied in during image/video compresses according to what comprising in reference picture (being also referred to as the forward direction image).In the case, each is predicted image (perhaps picture or frame) is divided into non-overlapped rectangular block.Estimation in the reference picture (Motion Estimation, ME) the derive motion vector employed vector of prediction of the skew of the coordinate being used for providing from the coordinate of predicted picture to reference picture (that is, for) of each piece are provided.Then, with reference to by the corresponding blocks in the reference frame of the motion vectors point of being derived, use motion compensation (MC) to predict each piece.ME and MC both are the known methods of those skilled in the art.The method can help to eliminate redundant information, and the result, can need less bit to describe residual error (its be between original and predicted poor).Yet in fact this ME/MC Forecasting Methodology is not to be used to predict the final solution of frame in the future, and this is because it is based on the hypothesis that the motion object of being caught is being carried out translational motion, and this is always ungenuine.In addition, for the image that relates to nongausian process was estimated, the ME/MC technology can not fully extract and will help to predict that the institute about the past of frame might information in the future.

Nowadays, need a kind of image prediction solution prior art shortcoming, that can easily on the existing communication architecture, realize that is used to overcome.

Summary of the invention

The objective of the invention is to overcome the deficiency and/or prior art is made improvement.

In order to reach such degree, the present invention proposes a kind of method that is used for calculating predicted frame according to first and second reference frames, for each block of pixels in the predicted frame, described method comprises following action:

A) first block of pixels in juxtaposed first reference frame of block of pixels in definition and the predicted frame;

B) along described and put the motion vector of piece and be defined in second reference frame and first and put the second corresponding block of pixels of block of pixels from described first reference frame to second reference frame;

C1) calculate be used to allow with and the pixel of putting piece be transformed to first coefficient sets of second pixel;

D) pixel of using first coefficient sets and coming in comfortable first reference frame to calculate predicted frame piece with juxtaposed the 4th pixel of second block of pixels.

The invention still further relates to a kind of system according to claim 7.

The invention still further relates to a kind of device according to claim 4.

The invention still further relates to a kind of computer program according to claim 10.

The advantage of the method that proposes is that it can make full use of redundancy adaptively, is adjusted at the movable information of deriving between the successive frame with the characteristic according to pixel in the local spatial time zone.

Compare with existing solution, another advantage of the method that proposes is that it can regulate interpolation coefficient (to come predict pixel according to the existing pixel in the previous frame) adaptively, thus the unstable statistical attribute of match video signal.Interpolation coefficient has been played the part of the key player for precision of prediction.Described coefficient is accurate more, and then predicted frame is just reliable more.These coefficients may relate to the heavy burden of the bit rate aspect of video compression.Thereby the method according to this invention has proposed to be used for deriving by the high similitude between the same object that makes full use of consecutive frame the algorithm of first more severity factor, has therefore discharged the great burden that transmits this coefficient.

Description of drawings

Now, will be individually by means of example and only describe embodiments of the invention with reference to the accompanying drawings, in described accompanying drawing, provide corresponding Reference numeral, and wherein to same parts:

Fig. 1 schematically illustrates the example according to the pixel prediction in the predicted image that carries out according to the pixel in the reference picture of the embodiment of the invention;

Fig. 2 schematically illustrates piece and the frame that uses in the method according to the embodiment of the invention;

Fig. 3 A schematically illustrates the method according to the embodiment of the invention;

Fig. 3 B schematically illustrates the method according to the embodiment of the invention;

Fig. 3 C schematically illustrates the method for the additional embodiment according to the present invention;

Fig. 4 A schematically illustrates integer (integer) sample of 1/4th sample brightness interpolations (quarter sample luma interpolation) that are used for traditional interpolating method and the example of mark (fractional) sampling location;

Fig. 4 B schematically illustrates the example that is used for reciprocal fraction pixel (fractional pixel) is carried out the spatial neighborhood of interpolation according to the embodiment of the invention;

Fig. 5 schematically illustrate according to the embodiment of the invention from the back to transformation to the forward direction coefficient;

Fig. 6 is by the comparison of the method according to this invention with the side information of existing scheme generation based on the motion extrapolation in DVC coding-" George Foreman (foreman) ";

Fig. 7 is by the comparison of the method according to this invention with the side information of existing scheme generation based on the motion extrapolation in DVC coding-" Mobile (Mobile) ";

Fig. 8 is by the comparison of the method according to this invention with the reconstruction WZ frame of existing scheme generation based on the motion extrapolation in DVC coding-" George Foreman ";

Fig. 9 is by the comparison of the method according to this invention with the reconstruction WZ frame of existing scheme generation based on the motion extrapolation in DVC coding-" Mobile ";

Figure 10 is the comparison of the performance when replacing hopping model (skip model) for sequence " Mobile " by the method according to this invention; And

Figure 11 is when the comparison for the performance of sequence " Tan Pute (Tempete) " when replacing hopping model by the method according to this invention.

Embodiment

Below be the description of example embodiment, when making in combination with accompanying drawing, the description of these example embodiment will prove above-mentioned feature and advantage, and introduce feature and advantage in addition.

In the following description, for the purpose of explaining rather than limiting, propose specific detail (such as, framework, interface, technology, device etc.) describe being used to.Yet, will be apparent that for those skilled in the art other embodiment that break away from these details will be understood that to be in the scope of claims.

And, for purpose clearly, omitted the detailed description of well known device, system and method, thereby do not made the description of the invention fuzzy.In addition in detail, other entities in router, server, node, base station, gateway or the communication network are not described in detail, this is because their realization has exceeded the scope of native system and method.

In addition, should understand clearly, comprised accompanying drawing for the n-lustrative purpose, and described accompanying drawing not show the scope of native system.

The method according to this invention has proposed a kind of model that is used for coming based on the observation that image is formerly made predicted picture (that is so-called predicted or current images/frame).In the method according to the invention, be that unit carries out this prediction with the block of pixels, and can carry out this prediction for each piece of predicted image.In the text, image can be assimilated (assimilate) and be (pixel) piece.By in first image and second image and put (collocated) piece, one will understand that the piece that in two images, is in the identical position.For example, in Fig. 2, piece B _t(k, l) and B _T-1(k l) is and puts piece.

Fig. 3 A has described the illustrative embodiment of the method according to this invention, and is wherein known and calculate this predicted frame at predicted frame 260 (that is the frame that predict) first reference frame 200 and second reference frame 210 before according to the both.All images can be in the image stream.

In this illustrative embodiment, for each block of pixels that will predict in the predicted frame 260, this method comprises the action 220 that is used for allowing to define first block of pixels in juxtaposed first reference frame of block of pixels (that is, the 3rd) that will predict with predicted frame.Then, action 230 allows along first and puts the motion vector of block of pixels from described first reference frame to second reference frame and be defined in second reference frame with described and put the second corresponding block of pixels of piece.Motion vector is the vector that is used for inter prediction (inter prediction), the skew of the coordinate this inter prediction is used for providing from the coordinate of predicted picture to reference picture.It is used for showing this macro block in the predicted picture or pixel (or a similar macro block or pixel) based on reference picture macro block or locations of pixels.Because first and second reference pictures are known pictures, thus can with easily for those skilled in the art can with technology be used to derive from first reference frame to motion vector second reference frame and that put piece, and therefore, can define second.Subsequently, can in action 240, calculate be used to allow with and the pixel of putting piece be transformed to first coefficient sets of second pixel.Finally, action 250 pixel that allow to use first coefficient sets and come in comfortable first reference frame to calculate predicted frame piece with juxtaposed the 4th pixel of second block of pixels.

In the method according to the invention, use first coefficient sets, the block of pixels of deriving and to predict according to the 4th block of pixels in first reference frame.Thereby, since with second and the 4th respectively and put in second and first reference frame, so use the 4th block of pixels that will predict in the predicted frame of deriving to hint: the motion vector that is used to define second is with to be used to set up the motion vector that concerns between the 4th and the block of pixels that will predict identical.

Fig. 1 has schematically described along following motion vector 130 and has predicted pixel 111 in the predicted frame 110 according to the pixel in the reference frame 120, and the pixel 111 that this motion vector 130 is used for predicting is carried out related (link) with its respective pixel 122 of reference frame 120.

As shown in Figure 1, for each pixel 111 in the predicted frame 110, along derive respective pixel 121 in the reference frame 120 of (shown in Figure 1) movement locus (trajectory) by motion vector 130.Defined is the square space neighborhood 125 at center with respective pixel 121 in reference frame 120.Thereby, the pixel in the predicted frame 111 is approximately the linear combination of the pixel 122 of corresponding spatial neighborhood 125 in the reference frame 120.This interpolation can be handled and be expressed as:

${\hat{Y}}_t(m,n)=\underset{-r\≤(i,j)\≤r}{\mathrm{\Σ}}{X}_{t-1}(\tilde{m}+i,\tilde{n}+j)\·{\mathrm{\α}}_{i,j}+n_t(m,n)---\left(1\right)$

Wherein:

- Showed and be positioned at coordinate (m, the predicted pixel of n) locating 111 in the predicted frame 110;

-X _T-1Performance corresponding to pixel in the reference frame 120;

-

Showed and passed through

, be positioned at (m, the position of the respective pixel 121 in the motion vector 130 of the corresponding predicted pixel of n) locating 111 reference frame 120 pointed in the predicted frame 110;

-α _{I, j}It is interpolation coefficient;

-n _t(m n) is additive white Gaussian noise.

Can use method known to those skilled in the art (such as, like this mean square error (MSE) method and lowest mean square (LMS) method of further explaining of the back) interpolation coefficient of deriving.

The radius r of interpolation set or interpolation filter (that is the interpolation coefficient that, is used for square space neighborhood 125) can be used for the dimension definitions of interpolation filter: (2r+1) * (2r+1).For example, in Fig. 1, radius is r=1, and interpolation filter is of a size of 3 * 3.

Fig. 2 has described the frame that uses in the method according to the invention.Use the first reference frame X _T-1211 and the second reference frame X _T-2222 derive and are used to derive predicted frame Y _tThe interpolation coefficient of each block of pixels in 205.

With the first reference frame X _T-1First B in 211 _T-1(k l) 212 is defined as predicted frame Y _tThe block of pixels that will predict in 205 (the 3rd) B _t(k, l) 201 and put piece.

Since known and defined first and second reference frames both, so existing method well known by persons skilled in the art allow along and put (or first) piece B _T-1(k, l) 212 from the first reference frame X _T-1To the second reference frame X _T-2Motion vector v _{T-1, t-2}(k l) defines second

Thereby, can be according to the known pixels in first and second reference frames, definition first interpolation coefficient set as described in Fig. 1.

Then, together with on this in previous incompatible use second reference frame 222 of first coefficient set that obtains

First reference frame in and put piece

(with reference to figure 3A, being also referred to as the 4th) is to derive block of pixels (or the 3rd) B that will predict in the predicted frame 205 _t(k, l) the predicted pixel in 201.

Fig. 3 B has described the illustrative embodiment of the method according to this invention.

In action 300, at predicted frame Y _tThe block of pixels B that middle selection will be predicted _t(k, l).Then, definition and Y in action 310 _tIn the juxtaposed B of block of pixels that will predict _T-1(k, l).Because it is known and defined X _T-1And X _T-2Both, thus in action 320 can along and put piece B _T-1(k, l) 212 from the first reference frame X _T-1To the second reference frame X _T-2Motion vector v _{T-1, t-2}(k l) is defined in X _T-2In with B _T-1(k, l) second of correspondence

The action 330 in the definition with Juxtaposed X _T-1In the 4th

Under the situation of application drawing 1 described method, in action 340, can pass through according to second

In pixel and approximate first B _T-1(k, the l) pixel in obtain the set of first interpolation coefficient.In other words, suppose B _T-1(k, l) each pixel in be approximately with by the motion v _{T-1, t-2}(k, l) respective pixel of Zhi Xianging is the piece at center

In the linear combination of square space neighborhood:

${\hat{Y}}_{t-1}(m,n)=\underset{-r\≤(i,j)\≤r}{\mathrm{\Σ}}{X}_{t-2}(\tilde{m}+i,\tilde{n}+j)\·{\mathrm{\α}}_{\mathrm{ij}}+n_{t-1}(m,n)---\left(2\right)$

The approximate definition of depending on interpolation coefficient of this pixel.In fact, these interpolation coefficients should be chosen to be optimum interpolation coefficient.

In equation (2), known X _T-2In pixel.In addition, because known Y also _T-1In pixel, so can be with the pixel of being undertaken by equation (2) approximate and Y _T-1In corresponding actual pixels compare so that derive interpolation coefficient α _{I, j}In the illustrative embodiment of the method according to this invention, so go up mentionedly, use mean square error (MSE) to carry out this relatively, to define resulting mean square error:

${\mathrm{\ϵ}}^2(k,l)=\underset{(m,n)\∈}{\mathrm{\Σ}}\underset{B_{t-1}(k,l)}{\mathrm{\Σ}}E\left({\left|\right|Y_{t-1}(m,n)-{\hat{Y}}_{t-1}(m,n)\left|\right|}^2\right)---\left(3\right)$

MSE as performance standard can be considered as for by based on removing the measurement that measurable information reduces the energy of how many these signals from the observation of signal.Because the target of fallout predictor is to remove this measurable information, so preferable fallout predictor is corresponding to less MSE.

Then, can use lowest mean square (LMS) method to derive optimum interpolation coefficient.

Then, make following hypothesis in action in 345, promptly can use the first identical coefficient sets, be similar to pixel in the piece that will predict according to the pixel in the 4th, this is owing to have the highly redundant degree between these two references and predicted frame).Suppose to use equation (3) and the optimum interpolation coefficient of deriving is α _{I, j}, then as previously explained, can use same factor and equation (1) to make B as follows _t(k, prediction l):

${\hat{Y}}_t(m,n)=\underset{-r\≤(i,j)\≤r}{\mathrm{\Σ}}{X}_{t-1}(\tilde{m}+i,\tilde{n}+j)\·{\mathrm{\α}}_{i,j}+n_t(m,n)---\left(4\right)$

Wherein, α _{I, j}It is the interpolation coefficient that in equation (2) and (3), obtains.

What can emphasize is, the frame in the frame stream is near more, and then redundancy is high more, and thereby, this hypothesis is also just good more.What can emphasize is to be to use identical motion vector in fact, promptly

V _t，t-1(k，l)＝V _t-1，t-2(k，l)

Come basis In pixel derivation B _t(k, the l) prediction of the pixel in (action 350) and according to

In pixel derivation B _T-1(k, l) prediction of the pixel in.

In additional embodiment of the present invention,, can derive the set of second interpolation coefficient, so that increase the precision of the pixel prediction in the block of pixels that will predict in the predicted frame with reference to figure 3C.

In fact, symmetrically, can use second interpolation coefficient to gather in action 245 will

In pixel approximate or be expressed as B _T-1(k, the linear combination of pixel l):

${\hat{X}}_{t-2}(\tilde{m},\tilde{n})=\underset{-r\≤(i,j)\≤r}{\mathrm{\Σ}}{X}_{t-1}(m+i,n+j)\·{\mathrm{\β}}_{i,j}+n_{t-2}(\tilde{m},\tilde{n})---\left(5\right)$

Then, can suppose: the second identical coefficient sets can be used in described second interpolation coefficient set of action 255 uses, according to B _t(k, l) pixel in is similar to or expresses In pixel (making hypothesis once more: for example when in frame stream, being chosen to be consecutive frame, then between reference and predicted frame, have the highly redundant degree):

${\hat{X}}_{t-1}(\tilde{m},\tilde{n})=\underset{-r\≤(i,j)\≤r}{\mathrm{\Σ}}{Y}_t(m+i,n+j)\·{\mathrm{\β}}_{i,j}+n_{t-1}(\tilde{m},\tilde{n})---\left(6\right)$

Yet, here, because B _t(k, l) the pixel the unknown in (as the pixel that will predict) is so can not be expressed as them

In the linear combination of pixel.But, because mathematic(al) representation is linear combination, so can use symmetrical interpolation coefficient, the basis of the interpolation coefficient of second set

In pixel express B _t(k, l) pixel in:

${\mathrm{\β}}_{i,j}^{\′}={\mathrm{\β}}_{-i,-j}---\left(7\right)$

${\hat{Y}}_t(m,n)=\underset{-r\≤(i,j)\≤r}{\mathrm{\Σ}}{X}_{t-1}(\tilde{m}+i,\tilde{n}+j)\·{\mathrm{\β}}_{i,j}^{\′}+n_t(m,n)---\left(8\right)$

Wherein, β ' _{I, j}Be the contrary coefficient corresponding with the coefficient of in (5), deriving.

Finally, for this optional embodiment according to the inventive method, derive/obtain two interpolation coefficient set, this has hinted basis

The B of middle pixel _t(k, l) two expression of middle pixel/approximate.Thereby, for each pixel, can be by these two of same pixel approximately be averaged or average, be similar to according to described two and derive optimum prediction:

${\hat{Y}}_t(m,n)=$

$(\underset{-r\≤(i,j)\≤r}{\mathrm{\Σ}}{X}_{t-1}(\tilde{m}+i,\tilde{n}+j)\·{\mathrm{\α}}_{i,j}+\underset{-r\≤(i,j)\≤r}{\mathrm{\Σ}}{X}_{t-1}(\tilde{m}+i,\tilde{n}+j)\·{{\mathrm{\β}}^{\′}}_{i,j})/2+n_t(m,n)$

(9)。

In fact, equation (4) and (8) allow approximate frame Y in two different directions (forward direction and back to) _tIn (this has hinted α for m, the same pixel of n) locating _{I, j}≈ β ' _{I, j}Thereby, allow to improve this accuracy of predicting.

In the practice, for example under the situation of encoder/decoder system, the method according to this invention is based on the following fact, promptly the block of pixels both in first and second reference frames for encoder can with/known, thereby this allow to use data of deriving according to these reference frames to obtain predicted frame.Can also use the interpolation device that is used for calculating predicted frame to realize this method according to video flowing first and second reference frames.Described coding, decoding or interpolation device typically can be following electronic installations, and it comprises and be arranged to the processor that is carried in the executable instruction of storing on the computer-readable medium that this executable instruction makes described processor carry out this method.This interpolation device can also be the encoder/decoder part that is used for calculating according to video flowing first and second reference frames system of predicted frame, this system comprises conveyer, this conveyer is used for transmitting the video flowing that comprises described reference frame to interpolation device, to be used for further calculating predicted frame.

In described this illustrative embodiment, motion vector is pixel integer precision (integer accuracy) on this.Yet the method according to this invention also can realize subpixel accuracy.This because of (being described on Fig. 4 A) existing/known 1/4th pixel interpolating methods in, use the whole pixel nearest, come each sub-pixel is carried out interpolation along level and vertical direction, filter tap (for example, 6 tap filters mean that interpolation utilize 6 nearest whole pixels) by fixing with the sub-pixel of wanting interpolation.

On Fig. 4 A, illustrate the integral sample (having uppercase shaded block) and fractional sampling position (the unhatched blocks) that are used for 1/4th sample brightness interpolations (that is, with the resolution interpolation of luma samples be original resolution 4 times) in level and vertical direction with lowercase.

At subpixel accuracy in this case, can use the method according to this invention as follows, shown in Fig. 4 B (spatial neighborhood).For each sub-pixel, find nearest whole pixel, then by the weighted linear combination of whole pixel in the square neighborhood that is the center with recently whole pixel, come the sub-pixel of correspondence is carried out interpolation.By selected suitable interpolation coefficient, it has been realized and the identical result of existing subpixel interpolation method.Yet, in real image, only carry out interpolation, and under the situation of the multiple section (complex region) that uses existing/known solution, this interpolation may be always enough not accurate along level and vertical direction.In addition, in the conventional method, interpolation tap and filter are always fixed, and this has further limited interpolation result's precision.Yet, in the method according to the invention, can carry out interpolation, rather than be limited to along level or vertical direction and carry out interpolation along any direction, promptly along the coefficient of specific direction significantly greater than other directions.As example, if diagonally there is the edge, then relative to the coefficient in other positions along correspondence the filter coefficient angular direction, the method according to this invention is incited somebody to action, and thereby improved the interpolation precision (yet, conventional filter only can be carried out interpolation along level or vertical direction, and thereby can not be suitable for edge direction).In addition, can regulate interpolation coefficient adaptively according to the characteristic of pixel in the adjacent space neighborhood.

To propose the forecasting efficiency of model in order verifying, to use distributed video coding (DVC) extrapolation to make the example of realization.In DVC, final Wyner-Ziv (WZ) frame of rebuilding comprises the side information (SI) that adds the error of being proofreaied and correct by Parity Check Bits.As a result, the improvement of SI prediction has constituted one of most critical aspect of improving the DVC compression efficiency.Because if SI is high-quality, then the energy of residual, information (it need proofread and correct the error between SI and the primitive frame) reduces, and has caused transmitting the minimizing of Parity Check Bits, and thereby has reduced bit rate.Because the method according to this invention is suitable for only predicting present frame based on the available information in past, thus it can be implemented in during extrapolation among the DVC uses, and itself and existing scheme based on extrapolation can be compared.In DVC, because original pixels is unavailable at decoder-side, so carry out ME in the frame in the past.For example, as shown in Figure 2, for each the piece B in the predicted frame _t(k, l), we at first use and put piece B _T-1(k l) as current block, and finds its MV among the frame t-2, and handles and use MV to obtain B by carrying out MC among the frame t-1 then _t(k, prediction l).In the illustrative embodiment of the method according to this invention, we use with existing based on the identical MV of the scheme of motion extrapolation, and have described comparative result in Fig. 6-9.H.263+, we will use key frame, and wherein QP is set to 8, and use the Turbo encoder to come the WZ frame is encoded then.In Fig. 6 and Fig. 7, described SI relatively.Can easily observe: compare with existing motion extrapolation method, the method according to this invention can be improved the PSNR value of SI significantly.For example, the gain in the George Foreman QCIF sequence is greater than 1.5dB, and the gain in the Mobile QCIF sequence approximately is 3dB.Should significantly improve mainly and predict owing to the super ability of the method according to this invention.The WZ that Fig. 8 and 9 has presented the method according to this invention and existing motion extrapolation method compares.As can be seen, in George Foreman QCIF sequence, gain is greater than 1db, and the gain in Mobile QCIF sequence, this gains greater than 2.5dB.

Claims (10)

1. method that is used for calculating predicted frame according to first and second reference frames, for each block of pixels in the predicted frame, described method comprises following action:

A) definition with corresponding to first block of pixels in juxtaposed first reference frame of the 3rd block of pixels of block of pixels in the predicted frame;

B) be defined in second corresponding second reference frame block of pixels along described first motion vector with first block of pixels from described first reference frame to second reference frame;

C1) calculate first coefficient sets that is used to allow first pixel is transformed to second pixel;

D) use the pixel juxtaposed the 4th in first coefficient sets and next comfortable first reference frame, calculate the 3rd pixel with second block of pixels.

2. according to the method for claim 1, also comprise following action c2): calculate second coefficient sets that is used to allow second pixel is transformed to first pixel, this action d) also use described second coefficient sets.

3. according to the method for one of aforementioned claim, wherein said block of pixels is the square block of n * n pixel, wherein n is the integer greater than 1, first and second coefficient sets are corresponding to first and second n * n matrix, and wherein, at action d) in, this calculation procedure is considered described first matrix and described second transpose of a matrix.

4. interpolation device that is used for calculating predicted frame according to first and second reference frames of video flowing, described device is arranged to described first and second frames of selection from this video flowing, each block of pixels in the predicted frame further is arranged to described device:

A) definition with as first block of pixels in juxtaposed first reference frame of the 3rd block of pixels of block of pixels in the predicted frame;

B) be defined in second corresponding second reference frame block of pixels along described first motion vector with first block of pixels from described first reference frame to second reference frame;

C1) calculate first coefficient sets that is used to allow first pixel is transformed to second pixel;

D1) use the pixel juxtaposed the 4th in first coefficient sets and next comfortable first reference frame, calculate the 3rd pixel with second block of pixels.

5. according to the device of claim 4, further be arranged to:

C2) calculate second coefficient sets that is used to allow second pixel is transformed to first pixel;

D2) also use the pixel of the predicted frame of the incompatible calculating of second coefficient set.

6. according to the device of one of aforementioned claim 4 and 5, wherein said block of pixels is the square block of n * n pixel, wherein n is the integer greater than 1, first and second coefficient sets further are arranged to this device corresponding to first and second n * n matrix: consider that described first matrix and described second transpose of a matrix calculate the pixel of predicted frame.

7. system that is used for calculating predicted frame according to first and second reference frames of video flowing, described system comprises:

-conveyer is used to transmit this video flowing;

-interpolation device is arranged to:

-receive this video flowing from this conveyer;

-described first and second frames of selection from this video flowing,

Each block of pixels in the predicted frame further is arranged to described device:

A) definition with as first block of pixels in juxtaposed first reference frame of the 3rd block of pixels of block of pixels in the predicted frame;

B) be defined in second corresponding second reference frame block of pixels along described first motion vector with first block of pixels from described first reference frame to second reference frame;

C1) calculate first coefficient sets that is used to allow first pixel is transformed to second pixel;

D1) use first coefficient sets and come in comfortable first reference frame to calculate the 3rd pixel with juxtaposed the 4th pixel of second block of pixels.

8. according to the system of claim 7, wherein this interpolation device further is arranged to:

C2) calculate second coefficient sets that is used to allow second pixel is transformed to first pixel;

D2) also use the pixel of the predicted frame of the incompatible calculating of second coefficient set.

9. according to the system of one of aforementioned claim 7 and 8, wherein said block of pixels is the square block of n * n pixel, wherein n is the integer greater than 1, first and second coefficient sets further are arranged to this device corresponding to first and second n * n matrix: consider that described first matrix and described second transpose of a matrix calculate the pixel of predicted frame.

10. computer program that is used to be provided at the computer executable instructions of storing on the computer-readable medium, in the time of on being loaded into data processor, this computer executable instructions makes this data processor carry out the method that is used for calculating according to first and second reference frames predicted frame according to claim 1 to 3.

Images