CN100527843C - Method for obtaining image by decode - Google Patents

Abstract

The method comprises: getting the motion vector of relevant block in backward reference frame from bit-stream, which is used for derivation; deciding if the motion vector is over the forward reference frame which is what the B frame maximally appoints to; if not, the calculating the forward and backward motion factor with normal way; if yes, using the motion vector of forward reference frame capable of being obtained by B frame in same direction to replace the motion vector of relevant block of backward reference frame; calculating the forward and backward motion vector of B frame; averaging the pixels corresponding to the image reference block appointed by the forward and backward motion vector to get two-way forecast reference block; adding the two-way forecast reference block to the decoded block residues to get the video image of macro block corresponding to the B frame.

Description

The method of obtaining image by decode

Technical field

The present invention relates to the method for a kind of compression of digital video field obtaining image by decode, especially obtain the method for image reference piece under a kind of coded system of fixed reference frame number, and by asking for the image reference piece, the method for this image is obtained in decoding, belongs to the Image Compression field.

Background technology

Video coding and decoding technology is to realize high-quality, storage of low-cost multi-medium data and the key of transmitting efficiently.Now popular image encoding international standard all is the coding method of adopting motion compensation, discrete cosine transform and quantification based on the piece coupling to combine.The MPEG-1 that typically has International Standards Organization/International Electrotechnical Commissio first United Technologies group (ISO/IEC JTC1) to release, international standards such as MPEG-2 and MPEG-4, and the H.26x series that International Telecommunications Union (ITU-T) proposes is recommended.These video encoding standards have obtained extensive use in industrial quarters.

These video encoding standards have all adopted hybrid video coding (Hybrid Video Coding) strategy, generally include: four main modular such as prediction, conversion, quantification and comentropy coding.The major function of prediction module is to utilize the encoded image of building of laying equal stress on that the current image encoded of wanting is predicted (inter prediction), perhaps utilizes in the image the encoded image block of building (or macro block) of laying equal stress on that the current image encoded piece (or macro block) of wanting is predicted (infra-frame prediction); The major function of conversion module is that the image block that will import transforms to the another one space, the energy of input signal is concentrated on the low frequency transform coefficient as much as possible, and then reduce the correlation between the image block interior element, helps compression; The major function of quantization modules is that the coefficient with conversion is mapped on the finite element collection that helps encoding; The major function of comentropy coding module is according to statistical law, and the conversion coefficient after quantizing is represented with variable length code.

Major function based on the prediction of motion compensation is to eliminate video sequence redundancy in time.Most of code efficiency of video coding comes from prediction module.Video coding process is exactly the process that each two field picture of video sequence is encoded.

Video decoding system comprises similar module, mainly be will the input code stream by processes such as entropy decoding, inverse quantization, inverse transformation, each two field picture of decoding and rebuilding.

Because video coding system commonly used is that elementary cell is carried out with the macro block to the coding of each two field picture.When each two field picture of coding, can be divided into situations such as (I frame) coding in the frame, prediction (P frame) coding and bi-directional predicted (B frame) coding again.In general, during coding, I frame, P frame and B frame are encoded to intert and are carried out, such as the order according to IBBPBBP.

The coding of B frame can be so that encoding compression efficient reaches 200: 1 above code checks.Macro block in the B frame is encoded and need be comprised: direct (Direct), forward prediction (Forward Prediction), back forecast (Backward Prediction) and bi-directional predicted (Bi-directional Prediction) four kinds of patterns.Since the B frame technique need carry out simultaneously forward direction and back to estimation, therefore need higher computational complexity, to introduce extra identification information for before and after distinguishing to motion vector simultaneously.

In traditional video encoding standard (as MPEG-x series and series H.26x), the reference frame of B frame all has only behind a forward reference frame and one to reference frame, and the P frame then has only a forward reference frame.In order to utilize the relativity of time domain between the image more fully, allowed P frame and B frame that a plurality of forward reference frame are arranged.But this also will increase spatiotemporal expense simultaneously greatly, so a kind of compromise mode is to adopt fixing reference frame number to limit bigger time and space expense, in fact since the time domain epigraph between correlation along with the distance far away more, correlation is just weak more, so this qualification is rational.

The Direct coding mode is a kind of to forward direction and back to the coding mode of all predicting, derived to the motion vector of reference picture by the back to motion vector and obtain in the front and back of B frame, can motion vector information not encoded, therefore can reduce the shared bit number of encoding motion vector information, improve code efficiency effectively, therefore use very extensive.

For P, the B frame, when adopting the fixed reference frame number to encode, the forward direction of considering the P frame with reference to will more than the forward reference frame number of B frame (the B frame must comprise one after to reference frame, and the P frame does not have), therefore can cause the unmatched problem of motion vector, that is: for the B frame, when carrying out the motion vector derivation, need use the motion vector of back each piece in reference frame, because the back is the P frame to reference frame, may cause the reference frame of motion vector points to exceed the maximum forward reference frame that can point to of B frame like this; Such as, as shown in Figure 1, when the fixed reference frame number is 2 frames, the reference frame that the B frame can point to is its 2 adjacent P frames, and as 2 P frames before the reference frame of the P of reference frame frame is this P frame behind the B frame, be P_REF_1 and P_REF_0, when the P frame P_REF_1 of the motion vector points forefront of this P frame, this P frame has exceeded the maximum forward reference frame that can point to of B frame, calculate the motion vector points B frame P_REF_1 that is beyond one's reach, therefore, the real reference block that the B frame can't obtain to encode, can make coding deviation occur, cause the serious distortion of image.

Summary of the invention

When the purpose of this invention is to provide a kind of fixing reference frame number, under the Direct coding mode, obtain the method for real image reference piece, and accordingly by asking for the image reference piece, the method for this image is obtained in decoding.

For achieving the above object, the invention provides a kind of method of obtaining image by decode, may further comprise the steps:

Step 1, from code stream, obtain the back and in reference frame, be used to motion vector MV that derive, relevant block;

Step 2, judge whether this motion vector has exceeded the maximum forward reference frame that can point to of B frame, does not exceed then execution in step 3, otherwise execution in step 4;

Step 3, calculate the forward motion vector MV of current B frame by following formula _FWith reverse vector M V _B:

${\mathrm{MV}}_F=\frac{\mathrm{tb}}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

Tb is current B frame and the distance of forward reference frame on time domain, and td is a forward reference frame and back to the distance of reference frame on time domain;

Step 4, calculate the forward motion vector MV of current B frame by following formula _FWith reverse vector M V _B:

${\mathrm{MV}}_F=\frac{{\mathrm{tb}}^l}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

Tb ' is the time domain distance between current B frame and its forward reference frame that can point to;

Step 5, to MV _FAnd MV _BThe pixel of image reference piece correspondence pointed is averaged, and tries to achieve bi-directional predicted reference block;

Step 6, with described bi-directional predicted reference block and decoded residual error addition, obtain current B frame the video image of corresponding macro block.

As shown from the above technical solution, the present invention adopts the method for asking for the decoding of image reference piece by coupling fixed reference frame number coded system, has the fixed reference of cooperation frame coding mode, overcome motion vector and do not match, guarantee the beneficial effect of the efficient of coding to greatest extent.

Below by drawings and Examples, technical scheme of the present invention is described in further detail.

Description of drawings

Fig. 1 is the schematic diagram when the P frame points to reference frame P_REF_1 in the prior art;

Schematic diagram when Fig. 2 is embodiment of the invention P frame sensing reference frame P_REF_0;

Schematic diagram when Fig. 3 is embodiment of the invention P frame sensing reference frame P_REF_1.

Embodiment

Design philosophy of the present invention is at first will judge according to the motion vector to reference frame behind the B frame, whether the forward reference frame of judging this motion vector points has exceeded the maximum forward reference frame that can point to of B frame, if do not exceed, then adopt common mode to calculate the forward motion vector of B frame, obtain forward direction image reference piece; If exceeded scope, then adopt by dwindling the method for motion vector, making to calculate forward motion vector and do not exceed the maximum forward reference frame scope that can point to of B frame, can obtain real forward direction image reference piece.

Method of deciding to obtain under the reference frame number direct coding pattern image reference piece of the present invention is carried out image encoding at coding side, comprises the steps:

Step 1, carry out estimation, obtain the motion vector MV of back to the reference frame relevant block at each piece of current B frame;

Step 2, judge whether this motion vector has exceeded the maximum forward reference frame that can point to of B frame, if do not exceed, then execution in step three; Otherwise, execution in step four;

Step 3, can calculate current block by following formula front and back to motion vector:

If MV _FWith MV _BBe the forward and backward motion vector of current block,

${\mathrm{MV}}_F=\frac{\mathrm{tb}}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

Tb is current B frame and the distance of forward reference frame on time domain, and td is a forward reference frame and back to the distance of reference frame on time domain;

Step 4, can calculate current block by following formula front and back to motion vector:

If MV _FWith MV _BBe the forward and backward motion vector of current block,

${\mathrm{MV}}_F=\frac{{\mathrm{tb}}^l}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

Tb is current B frame and the distance of forward reference frame on time domain, and td is a forward reference frame and back to the distance of reference frame on time domain; Tb ' is the time domain distance between current B frame and its forward reference frame that can point to;

Step 5, MV _BWith MV _FTwo image blocks pointed are the pairing image reference piece of current block just.

After obtaining the image reference piece, the pixel of two reference blocks is averaged, tries to achieve final bi-directional predicted reference block, again that B frame macro block to be encoded and bi-directional predicted reference block is poor, acquisition piece residual error; The piece residual error is encoded, just finished coding this B frame.

The method of corresponding obtaining image by decode provided by the present invention may further comprise the steps:

Step 1, from code stream, obtain the back and in reference frame, be used to motion vector MV that derive, relevant block;

Step 2, judge whether this motion vector has exceeded the maximum forward reference frame that can point to of B frame, does not exceed then execution in step 3, otherwise execution in step 4; Its determination methods can be: whether the time domain that relatively obtains the maximum forward reference frame that can point to of B frame is more than or equal to the time domain of back to the forward reference frame of the motion vector points of reference frame relevant block, if then do not exceed; Otherwise, exceed;

Step 3, calculate the forward motion vector MV of current B frame by following formula _FWith reverse vector M V _B:

${\mathrm{MV}}_F=\frac{\mathrm{tb}}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

Tb is current B frame and the distance of forward reference frame on time domain, and td is a forward reference frame and back to the distance of reference frame on time domain;

Step 4, calculate the forward motion vector MV of current B frame by following formula _FWith reverse vector M V _B:

${\mathrm{MV}}_F=\frac{{\mathrm{tb}}^l}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

Tb is current B frame and the distance of forward reference frame on time domain, and td is a forward reference frame and back to the distance of reference frame on time domain; Tb ' is the time domain distance between current B frame and its forward reference frame that can point to;

Step 5, to MV _FAnd MV _BThe pixel of image reference piece correspondence pointed is averaged, and tries to achieve bi-directional predicted reference block;

Step 6, with described bi-directional predicted reference block and decoded residual error addition, obtain current B frame the video image of corresponding macro block.

The forward reference frame that the present invention adopts the B frame to point to replaces exceeding the forward reference frame that the B frame points to scope, goes beyond the scope thereby solved, and the B frame can't obtain the problem of image reference piece.Though the image reference piece that obtains here be not the back to the real image reference piece that points to of P frame, but because real image reference piece can't obtain, and the image reference piece that the inventive method obtains is the reference block with the real image reference piece equidirectional that points to of P frame, the two is because relativity of time domain and very approaching, so adopt this method can guarantee that code efficiency can not degenerate a lot.If adopt fixing image block (such as adopting monochromatic image block), then can't guarantee the effect of coding/decoding; If adopt other method to obtain image block, can't utilize the relativity of time domain of image, the effect of coding/decoding is unpredictable too.

In order to have guaranteed to greatest extent to utilize the relativity of time domain between the image, for tb ' in step 4 and the step 4 be the B frame can point to near the forward reference frame of P frame reference frame pointed, promptly when the B frame can not reach P frame reference frame pointed, the immediate with it forward reference frame that can point to the B frame replaced.

According to formula as can be seen, the present invention is in encoding-decoding process, be to select equidirectional motion vector, just length narrows down to the scope that the B frame can point to, the utilization that this way is best relativity of time domain, when the immediate with it forward reference frame that can point to the B frame replaced P frame forward reference frame pointed, relativity of time domain had obtained maximum utilization.

In the step 1 of the present invention, obtaining the back has much to the method for the motion vector MV of reference frame relevant block, because the back is the P frame to reference frame, so each macro block of P frame all has a motion vector MV, the key of step 1 is the correspondence between the macro block, it is corresponding with which macro block in the P frame to be macro block of B frame, has found corresponding macro block, and the motion vector MV of this macro block of P frame is the benchmark that the B frame calculates forward direction and backward motion vector.

The general identical corresponding method in position that adopts determines that the B frame is the corresponding macro block of P frame with the back to reference frame, promptly back macro block identical with the B frame position in reference P frame is as corresponding macro block, different algorithms may be different on corresponded manner, exceeds the scope that the B frame can point to but motion vector all may appear in any corresponded manner.

Coding/decoding method of the present invention goes for the coding/decoding of any predictive mode of B frame.Each macro block of current B frame have forward direction, back to etc. multiple predictive mode, below Direct coding mode just commonly used be example, be 2 with the fixed reference frame number, describe technical scheme of the present invention in detail.

At first, obtain the back to the motion vector of each piece of reference frame with respect to forward reference frame for B frame to be encoded, therefore this motion vector, may point to reference frame P_REF_0 (see figure 2) or P_REF_1 (see figure 3) owing to come from the P frame.

By the back motion vector that has obtained to each piece of reference frame, draw and be used for the vector M V (x that B frame current block direct mode motion vector is derived, y), this motion vector can be to come from the back to the reference picture piece consistent with the current block locus, the directmode partition of current block and co-located partition as shown in Figure 1, can certainly derive out with additive method, 1 expression present encoding piece among the figure, the identical corresponding blocks of 2 expression P frame position, the implication of the mark among other figure is identical.

When the reference frame that points to was P_REF_0, as shown in Figure 2, this reference frame did not exceed the scope of the forward reference frame that the B frame can point to, and the front and back of current block obtain according to the motion vector derivation to relevant block afterwards to motion vector:

${\mathrm{MV}}_F=\frac{\mathrm{tb}}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

MV _FWith MV _BThen be corresponding forward motion vector of current block and backward motion vector.Here tb is current B frame and the distance of forward reference frame on time domain, and td is a forward reference frame and back to the distance of reference frame on time domain, and MV represents that the back is to the motion vector of the corresponding macro block of reference frame with respect to forward reference frame.

When the reference frame that points to is P_REF_1, as shown in Figure 3, this reference frame has exceeded the scope of the forward reference frame that the B frame can point to, the image macro of this reference frame, the B frame can't obtain, so need replace the image block among the P_REF_1 with the image block among the P_REF_0, therefore, derive to the motion vector of relevant block for the back to motion vector and obtain as follows in the front and back of current block:

${\mathrm{MV}}_F=\frac{{\mathrm{tb}}^l}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

Tb ' is the time domain distance between current B frame and the forward reference frame P_REF_0, MV _FWith MV _BThen be corresponding forward motion vector of current block and backward motion vector.Here tb is current B frame and the distance of forward reference frame on time domain, and td is a forward reference frame and back to the distance of reference frame on time domain, and MV represents that the back is to the motion vector of reference frame respective macroblock with respect to forward reference frame.

Because MV _BWith MV _FTwo image blocks pointed are the pairing image reference piece of this macro block just, after obtaining the image reference piece, the pixel of two reference blocks is averaged, just can be in the hope of final bi-directional predicted reference block, again that B frame macro block to be encoded and bi-directional predicted reference block is poor, obtain the piece residual error; The piece residual error is encoded, just finished coding this B frame.

Accordingly, the process of obtaining image by decode is the inverse process of coding, and step is as follows:

At first to judge the predictive mode of current block,, carry out following step if when being direct mode; If be other coding mode, undertaken by corresponding decoding step;

For B frame to be decoded, from code stream, obtain the back and in reference frame, be used to motion vector MV that derive, relevant block, this motion vector may point to reference frame P_REF_0 (see figure 2) or P_REF_1 (see figure 3) corresponding to the different situations of coding.

When if the corresponding reference frame of motion vector MV does not exceed the maximum forward reference frame that can point to of B frame,, can pass through following formula acquisition front and back to motion vector referring to Fig. 2:

${\mathrm{MV}}_F=\frac{\mathrm{tb}}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

MV _FWith MV _BThen be corresponding forward motion vector of current block and backward motion vector.Here tb is current B frame and the distance of forward reference frame on time domain, and td is a forward reference frame and back to the distance of reference frame on time domain, and MV represents that the back is to the motion vector of reference frame respective macroblock with respect to forward reference frame.

When if the corresponding reference frame of motion vector MV exceeds the maximum forward reference frame that can point to of B frame,, then pass through following formula front and back to motion vector referring to Fig. 3:

${\mathrm{MV}}_F=\frac{{\mathrm{tb}}^l}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

Tb ' is the time domain distance between current B frame and the forward reference frame P_REF_0.

By to MV _BWith MV _FTwo corresponding pixels of prediction of indication are averaged, just can be in the hope of final bi-directional predicted reference block.

Bi-directional predicted reference block and corresponding piece residual error addition are formed the current block image block, just finish whole decode procedure, obtained the decoding video images of current block.

It should be noted that at last: above embodiment is only in order to technical scheme of the present invention to be described but not limit it, although the present invention is had been described in detail with reference to preferred embodiment, those of ordinary skill in the art is to be understood that: it still can make amendment or be equal to replacement technical scheme of the present invention, and these modifications or be equal to replacement and also can not make amended technical scheme break away from the spirit and scope of technical solution of the present invention.

Claims (3)

1, a kind of method of obtaining image by decode is characterized in that may further comprise the steps:

Step 1, from code stream, obtain the back and in reference frame, be used to motion vector that derive, relevant block;

Step 2, judge whether this motion vector has exceeded the maximum forward reference frame that can point to of B frame, does not exceed then execution in step 3, otherwise execution in step 4;

Step 3, calculate the forward motion vector and the reverse vector of current B frame by following formula:

${\mathrm{MV}}_F=\frac{\mathrm{tb}}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

MV is that described back is used to derive in reference frame, the motion vector of relevant block, MV _FBe forward motion vector, MV _BIt is the reverse vector; Tb is current B frame and the distance of forward reference frame on time domain, and td is a forward reference frame and back to the distance of reference frame on time domain;

Step 4, calculate the forward motion vector and the reverse vector of current B frame by following formula:

${\mathrm{MV}}_F=\frac{{\mathrm{tb}}^{/}}{\mathrm{td}}\×\mathrm{MV}$

${\mathrm{MV}}_B=\frac{\mathrm{tb}-\mathrm{td}}{\mathrm{td}}\×\mathrm{MV}$

Tb/ is the time domain distance between current B frame and its forward reference frame that can point to;

Step 5, the pixel of forward motion vector and reverse vector image reference piece correspondence pointed is averaged, tries to achieve bi-directional predicted reference block;

Step 6, with described bi-directional predicted reference block and decoded residual error addition, obtain current B frame the video image of corresponding macro block.

2, method according to claim 1 is characterized in that in the described step 1, and the motion vector that is used to derive in reference frame after obtaining from code stream is specially:

Choose back macro block identical with macro block position to be decoded in the current B frame in reference P frame as corresponding macro block, and obtain the motion vector of this macro block of P frame.

3, method according to claim 1 is characterized in that in the described step 2, judges whether back motion vector that be used to derive, relevant block to reference frame has exceeded the maximum forward reference frame that can point to of B frame and be specially:

Whether the time domain that relatively obtains the maximum forward reference frame that can point to of B frame is more than or equal to the time domain of back to the forward reference frame of the motion vector points of reference frame relevant block, if then do not exceed; Otherwise, exceed.

Images