CN101518089B

CN101518089B - Coding/decoding methods, coders/decoders, and method and device for finding optimally matched modules

Info

Publication number: CN101518089B
Application number: CN2007800003205A
Authority: CN
Inventors: 何芸; 许晓中; 郑建铧; 孟新建; 史舒娟; 郑萧桢
Original assignee: Tsinghua University; Huawei Technologies Co Ltd
Current assignee: Tsinghua University; Huawei Technologies Co Ltd
Priority date: 2006-09-01
Filing date: 2007-08-31
Publication date: 2010-12-01
Anticipated expiration: 2027-08-31
Also published as: CN101137065A; CN101518089A; WO2008025300A1

Abstract

The invention discloses two coding methods. One coding method comprises the steps of: performing a first prediction of a current image to be coded to obtain a first residual image and a first motion vector; and performing a second prediction to the first residual image to obtain a second residual image and a second motion vector and writing the second residual image, the second motion vector and the first motion vector into a compression code flow or into compression code flow after entropy coding. The invention also discloses two decoding methods, two coders, two decoders, a coding method, a coder-decoder, a method and a device for finding optimally matched modules from a plurality of multilook images. The technical proposal of the invention can eliminate the relevance of residual images.

Description

The method and the device of coding/decoding method, coder/decoder and searching best matching blocks

Technical field

The present invention relates to the image coding technique in the multimedia field, relate in particular to method for encoding images, coding/decoding method, encoder, decoder, decoding method, codec, and method and the device of between many views picture, seeking best matching blocks.

Background of invention

Present video encoding standard such as the (ITU of International Telecommunications Union, International TelecommunicationUnion) formulate H.261, H.263, H.263+, standard H.264, and (the MPEG of Motion Picture Experts Group, Moving Picture Experts Group) MPEG-1, MPEG-2, MPEG-3, the MPEG-4 etc. of Zhi Dinging are based upon on hybrid coding (Hybrid Coding) framework.So-called hybrid encoding frame is the video encoding method in a kind of incorporation time space, during coding, carries out in the frame earlier, the prediction of interframe, obtains predicted value, to eliminate the correlation of time-domain; According to the difference of predicted value and original image actual value, obtain residual image then, adopt discrete cosine transform method or other converter technique to carry out two-dimensional transform, to eliminate the correlation of spatial domain residual image; At last the data after the conversion are carried out entropy coding, to eliminate statistical redundancy, some side informations that comprise motion vector that the data behind the entropy coding are required with decoding are formed a compressed code flow together, for subsequent transmission and storage usefulness, reach the purpose of compressed video image.Correspondingly, when decoding, reconstruct image according to a series of decode procedures such as entropy decoding, inverse transformation and predictive compensations.

Two kinds of different coding methods of intraframe coding and interframe encode have been comprised in the hybrid encoding frame.When carrying out intraframe coding, original image is carried out two-dimensional transform, in transform domain, conversion coefficient is quantized then, carry out entropy coding at last; Perhaps in given frame, make full use of the spatial coherence of adjacent block, to a given block encoding time, at first can predict this given, then the difference of predicted value and actual value is carried out two-dimensional transform, quantification and entropy coding according to piece on every side.When carrying out interframe encode, adopt estimation to obtain motion vector, adopt inter prediction then based on motion compensation, obtain the difference of predicted value and original image actual value, it is residual image, residual image is carried out two-dimensional transform, in transform domain, conversion coefficient is quantized then, carry out entropy coding at last.

As seen, in above-mentioned intraframe coding and the interframe encode, all adopted Predicting Technique.Predicting Technique has made full use of in the frame and space, the temporal correlation of interframe, reduces code check on the basis of eliminating correlation, and improves the data volume compression ratio of compressed bit stream and original image.Especially in hybrid encoding frame, inter prediction is the effective ways that reduce the image temporal correlation, particularly (H.264/AVC at nearest video standard advanced video coding H.264, Advanced Video Coding) in, inter prediction has adopted wider piece size to divide, comprise from 16 * 16 to 4 * 4, and technology such as many reference image frames, therefore be widely used.In the video sequence that photographs at single camera, exist very big correlation between the adjacent image, i.e. redundancy.By utilizing the image encoded and the correlation between the current encoded image, image encoded reverts to reconstructed image, as the reference image current encoded image is predicted, and according to the difference between predicted value and the current encoded image actual value, obtain residual image, and residual image encoded, thereby can eliminate correlation between the adjacent image, reduce the coded data amount.

In the multi-video field, when same object being taken, also exist bigger correlation between the image of a plurality of video sequences that photographed equally as a plurality of video cameras, especially when position between the video camera near the time, correlation is stronger.Therefore when a plurality of video sequences are encoded simultaneously, can utilize in the current video sequence or in other video sequence image encoded as the reference image current image to be encoded is predicted, obtain residual image, and residual image encoded, thereby eliminate the correlation between the image of a plurality of video sequences, effectively reduce the redundancy between the image.

In the above-mentioned coding method, current image to be encoded has only been carried out once prediction, just the residual image that produces has been encoded then.And in fact, in the multi-video field, current image to be encoded is through once still having stronger correlation between back residual image that produces of prediction and the pairing residual image of reference picture, when the image of a plurality of video sequences being encoded according to existing coding method, relevance of residual images can't be eliminated, therefore also just code efficiency can't be better improved.

Summary of the invention

In view of this, provide a kind of method for encoding images, coding/decoding method, encoder, decoder, decoding method and codec in the embodiment of the invention,, improve code efficiency so that eliminate relevance of residual images.

In addition, also provide two kinds of method for encoding images, a kind of coding/decoding method, two kinds of encoders and a kind of decoder in the embodiment of the invention,, and further improve code efficiency so that eliminate relevance of residual images.

At last, provide two kinds in the embodiment of the invention again,, improve the search efficiency of best matching blocks so that utilize relevance of residual images at the method for seeking best matching blocks between many views picture and four kinds of devices of between many views picture, seeking best matching blocks.

First kind of method for encoding images that the embodiment of the invention provided comprises:

A, current image to be encoded is once predicted, obtained a residual image and a motion vector;

B, a residual image is carried out re prediction, obtain quadratic residue image and secondary motion vector, and quadratic residue image, secondary motion vector and a motion vector are write in the compressed bit stream.

First kind of picture decoding method that the embodiment of the invention provided comprises:

A, decoding obtains motion vector, secondary motion vector and quadratic residue image one time from compressed bit stream;

B, according to the secondary motion vector, the quadratic residue image is carried out motion compensation, obtain residual image one time, and, a residual image carried out motion compensation according to a motion vector, obtain reconstructed image.

First kind of encoder that the embodiment of the invention provided comprises: prediction module and coding module, and this encoder also comprises: the re prediction module, wherein,

A prediction module is used for current image to be encoded is once predicted, obtains a residual image and a motion vector;

The re prediction module is used to receive a residual image from a described prediction module, and a residual image that is received is carried out re prediction, obtains quadratic residue image and secondary motion vector;

Coding module, be used to receive from motion vector of a described prediction module with from the secondary motion vector and the quadratic residue image of described re prediction module, and a motion vector, secondary motion vector and the quadratic residue image that is received write in the compressed bit stream.

First kind of decoder that the embodiment of the invention provided comprises: decoder module, a rebuilding module, and this decoder also comprises: the secondary rebuilding module, wherein,

Decoder module is used for obtaining motion vector, secondary motion vector and quadratic residue image one time from the compressed bit stream decoding;

The secondary rebuilding module is used to receive quadratic residue image and secondary motion vector from described decoder module, according to the secondary motion vector that is received, the quadratic residue image is carried out motion compensation, obtains residual image one time;

A rebuilding module is used to receive from a residual image of described secondary rebuilding module and from a motion vector of decoder module, according to a motion vector that is received, a residual image is carried out motion compensation, obtains reconstructed image.

The decoding method that the embodiment of the invention provided comprises:

A, current image to be encoded is once predicted, obtain a residual image and a motion vector, and a residual image carried out re prediction, and obtain quadratic residue image and secondary motion vector, quadratic residue image, secondary motion vector and a motion vector are write compressed bit stream;

B, receive described compressed bit stream, and decoding obtains motion vector, secondary motion vector and quadratic residue image one time from compressed bit stream; According to the secondary motion vector, the quadratic residue image is carried out motion compensation, obtain residual image one time, and, a residual image is carried out motion compensation according to a motion vector, obtain reconstructed image.

The codec that the embodiment of the invention provided comprises: coding unit and decoding unit, wherein,

Coding unit, be used for current image to be encoded is once predicted, obtain a residual image and a motion vector, and a residual image carried out re prediction, obtain quadratic residue image and secondary motion vector, quadratic residue image, secondary motion vector and a motion vector are write compressed bit stream;

Decoding unit is used to receive described compressed bit stream, and decoding obtains motion vector, secondary motion vector and quadratic residue image one time from compressed bit stream; According to the secondary motion vector, the quadratic residue image is carried out motion compensation, obtain residual image one time, and, a residual image is carried out motion compensation according to a motion vector, obtain reconstructed image.

The second kind of method for encoding images that is provided in the embodiment of the invention comprises:

A, the image current to be encoded after first frame image to be encoded encoded are once predicted, obtain a residual image and a motion vector;

B, be unit with macro block or piece, at each macro block or the piece in the residual image, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, with this corresponding blocks as best matching blocks, and with a described motion vector as the secondary motion vector in the secondary reference picture, with this best matching blocks current macro or piece are predicted, obtain the quadratic residue piece, motion vector and quadratic residue piece and/or secondary motion vector are write in the compressed bit stream.

The third method for encoding images that is provided in the embodiment of the invention comprises:

B, be unit with macro block or piece, at each macro block or the piece in the residual image, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, with this piece as the search center starting point, seek best matching blocks, with this best matching blocks current macro or piece are predicted, and obtain quadratic residue piece and corresponding secondary motion vector, motion vector, quadratic residue piece and a secondary motion vector are write in the compressed bit stream.

The second kind of picture decoding method that provides in the embodiment of the invention comprises:

B, be unit with macro block or piece, at each macro block or the piece in the quadratic residue image, secondary motion vector according to this macro block or piece correspondence, or according to the motion vector as the secondary motion vector of this macro block or piece correspondence, this quadratic residue piece is carried out motion compensation, obtain residual block one time, and according to a motion vector of this residual block correspondence, this residual block is carried out motion compensation, obtain the reconstructed image piece, with the synthetic reconstructed image of all reconstructed image pieces of quadratic residue image.

The second kind of encoder that is provided in the embodiment of the invention comprises: prediction module and coding module is characterized in that this encoder also comprises: the re prediction module, wherein,

A prediction module is used for the image current to be encoded after first frame is once predicted, obtains a residual image and a motion vector;

The re prediction module, be used to receive a residual image from a prediction module, is unit to a residual image being received with macro block or piece, at each macro block or the piece in the residual image, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, with this corresponding blocks as best matching blocks, and with a described motion vector as the secondary motion vector in the secondary reference picture, with this best matching blocks current macro or piece are predicted, obtained the quadratic residue image;

Coding module, be used to receive from motion vector of a described prediction module with from the quadratic residue image and/or the secondary motion vector of described re prediction module, and motion vector being received and quadratic residue image and/or secondary motion vector are write in the compressed bit stream.

The third encoder that is provided in the embodiment of the invention comprises: prediction module and coding module is characterized in that this encoder also comprises: the re prediction module, wherein,

The re prediction module, be used to receive a residual image from a prediction module, is unit to a residual image being received with macro block or piece, at each macro block or the piece in the residual image, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, with this piece as the search center starting point, seek best matching blocks, with this best matching blocks current macro or piece are predicted, obtained quadratic residue image and corresponding secondary motion vector;

Coding module, be used to receive from motion vector of a described prediction module with from the quadratic residue image and the secondary motion vector of described re prediction module, and a motion vector, quadratic residue image and the secondary motion vector that is received write in the compressed bit stream.

The second kind of decoder that is provided in the embodiment of the invention comprises: decoder module, a rebuilding module is characterized in that this decoder also comprises: the secondary rebuilding module, wherein,

The secondary rebuilding module, be used to receive from the quadratic residue image of described decoder module and secondary motion vector or as a motion vector of secondary motion vector, according to the secondary motion vector that is received or as a motion vector of secondary motion vector, the quadratic residue image is carried out motion compensation, obtain residual image one time;

A rebuilding module is used to receive from a residual image of described secondary rebuilding module and from a motion vector of described decoder module, according to a motion vector that is received, a residual image is carried out motion compensation, obtains reconstructed image.

The third that provides in the embodiment of the invention sought the device of best matching blocks between many views picture, comprising:

The re prediction module, a residual image that is used for being received is a unit with macro block or piece, at each macro block or the piece in the residual image, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, with this corresponding blocks as best matching blocks.

The 4th kind of device of seeking best matching blocks between many views picture that provides in the embodiment of the invention comprises:

The re prediction module, a residual image that is used for being received is a unit with macro block or piece, at each macro block or the piece in the residual image, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, as the search center starting point, seek best matching blocks with this piece.

From such scheme as can be seen, the present invention by with the pairing residual image of some image of having encoded in each video sequence as the reference image, current image to be encoded is predicted through the residual image that produces after the predictive coding, to predict that again the quadratic residue image that the back obtains carries out the next code operation, thereby elimination relevance of residual images, further reduce the redundancy between the image, reduce the data volume of code stream, improve code efficiency.

Brief Description Of Drawings

Fig. 1 is the sequential schematic of the video sequence of a plurality of video cameras shootings.

Fig. 2 is the schematic diagram that concerns between current image to be encoded and the reference picture.

Fig. 3 is the flow chart of method for encoding images in the embodiment of the invention one.

Fig. 4 concerns schematic diagram between four images during re prediction in the flow process shown in Figure 3.

Fig. 5 is the flow chart of image reconstruction in the flow process shown in Figure 3.

Fig. 6 is the flow chart of picture decoding method in the embodiment of the invention one.

Fig. 7 be in the embodiment of the invention one encoder and decoder concern schematic diagram.

Fig. 8 is the structural representation of encoder in the embodiment of the invention one.

Fig. 9 is the structural representation of decoder in the embodiment of the invention one.

Figure 10 is the flow chart of method for encoding images in the embodiment of the invention two.

Figure 11 is the flow chart of image reconstruction in the flow process shown in Figure 10.

Figure 12 is the flow chart of picture decoding method in the embodiment of the invention two.

Figure 13 is the structural representation of encoder in the embodiment of the invention two.

Figure 14 is the structural representation of decoder in the embodiment of the invention two.

Figure 15 is the structural representation of encoder in the embodiment of the invention three.

Figure 16 is the structural representation of decoder in the embodiment of the invention three.

The codec of Figure 17 on the basis of encoder shown in Figure 8 and decoder shown in Figure 9, being assembled into.

Implement mode of the present invention

In the embodiment of the invention, utilize the strong correlation between the video sequence that a plurality of shot by camera come out, and the different video sequence still has the characteristics of very big correlation between the residual image that produces after the predictive coding, from the image that each video sequence had been encoded, select and the strong image of current image correlation to be encoded, and with the residual image of selected image correspondence as the reference image, residual image to current image correspondence to be encoded is predicted, the quadratic residue image that again prediction is obtained carries out the next code operation, thereby elimination relevance of residual images, further reduce the redundancy between the image, reduce data volume.

For describe clear for the purpose of, to carry out inter prediction and produce the process of residual image original image in the embodiment of the invention, it is estimation, be called once prediction, the residual image that is produced is called residual image one time, corresponding motion vector is called motion vector one time, corresponding one more residual block, a reference picture, a motion compensation etc.; With among the present invention on base of prediction, is that the inter prediction that reference picture carries out is called re prediction to residual image with a residual image of encoded image correspondence, the residual image that is produced is called the quadratic residue image, corresponding motion vector is called the secondary motion vector, corresponding quadratic residue piece in addition, secondary reference picture, secondary motion compensation etc.

Wherein, when a residual image was carried out re prediction, the embodiment of the invention had adopted two kinds of implementations, that is:

Mode one is directly carried out re prediction to a residual image, and the quadratic residue image that re prediction is produced carries out conversion or do not carry out conversion afterwards, carries out the next code operation with motion vector and secondary motion vector then.

Mode two is at first carried out conversion to a residual image, and to carrying out re prediction through a residual image after the conversion, the quadratic residue image that re prediction is produced afterwards carries out the next code operation together with motion vector and secondary motion vector.

Promptly for re prediction, mode one is to predict conversion more earlier, or only predicts not conversion; Mode two is that first conversion is predicted again.

Following examples are example with the situation that a plurality of video sequences are encoded simultaneously all, and the coding method in the embodiment of the invention, coding/decoding method, encoder, decoder, decoding method and codec are described in detail.

The a plurality of video sequences that photograph for different cameras, can be arranged in a two dimensional image array, as shown in Figure 1, Fig. 1 is the sequential schematic of the video sequence of a plurality of video cameras shootings, and abscissa is represented the video sequence that different cameras is taken among Fig. 1, looks 1 expression video sequence 1, look 2 expression video sequences 2, and the like, listed 4 video sequences among Fig. 1, ordinate represents to change in time each video sequence also to be increased.Wherein, in short at interval a period of time, especially synchronization exists very strong correlation between a plurality of images that different cameras photographs.Under the extreme case, the taken video sequence of different cameras might be identical.Therefore reference picture not only can be chosen in the current video sequence image encoded, and can choose in other video sequence image encoded, and as shown in Figure 2, Fig. 2 is the schematic diagram that concerns between current image to be encoded and the reference picture.The video image A2 that video image A1 that takes constantly at T1 for video camera A and T2 take constantly, and the video image B2 that takes constantly of the video image B1 that takes constantly at T1 of video camera B and T2, suppose A1, A2 and B1 to be encoded, when B2 is encoded, can choose with the B1 in the video sequence as the reference image, also can choose non-with the A1 in the video sequence or A2 as the reference image.In the practical application, reference picture also can have a lot, and can choose with other image in the video sequence or in other video sequence.

Embodiment one:

In the present embodiment, the implementation in the mode one is described in detail.A promptly direct residual image to a prediction generating carries out re prediction, and the quadratic residue image that re prediction produces is carried out next code handle.Fig. 3 is the flow chart of method for encoding images in the embodiment of the invention one.Generally speaking, when the original image to current image to be encoded carries out inter prediction, all be to be that unit predicts, and in last piece, carry the sign that current image to be encoded finishes, and then begin next one image to be encoded is carried out same inter prediction with macro block or piece.Therefore the method for encoding images flow process in the present embodiment is an example with a macroblock encoding process of current image to be encoded, and coding method of the present invention is elaborated.As shown in Figure 3, this flow process comprises the steps:

Step 301 is carried out inter prediction to current block to be encoded, and promptly once prediction obtains a residual block and a motion vector.

In this step, when carrying out inter prediction, can adopt the method that is provided in the standard H.264/AVC, also can adopt the method in other standard.

Generally speaking, at first given reference picture set, as the former frames in the video sequence, or in the adjacent video sequence with constantly several frames etc., as the case may be, can choose reference picture according to different modes, reference picture herein, promptly a reference picture is the reconstructed image of encoded image, and concrete image reconstruction process is referring to back flow process shown in Figure 4.Traversal is sought the best matching blocks of current block to be encoded in reference picture set afterwards, with this best matching blocks current piece to be encoded is predicted, obtains residual block and motion vector.Wherein, if find the reconstructed image of the reference picture of best matching blocks for synchronization encoded image in other video sequence, the vector that then obtains claims difference vector, the prediction piece that motion vector and difference vector all are meant current block in reference picture with respect to the displacement of current block.Succinct in order to describe, generally, motion vector and difference vector are referred to as motion vector.

Be that example is described still now with four images shown in Figure 2, suppose that current image to be encoded is B2, then when a macro block N to B2 encodes, can be with A1, the reconstructed image of A2 and B1 is as the reference image, in each reference picture, carry out estimation, i.e. once prediction, and in B1, found best matching blocks K at last, the residual block of once being predicted an i.e. residual block is N '=N-K, and corresponding motion vector i.e. motion vector is that (Kx Ky), and stores this motion vector, use during for next code and reconstructed image, simultaneously execution in step 302.A Forecasting Methodology for other macro block is identical therewith.

Step 302 is carried out inter prediction to a resulting residual block, and promptly re prediction obtains quadratic residue piece and secondary motion vector.

In this step, in order to eliminate relevance of residual images, to residual image with the residual image of encoded image as the reference image, promptly the secondary reference picture carries out re prediction.Forecasting Methodology to once predict similar.Reference picture herein is the reconstructed image of the corresponding residual image of encoded image, and concrete image reconstruction process is referring to back flow process shown in Figure 4.

Be example still with four images shown in Figure 2, suppose A2 with the reconstructed image of A1 as the reference image, obtain residual image A2 ' one time, B2 as the reference image, obtains residual image B2 ' with the reconstructed image of B1 one time.Concern schematic diagram when Fig. 4 is re prediction between four images.As shown in Figure 4, hypothesis is predicted a residual block N ' among the B2 ' as the reference image with A2 ' in this step, and in A2 ', find best matching blocks M, obtaining the quadratic residue piece is N "=N '-M; corresponding motion vector is the secondary motion vector for (Mx My), and stores this secondary motion vector; use while execution in step 303 during for next code and reconstructed image.Wherein, A2 ' is the reconstructed image of the corresponding residual image of encoded image.Re prediction method for other macro block is identical therewith.

Step 303 is carried out conversion to resulting quadratic residue piece, and with the conversion coefficient battle array after the conversion and once prediction obtain the secondary motion vector that motion vector and re prediction obtain and carry out entropy coding, and write in the compressed bit stream together; Utilize the factor arrays after the conversion to carry out image reconstruction simultaneously, as the reference picture of other image to be encoded.

In this step, when the quadratic residue piece is carried out conversion, comprise conversion and quantification, when carrying out entropy coding, also can comprise have a variety ofly in the specific implementation prior art, introduce no longer in detail herein operations such as the factor arrays after the conversion reorder.Wherein, entropy coding just adopts more a kind of method, in the practical application, other coding method can also be arranged.

So far, this macroblock encoding process finishes.In the practical application, after this macroblock encoding finished, current image to be encoded can judge in self image whether have the macro block that does not carry out above-mentioned coding, if exist, then continues to carry out the process of above-mentioned steps 301 to step 303; Otherwise, this two field picture end-of-encode.The subsequent frame image continues to carry out said process as current image to be encoded.Wherein, utilize the conversion coefficient battle array after the conversion to carry out the process of image reconstruction with reference to following flow process shown in Figure 5.

Fig. 5 is the image reconstruction schematic flow sheet in the method for encoding images shown in Figure 3.Process of reconstruction with a macro block of current image to be decoded is an example, and this flow process comprises the steps:

Step 501 is carried out inverse transformation to the conversion coefficient battle array after the conversion, obtains the quadratic residue piece.

In this step, when the conversion coefficient battle array after the conversion is carried out inverse transformation, comprise inverse quantization and inverse transformation.Inverse transformation herein is corresponding to the conversion in when coding in the step 303.

Step 502 is according to the secondary motion vector that obtains in the step 302, to resulting quadratic residue piece, when coding,, promptly in the secondary reference picture, carry out motion compensation as the residual image of reference image, obtain residual block one time, and this residual block is stored.

In this step, be that example is described still with four images shown in Figure 2, and suppose the macro block N of current image B 2 to be encoded is rebuild, then for the quadratic residue piece N that obtains after the inverse transformation ", according to the N of storage in the step 302 " corresponding secondary motion vector (Mx, My), from the reference residual image A2 ' that formerly rebuilds, search out corresponding compensation block M, "+M stores this residual block, and execution in step 503 to obtain residual block N '=N then one time.This process of reconstruction of other macro block is identical therewith.

All rebuild when the quadratic residue piece of all macro blocks of B2 and to finish,, obtain the reconstructed image of residual image B2 ', as the secondary reference picture of other image to be encoded according to the institute of the B2 that is stored residual block once.

Step 503 according to a motion vector that obtains in the step 301, is carried out motion compensation in the reference picture as the reference image to a resulting residual block when encoding, obtain the reconstructed image piece, and this reconstructed image piece is stored.

In this step, when the residual block N ' to B2 ' in the step 303 decoded, (Kx Ky) searched out compensation block K from the reference picture B1 that formerly rebuilds according to a motion vector of storing in the step 301, obtain reconstructed image piece N=N '+K, and this reconstructed image piece N is stored.This process of reconstruction of other macro block is identical therewith.

All rebuild when a residual block of all macro blocks of B2 and to finish, according to all reconstructed image pieces of the B2 that is stored, obtain the reconstructed image of image B 2, as a reference picture of other image to be encoded.

In actual applications, after image encoded, might be stored in the CD, in the hard disk, or in the network, or directly be transferred to receiving terminal, but which kind of situation no matter, when we need use or watch these images, all need it is decoded, could use then.Below in conjunction with Fig. 6 the picture decoding method among the present invention is described in detail.

Fig. 6 is the flow chart of picture decoding method in the embodiment of the invention one.Herein, also the decode procedure with a macro block of current image to be decoded is an example, and coding/decoding method of the present invention is elaborated.As shown in Figure 6, this flow process comprises the steps:

Step 601 is carried out entropy decoding to compressed bit stream, obtains motion vector and secondary motion vector, and the decoded conversion coefficient battle array of entropy is carried out inverse transformation, obtains the quadratic residue piece.

In this step, coding/decoding method is corresponding with coding method, compressed bit stream is carried out entropy decoding after, at first obtain motion vector and secondary motion vector; Can reorder to the decoded view data of entropy afterwards, and the view data after the ordering is carried out inverse transformation, wherein can comprise inverse quantization, inverse transformation etc. in the inverse transformation process, obtain current to be decoded of this image to be decoded, i.e. the quadratic residue piece.

Step 602 according to the secondary motion vector that obtains, to resulting quadratic residue piece, is carried out motion compensation in when coding in as the residual image of secondary reference picture, obtains residual block one time, and this residual block is stored.

In this step, be that example is described still, and suppose that current image to be decoded is B2, when then wherein macro block N being decoded with four images shown in Figure 2, "; the N that obtains according to decoding " corresponding secondary motion vector for the quadratic residue piece N that after compressed bit stream decoding, obtains (Mx, My), from decoded residual image A2 ' formerly, search out corresponding compensation block M, "+M stores this residual block, and execution in step 603 to obtain residual block N '=N then one time.This decode procedure of other macro block is identical therewith.

All decoding when the quadratic residue piece of all macro blocks of B2 finishes, and according to the institute of the B2 that is stored residual block once, obtains the reconstructed image of a residual image, can be used as the secondary reference picture of other image to be decoded.

Wherein, A2 ' is the secondary reference picture in when coding, and this moment, the secondary reference picture during as decoding was obtained during execution in step 602 when the decoding by formerly the A2 of decoded picture, i.e. the decoded reconstructed image of the corresponding residual image of A2.

Step 603 according to a motion vector that obtains, to a resulting residual block, is carried out motion compensation in when coding in as the image of a reference picture, obtains the reconstructed image piece, and this reconstructed image piece is stored.

In this step, when the residual block N ' to B2 ' in the step 602 decodes, a motion vector (Kx of the N ' correspondence that obtains according to decoding, Ky) from decoded reference picture B1 formerly, search out compensation block K, obtain reconstructed image piece N=N '+K, this reconstructed image piece is stored.This decode procedure of other macro block is identical therewith.

So far, current to be decoded decode procedure finishes.All decoding when a residual block of all macro blocks of B2 finishes, and according to all reconstructed image pieces of the B2 that is stored, obtains the reconstructed image of original image, for follow-up use.The reconstructed image that obtains simultaneously also can be used as a reference picture of other image to be decoded.

Wherein, B1 is a reference picture in when coding, and this moment, a reference picture during as decoding was obtained during execution in step 603 when the decoding by formerly the A2 of decoded picture, i.e. the decoded reconstructed image of A2.

Above method for encoding images among the present invention and coding/decoding method are described in detail, again coding/decoding system among the present invention is carried out a detailed description below.Fig. 7 is based on the schematic diagram that concerns of the coder-decoder of above method for encoding images and coding/decoding method and decoder among the embodiment one.

Wherein, encoder is used for original image is encoded, and generates compressed bit stream and uses for subsequent transmission and storage.Cataloged procedure comprises: current image to be encoded is once predicted, obtain a residual image and a motion vector, residual image with the encoded image correspondence is a reference picture, a residual image that obtains is carried out re prediction, obtain quadratic residue image and secondary motion vector, this quadratic residue image and corresponding secondary motion vector, a motion vector are carried out next code and handle.

Decoder is used for the compressed bit stream of image is decoded, and the reconstructed image that obtains original image is for follow-up use.Decode procedure comprises: according to the secondary motion vector that solves from code stream, to the quadratic residue image that from code stream, solves, carry out motion compensation the reconstruction residual image as the secondary reference picture during from coding, obtain residual image one time, according to a motion vector that from code stream, solves, to this residual image from when coding as carrying out motion compensation the reconstructed image of a reference picture, obtain the reconstructed image of original image.

Wherein, the internal structure of encoder and decoder is respectively as Fig. 8 and shown in Figure 9.

Fig. 8 is the structural representation of encoder in the embodiment of the invention one.As shown in Figure 8, this encoder comprises: prediction module, re prediction module, conversion module, coding module, inverse transform block, secondary rebuilding module and a rebuilding module.

And the coding flow process is: a current image to be encoded → prediction module → re prediction module → conversion module → coding module → code stream; Flow process is during reconstruction: the reconstructed image of inverse transform block → secondary rebuilding module → one time rebuilding module → present image.Wherein the secondary motion vector that produces of motion vector producing of prediction module and re prediction module is directly exported to coding module, when secondary is rebuild, what adopt is the secondary motion vector that the re prediction module produces, when once rebuilding, employing be the motion vector that prediction module produces.

Wherein, a prediction module is used for reading a reference picture set from a rebuilding module, is unit to current image to be encoded with macro block or piece, current block to be encoded is chosen best matching blocks from the reference picture set, with selected best matching blocks current block to be encoded is predicted, obtain a residual block and a corresponding motion vector, a residual block is exported to the re prediction module, a motion vector is exported to coding module, simultaneously this motion vector is stored, used for a rebuilding module.

The re prediction module is used to receive a residual block from a prediction module, from the secondary rebuilding module, read the set of secondary reference picture, a residual block that is received is chosen best matching blocks from the set of secondary reference picture, with selected best matching blocks this residual block is predicted, obtain quadratic residue piece and secondary motion vector, the quadratic residue piece is exported to conversion module, the secondary motion vector is exported to coding module, simultaneously this secondary motion vector is stored, used for the secondary rebuilding module.

Conversion module is used to receive the quadratic residue piece from the re prediction module, and the quadratic residue piece that is received is carried out conversion and quantification, further compressed image code check, and with conversion and the conversion coefficient battle array after quantizing export to coding module and inverse transform block.

Coding module can comprise and reordering and cataloged procedure such as entropy coding, be used to receive conversion coefficient battle array from conversion module, together with carrying out entropy coding, write in the code stream from motion vector of a prediction module with from the secondary motion vector of re prediction module.

Reference picture using in an above-mentioned prediction module and two modules of re prediction module and secondary reference picture are respectively the reconstructed image of encoded image and the reconstructed image of the corresponding residual image of encoded image, and when current image to be encoded is encoded, in order to provide reference picture for the next code image, also need the encoded image of current image to be encoded is rebuild, thus comprise the said inverse transform block in front in this decoder, be secondary rebuilding module and a rebuilding module.

Wherein, inverse transform block is used to receive the conversion from conversion module, the conversion coefficient battle array after the quantification, and the conversion coefficient battle array that is received is carried out inverse quantization and inverse transformation, obtains the quadratic residue piece of current encoded image, exports to the secondary rebuilding module.

The secondary rebuilding module is used to receive the quadratic residue piece from inverse transform block, and read the secondary motion vector that the re prediction module is stored, according to the secondary motion vector that is read, carry out motion compensation in the secondary reference picture when coding, obtain a residual block and export to rebuilding module one time, and this residual block is stored.

If all quadratic residue pieces of current encoded image all rebuild end, then according to the institute of the current encoded image of being stored once residual block obtain a residual image of current encoded image, can be used as the secondary reference picture of other forecasting institute need.If the set as the secondary reference picture is the first five two field picture, then next image to be encoded carries out five required two field pictures of re prediction can add the image that this reconstruction obtains for preceding four two field pictures, and a top two field picture this moment can be deleted.

Rebuilding module is used to receive a residual block from the secondary rebuilding module, and read the motion vector that prediction module is stored, according to a motion vector that is read, carry out motion compensation in the reference picture when coding, obtain the reconstructed image piece, and this reconstructed image piece is stored.

If an all residual block of current encoded image is all rebuild end, then obtain the reconstructed image of current encoded image according to all reconstructed image pieces of the current encoded image of being stored, can be used as the reference picture that other forecasting institute needs.If the set as a reference picture is the first five two field picture, five two field pictures that then next image to be encoded carries out forecasting institute need can add the image that this reconstruction obtains for preceding four two field pictures, and a top two field picture this moment can be deleted.

When wherein first image being encoded, reference picture can be sky, when promptly first image being encoded, can handle and directly carry out next code without prediction.

Fig. 9 is the structural representation of decoder in the embodiment of the invention one.As shown in Figure 9, this decoder comprises: decoder module, inverse transform block, secondary rebuilding module and a rebuilding module.

And decoding process is: the reconstructed image of decoder module → inverse transform block → secondary rebuilding module → one time rebuilding module → present image.Secondary motion vector when wherein secondary is rebuild and all preceding continuous decoder module decoding of a motion vector when once rebuilding obtain afterwards.

Wherein, decoder module is used for compressed bit stream is carried out that entropy is decoded, reordered etc., obtain current to be decoded in a current image to be decoded motion vector, secondary motion vector and quadratic residue block conversion coefficient battle array etc., and quadratic residue block conversion coefficient battle array exported to inverse transform block, the secondary motion vector is exported to the secondary rebuilding module, and one time motion vector is exported to rebuilding module one time.

Inverse transform block is used for receiving from decoder module quadratic residue block conversion coefficient battle array, and the quadratic residue block conversion coefficient battle array that is received is carried out inverse quantization and inverse transformation etc., obtains the quadratic residue piece, exports to the secondary rebuilding module.

The secondary rebuilding module is used to receive from the secondary motion vector of decoder module with from the quadratic residue piece of inverse transform block, and according to the secondary motion vector that is received, to carrying out motion compensation in the secondary reference picture of quadratic residue piece when encoding, obtain residual block one time, export to rebuilding module one time, and this residual block is stored.

If all quadratic residue pieces of current image to be decoded all rebuild end, then according to the institute of the current image of being stored to be decoded once residual block obtain a residual image of current image to be decoded, can be used as other required secondary reference picture of decoding.

Rebuilding module is used to receive from motion vector of decoder module with from a residual block of a rebuilding module, and according to a motion vector that is received, to carrying out motion compensation in the reference picture of a residual block when encoding, obtain the reconstructed image piece, and this reconstructed image piece is preserved.

If an all residual block of current image to be decoded is all rebuild end,, obtain the reconstructed image of original image, for follow-up use then according to all reconstructed image pieces of the current image of being stored to be decoded.The reconstructed image that obtains simultaneously also can be used as a reference picture of other image to be decoded.

In conjunction with the embodiments the ordinary circumstance of coding method of the present invention, coding/decoding method and coding/decoding system is described above.In the practical application, in the step 302 when a residual block N ' is carried out re prediction, in order in A2 ', to find best matching blocks M, can be directly with A2 ' as with reference to image, obtain motion vector (Mx, My).

Because if there is B1 is that reference picture has carried out once prediction with A1, and hypothesis macro block corresponding with the macro block N present position of B2 in B1 is N1, and the best matching blocks of N1 in A1 is M1, a corresponding difference vector be (M1x, M1y), the macro block that M1 corresponding position in B2 is arranged again is M, then the quadratic residue piece is N "=N '-M; and can with difference vector (M1x, M1y) as the secondary difference vector of N ' in A2 ' (Mx, My).(M1x M1y) has write code stream, and then this need not to write in the code stream again as if a difference vector.Suppose that B1 is that reference picture is when once predicting with A1, just when seeking best reference picture, carry out, then can be with a difference vector (M1x who produces in this time prediction, M1y) remain, during for re prediction, (M1x, M1y) the secondary difference vector as re prediction writes in the code stream with this difference vector.Also can adopt this method to handle for other macro block.

Equally, when in step 502, rebuilding, or when decoding in the step 602, keep image B 1 all difference vector in reference picture A1, when present image B2 macro block N to be decoded is decoded, at first decoding obtains motion vector (" and the pairing difference vector of macro block N1 of macro block N correspondence position in B1 (M1x; M1y); will (M1x M1y) as the secondary motion vector of N, searches out corresponding compensation block M in reference picture A2 '; the "+M that obtains residual block N '=N one time of Kx, Ky), and the quadratic residue piece N that obtains according to decoding, (Kx Ky) searches out compensation block K in B1, obtain reconstructed image piece N=N '+K by a motion vector again.Also can adopt this method to handle for other macro block.

Perhaps in said process, obtain (M1x, M1y) afterwards, not with this vector directly as the difference vector (Mx of N ' in A2 ', My), but can in A2 ', find macro block M, then with the center initial point of M as search according to this difference vector, near search best matching blocks M ' M, and then obtain M ' correspondence difference vector (M ' x, M ' y), when then decoding with the method described in embodiment one or the embodiment two.Certainly M ' also might be identical with M, and this moment also can be still so that (M1x is M1y) as the difference vector of N ' in A2 '.

Corresponding in encoder shown in Figure 8, may further include a difference vector memory module, be used for storing the difference vector that once prediction is produced, when carrying out re prediction, can from this difference vector memory module, retrieve required difference vector, as the secondary motion vector, perhaps as the search center point of best matching blocks.The specific implementation process is with introducing in the above-mentioned method.

Embodiment two:

In the present embodiment, implementation in the mode two is described in detail, promptly before carrying out re prediction, earlier a residual image is carried out conversion, then to a residual image after the conversion, carry out re prediction in the secondary reference picture after the conversion of secondary reference picture correspondence, afterwards the quadratic residue image that produces behind the re prediction is carried out next code and handle.

In the present embodiment, in order to distinguish with mode one, to be called one time residual image conversion coefficient battle array to the residual image that a residual image carries out producing after the conversion, correspondingly, to be called one time residual block conversion coefficient battle array to the residual block that a residual block carries out producing after the conversion, to carry out the quadratic residue image that re prediction produces to a residual image conversion coefficient battle array and be called quadratic residue image transform factor arrays, correspondingly, to carry out the quadratic residue piece that re prediction produces to a residual block conversion coefficient battle array and be called quadratic residue block conversion coefficient battle array, in addition, also has secondary reference picture conversion coefficient battle array etc.Equally, in decoding end, the quadratic residue image that obtains of will decoding from compressed bit stream is called quadratic residue image transform factor arrays, the residual image that the quadratic residue image is carried out producing after secondary is rebuild is called residual image conversion coefficient battle array one time, correspondingly, the quadratic residue piece that obtains in the quadratic residue image of will decoding from compressed bit stream is called quadratic residue block conversion coefficient battle array, will be called residual block conversion coefficient battle array one time to the residual block that the quadratic residue piece carries out producing after secondary is rebuild.

Figure 10 is the flow chart of method for encoding images among the embodiment two.Method for encoding images flow process in the present embodiment is an example with a macroblock encoding process of current image to be encoded still, and as shown in figure 10, this flow process comprises the steps:

Step 1001 is once predicted current block to be encoded, obtains a residual block and a motion vector.

Specific implementation process in this step can be with step 301 shown in Figure 3 among the embodiment one.

Step 1002 is carried out conversion to a residual block that obtains, and obtains residual block conversion coefficient battle array one time, and this residual block conversion coefficient battle array is carried out re prediction, obtains quadratic residue block conversion coefficient battle array, and the secondary motion vector.

In this step, be example still with a residual block N ' among the B2 ', at first N ' is carried out conversion and quantification, obtain the conversion coefficient battle array N ' after the conversion, and hypothesis M is the best matching blocks of the middle correspondence of reference picture A2 ', corresponding secondary motion vector is (Mx, My), and the correspondent transform factor arrays of M after conversion is M, then can be predicted N ' by M, obtains quadratic residue block conversion coefficient battle array N "=N '-M; Perhaps also can predict some coefficient of N ' by some coefficient of M ', as direct current (DC by M ', Direct Current) coefficient is predicted the DC coefficient of N ', interchange (AC by M ', Alternating Current) coefficient is predicted the AC coefficient of N ' etc., and concrete Forecasting Methodology is decided according to actual conditions.Simultaneously, the secondary motion vector that obtains is stored, used during for image reconstruction.

Step 1003, to quadratic residue block conversion coefficient battle array, and once the secondary motion vector that obtains of motion vector obtaining of prediction and re prediction carries out entropy coding, and writes in the compressed bit stream together; Utilize quadratic residue block conversion coefficient battle array to carry out image reconstruction simultaneously, as the reference picture of other image to be encoded.

Wherein, utilize quadratic residue block conversion coefficient battle array to carry out the process of image reconstruction with reference to following flow process shown in Figure 11.

Figure 11 is the image reconstruction schematic flow sheet in the method for encoding images shown in Figure 10.Process of reconstruction with a macro block of current image to be decoded is an example, and this flow process comprises the steps:

Step 1101 according to the secondary motion vector that obtains in the step 1002, to quadratic residue block conversion coefficient battle array, in the secondary reference picture conversion coefficient battle array as the reference image when coding, is carried out the secondary motion compensation, obtains residual block conversion coefficient battle array one time.

In this step, be that example is described still with four images shown in Figure 2, and suppose the macro block N of current image B 2 to be encoded is rebuild, then for the quadratic residue block conversion coefficient battle array N that obtains behind the re prediction "; " the corresponding secondary motion vector (Mx according to the N of storage in the step 1002, My), from the conversion coefficient battle array of the reference residual image A2 ' that formerly rebuilds, search out corresponding compensation block M, obtain residual block conversion coefficient battle array N '=N "+M then one time.This process of reconstruction of other macro block is identical therewith.

Step 1102 is carried out inverse transformation to a resulting residual block conversion coefficient battle array, obtains residual block again and again, and a resulting residual block is stored.

In this step, when being carried out inverse transformation, the conversion coefficient battle array after the conversion, then obtains N ' after the N ' process inverse transformation in the step 1101 corresponding to the conversion in when coding in the step 1002.

All rebuild when the quadratic residue block conversion coefficient battle array of all macro blocks of B2 and to finish,, obtain the reconstructed image of a residual image B2 ', as the secondary reference picture of other image to be encoded according to the institute of the B2 that is stored residual block once.

Step 1103 according to a motion vector that obtains in the step 1001, is carried out a motion compensation in the reference picture as the reference image to a resulting residual block when encoding, obtain the reconstructed image piece, and this reconstructed image piece is stored.

In this step, when the residual block N ' to B2 ' in the step 1102 decoded, (Kx Ky) searched out compensation block K from the reference picture B1 that formerly rebuilds according to a motion vector of storing in the step 1001, obtain reconstructed image piece N=N '+K, and this reconstructed image piece N is stored.This process of reconstruction of other macro block is identical therewith.

Figure 12 is the flow chart of picture decoding method in the embodiment of the invention two.Herein, also the decode procedure with a macro block of current image to be decoded is an example, and as shown in figure 12, this flow process comprises the steps:

Step 1201 is carried out entropy decoding to compressed bit stream and is obtained motion vector and secondary motion vector, and quadratic residue block conversion coefficient battle array.

In this step, coding/decoding method is corresponding with coding method, compressed bit stream is carried out entropy decoding after, obtain motion vector and secondary motion vector, and current to be decoded of this image to be decoded, i.e. quadratic residue block conversion coefficient battle array.

Step 1202 according to the secondary motion vector that obtains, to resulting quadratic residue block conversion coefficient battle array, is carried out motion compensation in the residual image conversion coefficient battle array as the secondary reference picture when coding, obtains residual block conversion coefficient battle array one time.

In this step, be that example is described still with four images shown in Figure 2, and suppose that current image to be decoded is B2, when then wherein macro block N being decoded, for the quadratic residue block conversion coefficient battle array N that after the compressed bit stream decoding, obtains ", N " the corresponding secondary motion vector (Mx that obtains according to decoding, My), from the conversion coefficient battle array of decoded residual image A2 ' formerly, search out corresponding compensation block M, obtain residual block conversion coefficient battle array N '=N "+M then one time.This decode procedure of other macro block is identical therewith.

Wherein, A2 ' is the secondary reference picture in when coding, and this moment, the secondary reference picture during as decoding was obtained during execution in step 1203 when the decoding by formerly the A2 of decoded picture, i.e. the decoded reconstructed image of the corresponding residual image of A2.When carrying out this step, at first A2 ' is carried out conversion, obtain the conversion coefficient battle array of A2 '.

Step 1203 is carried out inverse transformation to a resulting residual block conversion coefficient battle array, obtains residual block again and again, and a resulting residual block is stored.

In this step, when being carried out inverse transformation, a residual block conversion coefficient battle array that obtains, then obtains N ' after the N ' process inverse transformation in the step 1202 corresponding to the conversion in when coding in the step 1002.

All decoding when the quadratic residue block conversion coefficient battle array of all macro blocks of B2 finishes, and according to the institute of the B2 that is stored residual block once, obtains the reconstructed image of a residual image B2 ', as the secondary reference picture of other image to be decoded.

Step 1204 according to a motion vector that obtains, to a resulting residual block, is carried out motion compensation in when coding in as the image of a reference picture, obtains the reconstructed image piece, and this reconstructed image piece is stored.

In this step, when the residual block N ' to B2 ' in the step 1203 decodes, a motion vector (Kx of the N ' correspondence that obtains according to decoding, Ky) from decoded reference picture B1 formerly, search out compensation block K, obtain reconstructed image piece N=N '+K, this reconstructed image piece is stored.This decode procedure of other macro block is identical therewith.

Above-mentioned another method for encoding images and coding/decoding method among the present invention are described in detail, again the encoder based on above-mentioned method for encoding images and coding/decoding method among the present invention is elaborated below.

Figure 13 is the structural representation of encoder in the embodiment of the invention two.As shown in figure 13, this encoder comprises: prediction module, conversion module, re prediction module, coding module, secondary rebuilding module, inverse transform block and a rebuilding module.

And the coding flow process is: a current image to be encoded → prediction module → conversion module → re prediction module → coding module → code stream; Flow process is during reconstruction: the reconstructed image of secondary rebuilding module → inverse transform block → one time rebuilding module → present image.Wherein the secondary motion vector that produces of motion vector producing of prediction module and re prediction module is directly exported to coding module, when secondary is rebuild, what adopt is the secondary motion vector that the re prediction module produces, when once rebuilding, employing be the motion vector that prediction module produces.

Wherein, a prediction module is used for reading a reference picture set from a rebuilding module, is unit to current image to be encoded with macro block or piece, current block to be encoded is chosen best matching blocks from the reference picture set, with selected best matching blocks current block to be encoded is predicted, obtain a residual block and a corresponding motion vector, a residual block is exported to conversion module, a motion vector is exported to coding module, simultaneously this motion vector is stored, used for a rebuilding module.

Conversion module is used to receive a residual block from a prediction module, and a residual block that is received carried out conversion, obtain a residual block conversion coefficient battle array and export to the re prediction module, and the secondary reference picture that will read from the secondary rebuilding module carries out conversion, obtain secondary reference picture conversion coefficient battle array, export to the re prediction module.

The re prediction module is used to receive from residual block conversion coefficient battle array of a prediction module and secondary reference picture conversion coefficient battle array, conversion coefficient battle array with best matching blocks in the secondary reference picture conversion coefficient battle array is predicted this residual block conversion coefficient battle array, obtain quadratic residue block conversion coefficient battle array and secondary motion vector, export to coding module and secondary rebuilding module.Simultaneously, this secondary motion vector is stored, used for the secondary rebuilding module.

Coding module can comprise and reordering and cataloged procedure such as entropy coding, be used to receive quadratic residue block conversion coefficient battle array and secondary motion vector from the re prediction module, carry out entropy coding together with a motion vector, write in the code stream from a prediction module.

The secondary rebuilding module is used to receive the quadratic residue block conversion coefficient battle array from the re prediction module, and according to the secondary motion vector that reads from the re prediction module, to the quadratic residue block conversion coefficient battle array that is received, carry out motion compensation in the secondary reference picture conversion coefficient battle array when coding, obtain a residual block factor arrays and export to inverse transform block.

Inverse transform block is used to receive a residual block conversion coefficient battle array from the secondary rebuilding module, and a residual block conversion coefficient battle array that is received carried out inverse transformation, obtain residual block one time, export to rebuilding module one time, and this residual block is stored.

If all quadratic residue block conversion coefficient battle arrays of current encoded image are all rebuild and inverse transformation finishes, then according to the current encoded image of being stored once residual block obtain a residual image of current encoded image, can be used as the secondary reference picture that other forecasting institute needs.

Rebuilding module is used to receive a residual block from inverse transform block, and read the motion vector that prediction module is stored, according to a motion vector that is read, carry out motion compensation in the reference picture when coding, obtain the reconstructed image piece, and this reconstructed image piece is stored.

If an all residual block of current encoded image is all rebuild end, then obtain the reconstructed image of current encoded image according to all reconstructed image pieces of the current encoded image of being stored, can be used as the reference picture that other forecasting institute needs.

Figure 14 is the structural representation of decoder in the embodiment of the invention two.As shown in figure 14, this decoder comprises: decoder module, secondary rebuilding module, inverse transform block, rebuilding module and conversion module.

And decoding process is: the reconstructed image of decoder module → secondary rebuilding module → inverse transform block → one time rebuilding module → present image.Secondary motion vector when wherein secondary is rebuild and a motion vector when once rebuilding all are that the decoder module decoding obtains afterwards.

Wherein, decoder module, be used for compressed bit stream is carried out the entropy decoding, obtain current to be decoded in a current image to be decoded motion vector, secondary motion vector and quadratic residue block conversion coefficient battle array etc., and quadratic residue block conversion coefficient battle array and secondary motion vector exported to the secondary rebuilding module, a motion vector is exported to rebuilding module one time.

The secondary rebuilding module is used to receive quadratic residue block conversion coefficient battle array and the secondary motion vector from coding module, and the secondary reference picture conversion coefficient battle array when receiving this quadratic residue block conversion coefficient battle array coding from conversion module, according to the secondary motion vector that is received, to carry out motion compensation in the secondary reference picture conversion coefficient battle array of quadratic residue block conversion coefficient battle array when encoding, obtain the residual block factor arrays one time, export to inverse transform block.

Wherein, secondary reference picture conversion coefficient battle array is a residual image through having decoded after secondary rebuilding module and the inverse transform block, conversion module is used to read this residual image, and a residual image that is read carried out conversion, the conversion coefficient battle array after the conversion is exported to the secondary rebuilding module.

Inverse transform block is used to receive a residual block conversion coefficient battle array from the secondary rebuilding module, and a residual block conversion coefficient battle array that is received carried out inverse transformation, obtain residual block one time, export to reconstructed image one time, and this residual block is stored.

If all quadratic residue block conversion coefficient battle arrays of current image to be decoded are all rebuild and inverse transformation finishes, then according to institute's residual block once of the current image of being stored to be decoded, obtain a residual image of current image to be decoded, can be used as the required secondary reference picture of other decoding.

If an all residual block of current image to be decoded is all rebuild end, then according to all reconstructed image pieces of the current image of being stored to be decoded, obtain the reconstructed image of current image to be decoded, can be used as a required reference picture of other decoding.

Wherein, encoder among the embodiment two, in process to image reconstruction, after the secondary rebuilding module, obtain residual block conversion coefficient battle array one time, when this residual block conversion coefficient battle array is exported to inverse transform block, can store it, and according to institute's residual block conversion coefficient battle array once of storage current encoded image, obtain residual image conversion coefficient battle array one time, secondary reference picture conversion coefficient battle array as input re prediction module, then conversion module is after carrying out conversion to a residual block conversion coefficient battle array, needn't store it again, and this secondary reference picture conversion coefficient battle array also need not by obtaining after the conversion of secondary reference picture process conversion module.Equally, in decoder, also can do same processing, can need not comprise conversion module in the decoder this moment.

When carrying out conversion or conversion module when carrying out conversion, can comprise transform and quantization in the present embodiment, when carrying out next code or coding module when carrying out coding, also can comprise and reordering and entropy coding, perhaps other coded system; During same inverse transformation, or inverse transform block is when carrying out inverse transformation, during with conversion or conversion module corresponding, also can comprise inverse quantization and inverse transformation, preorder when decoding, or decoder module is corresponding with next code or coding module when carrying out decoding, can comprise entropy decoding and reorder, perhaps other decoding process.

Embodiment three

In the present embodiment, method for encoding images, the coding/decoding method in the Application Example one, and the method for encoding images among the embodiment two simultaneously, coding/decoding method.Accordingly, realize functions, be the implementation that on the basis of embodiment two, has added among the embodiment one for two kinds of the encoder of encoder in the foregoing description one and the embodiment two.Figure 15 is the structural representation of encoder in the embodiment of the invention three.This encoder comprises: prediction module, conversion module, re prediction module, coding module, secondary rebuilding module, inverse transform block and a rebuilding module.The coding flow process of representing comprising dotted portion: a current image to be encoded → prediction module → re prediction module → conversion module → coding module → code stream; Flow process during reconstruction: the reconstructed image of inverse transform block → secondary rebuilding module → one time rebuilding module → present image.And the coding flow process partly represented of solid line: a current image to be encoded → prediction module → conversion module → re prediction module → coding module → code stream; Flow process during reconstruction: the reconstructed image of secondary rebuilding module → inverse transform block → one time rebuilding module → present image.

Wherein the secondary motion vector that produces of motion vector producing of prediction module and re prediction module is directly exported to coding module, when secondary is rebuild, what adopt is the secondary motion vector that the re prediction module produces, when once rebuilding, employing be the motion vector that prediction module produces.

Figure 16 is the structural representation of decoder in the embodiment of the invention three.This decoder comprises: decoder module, secondary rebuilding module, inverse transform block, rebuilding module and conversion module.The decoding process of representing comprising dotted portion: the reconstructed image of decoder module → inverse transform block → secondary rebuilding module → one time rebuilding module → present image.And the decoding process partly represented of solid line: the reconstructed image of decoder module → re prediction module → inverse transform block → one time prediction module → present image.

When wherein decoding, a secondary motion vector when secondary is rebuild and a motion vector when once rebuilding all are that the decoder module decoding obtains afterwards.

Wherein, it still is the flow process of dotted portion that encoder adopts the flow process of solid line part, can be according to working control, perhaps according to the encoding and decoding effect, perhaps carry out respectively, adopt which kind of coded system according to other actual conditions, will corresponding which kind of decoding process, can be when encoder encodes, with the picture frame is unit, is unit etc. with the macro block perhaps, and a flag bit is set, adopt which kind of coded system, just carry out mark with flag bit; Decoder can be carried out corresponding decode procedure according to separating code mark when compressed bit stream is decoded.Wherein each functional module in the encoder is according to the implementation that reality adopted, and carries out function corresponding, when adopting the implementation of solid line part, according to carrying out among the embodiment two; When adopting the implementation of dotted portion, according to carrying out among the embodiment one.

In the actual coding process, about the entropy coding link, other coding method can also be arranged, in like manner, entropy decoding link also can be other coding/decoding method, and wherein, coding/decoding method is corresponding with coding method.And, also can not carry out entropy coding or other cataloged procedure, and directly motion vector, secondary motion vector and a quadratic residue image be write in the compressed bit stream.

Decoder among above-mentioned three embodiment, when carrying out the secondary reconstruction, the quadratic residue image of being imported also can be to carry out entropy coding compressed bit stream afterwards, need increase a decoder module this moment in encoder, be used for the compressed bit stream that the received code module generates, decoding obtains quadratic residue image, secondary motion vector and a motion vector from compressed bit stream, and then process of reconstruction can be with the process in the decoder.

Wherein, method for encoding images provided by the present invention and picture decoding method can use the formation decoding method simultaneously, comprising:

During coding, current image to be encoded is once predicted, obtain a residual image and a motion vector, and a residual image carried out re prediction, obtain quadratic residue image and secondary motion vector, quadratic residue image, secondary motion vector and a motion vector are write in the compressed bit stream send.

During decoding, receive compressed bit stream, and decoding obtains motion vector, secondary motion vector and quadratic residue image one time from compressed bit stream; According to the secondary motion vector, the quadratic residue image is carried out motion compensation, obtain residual image one time, and, a residual image is carried out motion compensation according to a motion vector, obtain reconstructed image.

For the specific implementation process of Code And Decode, also can be with embodiment one to embodiment three described several method.

Encoder provided by the present invention also can be assembled into a codec and use.Shown in Figure 17, the codec that is assembled on the basis of Figure 17 for encoder shown in Figure 8 in embodiment one and decoder shown in Figure 9, this codec comprises: coding unit and decoding unit.

Wherein, coding unit, be used for current image to be encoded is once predicted, obtain a residual image and a motion vector, and a residual image carried out re prediction, obtain quadratic residue image and secondary motion vector, quadratic residue image, secondary motion vector and a motion vector are write in the compressed bit stream send.

Decoding unit is used to receive compressed bit stream, and decoding obtains motion vector, secondary motion vector and quadratic residue image one time from compressed bit stream; According to the secondary motion vector, the quadratic residue image is carried out motion compensation, obtain residual image one time, and according to a motion vector, a residual image is carried out motion compensation, obtain reconstructed image, and decoding unit is further used for providing as the reconstructed image with reference to image for coding unit.

Wherein, coding unit can comprise: prediction module, re prediction module, conversion module and a coding module; Decoding unit can comprise: decoder module, inverse transform block, secondary rebuilding module and a rebuilding module.

This codec has further comprised decoder module for encoder shown in Figure 8, be used for when providing the reconstructed image of reference picture for cataloged procedure, decoding obtains quadratic residue image and motion vector and secondary motion vector from the compressed bit stream that coding generates, then this quadratic residue image is carried out inverse transformation, when carrying out image reconstruction, the secondary motion vector that needed secondary motion vector can be the re prediction module stores, it also can be the secondary motion vector that the decoder module decoding obtains, in like manner, a needed motion vector can be a motion vector of a prediction module storage, it also can be the motion vector that the decoder module decoding obtains, obtain if finish by decoder module, then prediction module and re prediction module can be not be stored the motion vector and the secondary motion vector that are produced.

This codec is for decoder shown in Figure 9, the part of decoding is mainly finished by decoder module, inverse transform block, secondary rebuilding module and a rebuilding module, and the function of each module and annexation are identical with functions of modules and annexation shown in Figure 9.

Current can also have other malformation for codec shown in Figure 17, perhaps can not comprise conversion module and inverse transform block etc., and decoder module and coding module can have a lot of coded systems and decoding process too.

Equally, also can be assembled into codec for the encoder among embodiment two and the embodiment three.And various malformations etc. can be arranged.

From above-mentioned each embodiment as seen, the present invention by with the pairing residual image of some image of having encoded in each video sequence as the reference image, current image to be encoded is predicted through the residual image that produces after the predictive coding, to predict that again the quadratic residue image that the back obtains carries out the next code operation, thereby elimination relevance of residual images, further reduce the redundancy between the image, reduce the data volume of code stream, improve code efficiency.

In addition, when current image to be encoded is predicted through the residual image that produces after the predictive coding, a residual image to current image to be encoded is a unit with macro block or piece, in residual image, seek best matching blocks as the secondary reference picture, when seeking best matching blocks, by utilizing a difference vector of correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, with this corresponding blocks as best matching blocks, and with a difference vector of institute's reference as the secondary difference vector in the secondary reference picture, thereby can reduce the data volume of code stream; Perhaps after finding corresponding blocks, as the search center starting point, seek best matching blocks, thereby further improve code efficiency with this piece.

At last, the present invention can also predict the conversion coefficient battle array after the residual image conversion, thereby for eliminating the method for correlation between the residual image, provide more choices, make coding more flexible, thereby can eliminate relevance of residual images to greatest extent.

The various embodiments described above are a kind of displaying of inventive concept, and non-limiting protection scope of the present invention.Within the spirit and principles in the present invention all, any modification of being made, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.

Claims

1. a method for encoding images is characterized in that, this method comprises:

2. the method for claim 1, it is characterized in that, described step B specifically comprises: with macro block or piece is unit, at each macro block or the piece in the residual image, in residual image set, seek best matching blocks as the secondary reference picture, with this best matching blocks current macro or piece are predicted, and obtained quadratic residue piece and corresponding secondary motion vector, quadratic residue piece, secondary motion vector and a motion vector are write in the compressed bit stream.

3. method as claimed in claim 2, it is characterized in that, seeking best matching blocks described in the step B in the residual image set as the secondary reference picture is: when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, with this corresponding blocks as best matching blocks, and with a described motion vector as the secondary motion vector in the secondary reference picture;

Perhaps

When carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image finds corresponding blocks in the secondary reference picture, as the search center starting point, seeks best matching blocks with this corresponding blocks.

4. the method for claim 1 is characterized in that, further comprise before the step B: with macro block or piece is unit, and each macro block in the residual image or piece are carried out conversion, obtains a residual block after the conversion;

Described step B is: at a residual block after each conversion, in as the image after the residual image conversion of secondary reference picture, seek best matching blocks, predict with the residual block of this best matching blocks after current conversion, and obtain quadratic residue piece and corresponding secondary motion vector, quadratic residue piece, secondary motion vector and a motion vector are write in the compressed bit stream.

5. as any described method in the claim 2 to 4, it is characterized in that described secondary reference picture is: the reconstructed image of the corresponding residual image of encoded image.

6. as any described method in the claim 1 to 4, it is characterized in that, write among the step B before the compressed bit stream, further comprise: quadratic residue image, secondary motion vector and a motion vector are carried out entropy coding.

7. a picture decoding method is characterized in that, this method comprises:

8. method as claimed in claim 7, it is characterized in that, described step B is: with macro block or piece is unit, at each macro block or the piece in the quadratic residue image, secondary motion vector according to this macro block or piece correspondence, this quadratic residue piece is carried out motion compensation, obtain residual block one time, and according to a motion vector of this residual block correspondence, this residual block is carried out motion compensation, obtain the reconstructed image piece, with the synthetic described reconstructed image of all reconstructed image pieces of quadratic residue image.

9. method as claimed in claim 8, it is characterized in that the described motion vector according to this residual block correspondence of step B carries out before the motion compensation this residual block, further comprise: a residual block that obtains is carried out inverse transformation, obtain a residual block after the inverse transformation.

10. encoder comprises: prediction module and coding module is characterized in that this encoder also comprises: the re prediction module, wherein,

11. encoder as claimed in claim 10 is characterized in that, this encoder also comprises:

The secondary rebuilding module, be used to receive quadratic residue image from described re prediction module, and read the secondary motion vector that obtains in the re prediction module, according to this secondary motion vector, the quadratic residue image that is received is carried out motion compensation, obtain residual image one time, the reference picture that a described residual image is used for described re prediction module when carrying out re prediction;

A rebuilding module, be used to receive a residual image from described secondary rebuilding module, and read a motion vector that obtains in prediction module, according to this motion vector, a residual image that is received is carried out motion compensation, obtain reconstructed image, the reference picture that described reconstructed image is used for a described prediction module when once predicting.

12. encoder as claimed in claim 11 is characterized in that, described encoder further comprises: conversion module and inverse transform block, wherein,

Conversion module is used to receive the quadratic residue image from the re prediction module, and the quadratic residue image that is received is carried out conversion, and the quadratic residue image after the conversion is exported to coding module and inverse transform block;

Inverse transform block is used to receive the quadratic residue image from conversion module, and the quadratic residue image that is received is carried out inverse transformation, obtains the quadratic residue image after the inverse transformation, exports to the secondary rebuilding module.

13. encoder as claimed in claim 11 is characterized in that, described encoder further comprises: conversion module and inverse transform block, wherein,

Conversion module is used to receive a residual image from a prediction module, and a residual image is carried out conversion, and a residual image after the conversion is exported to the re prediction module;

Inverse transform block is used to receive a residual image from the secondary rebuilding module, and a residual image that is received is carried out inverse transformation, and a residual image after the inverse transformation is exported to rebuilding module one time.

14. a decoder, this decoder comprises: decoder module, a rebuilding module is characterized in that this decoder also comprises: the secondary rebuilding module, wherein,

15. decoder as claimed in claim 14, it is characterized in that, this decoder further comprises: inverse transform block, be used to receive quadratic residue image from decoder module, the quadratic residue image that is received is carried out inverse transformation, and the quadratic residue image after the inverse transformation is exported to the secondary rebuilding module.

16. decoder as claimed in claim 14, it is characterized in that, this decoder further comprises: inverse transform block, be used to receive a residual image from the secondary rebuilding module, and a residual image that is received carried out inverse transformation, a residual image after the inverse transformation is exported to rebuilding module one time.

17. a decoding method is characterized in that, this method comprises:

18. a codec is characterized in that, this codec comprises: coding unit and decoding unit, wherein,

Coding unit, be used for current image to be encoded is once predicted, obtain a residual image and a motion vector, and a residual image carried out re prediction, obtain quadratic residue image and secondary motion vector, quadratic residue image, secondary motion vector and a motion vector are write in the compressed bit stream;

19. the codec shown in claim 18 is characterized in that, described coding unit comprises:

The re prediction module is used to receive a residual image from a prediction module, and a residual image that is received is carried out re prediction, obtains quadratic residue image and secondary motion vector;

Coding module, be used to receive from motion vector of a prediction module with from the secondary motion vector and the quadratic residue image of re prediction module, and a motion vector, secondary motion vector and the quadratic residue image that is received write in the compressed bit stream.

20., it is characterized in that described decoding unit comprises as codec as described in the claim 19:

The secondary rebuilding module is used to receive quadratic residue image and secondary motion vector from decoder module, according to the secondary motion vector that is received, the quadratic residue image is carried out motion compensation, obtains residual image one time;

A rebuilding module is used to receive from a residual image of secondary rebuilding module and from a motion vector of decoder module, according to a motion vector that is received, a residual image is carried out motion compensation, obtains reconstructed image.

21. a method for encoding images is characterized in that, this method comprises:

B, be unit with macro block or piece, at each macro block or the piece in the residual image, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, with this corresponding blocks as best matching blocks, and with a described motion vector as the secondary motion vector in the secondary reference picture, with this best matching blocks current macro or piece are predicted, obtained the quadratic residue piece; Motion vector and quadratic residue piece and/or secondary motion vector are write in the compressed bit stream.

22. method as claimed in claim 21, it is characterized in that, described in the steps A current image to be encoded is once predicted, obtaining a residual image and a motion vector comprises: with macro block or piece is unit, at each macro block or the piece in the current image to be encoded, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the reference picture of current macro or piece, find corresponding blocks, with this corresponding blocks as best matching blocks, and with a described motion vector as current macro or piece a motion vector in reference picture, with this best matching blocks current macro or piece are predicted, obtained residual block one time;

Perhaps, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the reference picture of current macro or piece, find corresponding blocks, with this piece as the search center starting point, seek best matching blocks, current macro or piece are predicted, and obtain a motion vector of residual block and correspondence with this best matching blocks.

23. a method for encoding images is characterized in that, this method comprises:

B, be unit with macro block or piece, at each macro block or the piece in the residual image, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the secondary reference picture, find corresponding blocks, as the search center starting point, seek best matching blocks with this piece, with this best matching blocks current macro or piece are predicted, and obtained quadratic residue piece and corresponding secondary motion vector; Motion vector, quadratic residue piece and a secondary motion vector are write in the compressed bit stream.

24. method as claimed in claim 23, it is characterized in that, described in the steps A current image to be encoded is once predicted, obtaining a residual image and a motion vector comprises: with macro block or piece is unit, at each macro block or the piece in the current image to be encoded, when carrying out disparity estimation, a motion vector according to correspondence position piece in the encoded image, in the reference picture of current macro or piece, find corresponding blocks, with this corresponding blocks as best matching blocks, and with a described motion vector as current macro or piece a motion vector in reference picture, with this best matching blocks current macro or piece are predicted, obtained residual block one time;

25. a picture decoding method is characterized in that, this method comprises:

26. an encoder comprises: prediction module and coding module is characterized in that this encoder also comprises: the re prediction module, wherein,

27. encoder as claimed in claim 26 is characterized in that, this encoder also comprises:

The secondary rebuilding module, be used to receive quadratic residue image from the re prediction module, and read a motion vector that obtains in the secondary motion vector that obtains in the re prediction module or the prediction module as the secondary motion vector, according to a described secondary motion vector or a described motion vector as the secondary motion vector, the quadratic residue image that is received is carried out motion compensation, obtain residual image one time, the reference picture when being used for described re prediction module and carrying out re prediction;

A rebuilding module, be used to receive a residual image from the secondary rebuilding module, and read a motion vector that obtains in prediction module, according to this motion vector, a residual image that is received is carried out motion compensation, obtain reconstructed image, the reference picture when being used for a described prediction module and once predicting.

28. an encoder comprises: prediction module and coding module is characterized in that this encoder also comprises: the re prediction module, wherein,

29. encoder as claimed in claim 28 is characterized in that, this encoder also comprises:

The secondary rebuilding module, be used to receive quadratic residue image from the re prediction module, and read the secondary motion vector that obtains in the re prediction module, according to described secondary motion vector, the quadratic residue image that is received is carried out motion compensation, obtain residual image one time, the reference picture that a described residual image is used for described re prediction module when carrying out re prediction;

A rebuilding module, be used to receive a residual image from the secondary rebuilding module, and read a motion vector that obtains in prediction module, according to this motion vector, a residual image that is received is carried out motion compensation, obtain reconstructed image, the reference picture that described reconstructed image is used for a described prediction module when once predicting.

30. a decoder, this decoder comprises: decoder module, a rebuilding module is characterized in that this decoder also comprises: the secondary rebuilding module, wherein,

31. a device of seeking best matching blocks between many views picture is characterized in that this device comprises:

32. a device of seeking best matching blocks between many views picture is characterized in that this device comprises: