WO2008025300A1 - A method for encoding/decoding, the corresponding apparatus for encoding/decoding and a method or apparatus for searching optimum matching block - Google Patents

Abstract

A method for image/video encoding. It comprises following steps: making motion-compensated prediction for present image to get the first residual images and the first motion vectors. Then making the second motion-compensated prediction for the first residual images to get the second residual images and the second motion vectors. Encoding the second residual images and the second motion vectors and the first motion vectors into bit stream, or after the entropy coding, then encoding them into bit stream. Furthermore, the corresponding apparatus for image/video encoding and/or decoding are disclosed. The solutioin can get rid of the relativity among the residual images.

Description

 Editing/decoding method, encoder/decoder and method and device for finding the best matching block

 The present invention relates to image coding techniques in the field of multimedia, and more particularly to an image coding method, a decoding method, an encoder, a decoder, a codec method, a codec, and a method and apparatus for finding a best matching block between multi-view images. Background of the invention

 Current video coding standards such as the H.261, H.263, H.263+, and H.264 standards developed by the International Telecommunication Union (ITU) and the Moving Picture Experts Group (MPEG) MPEG-1, MPEG-2, MPEG-3, MPEG-4, etc. are all built on the Hybrid Coding framework. The hybrid coding framework is a video image coding method with mixed time space. When encoding, first perform intra-frame and inter-frame prediction, and obtain prediction values to eliminate the correlation in the time domain. Then, based on the predicted values and the actual values of the original images. The difference is obtained, and the residual image is obtained. The residual image is subjected to two-dimensional transform by discrete cosine transform method or other transform method to eliminate the correlation of the spatial domain. Finally, the transformed data is entropy encoded to eliminate the statistical The redundancy, together with the entropy-encoded data and some side information including the motion vector required for decoding, form a compressed code stream for subsequent transmission and storage, and achieve the purpose of compressing the video image. Accordingly, at the time of decoding, an image is reconstructed in accordance with a series of decoding processes such as entropy decoding, inverse transform, and predictive compensation.

The hybrid coding framework includes two different coding methods: intraframe coding and interframe coding. When intra-coding is performed, the original image is two-dimensionally transformed, then the transform coefficients are quantized in the transform domain, and finally entropy encoded; or the spatial correlation of adjacent blocks is fully utilized in a given frame, When encoding a block, you can first make a block for the given block based on the surrounding block. The prediction then performs two-dimensional transformation, quantization and entropy coding on the difference between the predicted value and the actual value. When performing inter-frame coding, the motion vector is obtained by motion estimation, and then the inter-frame prediction based on motion compensation is used to obtain the difference between the predicted value and the actual value of the original image, that is, the residual image, and the residual image is two-dimensionally transformed, and then The transform coefficients are quantized in the transform domain, and finally entropy encoded.

 It can be seen that the above prediction techniques are used in both intraframe coding and interframe coding. The prediction technology makes full use of the spatial and temporal correlation between frames and frames, reduces the code rate based on the elimination of correlation, and improves the data compression ratio of the compressed code stream to the original image. Especially in the hybrid coding framework, interframe prediction is an effective method to reduce the temporal correlation of images, especially in the recent video standard H.264 Advanced Video Coding (H.264/AVC, Advanced Video Coding), inter-frame The prediction uses a wider range of block sizes, from 16x16 to 4x4, and multiple reference image frames, so it is widely used. As in a video sequence captured by a single camera, there is a large correlation between adjacent images, namely redundancy. By using the correlation between the already encoded image and the current encoded image, the already encoded image is restored to the reconstructed image, and the current encoded image is predicted as the reference image, and based on the predicted value and the current value of the currently encoded image. The difference is obtained, and the residual image is obtained, and the residual image is encoded, thereby eliminating the correlation between adjacent images and reducing the amount of encoded data.

 Also in the field of multi-view video, when multiple cameras shoot the same object, there is a large correlation between the images of the multiple video sequences captured, especially when the cameras are located close to each other. More relevant. Therefore, when encoding a plurality of video sequences simultaneously, the currently encoded image may be predicted by using the encoded image in the current video sequence or other video sequences as a reference image to obtain a residual image, and the residual image is encoded. Thereby eliminating the correlation between the images of the plurality of video sequences, effectively reducing the redundancy between the images.

In the above encoding method, only one prediction is performed on the current image to be encoded, and then The resulting residual image is encoded. In fact, in the field of multi-view video, there is still a strong correlation between the residual image generated by the current image to be encoded and the residual image corresponding to the reference image, and the existing coding method is more When the images of the video sequences are encoded, the correlation between the residual images cannot be eliminated, and thus the coding efficiency cannot be improved. Summary of the invention

 In view of this, an embodiment of the present invention provides an image coding method, a decoding method, an encoder, a decoder, a codec method, and a codec, so as to eliminate correlation between residual images and improve coding efficiency.

 In addition, two image encoding methods, one decoding method, two encoders and a decoder are also provided in the embodiment of the present invention, so as to eliminate correlation between residual images and further improve coding efficiency.

 Finally, in the embodiment of the present invention, two methods for finding the best matching block between multi-view images and four devices for finding the best matching block among the multi-view images are provided, so as to utilize the correlation between the residual images. Improve the search efficiency of the best matching block.

 The first image coding method provided by the embodiment of the present invention includes:

 A. Performing a prediction on the current image to be encoded to obtain a residual image and a motion vector;

 B. Perform secondary prediction on the primary residual image to obtain a secondary residual image and a secondary motion vector, and write the secondary residual image, the secondary motion vector, and the primary motion vector into the compressed code stream.

 The first image decoding method provided by the embodiment of the present invention includes:

A. Decoding from the compressed code stream to obtain a motion vector, a quadratic motion vector, and a quadratic residual image; B. Perform motion compensation on the second residual image according to the secondary motion vector to obtain a residual image, and perform motion compensation on the primary residual image according to the primary motion vector to obtain a reconstructed image.

 The first type of encoder provided by the embodiment of the present invention includes: a primary prediction module and an encoding module, and the encoder further includes: a secondary prediction module, where

 a prediction module, configured to perform a prediction on the current image to be encoded, to obtain a residual image and a motion vector;

 a secondary prediction module, configured to receive a primary residual image from the primary prediction module, perform secondary prediction on the received primary residual image, and obtain a secondary residual image and a secondary motion vector;

 An encoding module, configured to receive a primary motion vector from the primary prediction module and a secondary motion vector and a secondary residual image from the secondary prediction module, and receive the received primary motion vector, secondary motion vector, and The secondary residual image is written into the compressed code stream.

 The first decoder provided by the embodiment of the present invention includes: a decoding module, a primary reconstruction module, and the decoder further includes: a secondary reconstruction module, where

 a decoding module, configured to decode a motion vector, a secondary motion vector, and a secondary residual image from the compressed code stream;

 a quadratic reconstruction module, configured to receive a secondary residual image and a secondary motion vector from the decoding module, perform motion compensation on the secondary residual image according to the received secondary motion vector, to obtain a residual image;

 And a primary reconstruction module, configured to receive a primary residual image from the secondary reconstruction module and a primary motion vector from the decoding module, and perform motion compensation on the primary residual image according to the received primary motion vector to obtain a reconstructed image.

 The coding and decoding method provided by the embodiment of the present invention includes:

A. Perform a prediction on the current image to be encoded, and obtain a residual image and a single operation. a motion vector, and performing a second prediction on the primary residual image to obtain a secondary residual image and a secondary motion vector, and writing the secondary residual image, the secondary motion vector, and the primary motion vector into the compressed code stream;

B. receiving the compressed code stream, and decoding a motion vector, a quadratic motion vector, and a quadratic residual image from the compressed code stream; performing motion compensation on the second residual image according to the secondary motion vector, and obtaining once The residual image is subjected to motion compensation for the primary residual image according to the primary motion vector to obtain a reconstructed image.

 A codec according to an embodiment of the present invention includes: a coding unit and a decoding unit, where

 a coding unit, configured to perform a prediction on the current image to be encoded, obtain a residual image and a motion vector, and perform a second prediction on the first residual image to obtain a secondary residual image and a secondary motion vector, which will be quadratic The residual image, the secondary motion vector, and the primary motion vector are written into the compressed code stream;

 a decoding unit, configured to receive the compressed code stream, and obtain a motion vector, a quadratic motion vector, and a quadratic residual image from the compressed code stream; perform motion compensation on the second residual image according to the secondary motion vector , obtaining a residual image, and performing motion compensation on the primary residual image according to the primary motion vector to obtain a reconstructed image.

 The second image encoding method provided in the embodiment of the present invention includes:

 A. Performing a prediction on the current image to be encoded encoded by the image to be encoded in the first frame to obtain a residual image and a motion vector;

B. In the macroblock or block unit, when performing disparity estimation for each macroblock or block in the primary residual image, find a corresponding in the secondary reference image according to the primary motion vector of the corresponding position block in the encoded image. Blocking, using the corresponding block as a best matching block, and using the primary motion vector as a secondary motion vector in the secondary reference image, and predicting the current macroblock or block by using the best matching block to obtain a second The residual block writes the primary motion vector and the secondary residual block and/or the secondary motion vector into the compressed code stream. A third image encoding method provided in the embodiment of the present invention includes:

A. Performing a prediction on the current image to be encoded encoded by the image to be encoded in the first frame to obtain a residual image and a motion vector;

 B. In the macroblock or block unit, when performing disparity estimation for each macroblock or block in the primary residual image, find a corresponding in the secondary reference image according to the primary motion vector of the corresponding position block in the encoded image. Block, using the block as a starting point of the search center, searching for the best matching block, using the best matching block to predict the current macroblock or block, and obtaining a quadratic residual block and a corresponding secondary motion vector, which will be a motion Vectors, quadratic residual blocks, and quadratic motion vectors are written into the compressed code stream.

 The second image decoding method provided in the embodiment of the present invention includes:

 A. Decoding from the compressed code stream to obtain a motion vector, a quadratic motion vector, and a quadratic residual image;

 B, in macroblock or block unit, for each macroblock or block in the quadratic residual image, according to the secondary motion vector corresponding to the macroblock or block, or the corresponding secondary motion of the macroblock or block The primary motion vector of the vector is motion-compensated for the second residual block to obtain a residual block, and the primary motion vector corresponding to the primary residual block is compensated for, and the first residual block is motion compensated to obtain a reconstructed image. Block, all reconstructed image blocks of the secondary residual image are combined to reconstruct the image.

 The second type of encoder provided in the embodiment of the present invention includes: a primary prediction module and a coding module, wherein the encoder further includes: a secondary prediction module, where

 a prediction module, configured to perform a prediction on the current image to be encoded after the first frame, to obtain a residual image and a motion vector;

a secondary prediction module, configured to receive a primary residual image from a primary prediction module, and perform a parallax for each macroblock or block in the primary residual image in units of macroblocks or blocks for the received primary residual image. Estimating, based on a motion of the corresponding position block in the encoded image Vector, finding a corresponding block in the secondary reference image, using the corresponding block as a best matching block, and using the primary motion vector as a secondary motion vector in the secondary reference image, using the best matching block for the current The macroblock or block performs prediction to obtain a quadratic residual block;

 An encoding module, configured to receive a primary motion vector from the primary prediction module and a secondary residual image and/or a secondary motion vector from the secondary prediction module, and receive the received primary motion vector and the second residual The difference image and/or the secondary motion vector are written into the compressed code stream.

 The third encoder provided in the embodiment of the present invention includes: a primary prediction module and a coding module, wherein the encoder further includes: a secondary prediction module, where

 a prediction module, configured to perform a prediction on the current image to be encoded after the first frame, to obtain a residual image and a motion vector;

 a secondary prediction module, configured to receive a primary residual image from a primary prediction module, and perform a parallax for each macroblock or block in the primary residual image in units of macroblocks or blocks for the received primary residual image. In the estimation, according to the primary motion vector of the corresponding position block in the encoded image, the corresponding block is found in the secondary reference image, and the block is used as the search center starting point to find the best matching block, and the current matching macro is used to match the current macro. The block or block performs prediction to obtain a quadratic residual block and a corresponding secondary motion vector;

 An encoding module, configured to receive a primary motion vector from the primary prediction module and a secondary residual image and a secondary motion vector from the secondary prediction module, and receive the received primary motion vector and the secondary residual image And the secondary motion vector is written into the compressed code stream.

 The second decoder provided in the embodiment of the present invention includes: a decoding module and a primary reconstruction module, wherein the decoder further includes: a secondary reconstruction module, where

 a decoding module, configured to decode a motion vector, a secondary motion vector, and a secondary residual image from the compressed code stream;

a quadratic reconstruction module, configured to receive a secondary residual image and a secondary motion vector from the decoding module or a primary motion vector as a secondary motion vector, according to the received secondary motion The vector or the primary motion vector of the secondary motion vector performs motion compensation on the secondary residual image to obtain a residual image;

 a primary reconstruction module, configured to receive a primary residual image from the secondary reconstruction module and a primary motion vector from the decoding module, and perform motion compensation on the primary residual image according to the received primary motion vector to obtain a reconstructed image. .

 The first method for finding the best matching block among the multi-view images provided in the embodiment of the present invention includes:

 When performing primary and/or secondary disparity estimation on the macroblock or block of the current image, finding a corresponding block in the reference image of the current macroblock or block according to the primary motion vector of the corresponding position block in the encoded image, and the corresponding block As the primary and/or second best matching block of the current macroblock or block.

 The second method for finding the best matching block among the multi-view images provided in the embodiment of the present invention includes:

 When performing one-time and/or quadratic disparity estimation on the macroblock or block of the current image, finding a corresponding block in the reference image of the current macroblock or block according to the primary motion vector of the corresponding position block in the encoded image, with the corresponding block As the starting point of the search center, find the primary and/or secondary best matching block of the current macroblock or block.

 The first device for finding the best matching block among the multi-view images provided in the embodiment of the present invention includes:

 a prediction module, configured to: in a macroblock or a block for the current image to be encoded, for each macroblock or block in the current image to be encoded, when performing disparity estimation, according to a motion vector of a corresponding position block in the encoded image Find the corresponding block in the reference image and use the corresponding block as the best matching block.

 The second device for finding the best matching block among the multi-view images provided in the embodiment of the present invention includes:

a prediction module, configured to use a macroblock or a block as a unit for the current image to be encoded, For each macroblock or block in the pre-coded image, when performing disparity estimation, according to the primary motion vector of the corresponding position block in the encoded image, find the corresponding block in the reference image, and use the block as the starting point of the search center to find the most Good match block.

 The third device for finding the best matching block among the multi-view images provided in the embodiment of the present invention includes:

 a secondary prediction module, configured to perform, in a macroblock or a block, the received residual residual image, for each macroblock or block in the primary residual image, according to the corresponding position block in the encoded image The primary motion vector finds the corresponding block in the secondary reference image, and uses the corresponding block as the best matching block.

 The fourth device for finding the best matching block among the multi-view images provided in the embodiment of the present invention includes:

 a secondary prediction module, configured to perform, in a macroblock or a block, the received residual residual image, for each macroblock or block in the primary residual image, according to the corresponding position block in the encoded image The primary motion vector finds the corresponding block in the secondary reference image, and uses the block as the starting point of the search center to find the best matching block.

 As can be seen from the foregoing solution, the present invention predicts a residual image generated by predicting and encoding the current image to be encoded by using a residual image corresponding to some of the already encoded images in each video sequence as a reference image, and then The secondary residual image obtained after the prediction is subjected to a subsequent encoding operation, thereby eliminating the correlation between the residual images, further reducing the redundancy between the images, reducing the data amount of the code stream, and improving the encoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

 Figure 1 is a sequence diagram of a video sequence taken by multiple cameras.

 FIG. 2 is a schematic diagram of the relationship between the current image to be encoded and the reference image.

FIG. 3 is a flowchart of an image encoding method according to Embodiment 1 of the present invention. FIG. 4 is a schematic diagram showing the relationship between four images in the second prediction in the flow shown in FIG. 3. FIG. 5 is a flow chart of image reconstruction in the flow shown in FIG.

 FIG. 6 is a flowchart of an image decoding method according to Embodiment 1 of the present invention.

 FIG. 7 is a schematic diagram showing the relationship between an encoder and a decoder according to Embodiment 1 of the present invention.

 FIG. 8 is a schematic structural diagram of an encoder according to Embodiment 1 of the present invention.

 FIG. 9 is a schematic structural diagram of a decoder according to Embodiment 1 of the present invention.

 FIG. 10 is a flowchart of an image encoding method according to Embodiment 2 of the present invention.

 Figure 11 is a flow chart of image reconstruction in the flow shown in Figure 10.

 FIG. 12 is a flowchart of an image decoding method according to Embodiment 2 of the present invention.

 FIG. 13 is a schematic structural diagram of an encoder according to Embodiment 2 of the present invention.

 FIG. 14 is a schematic structural diagram of a decoder according to Embodiment 2 of the present invention.

 FIG. 15 is a schematic structural diagram of an encoder according to Embodiment 3 of the present invention.

 FIG. 16 is a schematic structural diagram of a decoder in Embodiment 3 of the present invention.

 Fig. 17 is a codec assembled on the basis of the encoder shown in Fig. 8 and the decoder shown in Fig. 9. Mode for carrying out the invention

 In the embodiment of the present invention, the strong correlation between the video sequences captured by the plurality of cameras and the residual images generated by the different video sequences after the predictive coding are still highly correlated. Selecting an image with strong correlation with the current image to be encoded, and using the residual image corresponding to the selected image as a reference image, predicting the residual image corresponding to the current image to be encoded, and then predicting The obtained quadratic residual image is subjected to a subsequent encoding operation, thereby eliminating the correlation between the residual images, further reducing the redundancy between the images, and reducing the amount of data.

For the sake of clarity of description, the original image will be inter-predicted in the embodiment of the present invention. And the process of generating the residual image, that is, the motion estimation, called the first prediction, the generated residual image is called the primary residual image, and the corresponding motion vector is called the primary motion vector, and the corresponding residual block is once again. Reference image, primary motion compensation, etc.; in the present invention, on the primary prediction, the inter prediction of the primary residual image with the primary residual image corresponding to the encoded image as the reference image is called secondary prediction. The generated residual image is called a quadratic residual image, and the corresponding motion vector is called a secondary motion vector, and correspondingly, a secondary residual block, a secondary reference image, a secondary motion compensation, and the like.

 In the second prediction of the primary residual image, the embodiment of the present invention adopts two implementation manners, namely:

 In the first method, the primary residual image is directly predicted twice, and then the secondary residual image generated by the secondary prediction is transformed or not transformed, and then the subsequent motion operation is performed with the primary motion vector and the secondary motion vector.

 In the second method, the primary residual image is first transformed, the transformed primary residual image is subjected to secondary prediction, and then the secondary residual image generated by the secondary prediction is further followed by the primary motion vector and the secondary motion vector. Encoding operation.

 That is to say, for the second prediction, the first method is to predict the retransformation first, or to predict only the non-transformation; the second method is to transform and re-predict.

 In the following embodiments, the encoding method, the decoding method, the encoder, the decoder, the encoding and decoding method, and the codec in the embodiment of the present invention are described in detail by taking the case of simultaneously encoding a plurality of video sequences as an example.

For multiple video sequences captured by different cameras, they can be arranged into a two-dimensional image array. As shown in Figure 1, Figure 1 is a sequence diagram of video sequences captured by multiple cameras. The abscissa in Figure 1 indicates the shooting of different cameras. The video sequence, view 1 represents video sequence 1, view 2 represents video sequence 2, and so on, four video sequences are listed in Figure 1, and the ordinate indicates that each video sequence is also increasing over time. Among them, in a short interval Inside, especially at the same time, there is a strong correlation between multiple images captured by different cameras. In extreme cases, the video sequences captured by different cameras may be identical. Therefore, the reference image can not only select the encoded image in the current video sequence, but also select the encoded image in other video sequences, as shown in FIG. 2, FIG. 2 is a schematic diagram of the relationship between the current image to be encoded and the reference image. For the video image A2 taken at the time of the video images A1 and T2 taken by the camera A at time T1, and the video image B2 taken at the time of the video images B1 and T2 taken by the camera B at time T1, it is assumed that Al, A2, and B1 have been encoded. When encoding B2, B1 in the same video sequence may be selected as the reference image, or A1 or A2 in the different video sequence may be selected as the reference image. In practical applications, there may be many reference images, and other images in the same video sequence or in other video sequences may also be selected.

 Embodiment 1:

 In this embodiment, the implementation manner in the first method is described in detail. That is, the primary residual image generated by the primary prediction is directly predicted, and the secondary residual image generated by the secondary prediction is subjected to subsequent encoding processing. FIG. 3 is a flow chart of an image encoding method according to Embodiment 1 of the present invention. In general, when inter-prediction is performed on the original image of the current image to be encoded, the prediction is performed in units of macroblocks or blocks, and the flag of the end of the current image to be encoded is carried in the last block, and then starts. Perform the same inter prediction on the next image to be encoded. Therefore, the image encoding method in this embodiment takes the encoding process of one macroblock of the current image to be encoded as an example, and the encoding method of the present invention is described in detail. As shown in Figure 3, the process includes the following steps:

 Step 301: Perform inter prediction on the current block to be coded, that is, one prediction, and obtain a residual block and a motion vector.

In this step, when inter-frame prediction is performed, the method provided in the H.264/AVC standard may be used, or the methods in other standards may be used. In general, the reference image set is given first, like the first few frames in the video sequence, or several frames in the adjacent video sequence, etc. According to the specific situation, the reference image may be selected according to different manners, where reference is made here. The image, that is, the primary reference image, is the reconstructed image of the encoded image. For the specific image reconstruction process, refer to the flow shown in FIG. 4 later. Then, the best matching block of the current block to be coded is traversed in the reference image set, and the block to be encoded is predicted by using the best matching block to obtain a residual block and a motion vector. Wherein, if the reference image of the best matching block is the reconstructed image of the encoded image at the same time in other video sequences, the obtained vector is called the disparity vector, and the motion vector and the disparity vector refer to the prediction block of the current block in the reference image. The displacement relative to the current block. For the sake of brevity, in general, motion vectors and disparity vectors are collectively referred to as motion vectors.

 The four images shown in FIG. 2 are still taken as an example. If the current image to be encoded is B2, the reconstructed image of Al, A2, and B1 can be used as a reference image when encoding a macroblock N of B2. Motion estimation is performed in each reference image, that is, one prediction, and finally the best matching block K is found in B1, and the residual block obtained once for prediction, that is, the first residual block is N' = N - K , and the corresponding motion The vector, that is, the primary motion vector is (Kx, Ky), and the primary motion vector is stored for subsequent encoding and reconstruction of the image, and step 302 is performed. The same prediction method for other macroblocks is the same.

 Step 302: Perform inter prediction, that is, secondary prediction on the obtained primary residual block, and obtain a quadratic residual block and a secondary motion vector.

 In this step, in order to eliminate the correlation between the residual images, the residual image is subjected to secondary prediction using the residual image of the encoded image as a reference image, that is, a secondary reference image. The prediction method is similar to one prediction. The reference image here is a reconstructed image of the residual image corresponding to the encoded image. For the specific image reconstruction process, refer to the flow shown in FIG. 4 later.

Still taking the four images shown in FIG. 2 as an example, it is assumed that A2 uses the reconstructed image of A1 as a reference image to obtain a residual image Α2', and B2 uses the reconstructed image of B1 as a reference image. Go to the residual image B2' once. Figure 4 is a diagram showing the relationship between four images in the second prediction. As shown in FIG. 4, in this step, the first residual block N' in B2' is predicted with A2' as the reference image, and the best matching block M is found in A2', and the quadratic residual block is obtained. N" = Ν' - Μ , the corresponding motion vector, that is, the secondary motion vector is (Mx, My), and the secondary motion vector is stored for subsequent encoding and reconstruction of the image, and step 303 is performed. A2' is a reconstructed image of the coded image corresponding residual image. The second prediction method for other macroblocks is the same.

 Step 303: Perform transform on the obtained quadratic residual block, and entropy encode the transformed transform coefficient matrix and the secondary motion vector obtained by one prediction and the secondary motion, and write the compressed code together In the stream; image reconstruction is performed simultaneously using the transformed coefficient matrix as a reference image of other images to be encoded.

 In this step, when the quadratic residual block is transformed, including transform and quantization, and entropy coding may be performed, operations such as reordering the transformed coefficient matrix may be included, and the specific implementation manner is various in the prior art. It is not covered here. Among them, entropy coding only uses a more method, and in practice, there may be other coding methods.

 At this point, the encoding process of the macroblock ends. In an actual application, after the encoding of the macroblock is completed, the current image to be encoded may determine whether there is a macroblock in the image that is not subjected to the foregoing encoding, and if yes, continue the process of step 301 to step 303; otherwise, the frame The image encoding ends. The subsequent frame image continues to perform the above process as the current image to be encoded. The process of image reconstruction using the transformed transform coefficient matrix refers to the flow shown in FIG. 5 below.

 FIG. 5 is a schematic diagram of an image reconstruction process in the image encoding method shown in FIG. 3. Taking the reconstruction process of a macroblock of the current image to be decoded as an example, the process includes the following steps:

Step 501: Perform inverse transformation on the transformed transform coefficient matrix to obtain a quadratic residual block. In this step, when inverse transform is performed on the transformed transform coefficient matrix, inverse quantization and inverse transform are included. The inverse transform here corresponds to the transform at the time of encoding in step 303.

 Step 502: According to the secondary motion vector obtained in step 302, perform motion compensation on the obtained secondary residual block as a residual image of the reference image, that is, in the secondary reference image, to obtain a residual. Block, and store the one-time residual block.

 In this step, the four images shown in FIG. 2 are still taken as an example, and assuming that the macroblock N of the current image to be encoded B2 is reconstructed, the quadratic residual block obtained after the inverse transform is obtained.

N", according to the secondary motion vector (Mx, My) corresponding to N" stored in step 302, the corresponding compensation block M is found from the previously reconstructed reference residual image A2', and then the residual block N is obtained once. ' = N" + M , the primary residual block is stored, and step 503 is performed. The current reconstruction process of other macroblocks is the same.

 When the quadratic residual blocks of all the macroblocks of B2 are reconstructed, the reconstructed image of the residual image B2' is obtained as the second reference image of the other image to be encoded according to the stored residual block of B2.

 Step 503: Perform motion compensation on the obtained primary residual residual block as a reference image in the encoding according to the primary motion vector obtained in step 301, obtain a reconstructed image block, and store the reconstructed image block.

 In this step, when decoding the residual block N' of B2' in step 303, the compensation block K is found from the previously reconstructed reference image B1 according to the primary motion vector (Kx, Ky) stored in step 301. A reconstructed image is obtained 3⁄4 Ν = Ν' + Κ , and the reconstructed image block 存储 is stored. This reconstruction process for other macroblocks is the same.

When all the residual blocks of all the macroblocks of Β2 are reconstructed, according to all the reconstructed image blocks of Β2 stored, the reconstructed image of image Β2 is obtained as a reference image of other images to be encoded. In practical applications, after encoding the image, it may be stored in the optical disc, in the hard disk, or in the network, or directly transmitted to the receiving end, but in any case, when we need to use or view these images, we need Decode it before it can be used. The following will be combined with the map

6 The image decoding method in the present invention will be described in detail.

 FIG. 6 is a flowchart of an image decoding method according to Embodiment 1 of the present invention. Here, the decoding process of the macroblock of the current image to be decoded is taken as an example, and the decoding method of the present invention is described in detail. As shown in Figure 6, the process includes the following steps:

 Step 601: Entropy decoding the compressed code stream to obtain a motion vector and a quadratic motion vector, and inverse transforming the entropy decoded transform coefficient matrix to obtain a quadratic residual block.

 In this step, the decoding method corresponds to the encoding method. After entropy decoding the compressed code stream, the motion vector and the secondary motion vector are first obtained. Then, the entropy decoded image data can be reordered, and the sorted The image data is inverse transformed, wherein the inverse transform process may include inverse quantization, inverse transform, etc., to obtain a current block to be decoded of the image to be decoded, that is, a secondary residual block.

 Step 602: Perform motion compensation on the obtained residual residual block as a residual image of the secondary reference image at the time of encoding according to the obtained secondary motion vector, obtain a residual block, and obtain the primary residual. The block is stored.

 In this step, the four images shown in FIG. 2 are still taken as an example, and it is assumed that the current image to be decoded is B2, and when the macroblock N is decoded, the second obtained after decoding from the compressed code stream The residual block N", according to the decoded N" corresponding secondary motion vector (Mx, My), finds the corresponding compensation block M from the previously decoded residual image A2', and then obtains a residual Block N' = N" + M , the primary residual block is stored, and step 603 is performed. The current decoding process of other macroblocks is the same.

When the secondary residual block of all macroblocks of B2 is decoded, ^^ is stored according to all the stored B2 One residual block, a reconstructed image of the residual image is obtained, which can be used as a secondary reference image of other images to be decoded.

 Wherein, A2' is a secondary reference image at the time of encoding. At this time, as the secondary reference image at the time of decoding, the previous decoded image A2 is obtained when the step 602 is performed at the time of decoding, that is, after the A2 corresponding residual image is decoded. Rebuild the image.

 Step 603: Perform motion compensation on the obtained primary residual vector for the obtained primary residual block as the primary reference image at the time of encoding, obtain a reconstructed image block, and store the reconstructed image block.

 In this step, when decoding the residual block N' of B2' in step 602, the primary motion vector (Kx, Ky) corresponding to N' obtained by decoding is found from the previously decoded reference image B1. Compensating block K, obtaining a reconstructed image 3⁄4 N = N' + K , and storing the reconstructed image block. This decoding process of other macroblocks is the same.

 So far, the decoding process of the current block to be decoded ends. When all the residual blocks of all macroblocks of B2 are decoded, the reconstructed image of the original image is obtained for subsequent use according to all the reconstructed image blocks of B2 stored. The reconstructed image obtained at the same time can also serve as a reference image for other images to be decoded.

 Here, B1 is a primary reference image at the time of encoding, and is obtained as a primary reference image at the time of decoding, and is obtained by performing the step 603 at the time of decoding by the prior decoded image A2, that is, the reconstructed image after A2 decoding.

 The image encoding method and the decoding method in the present invention have been described in detail above, and a detailed description will be given below of the codec system of the present invention. Fig. 7 is a diagram showing the relationship between a codec and a decoder based on the above image encoding method and decoding method in the first embodiment.

The encoder is used to encode the original image to generate a compressed code stream for subsequent transmission and storage. The encoding process includes: performing a prediction on the current image to be encoded, and obtaining once The residual image and the primary motion vector, the residual image corresponding to the encoded image is used as a reference image, and the obtained primary residual image is subjected to secondary prediction to obtain a secondary residual image and a secondary motion vector, and the second residual is obtained. The difference image and the corresponding secondary motion vector and the primary motion vector are subjected to subsequent encoding processing.

 The decoder is used to decode the compressed code stream of the image to obtain a reconstructed image of the original image for subsequent use. The decoding process includes: performing motion compensation on the quadratic residual image solved from the code stream from the reconstructed residual image as the secondary reference image at the time of encoding according to the secondary motion vector solved from the code stream The primary residual image is motion-compensated from the reconstructed image as the primary reference image from the time of encoding based on the primary motion vector solved from the code stream to obtain a reconstructed image of the original image.

 The internal structure of the encoder and the decoder are shown in Figures 8 and 9, respectively.

 FIG. 8 is a schematic structural diagram of an encoder according to Embodiment 1 of the present invention. As shown in FIG. 8, the encoder includes: a primary prediction module, a secondary prediction module, a transformation module, an encoding module, an inverse transformation module, a secondary reconstruction module, and a primary reconstruction module.

 Moreover, the encoding process is: current image to be encoded → one prediction module → secondary prediction module → transformation module → coding module → code stream; the process of reconstruction is: inverse transformation module → secondary reconstruction module → primary reconstruction module → current image Rebuild the image. The primary motion vector generated by one prediction module and the secondary motion vector generated by the secondary prediction module are directly output to the coding module, and the secondary motion vector generated by the secondary prediction module is used in the secondary reconstruction, and is used in one reconstruction. Is a motion vector generated by a prediction module.

The primary prediction module is configured to read the reference image set once from the primary reconstruction module, and select the best matching block from the reference image set for the current to-be-coded block in units of macroblocks or blocks. The selected best matching block predicts the current block to be coded, obtains a residual block and a corresponding primary motion vector, outputs the primary residual block to the secondary prediction module, and outputs the primary motion vector to the coding module, and simultaneously One motion vector Stored for use by a rebuild module.

 The secondary prediction module is configured to receive a primary residual block from the primary prediction module, read the secondary reference image set from the secondary reconstruction module, and select the best match from the secondary reference image set for the received primary residual block. Block, predicting the primary residual block by using the selected best matching block, obtaining a secondary residual block and a secondary motion vector, outputting the secondary residual block to the transform module, and outputting the secondary motion vector to the coding The module stores the secondary motion vector for use by the secondary reconstruction module.

 The transform module is configured to receive the secondary residual block from the secondary prediction module, transform and quantize the received secondary residual block, further compress the image code rate, and output the transformed and quantized transform coefficient matrix to Encoding module and inverse transform module.

 The encoding module may include an encoding process such as reordering and entropy encoding for receiving a transform coefficient matrix from the transform module, together with the primary motion vector from the primary prediction module and the secondary motion vector from the secondary prediction module, entropy encoding, writing In the code stream.

 The primary reference image and the secondary reference image used in the two modules of the primary prediction module and the secondary prediction module are respectively a reconstructed image of the encoded image and a reconstructed image of the corresponding image of the encoded image, and are currently encoded When the image is encoded, in order to provide a reference image for the subsequent encoded image, it is also necessary to reconstruct the encoded image of the current image to be encoded, so the decoder includes the inverse transform module described above, that is, the secondary reconstruction module and one reconstruction. Module.

 The inverse transform module is configured to receive the transformed and quantized transform coefficient matrix from the transform module, and perform inverse quantization and inverse transform on the received transform coefficient matrix to obtain a quadratic residual block of the current encoded image, and output the same to the second Secondary reconstruction module.

The quadratic reconstruction module is configured to receive the secondary residual block from the inverse transform module, and read the secondary motion vector stored by the secondary prediction module, according to the read secondary motion vector, the secondary reference at the time of encoding Motion compensation in the image, get a residual block output to a re-modeling Block, and store the one-time residual block.

 If all the secondary residual blocks of the current coded image are reconstructed, the first residual image of the current coded image is obtained according to all the first time residual blocks of the stored current coded image, and can be used as a secondary reference image required for other predictions. . If the set of secondary reference images is the first five frames of images, the five frames of images required for the second prediction of the next image to be encoded may be the images of the first four frames plus the reconstructed image, the first frame. The image can now be deleted.

 The primary reconstruction module is configured to receive a primary residual block from the secondary reconstruction module, and read a motion vector stored by the prediction module once, and perform motion compensation in a reference image during encoding according to the read one motion vector. , the reconstructed image block is obtained, and the reconstructed image block is stored.

 If all the residual blocks of the current coded picture are reconstructed, the reconstructed image of the current coded picture is obtained from all the reconstructed image blocks of the stored current coded image, which can be used as a reference image for other predictions. If the set of the first reference image is the first five frames of images, the five frames of images required for the next image to be encoded may be the image of the first four frames plus the reconstructed image. The first image of the frame is the first image. Can be deleted.

 When the first image is encoded, the reference image may be empty, that is, when the first image is encoded, the subsequent encoding process may be directly performed without prediction.

 FIG. 9 is a schematic structural diagram of a decoder according to Embodiment 1 of the present invention. As shown in FIG. 9, the decoder includes: a decoding module, an inverse transform module, a secondary reconstruction module, and a primary reconstruction module.

 Moreover, the decoding process is: decoding module → inverse transform module → secondary reconstruction module → primary reconstruction module → reconstructed image of the current image. The secondary motion vector in the case of secondary reconstruction and the primary motion vector in the case of one reconstruction are obtained after decoding of the preceding decoding module.

The decoding module is configured to perform entropy decoding, reordering, and the like on the compressed code stream, to obtain a primary motion vector, a secondary motion vector, and a quadratic residual block transform coefficient matrix of the current to-be-decoded block in the current image to be decoded, and The quadratic residual block transform coefficient matrix is output to the inverse transform module, The secondary motion vector is output to the secondary reconstruction module, and the primary motion vector is output to the reconstruction module once.

 The inverse transform module is configured to receive a quadratic residual block transform coefficient matrix from the decoding module, and perform inverse quantization and inverse transform on the received quadratic residual block transform coefficient matrix to obtain a quadratic residual block, and output the second residual block. Rebuild the module.

 The quadratic reconstruction module is configured to receive the secondary motion vector from the decoding module and the secondary residual block from the inverse transform module, and according to the received secondary motion vector, the secondary reference of the secondary residual block at the time of encoding Motion compensation is performed in the image, and a residual block is obtained, which is output to a reconstruction module, and the primary residual block is stored.

 If all the secondary residual blocks of the current to-be-decoded image are reconstructed, the first residual image of the current image to be decoded is obtained according to the stored first residual residual block of the currently-to-be-decoded image, which can be used as the second required for other decoding. Secondary reference image.

 The primary reconstruction module is configured to receive a primary motion vector from the decoding module and a primary residual block from the primary reconstruction module, and perform motion compensation on a reference image of the primary residual block at the time of encoding according to the received primary motion vector. A reconstructed image block is obtained and the reconstructed image block is saved.

 If all the residual residual blocks of the current image to be decoded are reconstructed, the reconstructed image of the original image is obtained for subsequent use according to all the reconstructed image blocks of the stored current image to be decoded. The reconstructed image obtained at the same time can also serve as a reference image for other images to be decoded.

 The general description of the encoding method, the decoding method, and the codec system of the present invention has been described above with reference to the embodiments. In practical applications, in the second prediction of the residual block N' in step 302, in order to find the best matching block M in A2', the motion vector (Mx, My can be obtained without directly using A2' as the reference image. ).

Because if there is B1 with A1 as the reference image, a prediction is made, and it is assumed that the macroblock corresponding to the position of the macroblock N of B2 in B1 is N1, and the best match of N1 in A1 The block is M1, the corresponding primary disparity vector is (Mlx, Mly), and the macroblock of the corresponding position of M1 in B2 is M, then the quadratic residual block is Ν" = Ν'-M, and can be once The disparity vector (Mlx, Mly) is used as the quadratic disparity vector (Mx, My) of N' in A2'. If the first disparity vector (Mlx, Mly) has been written into the code stream, it is not necessary to write the code stream again this time. Let B1 make a prediction with A1 as the reference image, but only when searching for the best reference image, then the first disparity vector (Mlx, Mly) generated in the prediction can be retained for the second prediction. The primary disparity vector (Mlx, Mly) is written into the code stream as a quadratic disparity vector of the second prediction. This method can also be used for other macroblocks.

 Similarly, when reconstructing in step 502, or decoding in step 602, all the disparity vectors of image B1 in reference image A1 are preserved. When decoding the current image B2 to be decoded macroblock N, decoding is first obtained to obtain a motion vector ( Kx, Ky), and according to the secondary residual block N" obtained by decoding and the disparity vector (Mlx, Mly) corresponding to the macroblock N1 of the corresponding position of the macroblock N in B1, (Mlx, Mly) is taken as N The secondary motion vector finds the corresponding compensation block M in the reference image A2', and obtains a residual block N' = N" + M, and then finds the compensation block K in B1 by the primary motion vector (Kx, Ky). , get the reconstructed image block N = N' + K. This method can also be used for other macroblocks.

 Or after obtaining (Mlx, Mly) in the above process, the vector is not directly used as the disparity vector (Mx, My) of N' in A2', but the macroblock M can be found in A2' according to the disparity vector. Then, using M as the central starting point of the search, searching for the best matching block Μ' near M, and then obtaining the disparity vector (Mx, My) corresponding to M', then decoding is the same as in the first embodiment or the second embodiment. Said method. Of course, M' may also be the same as M. At this time, (Mlx, Mly) can still be used as the disparity vector of N' in A2'.

Correspondingly in the encoder shown in FIG. 8, a parallax vector storage may be further included. a module, configured to store a disparity vector generated in one prediction, and may retrieve a required disparity vector from the disparity vector storage module when performing secondary prediction, as a secondary motion vector, or as a search center of a best matching block point. The specific implementation process is described in the above method.

 Embodiment 2:

 In this embodiment, the implementation manner in the second method is described in detail, that is, before performing the secondary prediction, the residual residual image is first transformed, and then the transformed primary residual image is corresponding to the secondary reference image. The second prediction is performed in the transformed secondary reference image, and then the secondary residual image generated after the secondary prediction is subjected to subsequent encoding processing.

 In this embodiment, in order to distinguish from the mode, the primary residual image generated by transforming the primary residual image is referred to as a primary residual image transform coefficient matrix, and correspondingly, the primary residual block is transformed. The primary residual block is called the primary residual block transform coefficient matrix, and the secondary residual image generated by the secondary prediction of the primary residual image transform coefficient matrix is called a quadratic residual image transform coefficient matrix, and correspondingly, The quadratic residual block generated by the quadratic residual block transform coefficient matrix for secondary prediction is called a quadratic residual block transform coefficient matrix, and there is also a quadratic reference image transform coefficient matrix and the like. Similarly, at the decoding end, the secondary residual image decoded from the compressed code stream is referred to as a quadratic residual image transform coefficient matrix, and the primary residual image generated after the secondary reconstruction of the secondary residual image is called The residual residual image transform coefficient matrix, correspondingly, the quadratic residual block in the quadratic residual image decoded from the compressed code stream is called a quadratic residual block transform coefficient matrix, and the second residual block is performed. The primary residual block generated after the second reconstruction is called the primary residual block transform coefficient matrix.

 FIG. 10 is a flowchart of an image encoding method in Embodiment 2. The image encoding method in this embodiment still takes the encoding process of a macroblock of the current image to be encoded as an example. As shown in FIG. 10, the process includes the following steps:

Step 1001: Perform a prediction on the current block to be coded to obtain a residual block and a motion vector. The specific implementation process in this step may be the same as the step 301 shown in FIG. 3 in the first embodiment. Step 1002: Perform a transformation on the obtained first residual block to obtain a residual block transform coefficient matrix, perform secondary prediction on the primary residual block transform coefficient matrix, obtain a quadratic residual block transform coefficient matrix, and quadratic motion. Vector.

 In this step, the first residual block N' in B2' is taken as an example. First, N' is transformed and quantized to obtain a transformed transform coefficient matrix Ν', and Μ is assumed to be the corresponding maximum in the reference image A2'. For a good matching block, the corresponding quadratic motion vector is (Mx, My), and the transformed transform coefficient matrix of M is ^ Ϊ, then N can be predicted by N , to obtain a quadratic residual block transform coefficient matrix. N ' = - ; or some coefficients of ^ ϊ ' can also be used to predict some coefficients of Ν, such as DC (Direct Current) coefficient of ^ 预测 to predict the DC coefficient of N, by AC (AC , Alternating Current ) Coefficient to predict the AC coefficient of N, etc. The specific prediction method depends on the actual situation. At the same time, the obtained secondary motion vector is stored for use in image reconstruction.

 Step 1003: Entropy coding the quadratic residual block transform coefficient matrix, and the primary motion vector obtained by one prediction and the secondary motion vector obtained by the secondary prediction, and writing together into the compressed code stream; The block transform coefficient matrix performs image reconstruction as a reference image of other images to be encoded.

 The process of image reconstruction using the quadratic residual block transform coefficient matrix refers to the flow shown in FIG. 11 below.

 FIG. 11 is a schematic diagram of an image reconstruction process in the image encoding method shown in FIG. Taking the reconstruction process of a macroblock of the current image to be decoded as an example, the process includes the following steps:

Step 1101, according to the quadratic motion vector obtained in step 1002, transforming the coefficient matrix of the quadratic residual block, and using the quadratic reference image transform coefficient matrix as the reference image during encoding, The line quadratic motion compensation obtains a residual block transform coefficient matrix.

 In this step, the four images shown in FIG. 2 are still taken as an example, and assuming that the macroblock N of the current image to be encoded B2 is reconstructed, the quadratic residual block transform coefficient matrix obtained after the second prediction is obtained. N", according to the corresponding secondary motion vector (Mx, My) stored in step 1002, find the corresponding compensation block ^ 从 from the transform coefficient matrix of the previously reconstructed reference residual image A2', and then obtain a residual The difference block transform coefficient matrix N = " + . This reconstruction process for other macroblocks is the same.

 Step 1102: Perform inverse transformation on the obtained primary residual block transform coefficient matrix to obtain a primary residual block, and store the obtained primary residual block.

 In this step, when the inverse transform of the transformed transform coefficient matrix is performed, corresponding to the transform at the time of encoding in step 1002, N1 is obtained by inverse transform in step 1101.

 When the quadratic residual block transform coefficient matrix of all macroblocks of B2 is reconstructed, a reconstructed image of the residual image B2' is obtained according to all stored residual blocks of B2, as a secondary reference of other images to be encoded. image.

 Step 1103, according to the primary motion vector obtained in step 1001, perform motion compensation on the obtained primary residual block as a reference image in the reference image at the time of encoding, obtain a reconstructed image block, and store the reconstructed image block. .

 In this step, when decoding the residual block N' of B2' in step 1102, the compensation block K is found from the previously reconstructed reference image B1 according to the primary motion vector (Kx, Ky) stored in step 1001. A reconstructed image is obtained 3⁄4 N = N' + K and the reconstructed image block N is stored. This reconstruction process for other macroblocks is the same.

When all the residual blocks of all macroblocks of B2 are reconstructed, according to all the reconstructed image blocks of B2 stored, the reconstructed image of image B2 is obtained as a reference image of other images to be encoded. FIG. 12 is a flowchart of an image decoding method according to Embodiment 2 of the present invention. Here, the decoding process of one macroblock of the current image to be decoded is taken as an example. As shown in FIG. 12, the process includes the following steps:

 Step 1201: Entropy decoding the compressed code stream to obtain a motion vector and a quadratic motion vector, and a quadratic residual block transform coefficient matrix.

 In this step, the decoding method corresponds to the encoding method, and after performing entropy decoding on the compressed code stream, a motion vector and a secondary motion vector are obtained, and the current block to be decoded of the image to be decoded, that is, a quadratic residual block transform. Coefficient matrix.

 Step 1202: According to the obtained quadratic motion vector, perform motion compensation on the obtained quadratic residual block transform coefficient matrix, and perform motion compensation in the residual image transform coefficient matrix of the secondary reference image at the time of encoding, to obtain a residual block transform. Coefficient matrix.

 In this step, the four images shown in FIG. 2 are still taken as an example, and it is assumed that the current image to be decoded is B2, and when the macroblock N is decoded, the second obtained after decoding from the compressed code stream The residual block transform coefficient matrix, according to the corresponding quadratic motion vector (Mx, My) obtained by decoding, finds the corresponding compensation block ^ 中 from the transform coefficient matrix of the previously decoded residual image A2', and then Obtain a residual block transform coefficient matrix N = " + . This decoding process of other macroblocks is the same.

 Wherein, A2' is a secondary reference image at the time of encoding. At this time, as the secondary reference image at the time of decoding, the previous decoded image A2 is obtained when the step 1203 is performed at the time of decoding, that is, after the A2 corresponding residual image is decoded. Rebuild the image. When this step is performed, A2' is first transformed to obtain a transform coefficient matrix of A2'.

 Step 1203: Perform inverse transformation on the obtained primary residual block transform coefficient matrix to obtain a primary residual block, and store the obtained primary residual block.

In this step, the inverse transformation of the obtained residual block transform coefficient matrix corresponds to the step In the conversion at the time of encoding in step 1002, N is inversely transformed in step 1202 to obtain N'. When the quadratic residual block transform coefficient matrix of all macroblocks of B2 is decoded, a reconstructed image of the residual image B2' is obtained according to all stored residual blocks of B2, as a secondary reference of other images to be decoded. image.

 Step 1204: According to the obtained primary motion vector, perform motion compensation on the obtained primary residual block as an image of the primary reference image at the time of encoding, obtain a reconstructed image block, and store the reconstructed image block.

 In this step, when decoding the residual block N' of B2' in step 1203, the primary motion vector (Kx, Ky) corresponding to N' obtained by decoding is searched from the previously decoded reference image B1. To the compensation block K, a reconstructed image block N = N' + K is obtained, and the reconstructed image block is stored. This decoding process of other macroblocks is the same.

 So far, the decoding process of the current block to be decoded ends. When all the residual blocks of all macroblocks of B2 are decoded, the reconstructed image of the original image is obtained for subsequent use according to all the reconstructed image blocks of B2 stored. The reconstructed image obtained at the same time can also serve as a reference image for other images to be decoded.

 Further, the image encoding method and the decoding method in the present invention are described in detail. Hereinafter, the encoder and the decoder based on the image encoding method and the decoding method in the present invention will be described in detail.

 FIG. 13 is a schematic structural diagram of an encoder according to Embodiment 2 of the present invention. As shown in FIG. 13, the encoder includes: a primary prediction module, a transformation module, a secondary prediction module, an encoding module, a secondary reconstruction module, an inverse transformation module, and a primary reconstruction module.

Moreover, the encoding process is: current image to be encoded → one prediction module → transformation module → secondary prediction module → encoding module → code stream; the process of reconstruction is: secondary reconstruction module → inverse transformation module → primary reconstruction module → current image Rebuild the image. One of the prediction modules is generated The secondary motion vector generated by the primary motion vector and the secondary prediction module is directly output to the coding module, and the secondary motion vector generated by the secondary prediction module is used in the secondary reconstruction, and the primary prediction vector is used in one reconstruction. The resulting motion vector.

 The primary prediction module is configured to read the reference image set once from the primary reconstruction module, and select the best matching block from the reference image set for the current to-be-coded block in units of macroblocks or blocks. The selected best matching block predicts the current block to be coded, obtains a residual block and a corresponding primary motion vector, outputs a residual block to the transform module, and outputs a motion vector to the coding module, and simultaneously performs the motion. The vector is stored for use by a rebuild module.

 The transform module is configured to receive a residual block from the primary prediction module, and transform the received residual residual block to obtain a residual block transform coefficient matrix output to the secondary prediction module, and the secondary reconstruction module The read secondary reference image is transformed to obtain a quadratic reference image transform coefficient matrix, which is output to the secondary prediction module.

 The secondary prediction module is configured to receive a primary residual block transform coefficient matrix and a secondary reference image transform coefficient matrix from the primary prediction module, and use the transform coefficient matrix of the best matching block in the quadratic reference image transform coefficient matrix to the primary residual The block transform coefficient matrix is predicted, and a quadratic residual block transform coefficient matrix and a quadratic motion vector are obtained, which are output to an encoding module and a quadratic reconstruction module. At the same time, the secondary motion vector is stored for use by the secondary reconstruction module.

 The encoding module may include an encoding process such as reordering and entropy encoding, and is configured to receive a quadratic residual block transform coefficient matrix and a secondary motion vector from the secondary prediction module, and perform entropy encoding together with a primary motion vector from the primary prediction module. Write to the stream.

The quadratic reconstruction module is configured to receive a quadratic residual block transform coefficient matrix from the secondary prediction module, and transform the coefficient matrix of the received quadratic residual block according to the secondary motion vector read from the secondary prediction module, The motion compensation is performed in the quadratic reference image transform coefficient matrix at the time of encoding, and the residual block coefficient matrix output is obtained to the inverse transform module. The inverse transform module is configured to receive a residual block transform coefficient matrix from the quadratic reconstruction module, and inversely transform the received residual block transform coefficient matrix to obtain a residual block, which is output to a reconstruction module, and This one-time residual block is stored.

 If all the quadratic residual block transform coefficient arrays of the current coded image are reconstructed and the inverse transform ends, the first residual image of the current coded image is obtained according to all the first residual blocks of the stored current coded image, and can be used as other predictions. The required secondary reference image.

 The primary reconstruction module is configured to receive a primary residual block from the inverse transform module, and read a motion vector stored by the prediction module once, and perform motion compensation in a reference image during encoding according to the read one motion vector. A reconstructed image block is obtained and the reconstructed image block is stored.

 If all the residual blocks of the current coded picture are reconstructed, the reconstructed image of the current coded picture is obtained from all the reconstructed image blocks of the stored current coded image, which can be used as a reference image for other predictions.

 When the first image is encoded, the reference image may be empty, that is, when the first image is encoded, the subsequent encoding process may be directly performed without prediction.

 FIG. 14 is a schematic structural diagram of a decoder according to Embodiment 2 of the present invention. As shown in FIG. 14, the decoder includes: a decoding module, a quadratic reconstruction module, an inverse transform module, a primary reconstruction module, and a transformation module.

 Moreover, the decoding process is: decoding module→secondary reconstruction module→inverse transformation module→one reconstruction module→reconstructed image of the current image. The secondary motion vector in the case of secondary reconstruction and the primary motion vector in the case of one reconstruction are obtained after decoding by the decoding module.

The decoding module is configured to perform entropy decoding on the compressed code stream to obtain a primary motion vector, a secondary motion vector, and a quadratic residual block transform coefficient matrix of the current to-be-decoded block in the current image to be decoded, and the second residual The difference block transform coefficient matrix and the quadratic motion vector are output to the quadratic reconstruction module, and the primary motion vector is output to the reconstruction module once. The quadratic reconstruction module is configured to receive a quadratic residual block transform coefficient matrix and a quadratic motion vector from the encoding module, and receive the quadratic reference image transform coefficient matrix when the quadratic residual block transform coefficient matrix encoding from the transform module is received According to the received quadratic motion vector, the quadratic residual block transform coefficient matrix is motion-compensated in the quadratic reference image transform coefficient matrix at the time of encoding, and a residual block coefficient matrix is obtained, which is output to the inverse transform module.

 The secondary reference image transform coefficient matrix is a primary residual image that has been decoded after being subjected to the secondary reconstruction module and the inverse transform module, and the transform module is configured to read the primary residual image and read the residual once. The image is transformed, and the transformed transform coefficient matrix is output to the quadratic reconstruction module.

 The inverse transform module is configured to receive a residual block transform coefficient matrix from the quadratic reconstruction module, and inversely transform the received residual block transform coefficient matrix to obtain a residual block, which is output to a reconstructed image, and This one-time residual block is stored.

 If all the secondary residual block transform coefficient arrays of the current image to be decoded are reconstructed and the inverse transform ends, the first residual image of the current image to be decoded is obtained according to the stored first residual residual block of the currently decoded image, As a secondary reference image required for other decoding.

 The primary reconstruction module is configured to receive a primary residual block from the inverse transform module, and read a motion vector stored by the prediction module once, and perform motion compensation in a reference image during encoding according to the read one motion vector. A reconstructed image block is obtained and the reconstructed image block is stored.

 If all the residual blocks of the current image to be decoded are reconstructed, the reconstructed image of the current image to be decoded is obtained according to all the reconstructed image blocks of the currently decoded image to be decoded, and can be used as a reference image for other decoding.

The encoder in the second embodiment obtains a residual block transform coefficient matrix after the second reconstruction module in the process of reconstructing the image, and outputs the primary residual block transform coefficient matrix to the inverse transform module. At the same time, it can be stored and based on the current compilation All the residual block transform coefficient arrays of the code image are obtained, and a residual image transform coefficient matrix is obtained. As a secondary reference image transform coefficient matrix of the input quadratic prediction module, the transform module transforms the first residual block transform coefficient matrix. After that, it is not necessary to store it, and the quadratic reference image transform coefficient matrix does not need to be obtained by transforming the secondary reference image through the transform module. Similarly, in the decoder, the same processing can be performed, and the transform module does not need to be included in the decoder.

 In this embodiment, when performing transform or when transform module performs transform, it may include transform and quantization, and may perform re-sequencing and entropy coding, or other coding modes when performing subsequent coding or coding module execution; Or when the inverse transform module performs the inverse transform, corresponding to the transform or transform module, may also include inverse quantization and inverse transform, when pre-decoding, or when the decoding module performs decoding, corresponding to the subsequent encoding or encoding module, may include Entropy decoding and reordering, or other decoding methods.

 Embodiment 3

In this embodiment, the image encoding method and the decoding method in the first embodiment and the image encoding method and the decoding method in the second embodiment can be simultaneously applied. Correspondingly, the encoder and the decoder have the two implementation functions of the encoder and the decoder in the first embodiment and the second embodiment, and the implementation manner in the first embodiment is added to the second embodiment. . FIG. 15 is a schematic structural diagram of an encoder according to Embodiment 3 of the present invention. The encoder comprises: a primary prediction module, a transformation module, a secondary prediction module, an encoding module, a secondary reconstruction module, an inverse transformation module, and a primary reconstruction module. The coding process represented by the dotted line part is included: current image to be coded → primary prediction module → secondary prediction module → transformation module → coding module → code stream; reconstruction process: inverse transformation module → secondary reconstruction module → primary reconstruction module → current The reconstructed image of the image. And the coding process represented by the solid line part: current image to be encoded → one prediction module → transformation module → secondary prediction module → coding module → code stream; reconstruction process: secondary reconstruction module → inverse transformation module → primary reconstruction module → current The reconstructed image of the image. The primary motion vector generated by one prediction module and the secondary motion vector generated by the secondary prediction module are directly output to the coding module, and the secondary motion vector generated by the secondary prediction module is used in the secondary reconstruction, and is used in one reconstruction. Is a motion vector generated by a prediction module.

 FIG. 16 is a schematic structural diagram of a decoder in Embodiment 3 of the present invention. The decoder comprises: a decoding module, a secondary reconstruction module, an inverse transformation module, a primary reconstruction module, and a transformation module. The decoding process represented by the dotted line portion is included: decoding module → inverse transform module → quadratic re-modeling block → one-time reconstruction module → reconstructed image of the current image. And the decoding process represented by the solid line part: decoding module → secondary prediction module → inverse transform module → primary prediction module → reconstructed image of the current image.

 In the decoding, the secondary motion vector at the time of secondary reconstruction and the primary motion vector at the time of one reconstruction are obtained after decoding by the decoding module.

 Wherein, the encoder and the decoder adopt the flow of the solid line part or the flow of the dotted line part, which may be performed according to actual control, or according to the codec effect, or according to other actual conditions, and which encoding mode is used, which decoding is required In the mode, when the encoder is encoded, the image frame is used as a unit, or a macroblock is used as a unit, a flag bit is set, and which encoding mode is used, the flag bit is used for marking; the decoder decodes the compressed code stream. The corresponding decoding process can be performed according to the decoding flag. The function modules in the encoder and the decoder perform corresponding functions according to the actual implementation manner. For example, when the implementation of the solid line part is adopted, the implementation is performed according to the second embodiment; when the implementation of the dotted line part is adopted, Executed in the first embodiment.

In the actual coding process, there may be other coding methods for the entropy coding step. Similarly, the entropy decoding link may also be other decoding methods, wherein the decoding method corresponds to the coding method. Moreover, the primary motion vector, the secondary motion vector, and the secondary residual image may be directly written into the compressed code stream without performing entropy coding or other encoding processes. In the decoder of the above three embodiments, when the secondary reconstruction is performed, the input secondary residual image may also be a compressed code stream after entropy coding. In this case, a decoding module needs to be added to the encoder. After receiving the compressed code stream generated by the encoding module and decoding the second residual image, the secondary motion vector and the primary motion vector from the compressed code stream, the reconstruction process may be the same as the process in the decoder.

 The image encoding method and the image decoding method provided by the present invention can be used simultaneously to form a codec method, including:

 When encoding, the current image to be encoded is predicted once, and a residual image and a motion vector are obtained once, and a residual image is secondarily predicted to obtain a secondary residual image and a secondary motion vector, and the second residual is obtained. The image, the secondary motion vector, and the primary motion vector are written into the compressed code stream for transmission.

 During decoding, the compressed code stream is received, and a motion vector, a quadratic motion vector, and a quadratic residual image are decoded from the compressed code stream; and the second residual image is motion compensated according to the secondary motion vector to obtain a residual The difference image is subjected to motion compensation for the primary residual image according to the primary motion vector to obtain a reconstructed image.

 For the specific implementation process of encoding and decoding, several methods described in the first embodiment to the third embodiment are also possible.

 The encoder and decoder provided by the present invention can also be assembled into a codec for use. As shown in FIG. 17, FIG. 17 is a codec assembled on the basis of the encoder shown in FIG. 8 and the decoder shown in FIG. 9 in the first embodiment, the codec comprising: a coding unit and a decoding unit. .

The coding unit is configured to perform a prediction on the current image to be encoded, obtain a residual image and a motion vector, and perform a second prediction on the first residual image to obtain a secondary residual image and a secondary motion vector. The quadratic residual image, the secondary motion vector, and the primary motion vector are written into the compressed code stream for transmission. a decoding unit, configured to receive a compressed code stream, and obtain a motion vector, a quadratic motion vector, and a quadratic residual image from the compressed code stream; perform motion compensation on the second residual image according to the secondary motion vector, The residual image is once, and the residual image is motion compensated according to the primary motion vector to obtain a reconstructed image, and the decoding unit is further configured to provide the reconstructed image as a reference image for the encoding unit.

 The coding unit may include: a primary prediction module, a secondary prediction module, a transformation module, and an encoding module; the decoding unit may include: a decoding module, an inverse transform module, a secondary reconstruction module, and a primary reconstruction module.

 The codec further includes a decoding module for the encoder shown in FIG. 8 for decoding a quadratic residual image from the compressed stream generated by the encoding when the reconstructed image of the reference image is provided for the encoding process. a motion vector and a quadratic motion vector, and then inversely transforming the second residual image. When performing image reconstruction, the required secondary motion vector may be a secondary motion vector stored by the secondary prediction module, or may be The secondary motion vector obtained by the decoding module is decoded. Similarly, the required primary motion vector may be a motion vector stored by the prediction module, or may be a motion vector decoded by the decoding module. If the decoding module obtains the motion vector, Then, the primary prediction module and the secondary prediction module may not store the generated primary motion vector and secondary motion vector.

 The codec for the decoder shown in FIG. 9 is mainly decoded by the decoding module, the inverse transform module, the secondary reconstruction module, and the primary reconstruction module, and the functions and connection relationships of the respective modules and FIG. The module functions shown are the same as the connection relationship.

 Currently, the codec shown in FIG. 17 may have other structural modifications, or may not include a transform module and an inverse transform module. The decoding module and the encoding module may also have many encoding modes and decoding modes.

Also, the encoder and decoder in the second embodiment and the third embodiment can be assembled into a codec. And there are also various structural deformations and the like. As can be seen from the above embodiments, the present invention predicts a residual image generated by predicting and encoding an image to be encoded by using a residual image corresponding to some already encoded images in each video sequence as a reference image, and then The secondary residual image obtained after the prediction is subjected to a subsequent encoding operation, thereby eliminating the correlation between the residual images, further reducing the redundancy between the images, reducing the data amount of the code stream, and improving the encoding efficiency.

 In addition, when predicting a residual image generated after the current image to be encoded is subjected to predictive coding, the primary residual image of the current image to be encoded is in units of macroblocks or blocks, and is used as a residual image of the secondary reference image. Finding the best matching block, looking for the best matching block, finding the corresponding block in the secondary reference image by using the first-order disparity vector of the corresponding position block in the encoded image, and using the corresponding block as the best matching block, and The reference one-time disparity vector is used as the quadratic disparity vector in the secondary reference image, so that the data amount of the code stream can be reduced; or after finding the corresponding block, the block is used as the search center starting point to find the best matching block, thereby Further improve coding efficiency.

 Finally, the present invention can also predict the transform coefficient array after one residual image transformation, thereby providing more options for eliminating the correlation between residual images, and making the coding more flexible, thereby maximizing the Eliminate correlation between residual images.

 The above embodiments are merely illustrative of the inventive concept and are not intended to limit the scope of the invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims

Claim

An image encoding method, the method comprising:

 A. Performing a prediction on the current image to be encoded to obtain a residual image and a motion vector;

 B. Perform secondary prediction on the primary residual image to obtain a secondary residual image and a secondary motion vector, and write the secondary residual image, the secondary motion vector, and the primary motion vector into the compressed code stream.

 2. The method according to claim 1, wherein the step B specifically comprises: in a macroblock or a block, for each macroblock or block in the primary residual image, as a secondary reference image Find the best matching block in the residual image set, use the best matching block to predict the current macroblock or block, and obtain the quadratic residual block and the corresponding secondary motion vector, and the second residual block, The secondary motion vector and the primary motion vector are written into the compressed code stream.

 The method according to claim 2, wherein the finding the best matching block in the residual image set as the secondary reference image in step B is: when performing the disparity estimation, according to the corresponding in the encoded image a primary motion vector of the position block, finding a corresponding block in the secondary reference image, using the corresponding block as a best matching block, and using the primary motion vector as a secondary motion vector in the secondary reference image;

 Or

 When the disparity estimation is performed, the corresponding block is found in the secondary reference image according to the primary motion vector of the corresponding position block in the encoded image, and the corresponding block is used as the search center starting point to find the best matching block.

The method according to claim 1, wherein the step B further comprises: transforming each macroblock or block in the primary residual image in units of macroblocks or blocks to obtain a transformed one time. Residual block The step B is: searching for a best matching block in the image after the residual image transformed as the secondary reference image for each transformed residual residual block, and using the best matching block for the current transformed one. The residual block performs prediction, and obtains a quadratic residual block and a corresponding secondary motion vector, and writes the secondary residual block, the secondary motion vector, and the primary motion vector into the compressed code stream.

 The method according to any one of claims 2 to 4, wherein the secondary reference image is: a reconstructed image of the encoded image corresponding to the primary residual image.

 The method according to any one of claims 1 to 4, wherein before the compressing the code stream is written in the step B, the method further comprises: performing the second residual image, the secondary motion vector, and the primary motion vector Entropy coding.

 7. An image decoding method, the method comprising:

 A. Decoding from the compressed code stream to obtain a motion vector, a quadratic motion vector, and a quadratic residual image;

 B. Perform motion compensation on the second residual image according to the secondary motion vector to obtain a residual image, and perform motion compensation on the primary residual image according to the primary motion vector to obtain a reconstructed image.

 The method according to claim 7, wherein the step B is: according to the macroblock or the block, for each macroblock or block in the secondary residual image, according to the macroblock or block Corresponding secondary motion vector, performing motion compensation on the second residual block to obtain a residual block, and performing motion compensation on the primary residual block according to the primary motion vector corresponding to the primary residual block to obtain a reconstructed image Block, synthesize all reconstructed image blocks of the secondary residual image into the reconstructed image.

The method according to claim 8, wherein the step B: before performing motion compensation on the primary residual block according to the primary motion vector corresponding to the primary residual block, further comprising: The difference block is inverse transformed to obtain a first residual block after the inverse transform.

An encoder, comprising: a primary prediction module and an encoding module, wherein the encoder further comprises: a secondary prediction module, wherein

 a prediction module, configured to perform a prediction on the current image to be encoded, to obtain a residual image and a motion vector;

 a secondary prediction module, configured to receive a primary residual image from the primary prediction module, perform secondary prediction on the received primary residual image, and obtain a secondary residual image and a secondary motion vector;

 An encoding module, configured to receive a primary motion vector from the primary prediction module and a secondary motion vector and a secondary residual image from the secondary prediction module, and receive the received primary motion vector, secondary motion vector, and The secondary residual image is written into the compressed code stream.

 The encoder according to claim 10, wherein the encoder further comprises: a quadratic reconstruction module, configured to receive a quadratic residual image from the secondary prediction module, and read a secondary prediction a secondary motion vector obtained in the module, performing motion compensation on the received secondary residual image according to the secondary motion vector, to obtain a primary residual image, wherein the primary residual image is used by the secondary prediction module a reference image at the time of secondary prediction;

 a primary reconstruction module, configured to receive a primary residual image from the secondary reconstruction module, and read a primary motion vector obtained in the prediction module, and perform motion compensation on the received primary residual image according to the primary motion vector And obtaining a reconstructed image, where the reconstructed image is used for a reference image when the primary prediction module performs one prediction.

 The encoder according to claim 11, wherein the encoder further comprises: a transform module and an inverse transform module, where

 a transform module, configured to receive a secondary residual image from the secondary prediction module, transform the received secondary residual image, and output the transformed secondary residual image to the encoding module and the inverse transform module;

An inverse transform module, configured to receive a secondary residual image from the transform module, for the received The quadratic residual image is inversely transformed to obtain an inversely transformed quadratic residual image, which is output to the quadratic reconstruction module.

 The encoder according to claim 11, wherein the encoder further comprises: a transform module and an inverse transform module, where

 a transform module, configured to receive a residual residual image from the primary prediction module, transform the primary residual image, and output the transformed primary residual image to the secondary prediction module;

 The inverse transform module is configured to receive a primary residual image from the secondary reconstruction module, inversely transform the received primary residual image, and output the inverse transformed primary residual image to a reconstruction module.

 A decoder, the decoder comprising: a decoding module, a primary reconstruction module, wherein the decoder further comprises: a secondary reconstruction module, wherein

 a decoding module, configured to decode a motion vector, a secondary motion vector, and a secondary residual image from the compressed code stream;

 a quadratic reconstruction module, configured to receive a secondary residual image and a secondary motion vector from the decoding module, perform motion compensation on the secondary residual image according to the received secondary motion vector, to obtain a residual image;

 And a primary reconstruction module, configured to receive a primary residual image from the secondary reconstruction module and a primary motion vector from the decoding module, and perform motion compensation on the primary residual image according to the received primary motion vector to obtain a reconstructed image.

 The decoder according to claim 14, wherein the decoder further comprises: an inverse transform module, configured to receive a quadratic residual image from the decoding module, and perform inverse on the received quadratic residual image Transform, and output the inverse transformed quadratic residual image to the quadratic reconstruction module.

The decoder according to claim 14, wherein the decoder further comprises: an inverse transform module, configured to receive a primary residual image from the secondary reconstruction module, and The received residual residual image is inversely transformed, and the inverse transformed primary residual image is output to the primary reconstruction module.

 17. A codec method, the method comprising:

 A. Perform a prediction on the current image to be encoded, obtain a residual image and a motion vector, and perform a second prediction on the first residual image to obtain a quadratic residual image and a secondary motion vector, and obtain a quadratic residual image. , the secondary motion vector and the primary motion vector are written into the compressed code stream;

B. receiving the compressed code stream, and decoding a motion vector, a quadratic motion vector, and a quadratic residual image from the compressed code stream; performing motion compensation on the second residual image according to the secondary motion vector, and obtaining once The residual image is subjected to motion compensation for the primary residual image according to the primary motion vector to obtain a reconstructed image.

 18. A codec, the codec comprising: a coding unit and a decoding unit, where

 a coding unit, configured to perform a prediction on the current image to be encoded, obtain a residual image and a motion vector, and perform a second prediction on the first residual image to obtain a secondary residual image and a secondary motion vector, which will be quadratic The residual image, the secondary motion vector, and the primary motion vector are written into the compressed code stream;

 a decoding unit, configured to receive the compressed code stream, and obtain a motion vector, a quadratic motion vector, and a quadratic residual image from the compressed code stream; perform motion compensation on the second residual image according to the secondary motion vector , obtaining a residual image, and performing motion compensation on the primary residual image according to the primary motion vector to obtain a reconstructed image.

 The codec of claim 18, wherein the encoding unit comprises:

 a prediction module, configured to perform a prediction on the current image to be encoded, to obtain a residual image and a motion vector;

a secondary prediction module, configured to receive a residual image from a prediction module, The received residual residual image is subjected to secondary prediction to obtain a secondary residual image and a secondary motion vector;

 An encoding module, configured to receive a primary motion vector from a primary prediction module and a secondary motion vector and a secondary residual image from the secondary prediction module, and receive the received primary motion vector, secondary motion vector, and quadratic residual The image is written to the compressed code stream.

 The codec according to claim 19, wherein the decoding unit comprises: a decoding module, configured to decode, obtain a primary motion vector, a secondary motion vector, and a secondary residual image from the compressed code stream;

 a quadratic reconstruction module, configured to receive a secondary residual image and a secondary motion vector from the decoding module, and perform motion compensation on the secondary residual image according to the received secondary motion vector to obtain a residual image;

 The primary reconstruction module is configured to receive a primary residual image from the secondary reconstruction module and a primary motion vector from the decoding module, and perform motion compensation on the primary residual image according to the received primary motion vector to obtain a reconstructed image.

 21. An image encoding method, the method comprising:

 A. Performing a prediction on the current image to be encoded encoded by the image to be encoded in the first frame to obtain a residual image and a motion vector;

 B. In the macroblock or block unit, when performing disparity estimation for each macroblock or block in the primary residual image, find a corresponding in the secondary reference image according to the primary motion vector of the corresponding position block in the encoded image. Blocking, using the corresponding block as a best matching block, and using the primary motion vector as a secondary motion vector in the secondary reference image, and predicting the current macroblock or block by using the best matching block to obtain a second Residual block; Writes a primary motion vector and a secondary residual block and/or a secondary motion vector into the compressed code stream.

The method according to claim 21, wherein the step of predicting the current image to be encoded in step A, obtaining a residual image and a motion vector include: In the macroblock or block unit, when performing disparity estimation for each macroblock or block in the current image to be encoded, according to the primary motion vector of the corresponding position block in the encoded image, in the reference image of the current macroblock or block Finding a corresponding block, using the corresponding block as a best matching block, and using the primary motion vector as a primary motion vector of the current macroblock or block in the reference image, and using the best matching block to predict the current macroblock or block , get a residual block;

 Or, when performing the disparity estimation, according to the primary motion vector of the corresponding position block in the encoded image, find the corresponding block in the reference image of the current macroblock or the block, and use the block as the starting point of the search center to find the best matching block, The best matching block predicts the current macroblock or block and obtains a residual block and a corresponding primary motion vector.

 23. An image encoding method, the method comprising:

 A. Performing a prediction on the current image to be encoded encoded by the image to be encoded in the first frame to obtain a residual image and a motion vector;

 B. In the macroblock or block unit, when performing disparity estimation for each macroblock or block in the primary residual image, find a corresponding in the secondary reference image according to the primary motion vector of the corresponding position block in the encoded image. Block, using the block as a starting point of the search center, searching for the best matching block, using the best matching block to predict the current macroblock or block, and obtaining a quadratic residual block and a corresponding secondary motion vector; Vectors, quadratic residual blocks, and quadratic motion vectors are written into the compressed code stream.

The method according to claim 23, wherein in step A, the current image to be encoded is predicted once, and the first residual image and the primary motion vector are obtained: in a macroblock or a block, for the current When performing the disparity estimation for each macroblock or block in the image to be encoded, according to the primary motion vector of the corresponding position block in the encoded image, the corresponding block is found in the reference image of the current macroblock or the block, and the corresponding block is regarded as the most Preferably, the first motion vector is used as a primary motion vector of the current macroblock or block in the reference image, and the current macroblock or block is predicted by the best matching block to obtain a residual block; Or, when performing the disparity estimation, according to the primary motion vector of the corresponding position block in the encoded image, find the corresponding block in the reference image of the current macroblock or the block, and use the block as the starting point of the search center to find the best matching block, The best matching block predicts the current macroblock or block and obtains a residual block and a corresponding primary motion vector.

 25. An image decoding method, the method comprising:

 A. Decoding from the compressed code stream to obtain a motion vector, a quadratic motion vector, and a quadratic residual image;

 B, in macroblock or block unit, for each macroblock or block in the quadratic residual image, according to the secondary motion vector corresponding to the macroblock or block, or the corresponding macroblock or block corresponding to the second The primary motion vector of the secondary motion vector is motion-compensated for the second residual block, and the residual block is obtained once, and the primary motion vector corresponding to the primary residual block is obtained, and the motion compensation is performed on the primary residual block to obtain motion compensation. The image block is reconstructed, and all reconstructed image blocks of the secondary residual image are combined to reconstruct the image.

 An encoder, comprising: a primary prediction module and an encoding module, wherein the encoder further comprises: a secondary prediction module, wherein

 a prediction module, configured to perform a prediction on the current image to be encoded after the first frame, to obtain a residual image and a motion vector;

 a secondary prediction module, configured to receive a primary residual image from a primary prediction module, and perform a parallax for each macroblock or block in the primary residual image in units of macroblocks or blocks for the received primary residual image. In the estimation, according to the primary motion vector of the corresponding position block in the encoded image, the corresponding block is found in the secondary reference image, the corresponding block is taken as the best matching block, and the primary motion vector is used as the secondary reference image. a quadratic motion vector, using the best matching block to predict the current macroblock or block to obtain a quadratic residual block;

An encoding module, configured to receive a primary motion vector from the primary prediction module and a secondary residual image and/or a secondary motion vector from the secondary prediction module, and receive the received one The secondary motion vector and the secondary residual image and/or the secondary motion vector are written into the compressed code stream.

 The encoder according to claim 26, wherein the encoder further comprises: a quadratic reconstruction module, configured to receive a quadratic residual image from the secondary prediction module, and read the secondary prediction module The obtained quadratic motion vector or the primary motion vector obtained in the primary prediction module as the secondary motion vector, according to the secondary motion vector or the primary motion vector as the secondary motion vector, the received secondary residual Performing motion compensation on the difference image to obtain a residual image for the reference image when the secondary prediction module performs secondary prediction; a primary reconstruction module, configured to receive a residual residual image from the secondary reconstruction module, and read The primary motion vector obtained in the primary prediction module performs motion compensation on the received primary residual image according to the primary motion vector to obtain a reconstructed image, which is used for the reference image when the primary prediction module performs one prediction.

 An encoder, comprising: a primary prediction module and an encoding module, wherein the encoder further comprises: a secondary prediction module, wherein

 a prediction module, configured to perform a prediction on the current image to be encoded after the first frame, to obtain a residual image and a motion vector;

 a secondary prediction module, configured to receive a primary residual image from a primary prediction module, and perform a parallax for each macroblock or block in the primary residual image in units of macroblocks or blocks for the received primary residual image. In the estimation, according to the primary motion vector of the corresponding position block in the encoded image, the corresponding block is found in the secondary reference image, and the block is used as the search center starting point to find the best matching block, and the current matching macro is used to match the current macro. The block or block performs prediction to obtain a quadratic residual block and a corresponding secondary motion vector;

 An encoding module, configured to receive a primary motion vector from the primary prediction module and a secondary residual image and a secondary motion vector from the secondary prediction module, and receive the received primary motion vector and the secondary residual image And the secondary motion vector is written into the compressed code stream.

The encoder according to claim 28, wherein the encoder further comprises: a quadratic reconstruction module, configured to receive a quadratic residual image from the secondary prediction module, and read a secondary motion vector obtained in the secondary prediction module, and according to the secondary motion vector, the received secondary residual Performing motion compensation on the difference image to obtain a primary residual image, where the primary residual image is used for a reference image when the secondary prediction module performs secondary prediction;

 a reconstruction module, configured to receive a residual image from the secondary reconstruction module, and read a motion vector obtained in the prediction module once, and perform motion compensation on the received primary residual image according to the primary motion vector, Reconstructing an image, the reconstructed image being used for a reference image when the primary prediction module performs one prediction.

 30. A decoder, the decoder comprising: a decoding module, a primary reconstruction module, wherein the decoder further comprises: a secondary reconstruction module, wherein

 a decoding module, configured to decode a motion vector, a secondary motion vector, and a secondary residual image from the compressed code stream;

 a quadratic reconstruction module, configured to receive a secondary residual image and a secondary motion vector from the decoding module or a primary motion vector as a secondary motion vector, according to the received secondary motion vector or as a secondary motion vector One motion vector, motion compensation of the second residual image to obtain a residual image;

 a primary reconstruction module, configured to receive a primary residual image from the secondary reconstruction module and a primary motion vector from the decoding module, and perform motion compensation on the primary residual image according to the received primary motion vector to obtain a reconstructed image. .

 31. A method of finding a best matching block between multi-view images, the method comprising:

 When performing primary and/or secondary disparity estimation on the macroblock or block of the current image, finding a corresponding block in the reference image of the current macroblock or block according to the primary motion vector of the corresponding position block in the encoded image, and the corresponding block As the primary and/or second best matching block of the current macroblock or block.

32. A method for finding a best matching block between multi-view images, characterized in that The law includes:

 When performing one-time and/or quadratic disparity estimation on the macroblock or block of the current image, finding a corresponding block in the reference image of the current macroblock or block according to the primary motion vector of the corresponding position block in the encoded image, with the corresponding block As the starting point of the search center, find the primary and/or secondary best matching block of the current macroblock or block.

 33. An apparatus for finding a best matching block between multi-view images, the apparatus comprising:

 a prediction module, configured to: in a macroblock or a block for the current image to be encoded, for each macroblock or block in the current image to be encoded, when performing disparity estimation, according to a motion vector of a corresponding position block in the encoded image Find the corresponding block in the reference image and use the corresponding block as the best matching block.

 34. An apparatus for finding a best matching block between multi-view images, the apparatus comprising:

 a prediction module, configured to: in a macroblock or a block for the current image to be encoded, for each macroblock or block in the current image to be encoded, when performing disparity estimation, according to a motion vector of a corresponding position block in the encoded image Find the corresponding block in the reference image and use the block as the starting point of the search center to find the best matching block.

 35. An apparatus for finding a best matching block between multi-view images, the apparatus comprising:

 a secondary prediction module, configured to perform, in a macroblock or a block, the received residual residual image, for each macroblock or block in the primary residual image, according to the corresponding position block in the encoded image The primary motion vector finds the corresponding block in the secondary reference image, and uses the corresponding block as the best matching block.

36. An apparatus for finding a best matching block between multi-view images, the apparatus comprising: a secondary prediction module, configured to perform, in a macroblock or a block, the received residual residual image, for each macroblock or block in the primary residual image, according to the corresponding position block in the encoded image The primary motion vector finds the corresponding block in the secondary reference image, and uses the block as the starting point of the search center to find the best matching block.