EP2232869A1

EP2232869A1 - Motion encoding without the transmission of motion information and motion decoding

Info

Publication number: EP2232869A1
Application number: EP08858284A
Authority: EP
Inventors: Joël JUNG; Guillaume Laroche
Original assignee: France Telecom SA
Current assignee: Orange SA
Priority date: 2007-11-28
Filing date: 2008-11-19
Publication date: 2010-09-29
Also published as: US8731045B2; JP2011505095A; WO2009071803A1; CN101919251B; JP5324594B2; US20100266046A1; CN101919251A

Abstract

The invention relates to a method for encoding a portion of a current image, including: an estimation (10) of motion between the current image portion and a plurality of candidate image portions in order to form motion components, an evaluation (14) of a performance criterion for each candidate image portion, and the selection (16) of a reference image portion using said performance criteria. The method is characterised in that, for at least one motion component, the motion estimation, the performance criteria evaluation and the reference image portion selection only use information considered to be available at the decoder and no motion information is inserted into an output flow intended for a decoder. The invention also relates to the corresponding decoding method.

Description

MOTION CODING WITHOUT TRANSMISSION D ¹ MOTION INFORMATION, AND DECODING

The present invention relates to image coding and decoding techniques and more particularly to the encoding and decoding of the motion.

Several existing methods make it possible to encode a current image portion with respect to other previously decoded image portions according to an "INTER" encoding mode. The current image portion is represented by chrominance luminance information and motion information between that current image portion and a reference image portion. This makes it possible to exploit the temporal correlations between the images.

The motion information includes one or more components such as a translational motion vector, a rotation, a pixellic component, and a subpixel component. The encoding generally includes determining information for the reconstruction of the motion components. For example, a prediction of a motion vector is performed and the information for reconstruction includes the identifier of the prediction function used as well as a residue corresponding to the difference between the prediction and the original motion vector.

The information for the reconstruction also includes other information of various types that make it possible to obtain the components of the movement at the decoder. For example, this information is called related or "side information" and includes size parameters, function identifiers to use or other.

This information for the reconstruction is transmitted to the decoder.

The decoder reconstructs the different components of the estimated motion, i.e. applies the inverse prediction, combines the result with the residue and the related information. The decoder then applies the motion components with the reference image portion and combines the result with the luminance and chrominance information to obtain the decoded picture.

The motion components are information whose size increases with the resolution of the images and the information for the reconstruction of these components occupies a significant part of the bandwidth.

This poses problems especially in environments where the bandwidth is limited, such as wireless environments.

An object of the present invention is to improve this situation by making it possible to transmit motion information using less bandwidth.

For this purpose, an object of the present invention is a method of encoding at least a portion of a current image comprising a motion estimation between the current image portion and a plurality of candidate image portions to form images. motion components, evaluating a performance criterion for each candidate image portion, selecting a reference image portion from the candidate image portions using said performance criteria, and coding information in an output stream to a decoder, characterized in that, for at least one motion component, the motion estimation, the evaluation of performance criteria and the selection of a portion of a reference image use only information considered as available at said decoder and no motion information is inserted into said output stream.

Correspondingly, another object of the invention is a method of decoding at least a portion of a current image characterized in that it comprises a reception of a coding information stream of the portion of current image, a motion estimation for the current image portion from a plurality of candidate reference image portions and already decoded image portions, to form corresponding motion components, an evaluation of a performance criterion for each candidate image portion, selecting from said candidate image portions using said criteria to obtain said image portion reference and decoding of the current image portion from said reference portion and corresponding motion components.

Thus, the encoder uses only information available at the decoder and the decoder reproduces the calculations made at the coder. Consequently, for at least one motion component, no reconstruction information of said component must be transmitted, which saves bandwidth.

In a variant of the coding method, said estimation delivers several motion components and the method further comprises an information determination for the reconstruction of at least one of said motion components and the insertion of this information for the reconstruction in the stream. Release.

Correspondingly, the decoding method comprises receiving the reconstruction information and corresponding reconstruction of a motion component.

Thus, the process is combined with existing methods.

In a particular embodiment of the coding and decoding methods, the motion estimation and the evaluation of the performance criteria use correlations between fragments of the candidate picture portions and corresponding fragments that are neighbors and distinct from the portion of the picture. current image.

In another particular embodiment of the encoding and decoding methods, the motion estimation and the evaluation of the performance criteria use correlations between neighboring and distinct fragments of the candidate image portions and corresponding and distinct corresponding fragments. of the current image portion.

The invention also relates to computer programs and devices corresponding to these coding and decoding methods.

Other features and advantages of the present invention will appear in the description given below in a non-limiting manner and with reference to the appended drawings in which: - Figure 1 is a diagram showing two stations in communication respectively provided with an encoder and a video decoder;

FIG. 2 is a flowchart of the coding method according to one embodiment of the invention; FIGS. 3A, 3B and 3C show various embodiments of a particular step of the coding method; and

FIG. 4 is a flowchart of the decoding method according to one embodiment of the invention.

The invention is applicable to any system for coding and decoding images, such as, for example, the system shown with reference to FIG. 1. This system allows the coding of a video sequence of a digital television stream F and its transmission from a transmitter 1 containing a video encoder 2 comprising a controller or computer 3 associated with a memory 4. The transmission is provided to a receiving station 5 containing a decoder 6 also comprising a controller or computer 7 and a memory 8 .

For example, the transmitter 1 comprises an antenna transmitting on a digital television radio channel a data stream Φ in a format such as the so-called DVB format and the station 5 is a personal computer. With reference to FIG. 2, the method of the invention will now be described at the coding level.

The encoder 2 receives the stream F of image data of a video sequence. This stream is processed to encode each image or image portion. The term image portion generally refers to an element of the video sequence. Depending on the standards used, the term image may be replaced by the term frame and the term portion by the term block or macroblock.

The coding method begins with a motion estimation step 10 between the current image portion and candidate image portions. The candidate image portions are, for example, the portions corresponding to or close to the portion of the current image in a previous image. Motion components, such as motion vectors are obtained through estimation 10 for each candidate image portion.

In the embodiment described, motion estimation implements only information considered as available to the decoder.

By information considered available to the decoder here means information which the coder estimates or knows that they have been decoded. This is information previously transmitted and for which no error message has been received or for which a positive acknowledgment of receipt has been received, or information stored in both the encoder and the decoder.

With reference to FIGS. 3A to 3C, various motion estimates are described.

In the example described in FIG. 3A, the motion estimation is based on a technique called pairing or correlation of blocks or lines (in English "block / line matching"). For each candidate image portion CAND of an earlier image I (N-1) a motion vector Vmvt is estimated by considering blocks or lines L of the candidate image portion that are found in distinct but neighboring portions. of the portion to be coded CODE on the current picture I (N). These distinct and adjacent portions are image portions already transmitted and which the encoder believes are available at the decoder.

In the example described with reference to FIG. 3B, the same principle is applied to lines L which are distinct and close to the candidate image portion CAND and to corresponding lines that are distinct and close to the image portion to be encoded. CODE.

In the example described in FIG. 3C, the motion estimation is based on correlations between "pattern matching" contours of image fragments belonging to the CAND candidate image portions of a previous image. and the same outlines on fragments of the current image distinct and adjacent to the image portion to be coded CODE. In itself motion estimation algorithms for obtaining motion vectors from these data are known and will not be described in detail here. In the example, it is considered that the motion components comprise a translation vector and a rotation. At the end of the motion estimation step 10, motion components are obtained for each candidate image portion.

In the described embodiment, the motion estimation is followed by the determination of information for the reconstruction of each motion component. This is for example the determination of the most suitable prediction functions, a residual which represents the difference between the image portion to be encoded and a portion of the candidate image transformed by the prediction function or information so-called "side information". This information for reconstruction is calculated in a conventional manner per se. The method then comprises a step 14 of evaluating a performance criterion for each portion of candidate image. This evaluation implements a criterion using only information considered available at the level of the decoder.

Indeed, unlike the selections of the state of the art, the performance criterion can not be based on the distortion of the current image portion because it is not available at the decoder.

In the examples described with reference to FIGS. 3A and 3B, a usable performance criterion is the absolute sum of the differences between the lines L on the previous image and the corresponding lines on the current image.

In the example described with reference to FIG. 3C, a usable performance criterion is the absolute sum of the differences between the lines weighted by the contour information.

Thus a performance criterion is calculated for each portion of candidate image from information available at the decoder.

In the embodiment described, the information for the reconstruction is also available and can be used for calculation of the performance criterion. However, the reconstruction information used should be available at the decoder. For example, the residue can not be used.

The method then comprises a selection 16 for retaining a reference image portion from among the candidate image portions. This selection is based on the performance criteria previously obtained. Like the motion estimation 10 and the performance evaluation 14, the selection 16 uses only information considered as available at the decoder. In particular, the selection does not use the residue information.

For example, the selection consists in retaining the portion of candidate image with the best performance evaluation. Alternatively, the selection is based on the performance evaluation weighted by the available bandwidth. This allows you to select the best compromise between image quality and throughput.

At the end of the selection 16, a reference image portion is thus available as well as the associated motion components and information for the reconstruction of these components.

The coding method then comprises a step 18 encoding non-motion related information such as chrominance and luminance. In some embodiments, this step 18 uses the motion components or reconstruction information to encode information such as luminance. In one embodiment, the motion vector associated with the reference image portion is used. This step 18 is implemented in a conventional manner and will not be described in more detail.

Finally, the method comprises a step of emitting the stream Φ to the decoder. In the example described, none of the information for the reconstruction of the motion components is inserted into the output stream. In particular, neither the prediction residues of the motion vectors nor the identifiers of the reference image portion are inserted in the stream output, so that a significant fraction of the bandwidth is available.

The information relating to the coding of the other parameters of the image, such as the luminance and chrominance information, is inserted in this output stream in a conventional manner during this step 20.

Thus, the fact that the motion estimation, the evaluation of performance criteria and the selection of a reference image portion only use information considered as available at the level of the decoder makes it possible not to transmit the information for the first time. reconstruction of motion components. As a result, the transmission of motion information requires less bandwidth.

With reference to FIG. 4, the corresponding decoding method will now be described.

During a step 22, the stream Φ is received by the decoder. As previously indicated, in this embodiment the stream contains no information for the reconstruction of the motion components.

The decoding method begins with a motion estimation step 24 for the current image portion. This estimation is carried out identically to that carried out during the coding described with reference to step 10 of FIG. 2.

Thus, the motion estimation uses the information already decoded to evaluate a motion vector for each candidate image portion and the current image portion. The line correlation and edge correlation techniques described above with reference to FIGS. 3A to 3C are directly applicable.

Of course, the motion estimation method used at the decoder in step 24 must be the same as that used at the encoder in step 10.

In the same way as the motion estimation made to the coding, the motion estimation 24 delivers, for each portion of candidate reference image, motion components such as motion vectors. The decoding method then comprises a step 26 of evaluating a performance criterion for each portion of candidate image. This evaluation is carried out according to the same methods as the evaluation 14 implemented during the coding. Accordingly, a performance criterion is associated with each candidate image portion.

The estimation of performance criteria is followed by a selection 28 which is identical in all respects to the selection step 16 described above.

The motion estimation 24, the evaluation of performance criteria 26 and the selection 28 are therefore performed identically to the encoder and the decoder. This is made possible by the fact that these operations use only information considered as available at the decoder.

At the end of step 28, the decoder thus has a reference image portion and associated motion components without it being necessary to transmit any of the information necessary for the reconstruction of the components of the image. movement.

The decoding method finally comprises a step 30 of decoding the current image portion. During this step, the reference image portion is transformed by the associated motion components. This therefore makes it possible to obtain the motion information of the current image portion.

In addition, the other parameters of the current image portion, such as the luminance and chrominance information, are obtained in a conventional manner. Obtaining this information requires, in some embodiments, the use of the motion vector and the reference image portion as identified in steps 24 to 28.

Thus, the use of the same motion estimation, the same evaluation of the performance criteria and the same selection to the encoder and the decoder makes it unnecessary to transmit the motion information between the encoder and the decoder. In addition, other embodiments can be envisaged. In particular, other techniques of motion estimation, evaluation of performance criteria and selection are usable. In particular, techniques known as "error concealment" or "inpainting" can be implemented. In any case, these techniques must use only data considered available at the decoder.

As a variant, the step 12 of determining the information necessary for the reconstruction is not carried out during the coding. Such an embodiment makes it possible to reduce the calculations at the encoder. Conversely, the embodiment presented with reference to Figure 2 keeps the conventional order of operations and limits the changes to be made to existing equipment. In addition this allows performance evaluation using the information for reconstruction. In a variant, the method is applied to only a part of the motion components. Thus part of the motion components is encoded in a conventional manner, possibly using information available only at the encoder level. For this part of the motion components, the reconstruction information is transmitted. For the other part coded according to the principle of the invention, it is useless to transmit the information for the reconstruction. For example, the information for the reconstruction of the rotation component is transmitted while that for the translation component is not. In another example, the information for the reconstruction of a pixellic component is transmitted while those relating to a sub pixel component are not.

Furthermore, in a particular embodiment, an identifier of the matching method used during the motion estimation is transmitted to the decoder. Thus, the decoder uses only information that is available at its level but it receives an identifier of the method to be used.

In yet another variant, the candidate image portions are in the same image as the image portion to be encoded and decoded.

Claims

A method of encoding at least a portion of a current image

(CODE) comprising:

an estimate (10) of motion between the current image portion and a plurality of candidate image portions (CAND) to form motion components (Vmvt); an evaluation (14) of a performance criterion for each portion of candidate image;

selecting (16) a reference image portion from among the candidate image portions using said performance criteria; and an insertion (20) of coding information in an output stream to a decoder, characterized in that, for at least one movement component, the motion estimation, the evaluation of performance criteria and selecting a reference image portion use only information considered available at said decoder and no motion information is inserted into said output stream.

2. Method according to claim 1, characterized in that said estimate delivers several motion components and the method further comprises an information determination (12) for the reconstruction of at least one of said motion components and the insertion of this information for reconstruction in the output stream.

3. Method according to any one of claims 1 or 2, characterized in that the motion estimation and the evaluation of the performance criteria use correlations between fragments (L) portions candidate images and corresponding fragments that are adjacent and distinct from the current image portion.

4. Method according to any one of claims 1 or 2, characterized in that the motion estimation and the evaluation of the performance criteria use correlations between neighboring and distinct fragments (L) of the candidate image portions and neighboring and distinct corresponding fragments of the current image portion.

A computer program to be installed in a video processing apparatus (4) comprising instructions for implementing the steps of an encoding method according to any one of claims 1 to 4 during execution of the program by a computing unit of said apparatus.

6. Device (2) for encoding at least a portion of a current image comprising:

a motion estimator between the current image portion (CODE) and a plurality of candidate reference image portions (CAND) to form motion components (Vmvt);

a unit for evaluating a performance criterion for each portion of the candidate image,

a selector of a reference image portion among the candidate portions using said performance criteria; and a communication unit for inserting coding information in an output stream to a decoder, characterized in that, for at least one movement component, the motion estimator, the evaluation unit of The performance criteria and the reference image portion selector use only information considered available at said decoder and no motion information is inserted into the output stream.

7. A method of decoding at least a portion of a current image characterized in that it comprises:

a reception (22) of a stream of coding information of the current picture portion; a motion estimate (24) for the current image portion from a plurality of candidate reference image portions and already decoded image portions, to form corresponding motion components;

an evaluation (26) of a performance criterion for each portion of candidate image;

selecting (28) among said candidate image portions using said criteria to obtain said reference image portion; and decoding (30) the current image portion from said reference portion and corresponding motion components.

8. Method according to claim 7, characterized in that it further comprises the reception of information for the reconstruction, at least one component of movement and a corresponding reconstruction.

9. The method as claimed in claim 7, wherein the motion estimation and the evaluation of the performance criteria use correlations between fragments of the candidate image portions and corresponding and distinct corresponding fragments. of the current image portion.

Method according to one of claims 7 or 8, characterized in that the motion estimation and the evaluation of the performance criteria use correlations between neighboring and distinct fragments of the candidate image portions and corresponding fragments. neighbors and distinct from the current image portion.

A computer program to be installed in a video processing apparatus, comprising instructions for implementing the steps of a decoding method according to any one of claims 7 to 10, during execution of the program by a computing unit of said apparatus.

12. Device (6) for decoding at least one current image portion, characterized in that it comprises:

a communication unit for receiving coding information of the current image portion;

a motion estimator for the current image portion from a plurality of candidate reference image portions and already decoded image portions, to form corresponding motion components; a unit for evaluating a performance criterion for each portion of the candidate image;

a selector, from among said candidate image portions and using said performance criteria, of said reference image portion; and a decoder of the current image portion from said reference portion and corresponding motion components.