WO2009056752A1

WO2009056752A1 - Method and system for estimating future motion of image elements from the past motion in a video coder

Info

Publication number: WO2009056752A1
Application number: PCT/FR2008/051932
Authority: WO
Inventors: Pierre Larbier; Marc Baillavoine
Original assignee: Ateme Sa
Priority date: 2007-10-29
Filing date: 2008-10-27
Publication date: 2009-05-07
Also published as: FR2923125B1; FR2923125A1

Abstract

A method of estimating the motion of an image element contained in the current image (Pc) of a video sequence preceded by a past reference image (Pp) and followed by a future reference image (Pf)- Following the calculation (A) of at least one past motion estimation vector (I) of a block of pixels (Bc) of the current image, referenced in the past reference image (Pp), a step (B) is executed for predicting the motion of the block of pixels (Bc) towards the future reference image (Pf) by linear combination of the past motion estimation vector (II) and of at least one past motion estimation vector (III) of a block of pixels (Bcf) of the future reference image (Pf) deduced from the block of pixels (Bc) by the linear combination, so as to generate a predicted motion estimation vector (IV).

Description

Method and system for estimating the future motion of picture elements from the movement in a video encoder

The invention relates to a method and system for estimating the future motion of picture elements from the movement in a video encoder.

The current video encoders make it possible to generate video sequences, composed of Intra-frame images, called I-frames, coded without reference to the preceding images and thus to the past, of images predicted with respect to the past images, these predicted images being designated P-images, and hybrid predicted images, designated B-images, which are predicted both from previous images, past images, and from subsequent images, future images.

In order to predict the successive images relative to each other, a considered image of rank n is cut into blocks, typically 16 × 16 pixels. For each block B _n (x, y) of image of rank n, x and y denoting the coordinates of the block in this image, we search in the previous and / or following images of rank m, me [np; n + q] the block B _m (x ', y') which has the most similarity with the considered image block B _n (x, y). It is recalled that the notion of similarity is established by comparison of chrominance values and / or luminance of the constituent pixels of the blocks B _n (x, y) and B _n , (x ', y').

In the absence of movement of the block B _n (x, y) and the image element containing the latter, this block retains, in the image of rank m, the same position as in the image of rank n , x = x 'and y = y'.

In the presence of motion, motion estimation algorithms make it possible to determine the motion vector, designated motion vector, of B _n

(x, y) to B _m (x \ y ^* ). Many motion estimation algorithms are known from the state of the art. The use of these depends on hardware or technical equipment constraints, such as bandwidth, available memory size, etc. For a more detailed description of the motion estimation process, it is useful to refer to the article published by Hye-Yeon Cheong, Alexis Michael Tourapis "Fast motion estimation within coded H.264", IEEE International Conference on Multimedial Expo 2003.

In order to speed up the motion estimation algorithms, a known technique is to initialize the estimation search from a point x ', y' (coordinates of a block of pixels) of the image of rank m defining with the corresponding point (x, y) of the image of rank n a candidate motion vector, hereinafter referred to as the candidate, which is considered to be a reasonable approximation of motion.

The positions tested of the candidate are limited to a neighborhood of this candidate, defining for example a zone of 4x4 pixels. If the approximation is correct, the displacement of a block is also determined by testing at most 16 positions.

In the case of P-images, a candidate may, for example, be either the median

(or the average candidate motion vector) motion vectors of the blocks that surround it, either the movement vector of the block located at the same place in the previous image, or the null vector. Other definitions of the candidate movement vector can be retained.

Several candidates may be selected for research. The criteria for choosing the elimination of unsuccessful candidates are most often established due to constraints on the presence of necessary calculation. Thus, in the reference coder of the H264 standard, entitled ITU-T Rec

H.264 ISO / IEC 14496-10 Advanced Video Coding, several candidates are successively tested and then eliminated or preserved for the calculation of a final motion vector.

Currently, all the recent video coding standards implemented have, for effective coding of the motion vector, a motion predictor. This one is supposed to constitute an approximation of the movement and the vector motion actually encoded in the flow of sequences of video images is the difference between the final motion vector, last candidate found by the estimation of movement and the vector motion predicted, resulting of this predictor. However, the predicted motion vector is very often used as a candidate for computational cost saving purposes.

For a more detailed description of the implementation of the candidate and predicted motion vectors, reference may be made to Recommendation ITU-T Rec. H.264 ISO / IEC 14496-10 Advanced Video Coding, paragraph 8.4.1.

As an illustrative example, various candidates usually used in the search for future motion vectors, as recommended by the aforementioned recommendation H. 264, are described below:

-the direct vector: it is in fact the predictor vector used for bidirectional images. To calculate it, we take into account the motion vector B _n (x, y) of the current image of a co-localized block B _m (x, y) of the same position x, y in the future reference image , vector designated mvCol in FIG. 1, referenced in the past reference image. From the motion vector of the co-located block mvCol, a candidate vector for the motion vector of the block of the current image B _n (x, y) referenced in the future reference image is deduced by weighting the respective temporal distances. past and future reference images according to the relation:

- r- * - mvCol x tf mvLla = -

where tp represents the temporal distance separating the current image from the past reference image; tf represents the time distance separating the future reference image from the current image.

For a more detailed description of the direct vector, reference may be made to the above-mentioned recommendation H.264, paragraph 8.4.1.2.2.

It is also indicated that in the aforementioned recommendation H.264, the direct vector is, moreover, a predicted motion vector.

a candidate vector mvLlbissu of the motion estimation passed between a block of the current image B _n (x, y) referenced with respect to the reference image passed, by the vector mvLo for example, corresponding to the block Bp ( x ', y') on the image of past reference, weighted by the respective time distances of the past and future reference images respectively, according to the relationship

In this relation, tf and tp denote the temporal distances separating the current image from the future reference image respectively from the past reference image.

the zero vector mvo, collinear with the time axis.

The present invention, in particular, relates to the implementation of a method for predicting the future movement of image elements or blocks of image pixels from past motion vectors that are available at level of a video encoder, at the moment when the latter is searching for future motion vectors.

In particular, thanks to the implementation of the method, object of the present invention, the invention also relates to a video encoder in which the implementation of a motion predictor module makes the integration of a module superfluous future motion estimator, whose integration and operating cost is high.

Consequently, an object of the present invention is also the implementation of video coders intended for consumer equipment with a limited bandwidth, presenting a clear gain in equipment complexity and in manufacturing cost, while allowing a function to be provided. prediction of future motion, in the absence, however, of implementation of future motion estimation module.

The motion estimation method of an image element contained in the current image of a video sequence, this current image being preceded, in the coding order, by at least one future reference image, in which for this current image and for the future reference image, there is available a field of motion estimation vectors passed from each block of pixels constituting these images with respect to this past reference image, which conforms to to the object of the present invention, is remarkable in that following the calculation of at least one movement vector passed (mvLo) of at least one block of pixels (Bc) of the current image referenced in this past reference image, it includes at least one motion prediction step of this block of pixels of the current image to this image reference, by linear combination of this past motion estimation vector (mvLo) and at least one motion estimation vector passed from a block of pixels of the current image (Bc) through of this linear combination, to generate a predicted motion estimation vector (mvL ^ B) of this block of pixels of the current image referenced in the future reference image.

The video coder which is the subject of the present invention comprises at least one input memory for storing a video stream, a read module for the input memory and an encoding module delivering, via a port of input-output, an encoded video stream comprising motion estimation vectors of successive images constituting this video stream.

The aforementioned coding module includes a past motion estimation module delivering, for each current image, a set of past motion estimation vectors associated with a block of pixels of each successive image, excluding any module of estimation of future movement towards a future reference image.

The video encoder according to the invention is remarkable in that the coding module includes, at least, connected to the past motion estimation module, a storage memory of the motion estimation vectors, a motion predictor module of at least one block of pixels from this current image to this future reference image, by linear combination of a past motion estimation vector (mvLo) of a block of pixels of the current image referenced in this reference image past and at least one movement estimation vector passed from a block of pixels of the future reference image deduced from the block of pixels of the current image via this linear combination, to generate a vector predicted motion estimation (mvL _} B) of this block of pixels of the current image, referenced in the future reference image. A storage memory of each of the predicted motion estimation vectors (mvL _x B) is provided, which is connected to the input-output port.

The motion estimation method of an image element and the corresponding video encoder are capable of being implemented in the industrial field of coders and video coding, digital cameras or others.

They will be better understood by reading the description and by observing the following drawings in which: FIG. 1a represents, purely by way of illustration, a flowchart of the essential steps for implementing the method that is the subject of the present invention; FIG. 1b represents, for purely illustrative purposes, a time diagram illustrating the procedure for implementing the steps of the method that is the subject of the present invention, as illustrated in FIG. Figure Ic represents a purely illustrative implementation detail of the prediction step; FIG. 2a represents, for purely illustrative purposes, a complete flowchart of the steps enabling the implementation of the method that is the subject of the present invention in a nonlimiting preferred mode of implementation; FIG. 2b represents, in a purely illustrative manner, an implementation detail of the step of calculating the past motion estimation vector (mvLoP) of a block of pixels of the future reference image, deduced from FIG. current image via a linear combination; FIG. 3 represents, by way of illustration, a temporal diagram of a process for processing the appearance in an image prior to the current image of a fortuitous image element, Hé to this image element, when, for example, the displacement of this image element revealing this fortuitous image element such as the backdrop; FIG. 4a and FIG. 4b represent, in the form of functional blocks, the general architecture of a video encoder, in accordance with the subject of the present invention; FIG. 5 represents a comparison diagram of the signal-to-noise ratio SPNR expressed in logarithmic base, measured in dB, for different video flow rate values expressed in kb / s, for a conventional video coding incorporating a future motion estimation, respectively for a video coding according to the method object of the present invention, in the absence of future motion estimation, but replaced by a prediction of future motion.

A more detailed description of the motion estimation method of an image element contained in the current image of a video sequence, object of the present invention, will now be given in connection with FIGS. 1a, 1b and the figures. following.

With reference to the above figures, a current image, denoted Pc, preceded by at least one past reference image, denoted by Pp and followed by at least one future reference image, denoted Pf, is considered.

The method which is the subject of the invention is implemented in a video coder in which, for the current image Pc and for the future reference image Pf, there is a field of motion estimation vectors passed from each block of pixels, such as the current block Bc shown in Figure 1b, vis-à-vis the reference image passed Pp.

In FIG. 1a, the past motion vector field of each block of pixels is denoted {mvLo} and in FIG. 1b the increasing time axis t is represented from left to right and the y and x axes contained in a plane of each image are vertically represented in respectively orthogonal to the plane of the sheet.

With reference to the above figures, in step A, for each current block of pixels, the current image Bc of a past motion estimation vector, denoted mvLo, is available. In accordance with the conventional image processing technique, the above-mentioned motion estimation vector makes it possible to reference the current block Bc of the current image in the past reference image Pp. It is particularly understood that the block of pixels Bc the current image corresponds, by projection of motion, to the block of pixels Bcfp in the reference image passed Pp. The same is true for any block of pixels of the future reference image Pf with respect to the reference image passed Pp. With reference to FIG. 1a, step A is followed by a step of predicting the movement of each block of pixels of the current image towards the future reference image Pf.

The aforementioned prediction step, according to a remarkable aspect of the method which is the subject of the invention, is advantageously carried out by linear combination of the motion estimation vector passed mvLo and of at least one motion estimation vector passed from the block of motion. pixels, denoted Bcf, of the future reference image Pf, the aforementioned reference pixel block Bcf of the future reference image being deduced from the block of pixels of the current image Bc by means of the aforementioned linear combination, for generating a predicted motion estimation vector, the vector mvL, b, relating to each of the blocks of pixels of the current image referenced in the future reference image Pf.

With reference to FIG. 1a, it is indicated that the aforementioned linear combination includes at least the weighted inversion of the passed motion estimation vector, mvLo, of the block of pixels of the current image referenced with respect to the past reference image, designated block Bc in Figure Ib, to generate the predicted motion estimation vector mvL _λ B shown in Figure Ib.

Thus, with reference to FIGS. 1a and 1b, it is understood that the aforementioned linear combination comprises at least one term given by the relation (1): wvZ, B = k mvLo k = -tfc / tcp

In the previous relation, it is understood that the coefficient k of the aforementioned term performs the weighting of the amplitude of the vector mvLO pro rata of the time intervals tfc and tcp separating the future reference image from the current image respectively the image current of the passed reference image and the minus sign executes the 180 ° rotation of the motion estimation vector passed mvLo in order to perform the prediction of the movement of the current block Bc to the block Bcf referenced in the image of future reference Pf.

As furthermore shown in FIG. 1b, a specific treatment can then being executed on the block Bcf in the future reference image Pf, from the motion estimation vector passed from the aforementioned block referenced in the passed reference image Pf, via the motion estimation vector past, noted mvLoP in Figure la and Figure 1b. It is understood, in these conditions, that the motion estimation vector predicts mvL, B can then result from the implementation of a linear combination processing process, this processing process occurring between the motion estimation vectors passed mvLoet mvLoP as represented in FIG. 1b. The representative linear combination of this process of treatment is represented in FIG. 1a by the relation (2): wvZ, B-f (mvLoP, mvLo)

A more detailed description of a specific, non-limiting embodiment of the method that is the subject of the invention will now be given with reference to FIG. 2a and FIGS. 2b and 2c. The mode of implementation of FIG. 2a corresponds in fact to a specific implementation of step B of FIG.

With the observation of FIG. 2a, it is understood that the implementation of step A of FIG. 1a is again found, the latter then being followed by step B comprising a sub-step B 1 consisting of executing the relation ( 2) previously cited to perform the weighted inversion of the motion estimation vector passed mvLoet of a substep B2 corresponding to the calculation of the motion estimation vector passed mvLoP of the pixel block Bcf in the future reference image Pf It will be recalled that during the implementation of the method that is the subject of the invention in a video coder, the sub-calculation step B2 can be reduced to a simple call of the corresponding motion estimation vector stored and available in the coder. video above.

According to a remarkable aspect of the method which is the subject of the invention, the linear combination mentioned above includes, in the substep B3, calculating the difference of position between the position in the reference image Pp of the P1 end. motion estimation vector passed rnvLoP of the pixel block of the current image Bcf referenced in the future reference image Pf with respect to the reference image passed Pf, to generate the corresponding block of pixels Bcfp, and the position, in the same reference frame passed Pp, of the end P2 of the motion estimation vector passed mvLO of the block of pixels Bcp of the current image referenced in the reference image passed Pp.

The calculation carried out in the sub-step B3 thus makes it possible to generate a vector P _x P ₂ difference of past motion estimation, according to the relation (3):

^'EJP mvLoP ₁ = - (mvLo - mvl ₁ B) in aforementioned step B3 is then followed by a validation step of motion estimation vector predicted MVL _x B on inferiority criterion the magnitude of the difference vector of motion estimation passed, vector P ₁ P ₂ , to a threshold value S.

The step of validating the motion estimation vector predicted at the substep B4 is represented by the relation (4):

<s

On positive response to the test of the substep B4, the predicted motion estimation vector finally retained, denoted mvL _x , is the vector mvL _x B without further correction. On the contrary, on a negative response to the test of the substep B4, a reactualization of the motion estimation vector passed mvLo is then called for correction of the predicted motion estimation vector mvL _x B, as will be described more detailed later in the description.

A nonlimiting preferred mode of implementation of the method which is the subject of the invention will now be described with reference to FIG. 2b and FIG. 2c.

In Figure 2b, the same elements as previously described in connection with the previous figures have the same references. Thus, on the observation of FIG. 2b, there is of course step A, step B constituted by its sub-steps B1, B2, B3, B4 and B5 of FIG. 2a.

In addition, as shown in FIG. 2b above, and before the weighted inversion operation executed in sub-step B 1, the method which is the subject of the invention includes at least, after calculation of the vector motion estimation passed mvLodu block of pixels Bc of the current image referenced in the reference image passed Pp, a first test step, denoted BOl, of the validity of the motion estimation vector passed mvLo.

The above test step is represented by relation (5): Validity mvLo?

The above-mentioned test step may advantageously consist of the application of a criterion of an interframe coding efficiency greater than the intratrame coding efficiency. The application of the aforementioned test makes it possible to check whether the motion estimation vector has a meaning in the sense of the reference image and the evolution of the context, taking into account the estimated movement. It can be implemented from metrics of the sum of the absolute values of the different current and previous metric images applied to the variance and brightness of the pixels of the pixel blocks Bc and Bcfp previously described in the description. A known technique for performing the aforementioned BOl test can consist in comparing the variance of the block with the sum of the absolute values of the differences (SAD) between the current block Bc and the corresponding block referenced by the motion estimation vector in the image. Pp. For a more detailed description of this procedure, refer to Optimizing INTRA / INTER Coding Mode Decisions by Yen-Kuang Chen, Anthony Vetro, Huifang Sun and SY Kung Princeton University and Mitsubishi Electric ITA. , October 23, 1997.

Thus, it is indicated that, if the metric has a value less than a given variance value, interframe coding is carried out. Otherwise, we retain the intratrame coding.

With reference to FIG. 2b above, on a positive response to the first step test BO1 a second test step B02 is called to verify the significance of the motion represented by the motion estimation vector passed mvLo.

The significance of the aforementioned motion is a function of the contrast variance of the pixel block of the image Bc. It is estimated, indeed, that if the variance of the block is small, the motion estimation vector found for this block does not represent the movement of the block, because motion estimation processes can not validly operate from a block or block area with low variance. The test step B02 is represented by the relation (6): significant mvLo?

On positive response to the second test step B02, the weighted inversion sub step B1 is then called, followed by the substep B2.

With reference to FIG. 2b and FIG. 2a, the substep B2 is then followed by the substeps B3 and B4 including the substep B3 for calculating the motion estimation difference vector P ₁ P ₂ previously described in the description. and the test B4 in connection with FIG. 2a.

On positive response to the B4 test, the substep B5 motion vector correction predicts mvL _x B is then called.

Furthermore, as it also appears in FIG. 2b, on a negative response to the first of the validity test step BO1, the method, in the embodiment described, also includes a third test step BO1. for checking the similarity of the block of pixels of the current image Bc with respect to neighboring blocks.

More specifically, it is indicated that the similarity test performed in the BOI test step l consists in comparing the light characteristics of the aforementioned blocks, in order to discriminate the existence of at least one similar neighboring block. In FIG. 2B, the test step BOl 1 is noted by the relation (7):

Similarity with neighboring blocks?

On positive response to the test performed in step BO 1 1, the predicted motion estimation vector of the similar neighboring block is allocated to the pixel block of the image current Bc. Otherwise, in the absence of a similar neighboring block, a null motion estimation vector is allocated to the block of pixels of the current image Bc.

In step B04, the allocation operation of the predicted motion estimation vector of the neighboring block similar to the current block Bc is noted according to relation (8): / TIvL ₁ - / wvProperty

In step B03, the allocation operation of the zero motion vector is noted according to relation (9): mvL _x = mVo

Similarly, as shown in FIG. 2b, on a negative response to the second step of testing the significant character of the movement represented by the motion estimation vector passed mvLo, test step B02, the method which is the subject of the invention includes a fourth test step, denoted B021 consisting in checking the validity of the motion estimation vector of the block of pixels co-located on the future reference image Pf, on the criterion of efficiency the interframe coding greater than the efficiency intratrame coding.

In step B021, the validity test step is noted according to relation (10):

Validity mvCol?

On a positive response to the verification test step B021, the predicted motion estimation vector is allocated the direct vector from the collocation to the step B05.

In the above-mentioned step B05, the allocation operation is represented by the relation (11):

^ vZ ₁ = mvDirect

On the contrary, on a negative response to the verification test B021, a return to the call of the step of the third similarity test with neighboring blocks executed in the test step BO 1 1 is executed.

A more detailed description of the specific implementation mode of the sub-step B2 for calculating the motion estimation vector passed from the current block Bcf, referenced in the future reference image Pf, will now be given in connection with with Figure 2c.

In general, it is indicated that the calculation of the above-mentioned motion estimation vector mvLoP is capable of making the block of pixels Bc of the current image Pc correspond to a fictitious block of pixels Bcf, which does not correspond exactly to a block of pixels of the aforementioned future reference image Pf, the aforementioned block Bcf then corresponding to a plurality of adjacent blocks, and in particular to a plurality of adjacent block areas, themselves adjacent to the lines of delimiting the adjacent blocks considered in the plane of the future reference image Pf. In FIG. 2c, the abovementioned adjacent blocks are denoted by Bal, Ba2, Ba3 and

Ba4.

In this case, the step of calculating the positional difference of the end Pl of the motion estimation vector passed mvLoP in the past reference image is executed by weighting the areas occupied by the block of pixels of the current image in the future reference image Pf. Thus, the aforesaid estimation process consists of starting from the block of pixels pointed by the predicted motion estimation vector mvL _x B in the future reference image Pf allows for estimating this motion estimation vector as a pseudo-vector, barycentre common surfaces between the next block referenced to the sum of the adjacent physical blocks Bal, Ba2, Ba3 and Ba4 as shown in Figure 2c.

The motion estimation vector passed mvLoP then verifies the relation (12): r _{^ n} (Sa * mvA + Sb * mvB + Sc * mvC + Sd * mvD) mvLOP = -

(Sa + Sb + Sc + Sd)

In the previous relation, -mvA designates the movement estimation vector passed from the Bal block in the future reference image;

MvB denotes the motion estimation vector passed from the block Ba2 in the image of future reference;

MvC denotes the movement estimation vector passed from block Ba3 in the future reference image;

MvD denotes the motion estimation vector passed from block Ba4 in the future reference image;

-Sa, Sb, Sc, Sd, respectively designate the adjacent surfaces of the block of pixels actually pointed by the predicted motion estimation vector mvL _{ B.

A more detailed description of a specific mode of implementation of the method which is the subject of the invention in a particular situation in which a scene scanning motion at a substantially constant speed of a video camera equipped with a video coder and for an image element containing at least one block of pixels having, in the image plane, an accelerated movement, this block of pixels and the corresponding image element being integral and traversing in the image plane, between two separate images, an increasing distance over time will now be given in connection with Figure 3.

In the above figure, consider a succession of images encoded by a video encoder respecting a standard where the insertion of bidirectional images is allowed, these standards may correspond to MPEG2 or H.264 standards.

It is assumed that the current image, the current image, is a B-type image designated as image B 12, which is predicted with respect to the past reference image Pf and the future reference image Pf.

It is assumed that the past motion estimate has been made between the current image B1 2 and the past reference image Pp and has made it possible to obtain the corresponding motion estimation vector fields, as described previously in FIG. description.

It is thus assumed that the video camera performs a scene scanning movement, called "traveling" in English, at a constant speed, a point traveling the same distance between two successive images, as represented in FIG. 3 by the lower block streaked by example. To illustrate the mode of implementation of the method which is the subject of the invention, it is considered in FIG. 3 above a block of pixels represented in gray on the current image B 12 and designated Block 1.

Block 1 in the current frame represents the designated Object A in the passed reference picture.

This block is present in all the successive images. This block being accelerated, the weighted inversion of the vector mvLo, as described in step BI of FIG. 2b for example, generates the vector ItIvL ₁ B which does not result in the good block, because of the global translation of Object A. On the other hand, the above-mentioned vector THvL ₁ B results in a neighboring block, which belongs to the same object, and therefore follows the same movement.

Applying the process of calculating the motion estimation vector passed mvLoP to the block of the future reference image Pf, we find the vector mvLoP mentioned above, which represents the global movement of the object A in the reference image. future Pf and the reference image passed Pp.

As shown in FIG. 3, the distance "Ad" is then measured according to the process described in step B3 of FIG. 2a, this distance being of course measured in the reference image passed Pp.

According to a remarkable aspect of the method that is the subject of the invention, in its implementation as described with reference to FIG. 3, the motion difference vector passed P _x P ₁ -Ad represents the distance traveled in the image plane. by the picture element between the future reference picture and the current picture B 12.

Under these conditions, the validation step of the motion estimation vector predicts mvL _λ B advantageously comprises a step of correcting the vector by adding, in the future reference image Pf, a motion estimation vector of correction, noted Ad. equipollent to the motion estimation difference vector P ₁ P ₂ . The aforesaid operation may consist in putting back the motion estimation difference vector passed by vector summation to the end of the predicted vector mvL _x B to form the finally retained predicted vector, TtIvL ₁ as previously described in step B5 of Figure 2b. Under these conditions, the vector predicted mv £ thus generated perfectly represents the movement of the block between the future reference image Pf and the current image, the image B 12. Finally, the coding method of the present invention implements a linear combination of the general form satisfying relation (13):

TTIvL ₁ B = k mvLo + ε P ₁ P ₂

S ε = 1, otherwise. This relation is thus reduced to a linear combination of the motion estimation vectors passed mvLo of the current block Bc referenced with respect to the past reference image Pp and the motion estimation vector passed mvLoP of the current block Bc referenced by the vector motion estimation predicts mvL _x B in the future reference image Pf of the form TTIvL ₁ B = kmvLo + ε (mvLop - (mvLo - TnVL ₁ B)).

Another example of a remarkable treatment process of the method which is the subject of the invention will now be given in connection with the same FIG. 3 in the case of an appearance, in at least one image prior to the current image, or in the image noted Bl I in FIG. 3, of a fortuitous picture element linked to the picture element, that is to say to Object A and therefore to at least one block of pixels of the latter presenting the accelerated movement previously cited.

With reference to FIG. 3, it will be understood that the block designated Block 2 on the current image B 12 represents an appearance in the previous image, but the block designated Bloc 2 mentioned above does not exist on the last reference image, because it is in fact masked by the object A. However the current image B 12 is not predicted with respect to the previous image, image BI l, and it exists nevertheless on the future reference image, image Pf . In order to ensure a correct treatment of the Bloc 2, the appearance of which is fortuitous, the method that is the subject of the invention then makes it possible to analyze the surrounding blocks in order to find a similar block, as represented in FIG. 2b in step BOl 1. .

For this purpose, it is recalled as previously described in the description, that it implements only metrics that are already calculated in the metric encoder such as the variance of the pixel block considered, its brightness, its color for example .

In this case, the background being united, it is then easy to find a similar block of pixels, such as the pixel block shown striated in Figure 3 above, and having a valid motion estimation vector.

As shown in FIG. 2b, on a positive response to the similarity test BOI 1, then the block designated Bloc 2 is assigned the motion estimation vector of the similar block, as represented in step B 04 of FIG. Figure 2b.

Since the above-mentioned Block 2 has a correct motion estimation vector, the block of pixels designated Block 1 then has, because of the bound nature of these blocks, a correct motion estimation vector, without, however, executing any estimate of future movement as such.

The procedure described above in connection with FIG. 3 can be summarized as follows when no picture element similar to the fortuitous picture element exists in the passed reference picture Pp.

The aforementioned process then consists in discriminating, in the current image B 12 in the vicinity of the fortuitous image element, a similar neighboring image element and then in allocating to the fortuitous image element an estimation vector predicted motion pattern, designated mvL _x s * in FIG. 3, equips the predicted motion estimation vector mvL _x s of the similar neighbor image element.

The predicted motion estimation vector allocated to the fortuitous image element is thus directly deduced from the predicted motion estimation vector mvL _{ s' and thus allocated to the image element containing at least one block of pixels. presenting in the image plane an accelerated movement. A more detailed description of a video encoder, in accordance with the subject of the present invention, will now be given in connection with FIGS. 4a and 4b.

As represented in FIG. 4a, the video coder according to the invention comprises, in a conventional manner, at least one input memory, denoted EM, receiving the video stream, a read module of the input memory noted RM and a coding module noted CM.

The input memory EM and the read module RM are modules of conventional architecture, which, for this reason, will not be described in detail.

With regard to the coding module CM, it is indicated that it also comprises, in a conventional manner, a past motion estimation module and a memory of the set of past motion vectors, the set {mvLo} previously mentioned in the description.

The two aforementioned elements are also elements of conventional architecture, which, for this reason, will not be described in detail. However, according to a remarkable aspect of the video coder which is the subject of the invention, the coding module CM includes a past motion estimation module delivering for each current picture a set of past motion estimation vectors each associated with a block of motion. pixels of each successive image, excluding any future motion estimation module to a future reference image. In addition, as represented in FIG. 4b, the coding module CM includes, at least, connected to the past motion estimation module, a storage memory of the set of past motion estimation vectors, vectors

{mvLo}, with the reference MMPO.

As further shown in FIG. 4b, the aforementioned coding module includes a PM module that predicts the movement of at least one block of pixels from the current image to the future reference image Pf.

Of course, the PM module predicting the movement of at least one block of pixels from the current image to the future reference image operates by linear combination of a motion estimation vector passed mvL0 of a block of pixels. of the current image referenced in the reference image passed Pp and at least one motion estimation vector passed vector mvLo of a block of pixels, the block Bcf of the future reference image, deduced from the block of pixels of the current image Bc above via the linear combination mentioned above. Thus, the motion predictor module PM makes it possible to generate a predicted motion estimation vector of the block of pixels of the current image referenced in the future reference image Pf. The predicted motion estimation vector n. is other than the motion estimation vector mvL _x B.

Finally, the motion predictor module PM is further connected to a storage memory of each of the predicted motion estimation vectors forming a set of predicted motion estimation vectors {mvL _x }.

The aforementioned storage memory is connected to the input / output port denoted I / O in order to deliver a coded video stream and including motion estimation vectors of the successive images constituting the coded video stream. Of course, the motion predictor module PM operates in accordance with the method that is the subject of the invention as described above with reference to FIGS. 1a, 1b, 2a, 2b, 2c and 3.

For this purpose it comprises a weighted inversion module A of the motion estimation vector passed mvLo. It furthermore includes a module for calculating the difference in position between the position in the reference image Pp of the end P1 of the movement estimation vector passed from the block of pixels of the current image referenced in FIG. passed reference image Pp and the position, in the same past reference image, of the end P2 of the motion estimation vector mvLo of the block of pixels of the current image referenced in the reference image passed Pp to generate the vector P ₁ P ₂ past motion estimation difference, as described previously in the description.

In FIG. 4b, the module for calculating the aforementioned difference in position bears the reference B. The predicted motion estimation module PM comprises in in addition to a comparison module of the motion estimation difference vector passed to a threshold value, according to the relation (4) previously mentioned in the description. This comparison module is referenced C, D in Figure 4b.

Finally, as also shown in FIG. 4b, the motion predictor module PM also comprises a validity test module of each motion estimation vector passed on the criterion of interframe coding efficiency greater than the coding efficiency. intratram, a motion significantness test module represented by each motion estimation vector passed mvLo and a dedicated calculation module of the barycenter of motion estimation vectors passed in the passed reference image Pp, by weighting of surfaces occupied by the block of pixels of the current image in the future reference image, as described in connection with Figure 2c.

In FIG. 4b, the validity test, the significance test and the dedicated calculation modules of the center of gravity respectively bear the reference E, F, G. The invention also covers a computer program product comprising a sequence of instructions stored on a storage medium and executable by a computer or by the dedicated calculation unit of a video encoder as shown in FIG. 4b, in accordance with the subject of the present invention, this product being the computer program being remarkable in that, during the execution of the instructions constituting it, this program implements the motion estimation method of an image element contained in the current image of a video sequence, as well as as described previously in the description in conjunction with Figures la, Ib, 2a, 2b, 2c and 3.

As furthermore shown in FIGS. 4a and 4b, the above-mentioned computer program is implemented in modular form in the aforementioned PM predictor module. The aforementioned computer program object of the invention is then advantageously implemented in modular form in the aforementioned predictor module, in the form of separate software modules bearing the references A, B, C, D, E, F, G previously mentioned in the description. Finally, with reference to FIG. 5, it is indicated that the method and the video coder Objects of the present invention make it possible to eliminate the implementation of a future motion estimation module in video codecs which are very expensive to implement.

In the above figure, there is shown the distortion measurement due to the coding of a video stream represented by the PSNR (Peak Signal to Noise Ratio) signal-to-noise ratio in the case of a conventional coding, with estimation of future motion and in the case of the removal of the latter and its replacement by a future prediction according to the method object of the present invention.

It can indeed be seen in FIG. 5 that, despite this deletion, but thanks to the implementation of the PM predictor module instead of the latter, the quality degradation of the video stream is quite marginal, while at most of the order of a few hundredths of a dB in either case.

Claims

A method for estimating the motion of an image element contained in the current image of a video sequence, said current image being preceded, in the coding order, by at least one reference image passed and followed by at least one future reference image, in which, for said current image and for said future reference image, there is a field of motion estimation vectors passed from each block of pixels constituting these images vis-à-vis -vis of said past reference image, characterized in that, following the calculation of at least one past motion estimation vector (mvLo) of at least one block of pixels (Bc) of the current image referenced in said passed reference image, said method includes at least one step of predicting the movement of said at least one block of pixels from the current image to said future reference image, by linear combination of said past motion estimation vector (mvLo) and at least one motion estimation vector passed from a block of pixels (Bcf) of the future reference image, derived from said block of pixels of the current image (Bc) via said linear combination , to generate a predicted motion estimation vector (MvL ₁ B) of said at least one block of pixels of the current image referenced in said fututre reference image (Bcf).

2. Method according to claim 1, characterized in that said linear combination includes at least: the weighted inversion of said past motion estimation vector (mvLo) of said at least one block of pixels of the current image (Bc) referenced to said past reference image (Bcp) for generating said predicted motion estimation vector (mvL _x B).

The method of claim 2, characterized in that said linear combination further includes:

calculating the difference in position between the position, in the past reference image, of the end (Pj) of the past motion estimation vector

(mvLo) of said at least one block of pixels of the current image referenced in the future reference image (Bcf) with respect to the past reference image (Bcfp) and the position, in the same image of past reference, of the end (P ₂ ) of the vector past motion estimation (mvLo) of said at least one block of pixels of the current image referenced in the past reference image (Bcp), for generating a vector (PiP ₂ ) past motion estimation difference;

the validation of said predicted motion estimation vector (mvL _{ B), on the inferiority criterion of the amplitude of said difference vector (P ₁ P ₂ ) of motion estimation passed to a threshold value.

4. Method according to one of claims 2 or 3, characterized in that, prior to said weighted inversion, said method includes at least, after calculation of said at least one past motion estimation vector (mvLo) of at least a block of pixels of the current image referenced in said passed reference image,

a first step of testing the validity of said passed motion estimation vector, on the criterion of an Intertrame coding efficiency higher than the Intratrame coding efficiency; and, on a positive response to said first validity test step, a second step of testing the significance of the motion represented by said past motion estimation vector (mvLo), the significant character of the movement being a function of the variance in terms of contrast of said block of pixels of the current image and, on a positive response to said second test step, the call of said weighted inversion.

5. Method according to claim 3 or 4, characterized in that the step of calculating the position difference includes an estimation step, from said at least one block in pixels of the image referenced in the image of future reference (Bcf) of the past movement estimation vector (mvLo) in said past reference image by weighting the areas occupied by said pixel block of ^the current picture in said future reference image.

6. Method according to claim 4 or 5, characterized in that, on a negative response to said first validity test step, said method further includes a third similarity test step of said at least one block of pixels of the image. common to neighboring blocks, the similarity test of comparing the light characteristics of said blocks, in order to discriminate the existence of at least one similar neighbor block, the predicted motion estimation vector of said similar neighbor block being allocated to said pixel block of the current image on existence of said similar neighbor block and a zero motion estimation vector being allocated to said block of the current image, otherwise.

7. Method according to claims 4 and 6, characterized in that, on a negative response to said second step of testing the significance of the movement represented by said past motion estimation vector (mvLo), said method includes a fourth step of validity test of the motion estimation vector of the pixel block colocalised on said future reference image, on the criterion of an Intertrame coding efficiency higher than the Intratrame coding efficiency, the predicted motion estimation vector being allocated the direct vector, resulting from the collocation, on a positive response to said fourth validity test step and a call return of the step of the third similarity test with the neighboring pixel blocks being executed, if not.

8. Method according to one of claims 3 to 7, characterized in that for a scene scanning motion at a substantially constant speed of a camera and for a picture element containing at least one block of pixels having, in the image plane, an accelerated movement, said block of pixels and said picture element traversing in the image plane between two distinct images an increasing distance with time, said vector (P ₁ ^ P ₂ ) difference of past motion estimate represents the distance traveled in the image plane by said image element between said future reference image and said current image.

9. Method according to claims 3 and 8, characterized in that said step of enabling said predicted motion estimation vector [MvL ₁ B) comprises a step of correcting said vector by adding, in said future reference image, a correction motion estimation vector equipollent to said difference motion estimation vector (P _x P ₀ ).

10. Method according to claims 6, 8 and 9, characterized in that, on appearance in at least one image prior to said current image, a fortuitous image element, linked to said image element and at least one auditing block of pixels of the latter having an accelerated movement, no picture element similar to said fortuitous image element not existing in said passed reference image, said method consists of at least:

-discriminating, in said current image, in the vicinity of said fortuitous image element, a similar neighbor image element; -allocating to said fortuitous image element a predicted motion estimation vector (mvL _x s') equipping to the predicted motion estimation vector

(MvL ₁ S) of said similar neighbor picture element;

directly deduce from said predicted motion estimation vector (/ TIvL ₁ S ') allocated to said fortuitous image element a predicted homologous motion estimation vector allocated to said image element containing at least one block of pixels having, in the image plane, an accelerated movement.

A video encoder comprising at least one video stream storage input memory, an input memory read module, and an encoder module outputting via a video input / output port. (I / O) a coded stream comprising motion estimation vectors of the successive images constituting said video stream, said coding module including a past motion estimation module delivering for each current image a set of vectors of estimation of past motion each associated with a block of pixels of each successive image, excluding any future motion estimation module to a future reference image, characterized in that said encoding module includes at least one connected to said module of motion estimate passed,

a storage memory of said motion estimation vectors passed;

a motion predictor module of at least one block of pixels from said current image to said future reference image, by linear combination of a past motion estimation vector (mvLo) of a block of pixels of the current image referenced in said past reference image and at least one motion estimation vector passed from a block of pixels (Bcf) of the future reference image, derived from said block of pixels of the current image ( Bc) via said linear combination, to generate a vector predicted motion estimation (mvL _x B) of this block of pixels of the current image, referenced in said future reference image;

a storage memory of each of said predicted motion estimation vectors (mvL _x B), connected to said input / output port (I / O).

Video encoder according to claim 11, characterized in that said motion predictor module includes at least one weighted inversion module of said passed motion estimation vector (mvLo).

Video encoder according to claim 12, characterized in that it furthermore includes a module for calculating the difference in position between the position, in the passed reference image, of the end (P 1) of the vector of estimate passed

(mvLop) of the block of pixels of the current image referenced in the future reference image with respect to the past reference image and the position, in the same reference image passed (mvLo) of the block of pixels of the current image referenced in the passed reference image, for generating a vector (P ₁ P ₂ ) difference of past motion estimation;

means for comparing the difference vector (PiP ₂ ) with motion estimation passed to a threshold value.

Video encoder according to claim 12, characterized in that it further includes:

a validity test module of each passed motion estimation vector, based on Intertrame coding efficiency criteria higher than the Intratrame coding efficiency;

a module for testing the significant character of the movement represented by each past motion estimation vector (mvLo);

a dedicated calculation module of the barycentre of the motion estimation vectors passed (mvLoP) in said passed reference image, by weighting the areas occupied by the block of pixels of the current image in the future reference image.

15. Computer program product comprising a sequel of instructions stored on a storage medium and executable by a computer or dedicated computing unit of a video encoder, characterized in that during the execution of said instructions, said program implements the method for ^estimating of motion of an image element contained in the current image of a video sequence according to one of claims 1 to 10.