EP1312221A1

EP1312221A1 - Method for calculating an image interpolated between two images of a video sequence

Info

Publication number: EP1312221A1
Application number: EP01963072A
Authority: EP
Inventors: Nathalie Laurent-Chatenet; Alexandre Buisson
Original assignee: France Telecom SA
Current assignee: Orange SA
Priority date: 2000-08-24
Filing date: 2001-08-07
Publication date: 2003-05-21
Also published as: WO2002017645A1; JP4922536B2; JP2004507943A; US7342963B2; FR2813485A1; FR2813485B1; US20040037471A1

Abstract

The invention concerns a method for constructing at least an image interpolated between a preceding image and a following image of an animated sequence. The invention is characterised in that the method takes into account data concerning the presence of at least an occlusion zone in said preceding image and/or in said following image, called occlusion data.

Description

METHOD FOR CALCULATING AT LEAST ONE IMAGE INTERPOLED BETWEEN TWO IMAGES OF A SEQUE NCE VIDEO

The field of the invention is that of coding and / or decoding of 5 animated sequences of images. More specifically, the invention relates to the construction of one or more image (s) interpolated between two given images of an animated sequence.

The invention finds applications in all fields of imaging and image processing. There may be mentioned, by way of examples, and in a non-restrictive manner: video coding and / or decoding, in particular with a view to improving the performance of a coder / decoder of the MPEG4 or H263 + type for example, or of any other video coder / decoder; medical imagery; 15 - tracking moving objects; standards conversion, such as PAL / NTSC conversion for example.

The invention applies in particular, but not exclusively, to the temporal interpolation of frames and / or to bidirectional coding of video sequences, with a view in particular to overcoming the problems of bandwidth saturation and of adaptability to data terminals. varied natures.

Indeed, one of the main objectives of operators in the video field is now to allow a user to view animated sequences of satisfactory quality, and in particular in which the successive images 25 are linked in a fluid manner. However, the problem of the fluidity of the video sequences is highly dependent on that of the saturation of the transmission networks and of the capacity (in particular of storage) of the display terminals available to the user.

In order to satisfy user requests, and to overcome the problems of bandwidth saturation, it is therefore necessary to produce video streams covering a wide range of flow rates. However, video sequences transmitted at low or medium speed generally appear jerky when viewed.

It has therefore been envisaged to implement a frame frequency interpolation during the decoding of the transmitted bit streams, and / or a storage of additional information during coding in bidirectional mode, in order to generate more fluid video sequences, depending on the terminal. display, transmission network and / or user request.

Different frequency interpolation techniques, based on the estimation of a field of motion, have been considered, but none to date has shown satisfactory performance.

It is recalled that the movement between two images of a sequence is defined as a two-dimensional vector characterizing the difference in positions between a point of an image and the homologous point of the other image, under the assumption that these points correspond at the same physical point in each of the two images. The allocation of a displacement vector to each point of the image, or pixel, therefore defines a dense field of movement on the image, also called optical flow.

There are two types of motion field estimation methods used for frame frequency interpolation: - enumerative methods, also called mapping methods; differential methods.

The enumerative methods are based on a division of the domain of an image into regions of finite size, by regular or adapted partitioning. We then define a motion model, which can be, for example, affine, translational, polynomial, etc. and one proceeds by successive iterations, by applying a set of distinct values, included in a predetermined interval, to the parameters of this model.

The differential methods consist, for their part, of estimating a dense field of motion, by assigning a motion vector to each pixel of the image considered. They implement, on the one hand, calculation methods differential, and secondly, optimization techniques, based for example on gradient descents. Among these methods, we can notably cite methods based on the optical flow equation, pel-recursive methods (in English, "Physical Picture Element", physical image element), parametric methods or robust estimators for example .

Many works have been undertaken to improve existing image interpolation techniques. Among these, we can cite the method proposed by Ciro CAFFORIO and Fabio ROCCA ("Motion Compensation Image Interpolation", "Image Interpolation by motion compensation", IEEE Transactions on communications, Vol. 38, 2, Feb. 90 ), consisting of using a pel-recursive estimator for motion compensation.

It is recalled that the pel-recursive methods proceed by prediction-correction of the motion vectors associated with each of the chosen pixels of an image, according to a predetermined direction of travel. The prediction can be based on the value of the motion vector of the previous pixel, or on a linear combination of the motion vectors of the neighboring pixels of the pixel considered. The correction is then based on a minimization of the DFD (Displacement Frame Difference, in French "Difference between images") by a gradient descent method. A disadvantage of this technique of the prior art is that the algorithm implemented by CAFFORIO and ROCCA suffers from a strong interaction between the recursive directions and the direction of the displacement to be estimated.

To overcome this effect, we have considered using 4 separate recursive directions. This has minimized the disruptive effects, but this technique remains insufficiently effective.

Jack NIEWEGLOWSKI proposed a method based on the use of a geometric transformation, combined with a technique called "block matching" (in French, compression by repetition of zones) improved ("Motion Compensated Video Interpolation Using Digital Image arping", (in French "Motion compensated video interpolation using digital image distortion"), IEEE ICASSP'94).

Such a method makes it possible to take account of changes in scale but has the disadvantage of being not very robust, due to a decorrelation of the geometric transformation and of the compression by repetition of zones.

Q. WANG and LJ CLARK then considered implementing a parametric algorithm, based on the modeling of the movement of any region of an image by a model defined a priori ("Motion Estimation and Compensation for Image Sequence Coding", in French "Motion estimation and compensation for coding image sequences", Signal Processing: Image Communication, Vol.4, No. 2, April 92, pp. 161-174).

A disadvantage of this technique of the prior art is that it is not suitable for the estimation of regions of strong movement.

To overcome the various problems encountered by these motion estimation techniques, it has also been proposed to implement hierarchical motion estimation algorithms, such as those described by L. Bόrocsky and P. Csillag ("Multiresolution Motion Field Estimation for Wavelet Video Coding and Frame Interpolation ", in French" Multiresolution field of motion estimation for video wavelet coding and frame interpolation ", Final Workshop, Vigo, Spain, Oct. 94, pp. 163-166) .

Such hierarchical motion estimation algorithms, like the motion estimation methods of the prior art described above, run up against the problem of detecting the occlusion zones (compaction, occultation zones, or discovery of the mesh) within the images of the video sequence considered. Indeed, when different planes and objects overlap in a scene, occlusion zones appear, generating lines of discontinuity within the mesh associated with the image.

The most common technique used to manage the appearance and / or disappearance of occlusion zones within an image is the use of Ray. One such technique consists in associating, with each of the constituent pixels of the image, a label relating to the motion vector associated with this pixel.

Thus, a pixel receiving at least two motion vectors carries the label "occlusion", which means that it covers at least two pixels present in the previous image of the sequence. A pixel receiving a single motion vector carries the label "normal", thus indicating that it is present in the previous image and in the current image. Finally, a pixel receiving no motion vector bears the label "hole", so as to indicate that it belongs to an image discovery area. Such a technique is described in particular in "Interframe Interpolation of Cinematic Séquences" (in French "Inter-frame interpolation of animated sequences"), by J. Ribas-Corbera and J. Sklansky, Journal of Visual Communication and Image Representation, Vol. 4, No. 4, Dec. 93, pp. 392-406 and in "Motion

Compensating Interpolation for Improved Slow Motion "(in French" Motion compensating interpolation for improved slow motion ") by B. Chupeau and P. Salmon, Thomson-CSF / Laboratoires Électroniques

Rennes, France.

However, this technique of the prior art only uses this tag information to perform an a posteriori correction of the interpolated images, but does not take them into account during the motion estimation making it possible to construct such images.

The invention particularly aims to overcome these drawbacks of the prior art.

More specifically, an objective of the invention is to provide a technique for constructing an image interpolated between a previous image and a following image of an animated sequence, making it possible to improve the impression of fluidity when viewing the sequence.

Another objective of the invention is to implement a technique for constructing an interpolated image which is simple and inexpensive to implement. Another object of the invention is to provide a technique for constructing a robust interpolated image adapted to regions of strong movement.

The invention also aims to implement a technique for constructing an interpolated image within a video sequence making it possible to manage the appearance and / or disappearance of objects which correspond to occlusion zones ( reversal zones, discovery zones, settlement zones) within a mesh associated with the images of the sequence.

Yet another objective of the invention is to provide a technique for constructing an interpolated image, allowing a user having a terminal with reduced storage capacity to view a fluid video sequence.

Another object of the invention is to implement a technique for constructing an interpolated image, allowing a user to view a fluid video sequence, despite saturation of the bandwidth of the network by which it is transmitted.

The invention also aims to provide a technique for constructing an interpolated image which can be integrated into any video coder / decoder in order to improve its performance, in particular in terms of visual fluidity. These objectives, as well as others which will appear subsequently, are achieved using a method of constructing at least one image interpolated between a previous image and a following image of an animated sequence.

According to the invention, such a method takes account of information relating to the presence of at least one occlusion zone in said previous image and / or in said following image, called occlusion information.

Thus, the invention is based on a completely new and inventive approach to the interpolation of images within an animated sequence. Indeed, the invention is based in particular on the management of the occlusion zones (that is to say the reversal or uncovering zones) appearing within a mesh associated with an image of the sequence. The invention therefore makes it possible to significantly improve the visual fluidity of an animated sequence, by implementing a generation of intermediate images inserted between two given images of the sequence, and by blurring the artefacts and the lines of discontinuity introduced by the existence of one or more occlusion zone (s) within such images.

Advantageously, an image of said sequence is associated with a mesh.

Indeed, meshes are more and more frequently the support for representing objects, especially in a virtual reality scene. They thus represent the major part of the quantity of information describing such a scene. Conventionally, a mesh is defined by a set of vertices and faces (or a set of vertices, edges and orientations) defining a topology.

Such a mesh can for example be a flat mesh, or any other type of mesh making it possible to implement a motion estimation from the image with which it is associated.

According to an advantageous characteristic of the invention, said mesh is a hierarchical mesh having at least two levels of mesh nested within a pyramid of mesh. Thus, with reference to the example of FIG. 1, the use of a nested hierarchical mesh 11 makes it possible to obtain a robust coding, and reduces the cost of transmission and / or storage of the information. Indeed, there is a strong correlation between the information carried by a parent node and that carried by its child nodes within the mesh. Therefore, the values of the nodes 12, 13 belonging to the highest level of the mesh pyramid (corresponding to a coarse mesh of the image) are quantified, with a view to their transmission, for example to an encoder, and / or their storage on a data medium 15. On the other hand, for the lower levels of the mesh pyramid (corresponding to a finer mesh of the image), only the differential values of the son nodes 14, in view of their transmission and / or storage, as illustrated in FIG. 1.

According to an advantageous technique, said occlusion information is determined by the implementation of a step of estimation of forward and backward movement between said preceding and following images.

The implementation of a forward motion estimation step consists in determining a set of motion vectors associated with the vertices of a mesh representative of a first image at an instant t ,, and making it possible to construct the mesh associated with a second image at a later time X ^. Such a step therefore makes it possible in particular to detect the at least partial disappearance of an object between the images at times t, and t ^.

Similarly, the implementation of a rear estimation step makes it possible to determine the motion vectors to be applied to the vertices of the mesh associated with the image of the instant tj to obtain the image at the previous instant t, , and also makes it possible to detect a possible appearance of objects between the images at times t _x and tj.

We can therefore consider implementing a prior estimation step, so as to detect a possible disappearance of objects, or a rear estimation step, so as to detect a possible appearance of objects. The joint implementation of a front estimation step and a rear estimation step therefore makes it possible to detect the presence of any occlusion zone, corresponding to a reversal or a discovery of meshes, associated with a disappearance or an object appearance.

Preferably, in the presence of at least one occlusion zone in said previous image (respectively in said following image), such a method implements the following steps: for each occlusion zone, an exclusion step, in said previous image (respectively in said following image), of a region of said mesh including at least one mesh corresponding to said area occlusion, called encompassing region, so as to obtain an approximate previous image (respectively an approximate following image); a front (respectively rear) estimation step, so as to determine motion vectors making it possible to obtain an approximate next image (respectively preceding) from said approximate preceding image (respectively following), called forward motion vectors (respectively back) ; a step of constructing an intermediate image from said approximate previous (respectively following) image and said front (respectively rear) motion vectors.

Indeed, the presence of an occlusion zone can vitiate the calculation of the motion estimate between two given images of the sequence. An approximate image is therefore constructed, obtained by excluding from a given image at least one region of the mesh corresponding to at least one occlusion zone. The method then consists in carrying out a new estimate of the movement between the instants t _t and ,, after having excluded the faulty areas (that is to say the areas containing at least one mesh reversal), in order to minimize the prediction error between the two images corresponding to the times t, and ^ considered.

This new estimation, implemented from the approximate image, makes it possible to determine the optimal motion vectors for the continuous zone of the image (that is to say admitting a bijection between t, and t _j ) and thus to avoid that the values of the motion vectors obtained during the motion estimation are disturbed by the existence of zones of discontinuities.

According to an advantageous characteristic of the invention, said encompassing region is a k-disc of said occlusion zone where k≥l.

We recall the definition of a k-disk of the occlusion zone: the 1-disk of an occlusion zone is the occlusion zone itself; for all k> l, the k- disk of an occlusion zone corresponds to a region which contains the occlusion zone and whose border is located at a maximum distance of k-1 edges from the limit of the zone d occlusion considered. Preferably, said construction step consists in applying said front (respectively rear) motion vectors, weighted by a scalar less than 1, to said approximate preceding (respectively following) image. Indeed, the forward and / or backward motion estimation step makes it possible to determine the set of motion vectors making it possible to move from the previous image to the next image. Depending on the temporal position of the intermediate image that one wishes to construct between the preceding and following images, these vectors are weighted with a scalar of between 0 and 1. According to a first advantageous variant of the invention, such a method comprises a step of reconstructing a mesh associated with said intermediate image, consisting, for each occlusion zone, of implementing a step of determining colorimetric and / or photometric information associated with a part of said mesh corresponding to said encompassing region, so as to obtain said interpolated image.

An interpolated image is thus reconstructed from the intermediate image obtained previously, by determining luminance and / or chrominance information associated with the encompassing region.

Such a determination step can be implemented according to the approach proposed in patent document FR-98 12525 having for title "coding process of still or moving images with reduction and adaptation of the bit rate".

The object of this patent document is a method of coding a digital image, aiming to produce a bit stream representative of this image, the length of the bit train being a function of the desired representation. This method comprises the following steps: defining, on a domain of the image to be coded, a hierarchical mesh comprising a plurality of nested meshes whose mesh vertices can be pixels of said image; perform optimizations of luminance, chrominance, and positions on each mesh level; determine, for each mesh of said hierarchical mesh, a luminance difference between the image to be coded and an interpolated image obtained from the vertices of the nested mesh to which the mesh considered belongs, and - introduce the values (advantageously coded in differential compared to the preceding hierarchical level) of positions, luminance and chrominance of the vertices of the meshes whose luminance difference is greater than a threshold difference.

According to this technique, at the end of the meshing step, a quaternary tree structure, associated with the hierarchical mesh, is constructed to manipulate the values (colors and positions) of the vertices of the meshes. The tree presents a number of nodes equal to the number of triangles in the level of corresponding triangular mesh. Each node of the tree relates to a single triangle of the hierarchical mesh. Once this tree has been constructed, the data of the tree to be introduced into the bit stream representative of the image which will be transmitted and / or stored is selected. This selection depends on the desired quality.

To make this selection, a luminance difference between the image to be coded and the interpolated image is calculated for each triangle from the vertices of the nested mesh to which the mesh in question belongs. This difference is then compared to a threshold difference for each triangle. The value of the threshold deviation depends on the quality of representation desired.

The part of the tree relating to the triangles whose luminance difference is greater than the threshold difference is then introduced into the binary train, which therefore makes it possible to transmit the data relating to the image as a function of the local quality of these different triangular partitions.

Advantageously, said determining step implements on the one hand a finite element coding model, and on the other hand an alternative coding model. Indeed, when implementing a hierarchical coding, for example as described in the French patent document FR 98 12525, and when a portion of the image is very textured, it is then necessary to provide a number of large mesh levels to obtain photometric and / or colorimetric information of sufficient quality. In this case, the efficiency of hierarchical coding is low. In other words, hierarchical coding is well suited for relatively smooth images, but not for images with highly textured parts.

A new approach has therefore been envisaged, described for example in French patent document FR 99 06815 entitled "Hierarchical coding method and selective implementation of a coding based on reversible transformation, and corresponding decoding method". Such a method of coding an image to be coded, comprises the following steps: definition of a hierarchical mesh having at least two levels of nested mesh formed of meshes defined by vertices (which may be pixels of said image to be coded) ; determination, for each of said meshes, of error information between said image to be coded and an interpolated image obtained from the vertices of the meshes belonging to the mesh level of the mesh considered; stopping the refinement of the meshes having error information lower than a first predetermined threshold; implementation of a specific coding for the meshes presenting an error information higher than a second predetermined threshold; - Continuation of the refinement of the meshes having error information greater than said first predetermined threshold and less than said second predetermined threshold.

Thus, according to this method, which proves to be particularly advantageous in the context of the invention, two distinct coding modes are selectively used. The hierarchical coding, or with nested meshes, is the main coding, or coding basic, but it is not used systematically: for the image portions which require it (here, for the portions of the surrounding region which require it), another more effective coding is used.

The decision to use this specific coding is made by comparison with a threshold. More precisely, at each node considered in the tree representative of the hierarchical mesh, three possibilities are offered: stop the refinement, or the division, of the meshes (the quality reached for the portion of image or of the corresponding encompassing region being sufficient) , go to the next hierarchical level, keeping the same hierarchical coding, or use another coding (in particular a coding better suited to very textured portions).

It will be noted that it is possible to use several different types of specific coding (or the same coding with different parameters), to adapt the choice of coding even more precisely to the characteristics of the image portion or of the region portion. inclusive. According to a second advantageous variant of the invention, such a method comprises a step of reconstructing a mesh associated with said intermediate image, consisting, for each occlusion zone, in inserting into said mesh part of a mesh associated with said next image or said previous image corresponding to said encompassing region, so as to obtain said interpolated image.

Therefore, in the intermediate image, the content of the occlusion zones is replaced by the corresponding zones of the previous or next image. The interpolated image thus obtained is then made up of a part obtained by interpolation, and corresponding to the displacement of the mesh associated with the previous image (respectively following), and occlusion zones containing information of the next or previous image. .

According to an advantageous characteristic, such a method further comprises, for each occlusion zone, a step of estimating local movement between said interpolated image and a corresponding exact image, consisting in optimizing the position of at least one node of part of said mesh corresponding to said encompassing region in said interpolated image, so as to generate an improved first interpolated image.

At the coding level, such a movement is taken into account and coded only if the motion estimation determines the absence of discontinuity zones, and in particular the absence of mesh reversals, between the interpolated image and the corresponding exact image.

Advantageously, such a method further comprises a step of calculating an error image, so as to evaluate an error between said first improved interpolated image and said corresponding exact image. Thus, the motion estimation between the interpolated image and the corresponding exact image having determined the presence of occlusion zones, an error calculation is implemented.

According to an advantageous technique, said error being greater than a predetermined threshold for at least one region of said error image, such a method implements a step of determining colorimetric and / or photometric information associated with each of said regions, so as to generate a second improved interpolated image.

Advantageously, said determination step implements, on the one hand, a finite element coding model, and on the other hand, an alternative coding model.

As described above, such a determination step can be implemented according to the technique set out in the French patent document FR

99 06815 entitled "Hierarchical coding method and selective implementation of a coding based on reversible transformation, and corresponding decoding method".

According to an advantageous characteristic of the invention, said determining step is implemented on the basis of said error image or of said corresponding exact image, according to a predetermined cost criterion.

In fact, prior to the implementation of the step of determining colorimetric and / or photometric information, the cost of such a device is evaluated. step from the error image on the one hand, and from the exact image corresponding to the interpolated image on the other hand. We therefore favor the least expensive solution.

Preferably, said mesh is larger than said image it represents.

Such a technique is for example described in the French patent document n ° FR 00 09273 of July 13, 2000, entitled "Motion estimator for the coding and decoding of image sequences", in the name of the same applicants as the present application . According to the technique set out in this patent application, the use of a mesh of size larger than the image with which it is associated allows the implementation of a technique of motion estimation and compensation for coding and the decoding of image sequences, allowing the management of discoveries and settlement of meshes when moving objects. The use of a mesh of size larger than the size of the image makes it possible, according to the present invention, to improve the temporal interpolation of frames at the time of decoding.

The invention also relates to a coding method and a method for decoding an animated sequence, of the type implementing a step of interpolating at least one image interpolated between a previous image and a following image.

According to such coding and decoding methods, said interpolation step takes account of information relating to the presence of at least one occlusion zone in said previous image and / or in said following image, called occlusion information. .

The invention also relates to a terminal for viewing an animated sequence of images of the type comprising means for interpolating at least one image interpolated between a previous image and a following image, said interpolation means taking account of the presence of at least one occlusion zone in said previous image and / or in said following image. The invention also relates to a video signal and a data medium comprising information making it possible to reconstruct at least one image interpolated between a previous image and a following image of an animated sequence of images, said information taking account of the presence of at least one occlusion zone in said previous image and / or in said following image.

Other characteristics and advantages of the invention will appear more clearly on reading the following description of a preferred embodiment, given by way of simple illustrative and nonlimiting example, and of the appended drawings, among which: FIG. 1 , described above, presents an example of implementation of a nested triangular hierarchical mesh; FIG. 2 presents a block diagram of all the steps implemented according to the invention for constructing an image interpolated between a previous image and a following image of an animated sequence; Figure 3 illustrates in more detail the steps 11, 14 and 15, presented in Figure 2; FIG. 4 illustrates an example of application of the invention to the temporal interpolation of frames. The general principle of the invention is based on the management of occlusion zones during the construction of interpolated images within an animated sequence, with a view to bidirectional coding and / or temporal interpolation of frames at decoding.

We present, in relation to FIG. 2, an embodiment of the construction of an image interpolated between a previous image and a following image of an animated sequence, in the form of a block diagram presenting all the steps to implement according to the invention.

We consider two images of an animated sequence, called respectively I, and I ₂ , at two instants t, and t ^ successive. We seek to construct an interpolated image l between I, and I ₂ , at a time t 'intermediate between t, and t ^. During a step referenced 21, a forward motion estimate is made from image I, so as to determine the motion vectors to be applied to the vertices of the mesh (for example a hierarchical mesh as illustrated in FIG. 1) associated with I _x to obtain the image I ₂ . Such a step 21 can of course also consist in carrying out an estimate of backward movement from I ₂ , but, for the sake of simplicity of reading, we will only endeavor to describe an example of embodiment implementing a estimation step before during the step referenced 21.

Such a step referenced 21 makes it possible to detect the presence of possible occlusion zones, corresponding here to the at least partial disappearance of one or more objects in the image I,. It also makes it possible to determine the movement d to be applied to the mesh associated with I, to obtain I ₂ .

In the absence of occlusion zones, a displacement of, defined as follows, is applied to the mesh associated with I ,, during a step referenced 22:

We thus weight the displacement d, to be applied to the mesh associated with l _x to obtain I ₂ , by a scalar between 0 and 1, the value of which depends on the position of the image I 'with respect to the images I, and I ₂ . Thus, if one wishes to obtain an interpolated image close to the image I ,, we weight the displacement d by a scalar close to 0. If, on the other hand, we wish to obtain a temporally inteφolated image close to I ₂ , we weight the displacement d by a scalar close to l.

At the end of the step referenced 22, the desired interpolated image I ′ is therefore obtained, during a step referenced 23. If, during the step referenced 21, it detects the presence of at least an occlusion zone, I ₁ is excluded _; during a step referenced 24, the encompassing region or regions corresponding to the occlusion zone (s) detected, so as to avoid an error in the motion estimation which would be introduced by the presence of this or these zones d 'occlusion. We thus obtain an image I _j approximate, corresponding to image I, from which at least one occlusion zone has been excluded.

We then implement a forward motion estimation from I _j approximated during a step referenced 25, in order to determine the optimal motion vectors for the continuous area of the image (that is to say admitting a bijection between t _t and tj).

Steps 21, 24 and 25 are now described in more detail in relation to FIG. 3.

During the step referenced 21 of forward motion estimation from the image I _15, the presence of a turning area 31 is detected. During the step referenced 24, the turning area is therefore excluded 31 to the image I _l5 so as to obtain an image _L, approximated. One then carries out an estimation of motion before 25 from approximate I _x , so as to obtain the optimal motion vectors to be applied to the vertices of the mesh associated with I, approximated to obtain I ₂ .

During a step referenced 26, a displacement of is applied to the mesh associated with I, approximate, so as to obtain, during a step referenced 27, an intermediate image Ii _nteπnéd i _a i _re - Such a displacement of is obtained by weighting the optimal displacement d, determined during the step referenced 25, by a scalar between 0 and 1, according to a technique similar to that implemented during the step referenced 22.

In the case where the invention applies to frame frequency interpolation, a step referenced 291 is then implemented, consisting in inserting into Ii _nterméd i _area the _area or areas of I ₂ or I _x corresponding to the or to the encompassing regions excluded from I, during the step referenced 24. The content of the occlusion zones is thus replaced by the corresponding zones of the instant t ^ or of the instant t ,.

One thus obtains, during a step referenced 292, an image I 'consisting, on the one hand, of an interpolated part corresponding to the displacement of the mesh associated with, and on the other hand of the zone or zones, corresponding to the occlusion zone (s), and containing information of the image I ₂ or of the image I ,. In the case where the invention applies to bidirectional coding, one can envisage two variants of the embodiment allowing, from I _interméd i _a i _re to obtain the interpolated image I '.

A first embodiment consists, during a step referenced 281, to determine color information and / or associated with the photometric or regions I _Intermed i _a i _re corresponding to or encompassing regions excluded from I, during the step referenced 24. The entire mesh associated with Ij _nterméd i _a i _{re> is} thus reconstructed so as to obtain an interpolated image I ′ during a step referenced 282. Such a step 281 can for example be implemented according to the technique described in patent document FR-98 12525 having for title "coding process for still or moving images with reduction and adaptation of the bit rate".

A second embodiment consists, during a step referenced 291, for inclusion in Ii _ntermédia _ry the zone of I ₂ or I, corresponding to or encompassing regions excluded from I _x during the step referenced 24. The content of the occlusion zones is thus replaced by the corresponding zones of time t ^ or time t ,.

In general, the image I 'is obtained by inserting the content of the image I ₂ during the implementation of a prior estimate, and by inserting the content of the image ï during the implementation of an estimate of backward movement, in order to allow an overall recovery of the interpolated image V. However, in certain cases, it is possible to prefer to construct the image l 'by inserting the content of the image I ₂ (respectively L when the implementation of a rear estimate (respectively before), for example to obtain a better quality image l 'and / or to give the video sequence greater visual fluidity.

One thus obtains, during a step referenced 292, an image F consisting, on the one hand, of an interpolated part corresponding to the displacement of the mesh associated with I _x , and on the other hand of the zone or zones , corresponding to the occlusion zone (s), and containing information of the image I ₂ or of the image I ,. In order to obtain a better approximation of the image I ', an estimation of motion 293 is then made between I' and the corresponding exact interpolated image.

There is thus obtained, during a step referenced 294, a first improved interpolated image I, '. During a step referenced 295, an error image between I, ′ improved and the corresponding exact interpolated image is then calculated.

If the error is small (for example less than a predetermined error threshold), the image I is then inserted between the images I, and I ₂ of the animated sequence, during a step referenced 298. In the Otherwise, if the error is greater than a predetermined threshold for at least certain parts of the error image, a step 296 of determining colorimetric and / or photometric information associated with the zone (s) is implemented. the image on which the error is significant.

Again, such a step 296 can, for example, be implemented according to the technique described in patent document FR-98 12525 having for title "coding process of still or moving images with reduction and adaptation of the bit rate".

There is then obtained, during the step referenced 297, a second improved interpolated image I ₂ 'to be inserted between the images l _x and I ₂ of the sequence of images.

At the coding level, the information required by the coder then corresponds to the motion vectors d ^* , corresponding to the motion estimation between I 'and the corresponding exact image, and the chrominance and luminance information associated with the local correction d 'error, if necessary.

The interpolated image I ', I' _j improved or I ' ₂ improved obtained at the end of one of the steps referenced 23, 282, 294 or 297 can then be interposed between two successive images of an animated sequence, thus as illustrated in figure 4.

Thus, on decoding the sequence 40 for example, the decoder constructs an interpolated image 43, possibly from information transmitted by the coder, which he inserts between two images referenced 41 and 42, in order to increase the visual fluidity of the sequence 40.

Claims

1. A method of constructing at least one image interpolated between a previous image and a following image of an animated sequence, characterized in that it comprises at least one step of motion estimation taking account of information relating to the presence of at least one occlusion zone in said previous image and / or in said following image, say occlusion information.

2. Construction method according to claim 1, characterized in that an image of said sequence is associated with a mesh.

3. Construction method according to claim 2, characterized in that said mesh is a hierarchical mesh having at least two levels of mesh nested within a pyramid of mesh.

4. Construction method according to any one of claims 1 to 3, characterized in that said occlusion information is determined by the implementation of said motion estimation step, said motion estimation step being a forward and / or backward motion estimation step between said previous and next images.

5. Construction method according to any one of claims 1 to 4, characterized in that in the presence of at least one occlusion zone in said previous image (respectively in said following image), it implements the steps following: for each occlusion zone, a step of exclusion, in said previous image (respectively in said following image), of a region of said mesh including at least one mesh corresponding to said occlusion zone, called encompassing region, so as to obtain an approximate previous image (respectively an approximate following image); a front (respectively rear) estimation step, so as to determine motion vectors making it possible to obtain a following image

(respectively previous) approximated from said previous image (respectively following) approximated, called forward motion vectors

(respectively rear); a step of constructing an intermediate image from said approximate previous (respectively following) image and said front (respectively rear) motion vectors.

6. Construction method according to claim 5, characterized in that said encompassing region is a k-disc of said occlusion zone where k≥l.

7. Construction method according to any one of claims 5 and 6, characterized in that said construction step consists in applying said front (respectively rear) motion vectors, weighted by a scalar less than 1, to said previous image ( respectively next) approached.

8. Construction method according to any one of claims 5 to 7, characterized in that it comprises a step of reconstruction of a mesh associated with said intermediate image, consisting, for each occlusion zone, to be implemented a step of determining colorimetric and / or photometric information associated with a part of said mesh corresponding to said encompassing region, so as to obtain said interpolated image.

9. Construction method according to claim 8, characterized in that said determining step implements on the one hand a finite element coding model, and on the other hand an alternative coding model.

10. Construction method according to any one of claims 5 to 7, characterized in that it comprises a step of reconstruction of a mesh associated with said intermediate image, consisting, for each occlusion zone, to be inserted in said mesh part of a mesh associated with said next image or with said previous image corresponding to said encompassing region, so as to obtain said interpolated image.

11. Construction method according to claim 10, characterized in that it further comprises, for each occlusion zone, a step of estimating local movement between said interpolated image and a corresponding exact image, consisting in optimizing the position at least one node of a part of said mesh corresponding to said encompassing region in said inteφolated image, so as to generate a first improved interpolated image.

12. The construction method according to claim 11, characterized in that it further comprises a step of calculating an error image, so as to evaluate an error between said first improved integrated image and said corresponding exact image.

13. Construction method according to claim 12, characterized in that, said error being greater than a predetermined threshold for at least one region of said error image, it implements a step of determining colorimetric information and / or photometric associated with each of said regions, so as to generate a second improved integrated image.

14. Construction method according to claim 13, characterized in that said determining step implements, on the one hand, a finite element coding model, and on the other hand, an alternative coding model.

15. Construction method according to any one of claims 13 and 14, characterized in that said determining step is implemented from said error image or from said corresponding exact image, according to a criterion of predetermined cost.

16. Construction method according to any one of claims 2 to 15, characterized in that said mesh is larger than said image it represents.

17. A method of coding an animated sequence, of the type implementing an interpolation step of at least one image interpolated between a previous image and a following image, characterized in that said interpolation step comprises a sub- motion estimation step taking account of information relating to the presence of at least one occlusion zone in said previous image and / or in said following image, called occlusion information.

18. A method of decoding an animated sequence, of the type implementing an interpolation step of at least one image interpolated between a previous image and a next image, characterized in that said interpolation step comprises a sub- motion estimation step taking into account information relating to the presence at least one occlusion zone in said previous image and / or in said following image, say occlusion information.

19. Display terminal for an animated sequence of images of the type comprising means for integrating at least one image inteφolated between a previous image and a following image, characterized in that said interpolation means comprise means d motion estimation taking into account the presence of at least one occlusion zone in said previous image and / or in said following image.

20. Video signal comprising information making it possible to reconstruct at least one image interpolated between a previous image and a following image of an animated sequence of images, characterized in that said information takes account of the presence of at least one zone d occlusion in said previous image and / or in said following image.

21. Mesh comprising information making it possible to reconstruct at least one image interpolated between a previous image and a following image of an animated sequence of images recorded on a medium usable in a computer, characterized in that said information takes account of the presence at least one occlusion zone in said previous image and / or in said following image.