FR2820256A1 - Image coding and decoding method, e.g. for digital transmission over communications network, selectively using two coding modes, each optimizing the compression of an image as a function of different optimization criteria - Google Patents

Abstract

Information on the choice of an image coding mode is known from a decoder according to at least one of : (1) a predefined choice known to the coding and decoding units; (2) information relating to the choice included in a data stream including at least some of the coded images; (3) information relating to the choice included in a data stream and independent of the coded images; and (4) determination in an intrinsic manner by the decoder. Preferably the first coding mode involves transformation and time prediction on image blocks and may especially involve MPEG-4 coding, while the second coding involves a mesh operation, the mesh preferably being of triangular form. Independent claims are included for : (1) an image coding device; (2) an image decoding device; (3) an image storage device; (4) an image coding and/or decoding system; (5) a computer program product for image coding and decoding; (6) an image data signal; and (7) application of the image coding method to: (a) digital television; (b) real-time video transmission over an IP network; (c) real-time video transmission over a mobile network; and (d) the storage of image data.

Description

<Desc / Clms Page number 1>

Image coding and decoding methods, devices, systems, signals and corresponding applications.

 The technical field of the invention is that of bit rate reduction coding of moving image sequences, particularly video.

 Video coding applications are particularly numerous.

Let us quote in a non-exclusive way: - The transmission of digital TV; - Real-time video transmission over different types of networks:
IP, mobile, ("IP Streaming");
Computer storage of videos.

 The present document describes a new video coding method by hybridization of an MPEG type coding and a time interpolation coding based on a mesh representation, as well as the decoding method and the structure of the associated binary representation.

 By MPEG type coding is understood a coding based on a temporal prediction and a discrete cosine transformation based on a structure of rigid blocks, often of fixed size, but possibly of variable tille. The 2 representative standards for this coding family being the MPEG-4 version 1 to 4 and ITU-T / H standards. 263 until version 2.

The video coding and decoding schemes proposed to date fall into 2 categories: - Standardized codings, either by ISO / MPEG or by ITU-T, all based on the same type of techniques (time prediction and a transformation into a discrete cosine based on a block structure) - The codings under development proposed by research laboratories, which use a wide range of techniques: Coding by
Wavelets, by Regions, by Fractals, by Meshes, etc.

 Currently, MPEG-4 encoding is considered to be the state of the art not only of standardized encodings, but also of all published encodings.

 MPEG-4 or ITU-T / H encodings. 263 ++ are considered to be

<Desc / Clms Page number 2>

ayant atteint leurs limites, notamment à cause de la structure de blocs rigides de taille fixe utilisée comme support de l'ensemble des calculs et opérations de codage. En particulier, la prédiction temporelle des images au sein d'une séquence est insuffisamment exploitée.

having reached their limits, in particular because of the structure of rigid blocks of fixed size used to support all the calculations and coding operations. In particular, the temporal prediction of images within a sequence is insufficiently exploited.

 Furthermore, the published alternative codings have not yet reached a sufficient degree of optimization.

 Thus, in order to obtain coded video sequences at low bit rate, coders generally reduce the size of the images and time sub-sample the original video sequence. However, the second technique has the drawback of restoring jerky movements which are more or less annoying for the user depending on the level of subsampling.

 To avoid such jerks, it is important to reconstruct the missing images (not coded) with the decoder by temporal interpolation.

 However, current techniques for temporal interpolation of images do not make it possible to obtain satisfactory results, especially when they are implemented at the single decoder. Indeed, these techniques are the source of visual artefacts linked to block-based motion compensation techniques which define only one motion vector for all the pixels in a block.

 The objective of the invention is precisely to overcome the limitations of the prior techniques.

 More specifically, an objective of the invention is to provide a technique for coding, and decoding, image data, which makes it possible to obtain a reduced bit rate and / or a better quality of reconstructed image, compared with the techniques. known.

 This objective is achieved, according to the invention, using an image coding method, selectively implementing at least two image coding modes, each optimizing the compression of at least one image. a video sequence based on different optimization criteria.

 According to several advantageous embodiments, information on the choice of one of said coding modes can be known to a decoder according to

<Desc / Clms Page number 3>

moins une des techniques appartenant au groupe comprenant : choix prédéfini, connu au codage et au décodage ; - information représentative du choix incluse dans un flux de données comprenant au moins certaines des données d'images codées ; - information représentative du choix incluse dans un flux de données indépendant des données d'images codées ; - détermination du choix de façon intrinsèque, par le décodeur.

at least one of the techniques belonging to the group comprising: predefined choice, known in coding and decoding; - information representative of the choice included in a data stream comprising at least some of the coded image data; - information representative of the choice included in a data stream independent of the coded image data; - determination of the choice intrinsically, by the decoder.

 Advantageously, the method comprises a step of selecting a coding mode to be applied to said image, among at least: a first coding optimizing a photometric representation of an image; a second coding optimizing a representation of the movement between at least two images.

 Preferably, said second coding takes account of at least one previous image and / or at least one following image coded using said first coding.

 Advantageously, said second coding takes account of a motion vector field calculated from the immediately preceding image coded using said first coding and / or a motion vector field calculated from the image immediately following coded using said first coding.

 In this case, said motion vector fields can be used to determine a deduced motion vector field, associated with an image coded using said second coding.

 According to a preferred embodiment of the invention, said selection step is based on the implementation of a subsampling of fixed factor N, an image on N being coded using said first coding.

 Advantageously, this value N is variable, as a function of at least one predetermined criterion.

 According to a particular embodiment, said first coding implements a transformation on blocks of images and a temporal prediction by

<Desc / Clms Page number 4>

blocs. Il peut notamment s'agir d'un codage MPEG-4.

blocks. It can in particular be an MPEG-4 encoding.

In the latter case, type 1 (intra) and / or type P (predictive) images are preferably used (and not, preferably, type B images).

 According to another particular aspect of the invention, said second coding advantageously rests on the implementation of a mesh, and for example a triangular mesh.

 In this case, the method preferably comprises a step for managing the occlusion zones.

 The data produced can be gathered in a single flow.

Advantageously, it is possible to provide at least two data streams, which can be transmitted over independent transmission channels.

 Said data flows advantageously belong to the group comprising: a global header; image data encoded according to said first encoding; image data coded according to said second coding.

 The transmission of flows can therefore take place independently. This allows in particular a progressive and / or partial decoding of the images, according to the means and the needs.

Thus, according to a particular embodiment of the invention, the following aspects are used in particular:
Advanced optimization of the constituent modules of standard MPEG or ITU-T / H codings. 263 - Efficient temporal prediction and associated error coding for mesh-based techniques.

 Indeed, the mesh-based approach avoids the usual block effects thanks to the use of continuous motion fields. In addition, the mesh technique makes it possible to detect occultations of objects, as well as an error coding well suited to these areas. By additionally combining a coding

<Desc / Clms Page number 5>

d'erreur de type MPEG autour de ces zones, il est possible d'améliorer sensiblement l'efficacité de l'interpolation à un coût largement inférieur aux images bidirectionnelles (images de type B) proposées par MPEG.

MPEG type error around these areas, it is possible to significantly improve the efficiency of the interpolation at a cost much lower than the bidirectional images (type B images) offered by MPEG.

Thus, it is possible to effectively code the basic information at a low temporal resolution thanks to the coding of the MPEG type, with good quality, then to restore the full fluidity of the sequence thanks to the coding in interpolated mode by mesh.

les systèmes de codage, de transmission et/ou de décodage d'un signal d'images codé à l'aide du procédé de codage décrit ci-dessus (le choix d'un desdits modes de codage pouvant avantageusement être connue d'un décodeur selon au moins une des techniques appartenant au groupe comprenant : choix prédéfini, connu au codage et au décodage ; - information représentative du choix incluse dans un flux de données comprenant au moins certaines des données d'images codées ; - information représentative du choix incluse dans un flux de données indépendant des données d'images codées ; The invention also relates, of course, to: the methods of decoding an image signal coded using the coding method described above; the devices for coding an image signal coded using the coding method described above; the devices for decoding an image signal coded using the coding method described above (advantageously comprising means for determining at least part of a vector field and / or at least part of the occlusion zones, similar to those used during coding); devices for storing at least one image signal coded using the coding method described above;

the systems for coding, transmitting and / or decoding an image signal coded using the coding method described above (the choice of one of said coding modes can advantageously be known to a decoder according to at least one of the techniques belonging to the group comprising: predefined choice, known in coding and decoding; - information representative of the choice included in a data stream comprising at least some of the coded image data; - information representative of the choice included in a data stream independent of the coded image data;

<Desc / Clms Page number 6>

détermination du choix de façon intrinsèque, par le décodeur) ; les produits programme d'ordinateur pour le codage et/ou le décodage d'un signal d'images codé à l'aide du procédé de codage ; les supports de données d'un tel programme.

determination of the choice intrinsically, by the decoder); computer program products for encoding and / or decoding an image signal encoded using the encoding method; the data carriers of such a program.

 The invention also relates to the image data signals comprising data coded according to the method described above.

 Advantageously, this signal comprises at least one indicator indicating whether or not the process is activated.

 Preferably, the signal comprises data specifying the structure of the frames, at the start of the video sequence and / or in each signal frame.

 Advantageously, a sequence coded using said second coding begins with a header specifying the number of frames coded according to this second coding.

 According to a particular embodiment, the signal comprises at least two data streams, which can be transmitted over independent transmission channels.

 In this case, said data streams advantageously belong to the group comprising: a global header; image data encoded according to said first encoding; image data coded according to said second coding.

 The invention finds applications in numerous fields, and in particular in the fields belonging to the group comprising: digital television; real-time video over IP network; network real-time video to mobiles; storage of image data.

 Other characteristics and advantages of the invention will appear more clearly on reading the description of a preferred embodiment of

<Desc / Clms Page number 7>

l'invention, donné à titre de simple exemple illustratif et non limitatif, et des dessins annexés parmi lesquels : la figure lest un schéma de principe du codage de l'invention ; la figure 2 présente un exemple de structure de maillage hiérarchique pour le mouvement ;

la figure 3 illustre le principe de l'interpolation affine sur une maille triangulaire ; la figure 4 est un exemple d'occultation détectée par recouvrement de triangles ; la figure 5 illustre le processus de transformation d'un triangle quelconque de l'image en une matrice carrée symétrique ; la figure 6 illustre la transformation d'un triangle quelconque en un triangle isocèle rectangle ; la figure 7 illustre un maillage hiérarchique et la représentation par arbre quaternaire associée ; la figure 8 est un exemple de décision de codage pour le maillage hiérarchique ; la figure 9 présente une structure globale d'un train binaire selon l'invention ; la figure 10 présente un diagramme bloc d'un décodeur selon l'invention.

the invention, given by way of simple illustrative and nonlimiting example, and of the appended drawings among which: FIG. 1 is a block diagram of the coding of the invention; FIG. 2 presents an example of a hierarchical mesh structure for the movement;

FIG. 3 illustrates the principle of the affine interpolation on a triangular mesh; Figure 4 is an example of occultation detected by overlapping of triangles; FIG. 5 illustrates the process of transformation of any triangle of the image into a symmetrical square matrix; FIG. 6 illustrates the transformation of any triangle into a right isosceles triangle; FIG. 7 illustrates a hierarchical mesh and the representation by associated quaternary tree; FIG. 8 is an example of a coding decision for the hierarchical mesh; FIG. 9 shows an overall structure of a binary train according to the invention; FIG. 10 presents a block diagram of a decoder according to the invention.

 The embodiment of the invention described below essentially consists in the hybridization of an MPEG type coding, for example MPEG-4 with a mesh coding operating in interpolated mode, also called B mode or B images in the MPEG standards.

 Note that the MPEG-4 coding mentioned here can be replaced by any coder based on equivalent techniques, i.e. using a temporal prediction and a discrete cosine transformation based on a block structure, and the quantifications and entropy codings for the information generated.

In particular, ITU-T / H coding. 263 ++ can be substituted for MPEG-4 encoding.

<Desc / Clms Page number 8>

For each image of the sequence entering the coder, the latter decides according to a certain decision process (for example, a temporal subsampling of a fixed factor) to code it with the MPEG-4 encoding module, or well with the mesh-based encoding module.

 The coded images in mesh mode use as references for their temporal prediction the coded images in MPEG-4 mode located immediately before or immediately after the group of coded images in mesh mode to which they belong.

 The key point of the compression efficiency of the invention is that the mesh-based motion compensation leads to a very efficient time prediction, for a very low associated coding cost.

 Indeed, this technique: - Takes into account different types of movements in the images - Treats properly the overlaps and discoveries of zones due to the movements of objects.

 Figure 1 gives a general view of the principle of the coder.

 First of all, the incoming images are routed either to the MPEG encoding module, or to the mesh-based encoding module, according to a given decision mode, for example according to a predefined rhythm: 1 image on N is coded in MPEG, the others in interpolated mesh mode. We denote by Nk the numbers of the images coded in MPEG mode.

All other images! N <l <Nt, are encoded by an encoder based on a mesh, for example triangular, operating in interpolated mode called mode B. the general principle of this encoder is as follows:
1. Calculation of the front and rear motion fields between the images
Nk and N ,,,. these fields are modeled in the form of triangular meshes.

2. Detection of occultation zones, not predictable, in the images
It to code, from the knowledge of these fields of movement
3. Specific coding of these blackout zones according to one of the three

<Desc / Clms Page number 9>

modes possibles suivants : - Prédiction avec l'une des images de référence (Nk, N,,,, ou ces images compensées en mouvement avec leurs champs de mouvement) sans compensation de mouvement, puis codage de l'erreur de prédiction avec une technique basée maillage triangulaire - Prédiction avec l'une des images de référence (Nk, Nu, ou ces images compensées en mouvement avec leurs champs de mouvement) avec compensation de mouvement intra- image, puis codage de l'erreur de prédiction avec une technique basée maillage triangulaire - Codage intra-image basé avec une technique basée maillage triangulaire
4. Optionnellement, codage de type MPEG mode P de l'erreur résiduelle de prédiction ou de codage, limité à une zone autour de la zone d'occultation.

following possible modes: - Prediction with one of the reference images (Nk, N ,,,, or these motion compensated images with their motion fields) without motion compensation, then coding of the prediction error with a technique triangular mesh based - Prediction with one of the reference images (Nk, Nu, or these motion compensated images with their motion fields) with intra-image motion compensation, then coding of the prediction error with a based technique triangular mesh - Intra-image coding based on a triangular mesh based technique
4. Optionally, MPEG type P coding of the residual prediction or coding error, limited to an area around the occultation area.

Des champs de mouvement avant et arrière entre les images N et N, sont calculés, sous la forme de maillages hiérarchiques, par exemple triangulaires, r, et Tfk+1, comme indiqué figure 2. 1. Calculation of the front and rear motion fields between the N images; and N

Fields of forward and backward movement between the images N and N, are calculated, in the form of hierarchical meshes, for example triangular, r, and Tfk + 1, as indicated figure 2.

 Such meshes are obtained by dividing certain meshes, for example, the triangular meshes are divided into 4 sub-triangles, according to a certain criterion during the process of estimation of the movement. At each level of the hierarchy, division decisions or not are made for each mesh. Once these divisions have been decided, the adjacent meshes of the divided meshes are then divided so as to maintain a conforming mesh structure. The initial mesh, before division (top of the hierarchy), can be arbitrary.

 In the example in Figure 2, the motion estimator decides to divide triangles 3 and 8. This results in the division of triangles 2,4, 7 and 9. The process

<Desc / Clms Page number 10>

est itéré jusqu'à un niveau prédéfini de hiérarchie.

is iterated up to a predefined level of hierarchy.

où : . e dénote l'élément triangulaire de T contenant le point courant pde coordonnées x et y, . {ver(e)} dénote l'ensemble de ses trois noeuds ou sommets, numérotés i, j, k de positions et Pk 1

"'FI (1 = i, j, k) représente les coordonnées barycentriques du point p (x, y) dans l'élément triangulaire el, k avec :

ff,.)') =a ; ; ; , K ;. P,', t ; e9 ! . 11,... . ' .). , 'Pt (x, y) = 0 sinon

Un tel modèle définit un champ partout continu. De plus, il permet un contrôle fin de la précision de représentation, caractéristique essentielle pour la compression. In the case of triangular meshes, the expression of the field of motion defined by a triangular mesh T is given on each triangle e by:

or : . e denotes the triangular element of T containing the current point p of coordinates x and y,. {ver (e)} denotes all of its three nodes or vertices, numbered i, j, k with positions and Pk 1

"'FI (1 = i, j, k) represents the barycentric coordinates of the point p (x, y) in the triangular element el, k with:

ff ,.) ') = a; ; ; , K;. P, ', t; e9! . 11, .... '.). , 'Pt (x, y) = 0 otherwise

Such a model defines an everywhere continuous field. In addition, it allows fine control of the representation accuracy, an essential characteristic for compression.

 At each level of the hierarchy of meshes, the nodal motion vectors are calculated so as to minimize a prediction error. Different mesh-based motion estimators can be used, for example that described in patent FR nO 98 11227, or FR nO 99 15568.

 The important point is that the final mesh results from a hierarchical process starting from an initial mesh by divisions. This hierarchical character is in fact used for the differential coding of the nodal motion vectors between a node and its parent nodes (the ends of the arc on which it has been inserted). The structure of the mesh is recalculated at the decoder on the basis of the knowledge of the initial mesh, and of the mesh division indicators.

 Thus, at the end of the process, 2 motion meshes are obtained for each group of images comprised between the images Nk and Nk + 1, used for

<Desc / Clms Page number 11>

reconstruire l'ensemble des images du groupe.

reconstruct all of the group's images.

2. Detection of blackout areas
From these 2 meshes, the occultation zones, that is to say those that cannot be predicted in the image N, from the image Nk + 1 or vice versa, due to the overlap or the discovery of d objects are detected.

 These zones are simply defined by the overlapping triangles, once displaced by their nodal vectors.

 The figure illustrates the occultation detection based on the overlapping of triangles after displacement.

 The coder can continue the motion estimation by deactivating the triangles of the occultation zones, so as to obtain less biased displacement vectors.

b c'est 2 maillages de mouvement complets T\ et Tfk+l qui sont codés et insérés dans le train binaire. Le décodeur est alors capable de retrouver les zones d'occultation à partir de ces 2 maillages. This is however strictly internal to the coder's strategy, and ultimately,

b it is 2 complete motion meshes T \ and Tfk + l which are coded and inserted in the binary train. The decoder is then able to find the occultation zones from these 2 meshes.

These occultation zones are defined on the images Nk and Nk + 1 and once detected, the triangles belonging to them are labeled accordingly, both to the coder and to the decoder.

However, the coder needs to know these zones on the images Nk + 1 to Nk + 1. These are simply obtained by projection of the meshes T \ and Tfk + l on the image to be coded, by application of the nodal motion vectors, renormalized to take account of the temporal distance between the current image and the reference image Nk or Nk + 1.

prédiction est sélectionnée parmi 1, 1, mais aussi Ici'qui est l'image obtenue à l'instant 1 par compensation de mouvement avec le maillage T\ ou Tn à un niveau où il n'y a pas encore de recouvrement de maille. 3. Coding of shading areas:
For each occultation zone, the reference image for a possible

prediction is selected from 1, 1, but also Ici'qui is the image obtained at time 1 by motion compensation with the mesh T \ or Tn at a level where there is no mesh overlap yet.

More precisely, the choice between INk and IN (k + l) simply depends on the

<Desc / Clms Page number 12>

b maillage T\ ou Tf, ayant engendré la zone courante d'occultation. Ensuite, ce maillage est utilisé pour prédire l'image et donner I',. Le choix entre Ici et Ilk ou IN (k+l) se fait sur critère d'erreur de prédiction : l'image donnant l'erreur la plus faible est retenue. Ainsi, il suffit d'insérer 1 bit dans le flux binaire, par zone, pour coder le choix de la prédiction retenue.

b mesh T \ or Tf, having generated the current occultation zone. Then, this mesh is used to predict the image and give I ',. The choice between Here and Ilk or IN (k + l) is made on the basis of a prediction error criterion: the image giving the lowest error is retained. Thus, it suffices to insert 1 bit in the bit stream, per zone, to code the choice of the prediction retained.

 Denote I, the selected reference image.

The rest of the coding of these zones consists of 2 steps: - A prediction step - A coding step of the prediction error or the original texture in case of bad prediction
3. 1 Residual prediction of the texture of the occlusion zones
3 modes can be used, exclusively. The decision is made based on the criterion of least error.

Mode 1:
The values Y, U and V of a pixel in the area are simply that of the pixel with the same location of the reference image Ire Let î, the resulting image. We then code the prediction error between Î, and Il.

Mode 2:
An estimation of movement is then carried out between I, (the image to be coded) and Îl (the result of the prediction of mode 1) on the occultation zone. The resulting mesh, resulting from the last level of the mesh T ,, r = k or k + l, before the overlaps of meshes, is then coded as well as its nodal movements. Finally, the residual prediction error is coded according to a procedure defined below.

Mode 3:
No prediction is made and the original values of the pixels in the area are coded.

 4. Coding of the texture or the prediction error on the occultation zones.

 The original texture and the prediction error undergo the same coding, the principle of which is as follows:

<Desc / Clms Page number 13>

On suppose qu'un maillage triangulaire initial a pu être défini à partir du maillage de mouvement T, retenu pour la prédiction de la zone à coder. La façon de dériver ce maillage initial sera décrit plus loin.

It is assumed that an initial triangular mesh could be defined from the motion mesh T, retained for the prediction of the area to be coded. The way to derive this initial mesh will be described later.

The texture is then approximated on each mesh according to a choice: - The meshes rich in high frequencies are coded based on transform in discrete cosine, known as DCT - The smoother meshes are coded by a model of finite elements, affine .

 Again, we will take advantage of a hierarchical approach to reduce the cost of coding the representation by mesh.

 The approach adopted makes it possible to conserve the low coding cost associated with a regular hierarchy of meshes while allowing local adaptation to the content of the images made possible by the irregular decomposition of meshes.

 From the initial coarse mesh of the zone, the meshes are subdivided into 4 triangular sub-meshes up to a given level.

 On the last level, an optional permutation of the diagonals of the quadrilaterals generated by 2 adjacent triangles can be permuted, if this induces a reduction in the approximation error.

4. 1 Initialization of the texture mesh on the occultation zones
This mesh is simply given by the last level of Tr (mesh resulting from the displacement of Tk or Tek +, depending on the direction chosen) before the appearance of reversals on the zone considered. Thus, we have a texture mesh which fits naturally into the movement mesh, since extracted from the latter.

4. 2 Representations used for the texture on the triangles
2 representations are combined: affine interpolation and 4-triangular DCT.

Affine interpolation
The nodes of the triangular mesh carry the photometric information (color, error) and the interpolation for the points inside the triangle is

<Desc / Clms Page number 14>

effectuée par un élément fini de Lagrange, également appelé interpolation affine.

performed by a finite element of Lagrange, also called affine interpolation.

où (/=/,./, représente les coordonnées barycentriques du point p (x, y). v (p) peut être l'une des composantes photométriquesY, U ou V du point, ou encore l'erreur de prédiction pour ces composantes. The value v (p) of the point p (x, y) inside the triangle g, k defined by the 3 nodes PI, l = i, j, k is given by the following equation:

where (/=/,./, represents the barycentric coordinates of the point p (x, y). v (p) can be one of the photometric components Y, U or V of the point, or even the prediction error for these components.

 Several methods can be used for the calculation of the nodal values, in particular the methods of least squares.

Discrete cosine transformation (DCT) on triangles
The principle of the method consists in transforming any triangle into a right isosceles reference triangle. The content of this triangle is then symmetrized with respect to the hypotenuse to give a symmetrical square matrix (Figure 4).

 A classic (square) CSD is then applied to this matrix. We can show that the transformed matrix is also symmetrical. Only the coefficients of its lower triangle are then quantified and then statistically coded (entropy coding).

 Figure 4 describes the different stages of the process: selection of the triangle T, affine transformation of this into a right isosceles triangle T '. Due to the affine transformation, the pixels of the triangle are no longer located on a regular orthogonal grid, and the photometric values of the interior of the reference triangle should be resampled. For this, we use a process analogous to that of motion compensation in the image (in this case the affine transformation), using an interpolator, for example bilinear.

 The affine transformation F and its inverse FI are defined by the following equations:

<Desc / Clms Page number 15>

Les valeurs photométriques M (i, j) du triangle T' (donc du bloc M, symétrisé de T') sont obtenue par transformation inverse FI puis interpolation r de la texture de l'image à coder :

où : le coefficient M (iy, j x) est la valeur au point Q (X, Y) dont le transformé P (x, y) est F-' (Q) 1 r dénote l'interpolateur utilisé pour calculer la valeur de l'image au point P (x, y), de coordonnées potentiellement non entières.

The photometric values M (i, j) of the triangle T '(therefore of the block M, symmetrized of T') are obtained by inverse transformation FI then interpolation r of the texture of the image to be coded:

where: the coefficient M (iy, jx) is the value at point Q (X, Y) whose transform P (x, y) is F- '(Q) 1 r denotes the interpolator used to calculate the value of l image at point P (x, y), of potentially non-integer coordinates.

où If dénote la texture interpolée à partir des valeurs du bloc M', version quantifié de M. The reconstruction r of the Fest texture given by:

where If denotes the interpolated texture from the values of block M ', quantified version of M.

 This technique can only be applied to triangles with non-zero areas. But such triangles do not require texture coding by definition.

 Unlike the SADCT (DCT adapted to a shape), this transformation does not guarantee perfect reconstruction after reverse transformation, even in the absence of quantification.

où : . E est la partie entière par excès, . A@ est l'aire du triangle i. In order to reduce the reconstruction error, a scale factor is introduced for the calculation of the block Mi (of size N, x N,) for each triangle i:

or : . E is the whole part by excess,. A @ is the area of triangle i.

<Desc / Clms Page number 16>

 In fact, a = 1 achieves an interesting compromise, which is more effective for triangles close to an isosceles. Casa <1 is used in conjunction with the quantification step to compress the volume of information.

 Once the block Mi has been defined for each triangle, a conventional DCT transformation is applied to them, and the transformed coefficients are quantified according to several possible methods, for example a uniform scalar quantization, or even an incremental quantization with the frequency of the transformed coefficient. The use of well known MPEG or JPEG quantization matrices is also possible.

The expression of the DCT is given by:

k, , = if = 0 sinon linon

On a la relation F (u, v) = F (v, u) car : f (i, j) = f (j, i) (Vu, v, i, j = 0,"-, N-1) par définition.

k,, = if = 0 otherwise linon

We have the relation F (u, v) = F (v, u) because: f (i, j) = f (j, i) (Vu, v, i, j = 0, "-, N-1) by definition.

Consequently, one can be satisfied with calculating only the coefficients of the lower part of the transformed matrix.

4. 3 global texture coding
As indicated above, we use a uniform hierarchical mesh obtained by dividing each triangle of a given level of the hierarchy into 4 sub-triangles, by inserting nodes in the middle of the arcs. The process is repeated iteratively up to a maximum level. This hierarchy of triangles is also represented and managed by the coder in the form of a quaternary tree (Figure 5). Note that only the triangles included in the area to be coded are taken into account. The basic initial mesh construction process guarantees that any triangle in the mesh hierarchy belongs to the area to be coded.

 The process of mesh coding of a concealment area can be summarized as follows

<Desc / Clms Page number 17>

(D un maillage hiérarchique imbriqué est défini sur la zone à coder, par creation d'un maillage initial régulier puis sub-division itérative des triangles en 4 sous-triangles par insertion de nouveaux noeuds au milieu des arcs. Les valeurs aux noeuds sont calculées pour minimiser l'erreur d'approximation de la zone par le maillage (D les valeurs des pixels sont approchées par une interpolation affine sur le triangle les contenant à partir de ses valeurs aux noeuds.

(A nested hierarchical mesh is defined on the area to be coded, by creating a regular initial mesh then iterative sub-division of the triangles into 4 sub-triangles by insertion of new nodes in the middle of the arcs. The values at the nodes are calculated to minimize the error of approximation of the zone by the mesh (D the values of the pixels are approached by an affine interpolation on the triangle containing them starting from its values at the nodes.

 For each triangle of the hierarchy, we then evaluate the approximation error E then we decide on the different modes of representation and coding according to 2 thresholds: o, eut 2 (D if E <cfj, the affine interpolation is sufficient on the triangle; GO if Cri Es2, it is necessary to use a finer decomposition of the triangle to obtain a good approximation, always by affine interpolation. On,; G) if E <02, the triangle is textured and we code the error d refines using DCT.

 Finally, on the finest mesh, we test the error reduction provided by the diagonal permutation of the quadrilaterals formed by 2 adjacent triangles.

In the event of a positive result, this permutation is validated.

According to the coding modes chosen for the different triangles, the different information is coded as follows
The YUV nodal values are first predicted from the values of the parent nodes (ends of the arc where the current node has been inserted). The difference between the value of the node and its predicted value is then quantified.

 Finally, the structure of the quaternary tree (including the indicators of division or not of the triangles), the indicators of permutation of diagonals, the nodal differential values of YUV and the quantized DCT coefficients are coded by an arithmetic coder and inserted in the train binary.

5. Summary of the information coded in the bit stream of the coded frames by mesh
Each group of frames encoded in mesh mode between Nk + 1 and Nk + 1-1

<Desc / Clms Page number 18>

(où N N sont respectivement la trame précédente et la trame suivante encodée en mode MPEG) est représenté comme un tout dans le flux binaire.

(where NN are respectively the previous frame and the following frame encoded in MPEG mode) is represented as a whole in the bit stream.

noeuds) T\ et Tfk+l
La texture d'erreur de prédiction ou originale, pour chaque image du groupe
6. Structure globale du train binaire
Le train binaire global consiste en une succession de trames encodées en mode MPEG, et de groupes de trames encodés en mode maillage, comme indiqué figure 8. The information conveyed includes, in coded form:
A header for the whole group of frames, including among others the actual number of encoded frames
Motion meshes (structure and displacement vectors of

nodes) T \ and Tfk + l
The prediction or original error texture, for each image in the group
6. Overall structure of the binary train
The global bitstream consists of a succession of frames encoded in MPEG mode, and of groups of frames encoded in mesh mode, as shown in Figure 8.

 The global header of the bit stream representative of the coded sequence contains inter alia the indication of hybrid coding.

 The part of the bit stream corresponding to a group of coded frames in mesh mode begins with a header indicating inter alia the number of frames actually coded, possibly zero.

 The different data streams (bit streams) corresponding respectively to the global header of the coded sequence, to the MPEG encoded images and to the groups of i images encoded in mesh-interpolated mode can be sent on different independent channels if necessary. In particular, the coding method allows hierarchical (or scalable) decoding of the sequence, that is to say a decoding using only part of the total bit rate.

7. Decoding process
FIG. 9 gives a general view of the principle of decoding.

 First of all, the decoding of the header makes it possible to activate the hybrid decoding.

 Then, the decoder recognizes for each part of the bit stream corresponding to an autonomous entity whether it is an MPEG-4 encoded frame or a group of frames encoded by mesh.

<Desc / Clms Page number 19>

The MPEG-4 frames are supplied to the MPEG-4 decoding module, and the groups of coded frames in mesh mode are supplied to the mesh decoding module.

7. 1 Decoding based on mesh
First of all, the motion meshes T \ and Tf (, + I) for the group of images I @, NK <l <Nk + 1 are decoded.

 Then, the occultation zones for these images are found according to the same process as with the coder.

 The pixels outside the occultation zones are simply interpolated from the images INk and I, and from the motion fields T \ and T.

 The coarsest texture mesh (top of the hierarchy) is found for each occultation zone according to a process identical to that of the coder.

 The information associated with the corresponding hierarchical mesh (indicator for dividing triangles, decisions of affine interpolation or DCT coding, differential nodal YUV value and quantized DCT coefficients) is then decoded and the YUV values of the pixels of these areas reconstructed.

Claims (32)

CLAIMS 1. Image coding method, characterized in that it selectively implements at least two image coding modes, each optimizing the compression of at least one image of a video sequence according to criteria of optimization.  2. Coding method according to claim 1, characterized in that information on the choice of one of said coding modes is known to a decoder according to at least one of the techniques belonging to the group comprising: predefined choice, known to coding and decoding; information representative of the choice included in a data stream comprising at least some of the coded image data; information representative of the choice included in a data stream independent of the coded image data; determination of the choice intrinsically, by the decoder.  3. Coding method according to any one of claims 1 and 2, characterized in that it comprises a step of selecting a coding mode to be applied to said image, among at least: a first coding substantially optimizing a representation photometric of an image; a second coding substantially optimizing a representation of the movement between at least two images.  4. Coding method according to claim 3, characterized in that said second coding takes account of at least one previous image and / or at least one following image coded using said first coding.  5. Coding method according to claim 4, characterized in that said second coding takes into account a field of motion vectors calculated from the immediately preceding image coded using said first coding and / or a field motion vectors calculated from the immediately following image coded using said first coding. <Desc / Clms Page number 21>

6. Coding method according to claim 5, characterized in that said motion vector fields are used to determine a deduced motion vector field, associated with an image coded using said second coding.  7. Coding method according to any one of claims 3 to 6, characterized in that said selection step is based on the implementation of a subsampling of fixed factor N, an image on N being coded with using said first coding.  8. Coding method according to claim 7, characterized in that N is greater than 2.  9. Coding method according to any one of claims 7 and 8, characterized in that N is variable.  10. Coding method according to any one of claims 3 to 9, characterized in that said first coding implements a transformation on blocks of images and a temporal prediction by blocks.  11. Coding method according to claim 10, characterized in that said first coding is MPEG-4 coding.  12. Coding method according to claim 11, characterized in that the images delivered by said MPEG-4 coding comprise type 1 (intra) and / or type P (predictive) images.  13. Coding method according to any one of claims 3 to 12, characterized in that said second coding is based on the implementation of a mesh.  14. Coding method according to claim 13, characterized in that said mesh is triangular.  15. Coding method according to any one of claims 13 and 14, characterized in that it comprises a step for managing the occlusion zones.  16. Coding method according to any one of claims 1 to 15, characterized in that it produces at least two data streams, which can be transmitted over independent transmission channels. <Desc / Clms Page number 22>

17. Coding method according to claim 16, characterized in that said data streams belong to the group comprising: a global header; image data encoded according to said first encoding; image data coded according to said second coding.  18. A method of decoding a coded image signal using the coding method of any one of claims 1 to 17.  20. Device for coding an image signal coded using the coding method of any one of claims 1 to 17.  21. Device for decoding an image signal coded using the coding method of any one of claims 1 to 17.  22. Decoding device according to claim 21, characterized in that it comprises means for determining at least part of a vector field and / or at least part of the occlusion zones, similar to those implemented during coding.  23. Device for storing at least one image signal coded using the coding method of any one of claims 1 to 17.  24. System for coding, transmitting and / or decoding an image signal coded using the coding method of any one of claims 1 to 17.  25. The system as claimed in claim 24, characterized in that information on the choice of one of said coding modes is known to a decoder according to at least one of the techniques belonging to the group comprising: predefined choice, known for coding and for decoding; information representative of the choice included in a data stream comprising at least some of the coded image data; information representative of the choice included in a data stream independent of the coded image data; determination of the choice intrinsically, by the decoder. <Desc / Clms Page number 23>

26. A computer program product for coding and / or decoding an image signal coded using the coding method of any one of claims 1 to 17.  27. Data carrier carrying a computer program product for coding and / or decoding an image signal coded using the coding method of any one of claims 1 to 17.  28. Image data signal, characterized in that it comprises data coded according to the method of any one of claims 1 to 17.  29. Signal according to claim 28, characterized in that at least one indicator indicating whether the method according to any one of claims 1 to 14 is or is not activated.  30. Signal according to any one of claims 28 and 29, characterized in that it comprises data specifying the structure of the frames, at the start of the video sequence and / or in each signal frame.  31. Signal according to any one of claims 28 to 30, characterized in that a sequence coded using said second coding begins with a header specifying the number of frames coded according to this second coding.  32. Signal according to any one of claims 28 to 31, characterized in that it comprises at least two data streams, which can be transmitted on independent transmission channels.  33. Signal according to claim 32, characterized in that said data streams belong to the group comprising: a global header; image data encoded according to said first encoding; image data coded according to said second coding.  34. Application of the coding method according to any one of claims 1 to 17 to at least one of the fields belonging to the group comprising: digital television; real-time video over IP network; <Desc / Clms Page number 24>  network real-time video to mobiles; storage of image data.