Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/537—Motion estimation other than block-based
  • H04N19/543—Motion estimation other than block-based using regions
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/20—Analysis of motion
  • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments

Definitions

• Motion estimator for coding and decoding image sequences.
• the field of the invention is that of coding of images or sequences of images. More specifically, the invention relates to the estimation and compensation of the movement between two given images.
• the concept of movement between two images of a sequence is a concept well known in the scientific and technical community of image processing. This is defined as the two-dimensional vector characterizing the difference in positions between a point of an image and the homologous point of the other image, on the assumption that these points correspond to the same physical point in each of the two images .
• the assignment of a displacement vector to each point of the image, or pixel therefore defines a dense field of movement on the image, as illustrated in FIG. 1.
• the set 10 of the N * M vectors obtained for an image of size N * M is called the field of movement, or optical flow, between two images 11 and 12 corresponding respectively to the instants t and t + 1.
• the estimation of the movement between two images finds applications in all fields of imagery and image processing.
• - video coding the motion field is used for the prediction of an image from the previously decoded images.
• the representation by mesh of such a field is notably defined in different norms or standards, such as the MPEG-4 standard.
• medical imagery analysis of the movement of the human body, heart, etc. object tracking (for example in the context of road traffic control).
• the invention applies in particular to the processing of two-dimensional (2D) images, but also to the processing of images representative of multidimensional objects (in particular 3D).
• 3D multidimensional objects
• a first technique consists in calculating a dense field of motion for the image, and in assigning a displacement vector to each pixel.
• a second technique aims to calculate a displacement vector per rectangular block of the image.
• a third technique is based on the calculation of a mathematical model of the movement by region, possibly of arbitrary shape, of the image. Such a technique therefore also implements a segmentation of the image into regions, corresponding for example to the various video objects constituting the illustrated scene.
• a final technique described for example in patent application FR 2 783 123 entitled “Method for estimating the movement between two images", consists in calculating a field defined by a finite element model.
• a mesh is associated with the image, and an interpolation or extrapolation function makes it possible to calculate the value of the field at each point of the image, inside each of the meshes, as illustrated by the figure. 2.
• This technique is also based on the implementation of a differential method, which determines the movement parameters by optimization of a mathematical quality criterion (for example, a quadratic error between the image and its value predicted by the compensation of the movement).
• a mathematical quality criterion for example, a quadratic error between the image and its value predicted by the compensation of the movement.
• Such an optimization is carried out using a differential mathematical optimization method, in the case of criteria having the necessary mathematical properties (for example, a differentiable criterion).
• a triangular mesh is associated with the image 21.
• the value of the motion vectors of the vertices referenced 22, 23 and 24 of the mesh is optimized.
• one carries out an interpolation (or an extrapolation) in order to determine the vectors of movement of the points not corresponding to vertices of the mesh, such as the points referenced 25 and 26 for example.
• Such a motion estimation method based on the implementation of a hierarchical mesh representative of the image to be coded, is particularly suitable for the progressive transmission of data, and therefore represents an advantageous solution to the problems of band saturation. bandwidth and adaptability to communication networks, of different natures and capacities. However, such a method encounters certain difficulties when tracking movement.
• a drawback of this technique of the prior art is that it generates displacements of meshes, causing settlement or uncovering on certain areas of the image, as illustrated in FIG. 5.
• the deformable meshes define a continuous representation of a movement field (that is to say that the mesh follows the moving objects in the scene), while the real movement of a video sequence is of a discontinuous nature. Consequently, when an object moves for example from left to right in the image 51, the left zone 52, discovered by the displacement of the object, represents new information, and, therefore, is not more meshed.
• the right-hand area 53 supports a compaction of meshes. Similarly, when different planes and objects overlap in a scene, occultation zones appear, generating lines of discontinuity.
• the vertices of the mesh also carry photometric and / or colorimetric information which make it possible to generate a photometric and / or colorimetric field, and therefore to approximate the image.
• the appearance of non-meshed discovery areas therefore corresponds to the entry into the image of areas whose photometry and / or colorimetry cannot be approximated, that is to say black areas.
• This technique of the prior art also has the drawback of causing an overly approximation of the areas leaving the image that is too costly, corresponding to the areas of compacting of the meshes. Indeed, in the settlement zones, one obtains an over-representation of the field of movement, because too many meshs are used to make the approximation of a reduced portion of the field of movement. Such an over-representation does not harm the quality of approximation of the field of motion, but generates an additional cost of transmission.
• Another solution envisaged to compensate for the appearance of the areas of uncovering or of areas of settlement of the mesh consists in inserting new points on the black areas and in constructing a constrained Delaunay mesh.
• a disadvantage of this technique of the prior art is that it is costly in terms of time and speed of transmission and / or storage of information.
• a postprocessing consists in applying the motion vectors as estimated to the various nodes of the mesh, then in detecting the motion vectors having caused the appearance of defects in the mesh, and finally in correcting their value, so as to compensate for the mesh reversal phenomena.
• a second embodiment of a postprocessing consists of an iterative process: with each iteration, one applies a part of the estimated displacement to each of the nodes of the mesh, so as not to generate a reversal of meshes. The iterations are then repeated until a convergence of the process is obtained.
• the methods of postprocessing acting after the vectors of movement of the various nodes of the mesh were estimated, they do not allow an optimal management of the reversals of meshes. Indeed, in the post-processing methods, the motion vectors are corrected independently of their contribution to the minimization of the prediction error (for example of the quadratic error between the image and its value predicted by motion compensation ).
• Such a technique consists in undertaking an initial optimization of the motion vectors of the mesh, by letting the motion estimator create possible reversals between two successive instants t ⁇ and tj, so as to detect the discontinuity zones thus generated.
• the method then consists in carrying out a new estimation of the movement between the instants t x and t 2 , by excluding the faulty areas (that is to say the areas containing at least one mesh reversal), in order to minimize the prediction error between the two images corresponding to the instants t j and t ⁇ considered.
• This new estimation makes it possible to determine the optimal motion vectors for the continuous area of the image (that is to say admitting a bijection between t j and ⁇ ) and thus to avoid the values of the motion vectors obtained at during the initial optimization are not disturbed by the existence of discontinuity zones.
• the faulty areas are then approximated by a frequency or spatial method, when the method is applied to image compression for example, and permanently excluded from the optimization method, when the technique is applied to the monitoring of video objects for example .
• a drawback of this technique of the prior art is that it does not make it possible to manage the appearance of areas of discoveries and areas of compacting of meshes during a translation of an object within the image, as described above.
• an objective of the invention is to provide a technique for coding images represented using a mesh, allowing estimation and compensation of the movement within the image.
• Another objective of the invention is to implement an image coding technique making it possible to obtain a good approximation of the photometric and / or colorimetric surface of the image, in particular for highly textured areas.
• the invention also aims to provide a simple and robust image coding technique.
• the invention also aims to implement an image coding technique suitable for all areas in which the movement is estimated using meshes, and in particular the field of video coding (such as the MPEG4 and H263 + for example).
• Another object of the invention is to provide a technique for coding images presenting a reduced cost of transmitting information. Yet another objective of the invention is to implement an image coding technique ensuring visual fluidity of the movement of the image.
• Another object of the invention is to provide an image coding technique making it possible to effectively manage the appearance of areas of discovery and or of settlement zones during the movement of the objects constituting the image.
• Another object of the invention is to implement an image coding technique making it possible to manage the phenomena of inversion and mesh concealment.
• Another object of the invention is to provide an image coding technique making it possible to reduce the transmission and / or storage of motion vectors for the areas of mesh settlement.
• Yet another objective of the invention is to implement an image coding technique making it possible to ensure the consistency of the ratio of the information transmission rate to the image distortion rate.
• a method of coding a mesh representative of an image of an animated sequence such a mesh being larger than said image.
• image is meant here a two-dimensional image, or a multidimensional image, for example representative of a 3D object.
• the invention is based on a completely new and inventive approach to the estimation and compensation of movement for the coding and decoding of image sequences, allowing the management of discoveries and settlements of meshes during displacement. objects.
• the invention in particular proposes the coding of a mesh of dimension greater than that of the image with which it is associated, and therefore goes against the prejudices of the skilled person, who has always sought to reduce the amount of information to be coded and transmitted and / or stored on a data carrier. It seems indeed completely useless, a priori, for those skilled in the art of constructing a mesh larger than the image, since no information is available beyond the limits of the image.
• the invention thus makes it possible, during motion tracking, to maintain a mesh over all of the images in a video sequence.
• said image having an area of N * M pixels
• said mesh has an area of at least nN * nM pixels, n being an integer greater than or equal to 2.
• the invention ideally provides for using an infinite size mesh.
• the mesh extends on either side of the image, so that its surface is at least n times greater than that of the image, where n is a sufficiently large integer, defined before the implementation of the coding method according to the invention.
• said mesh is a hierarchical mesh having at least two levels of mesh nested within a pyramid of mesh.
• the use of a nested hierarchical mesh 41 makes it possible to obtain a robust coding, and reduces the cost of transmission and / or storage of information. Indeed, there is a strong correlation between the information carried by a parent node and that carried by its child nodes within the mesh. Therefore, the values of the nodes 42, 43 belonging to the highest level of the mesh pyramid (corresponding to a coarse mesh of the image) are quantified, with a view to their transmission, for example to an encoder, and / or their storage on a data carrier 45. On the other hand, for the lower levels of the pyramid of mesh (corresponding to a finer mesh of the image), only the differential values of the child nodes 44 are quantified, in view their transmission and / or storage, as illustrated in FIG. 4.
• said mesh is a triangular mesh consisting of an arrangement of vertices and triangular faces, each defined by three references to the vertices which it connects, and having three edges each connecting two of said vertices.
• the invention is of course also applicable to any other type of polygonal mesh, which may have undergone prior triangulation.
• such a coding method associates with at least some of the vertices and / or some of the faces of said mesh at least one of the following markers: an "inside” marker, a face carrying an “inside” marker if it has a non-empty intersection with said image, and a vertex bearing an "interior” marker if it belongs to an "interior” face; an "incoming” marker, a vertex carrying an "incoming” marker if it has an "inside” marker and is outside the image, and a face carrying an "incoming” marker if at least one of the three vertices that it links carries an "incoming” marker.
• such a coding method implements the succession of following steps: estimation of the movement of said vertices of said mesh, between two successive images of said sequence; construction of colorimetric and / or photometric information intended for vertices and / or faces of said mesh carrying said marker
• Such a construction step can be implemented according to the approach proposed in patent document FR-98 12525 having for title "coding process of still or moving images with reduction and adaptation of the bit rate".
• This technique relates to a method of coding an image digital, aimed at producing a binary train representative of this image, the length of the binary train being a function of desired representation.
• This method comprises the following steps: defining, on a domain of the image to be coded, a hierarchical mesh comprising a plurality of nested meshes whose mesh vertices can be pixels of said image; perform optimizations of luminance, chrominance, and positions on each mesh level; determine, for each mesh of said hierarchical mesh, a luminance difference between the image to be coded and an interpolated image obtained from the vertices of the nested mesh to which the mesh in question belongs, and introduce the values (advantageously differential coded) into the binary train compared to the preceding hierarchical level) of positions, luminance and chrominance of the vertices of the meshes whose luminance difference is greater than a threshold difference.
• a quaternary tree structure associated with the hierarchical mesh, is constructed to manipulate the values (colors and positions) of the vertices of the meshes.
• the tree presents a number of nodes equal to the number of triangles in the corresponding level of mesh. Each node of the tree relates to a single triangle of the hierarchical mesh.
• the data of the tree to be introduced into the bit stream representative of the image which will be transmitted and / or stored is selected. This selection depends on the desired quality.
• a luminance difference between the image to be coded and the interpolated image is calculated for each triangle from the vertices of the nested mesh to which the mesh in question belongs. This difference is then compared to a threshold difference for each triangle.
• the value of the threshold deviation depends on the quality of representation desired.
• the part of the tree relating to the triangles whose luminance difference is greater than the threshold difference is then introduced into the binary train, which therefore makes it possible to transmit the data relating to the image as a function of the local quality of these different triangular partitions.
• said estimation step comprises the following sub-steps: optimization of motion vectors of the vertices of said mesh carrying said "interior" marker, so as to minimize a predetermined error criterion for reconstruction of the following image; - interpolation and / or extrapolation of motion vectors from the vertices of said mesh not carrying said "interior” marker, in order to fluidize the overall displacement of said mesh.
• said interpolation and / or extrapolation step implements a double iterative front-back scanning, applied to the vertices of the lowest level of said grid pyramid for which said vectors movement of said vertices have been optimized.
• the invention therefore provides for artificially propagating the motion vectors from the vertices carrying an "interior” marker to the vertices not carrying an "interior” marker, by implementing such an iterative front-rear scan.
• such a coding method implements a step of moving each of the vertices of said mesh, by applying to it its own motion vector.
• said construction step implements on the one hand a finite element coding model, and on the other hand an alternative coding model, the latter being implemented for at least certain zones. of said image, according to a predetermined error criterion.
• hierarchical coding when implementing a hierarchical coding, for example as described in the French patent application FR 98 12525, and when a portion of the image is very textured, it is then necessary to provide a number of large mesh levels to obtain photometric and / or colorimetric information of sufficient quality. In this case, the efficiency of hierarchical coding is low. In other words, hierarchical coding is well suited for relatively simple images, but not for images with highly textured parts.
• Such a method of coding an image to be coded comprises the following steps: definition of a hierarchical mesh having at least two nested mesh levels formed of meshes defined by vertices (which may be pixels of said image to be encoded); determination, for each of said meshes, of error information between said image to be coded and an interpolated image obtained from the vertices of the meshes belonging to the mesh level of the mesh considered; stopping the refinement of the meshes having error information lower than a first predetermined threshold; implementation of a specific coding for the meshes presenting an error information higher than a second predetermined threshold; - Continuation of the refinement of the meshes having error information greater than said first predetermined threshold and less than said second predetermined threshold.
• the hierarchical coding or with nested meshes, is the main coding, or basic coding, but it is not used systematically: for the image portions which require it, another more efficient coding is used.
• the decision to use this specific coding is made by comparison with a threshold. More precisely, at each node considered in the tree representative of the hierarchical mesh, three possibilities are offered: stop the refinement, or the division, of the meshes (the quality reached for the corresponding image portion being sufficient), pass to the hierarchical level next, keeping the same hierarchical coding, or use another coding (in particular a coding better suited to highly textured portions).
• said construction step implementing a hierarchical coding model comprises the following sub-steps: temporary normal projection of said vertices carrying said marker
• Such a temporary normal projection of the vertices carrying the “incoming” marker on the edges of the image makes it possible to avoid taking account of pixels not belonging to the image, and therefore ensures the stability of the step of optimization of colorimetric and / or photometric information.
• said specific coding comprises the following sub-steps: - construction of a symmetrized image, obtained by applying to said image an axial symmetry with respect to each of its edges and a central symmetry with respect to each of its corners; optimization of said colorimetric and / or photometric information of said vertices and / or of said faces of said symmetrized image by application of said alternative model (DCT, fractals, matching pursuit).
• DCT fractals, matching pursuit
• such a coding method implements a step of memorizing photometric and / or colorimetric information associated with the vertices and / or the faces emerging from said image.
• the information relating to the zones leaving the image can be reused if such zones are brought to return to the image, without it being necessary to apply a new treatment to them.
• Such a situation is frequent in a video sequence: it is for example the case of a decoration located in the background of a character followed by a camera, if the character goes back and forth in the image.
• said construction step implements at least one iteration of the following successive sub-steps: - optimization of said colorimetric information and / or photometrics of said vertices of said "incoming" face, so as to minimize a predetermined error criterion; transmission to a terminal and / or storage on a data medium of said optimized colorimetric and / or photometric information.
• the nodal photometric and / or colorimetric values of the incoming meshes are re-optimized, then transmitted and / or stored as long as they are not fully entered in the image.
• said construction step implements at least one iteration of the following successive sub-steps: elaboration of a criterion making it possible to evaluating an image reconstruction quality obtained by taking into account the current photometric and / or colorimetric information of the vertices of said face; when said quality is not satisfactory: - optimization of said colorimetric and / or photometric information of said vertices of said face, so as to minimize a predetermined error criterion for reconstruction of the following image; transmission to a terminal and / or storage on a data medium of said optimized colorimetric and / or photometric information; transmission and / or storage of a tree of markers indicating to said terminal whether said colorimetric and / or photometric information of said vertices of said "incoming" face have been optimized or not.
• the quality of the surface approximation obtained is thus estimated taking into account the available photometric and / or colorimetric nodal values, and optimization and transmission and / or storage of the optimized values are only practiced in the event of a significant error. .
• Such a variant makes it possible to reduce the amount of information to be transmitted and / or stored.
• the implementation of such an alternative embodiment requires the transmission and / or storage of a tree of markers, indicating, for example to the decoder, whether the photometry and / or the colorimetry of the incoming mesh is to be updated before rendering the image.
• said step of estimating the movement further comprises a sub-step of constructing a tree of the motion vectors to be transmitted and / or stored, the belonging of a motion vector to said tree being determined from a criterion of relevance of the movement applied to said faces of said mesh.
• a sub-step of constructing a tree of the motion vectors to be transmitted and / or stored the belonging of a motion vector to said tree being determined from a criterion of relevance of the movement applied to said faces of said mesh.
• said relevance criterion results from a predetermined weighted combination of the following sub-criteria: a surface sub-criterion, making it possible to evaluate the ratio between the surface of a face and the average surface of the faces of said mesh belonging to the same level of said mesh pyramid; a compactness sub-criterion, making it possible to evaluate the relationship between the surface and the perimeter of a face of said mesh; a sub-criterion of antagonism of the movements, making it possible to evaluate the antagonism of the motion vectors of vertices of said mesh.
• all the motion vectors of said vertices of said mesh are transmitted and / or stored, a suitable decoder identifying according to a predetermined criterion the vectors to be taken into account.
• the vertex resulting from the fusion of the obscured vertex and the obscuring vertex is placed on the edge initially connecting the obscured vertex and the occulting vertex, for example in its middle.
• such a coding method implements a step of detecting at least one area of said image obscured during movement of said mesh, and a step of storing photometric information and / or colorimetric relating to said obscured area, in view of possible future use, said information not being processed.
• the invention also relates to a decoding method and to coding and decoding devices of a mesh representative of an image of an animated sequence, said mesh being of size larger than said image.
• the invention also relates to a signal representative of a mesh representative of an image of an animated sequence, said mesh being of size larger than said image.
• FIG. 1 already described above, illustrates the concept of field of motion
• - Figure 2 commented above, describes an example of approximation of the field of motion of Figure 1 by interpolation or extrapolation of inter-nodal movements
• FIG. 3, described above presents an example of mesh reversal, resulting from the existence of antagonistic motion vectors within the motion field presented in FIG. 1
• FIG. 4, described above illustrates an exemplary embodiment of hierarchical transmission of information
• FIG. 5 shows an example of the appearance of a zone of discovery and a zone of compaction of the mesh resulting from a movement within the image
• FIG. 6 illustrates the construction of a mesh of dimension greater than that of the image
• FIG. 7 presents an example of “interior” markers associated with certain faces and certain vertices of a mesh as illustrated in FIG. 6
• FIG. 8 illustrates the implementation of a temporary normal projection of vertices during the stage of construction of colorimetric and / or photometric information
• FIG. 9 shows an example of image symmetrization for the implementation of an alternative coding
• FIG. 10 illustrates an example of propagation of the motion vectors from the vertices carrying the marker "interior” towards the other vertices of the mesh, with a view to a global displacement of all the vertices of the pyramid of mesh;
• - Figure 11 presents an example of meshes carrying the marker "incoming" during several frames of information;
• FIG. 12 illustrates a phenomenon of packing of meshes on a zone of occultation of the mesh;
• FIG. 13 presents a first example of management of occultation phenomena by fusion of vertices;
• FIG. 14 illustrates a second example of managing the occultation phenomena by implementing a 3-manifold mesh.
• the invention therefore relates to an improvement in the motion estimation technique implementing a hierarchy of nested meshes (or hierarchical coding), as described in the preamble.
• provision is in fact made for using a mesh of size greater than the images of the sequence.
• An image 61 is described using a triangular mesh, consisting of a plurality of faces and vertices, which extends beyond the limits of image 61.
• some vertices 611 are located at the 'inside the image 61, and other vertices 612 are placed outside the limits of the image 61.
• such a mesh is a hierarchical mesh nested with infinite surface.
• Figure 7 shows an example of marking the vertices and the faces of a mesh using an "interior" marker.
• a vertex of the mesh 72 is "inside" at a given level of the mesh 72 if it belongs to an "inside" face of the level considered.
• the vertex referenced 722 bears the indication IN because it belongs to the "interior" face referenced 721.
• a vertex is "interior” (that is to say bears the indication IN in FIG. 7) while being outside of image 71.
• the vertex referenced 723 is " interior "because it belongs to the” interior "face referenced 721, but it is exterior to image 71.
• incoming marker
• a vertex is “incoming” if it is “inside” and that its photometric information and / or colorimetric are undefined or unsatisfactory, that is to say if it is “inside” but it is located outside of image 71.
• one face of the mesh 72 is “incoming "if at least one of its vertices is” incoming ". 2. Management of mesh stripping zones
• the photometric and / or colorimetric information associated with the faces and / or the vertices of the mesh entering the image for the first time is updated, in order to of make a complete approximation of the photometric and / or colorimetric surface of the image.
• Such an optimization notably consists in carrying out optimizations of luminance, chrominance and positions on each level of mesh. Then, for each mesh of the hierarchical mesh, a luminance difference between the image to be coded and an interpolated image is obtained, obtained from the vertices of the nested mesh to which the mesh considered belongs. Finally, we introduce into a binary train representative of the image the values (advantageously coded in differential compared to the previous hierarchical level) of positions, luminance and chrominance of the vertices of the meshes whose luminance difference is greater than a difference threshold.
• the procedure is, according to the invention, a temporary normal projection of the vertices " entering "on the edges of the image, as illustrated in FIG. 8.
• the vertices 821a, 822a and 823a of the representative mesh of the image 81 are projected onto the edges of the image at positions 821b, 822b, 823b.
• the projected vertices 821b, 822b, 823b are returned to their initial position, respectively 821a, 822a and 823a, but they are associated with the optimized color and / or photometric information after projection.
• the technique described in this patent application relates to a method of coding an image to be coded, comprising the following steps: definition of a hierarchical mesh having at least two nested mesh levels formed of meshes defined by vertices (which may be pixels of said image to be coded); - Determination, for each of said meshes, of error information between said image to be coded and an interpolated image obtained from the vertices of the meshes belonging to the mesh level of the mesh considered; stopping the refinement of the meshes having error information lower than a first predetermined threshold; implementation of a specific coding for the meshes presenting an error information higher than a second predetermined threshold; continuation of the refinement of the meshes having error information greater than said first predetermined threshold and less than said second predetermined threshold.
• the hierarchical coding is the main coding, or basic coding, but it is not used systematically: for the image portions which require it, another more efficient coding is used (DCT, fractals, matching pursuit, etc.).
• the images referenced 93, 95, 97 and 99 are thus obtained by applying to image 91 an axial symmetry with respect to each of the edges. of the image 91.
• the images referenced 92, 94, 96 and 98 are images of the image 91 by central symmetry with respect to each of the vertices of the image 91.
• Such propagation of the movement is, for example, implemented in a particular embodiment, according to the following algorithm: For all vertices S of L m if INTERIOR (S) then S becomes PROPAGATED otherwise S becomes non-PROPAGATED Iterate as long as there are undefined motion vectors For all vertices S of L m traveled from top-left to bottom- right
• a scan 104 of the image 102 is carried out with the aim of propagating the movement from the bottom to the top of the image. We thus attribute to the referenced vertex
• the "incoming" vertices of the mesh do not have photometric and / or colorimetric information, the latter being approximated from the photometric and / or colorimetric information available, associated with the portions of the "incoming" faces located inside the image.
• a first alternative embodiment consists in optimizing this information at each data frame.
• the nodal photometric and / or colorimetric values of the incoming faces 110 are reoptimized and transmitted and / or stored on a data medium each time the photometric and / or colorimetric surface of the image is approximated (i.e. at each instant t, t + 1, t + 2 and t + 3), as long as the incoming face 110 considered is not entirely included in the image 111.
• a second alternative embodiment, less costly in terms of data transmission consists in establishing a quality criterion, and in comparing, with this criterion, the quality of the photometric and / or colorimetric approximation of the surface of the image, obtained in taking into account the current nodal photometric values of the incoming faces. If such a comparison reveals a significant error in the approximation of the image, one then proceeds to an optimization and a transmission and / or storage on a data carrier of the photometric and / or colorimetric information of the "incoming" vertices. If, on the other hand, the quality of the approximation of the image is satisfactory, the current photometric and / or colorimetric values are kept for the "incoming" vertices.
• a tree of specific markers is also constructed, associated with the "incoming" vertices 112 to 114, making it possible to indicate to a decoder or to any other suitable terminal, whether the photometry and / or the colorimetry of the face " incoming "considered 110 must be updated before rendering image 111.
• the motion vectors of the vertices belonging to the levels L 0 to L m of the mesh pyramid are all optimized.
• presenting a reduced coding cost the whole is transmitted and / or stored on a data carrier motion vectors associated with the vertices of the settlement zone considered.
• the relevance of the motion vectors with regard to the overall motion estimation of the image is then evaluated at the time of decoding of the image.
• the "relevant" marker can take a TRUE value, when the movement of a vertex is relevant for the estimation of overall movement of the image, and must therefore be transmitted and / or stored on a data medium, or FALSE otherwise.
• We build the relevance tree from the faces and vertices of the hierarchical mesh belonging to a level of the pyramid of mesh less than or equal to L m .
• the value of the "relevant" marker is defined as follows:
• RELEVANT (T) CP (T)> SP
• Such a criterion is maximum and equal to 1 if T is equilateral, and it is zero if T is flat; a sub-criterion CA of antagonism of the movements of the vertices of the triangle considered.
• Such a criterion is worth 1 if the movements of the vertices of T are antagonistic, that is to say in the presence of a potential mesh reversal, and 0 if they are of the same direction and direction. If Dl, D2 and D3 are the motion vectors of the vertices of T, then:
• CP (T) [a * CS (T) + b * CC (T) + c * CA (T)] / [a + b + c] where a, b, and c are predetermined scalars.
• the "relevant" marker of S is initialized to FALSE For all the triangles T of the mesh pyramid traversed from the levels L 0 to L m If CP (T)> SP m
• the first method implemented according to the invention consists in detecting, at the time of decoding, the occultation zones determined by the faces 130, 133 of the mesh whose vertices 131, 132 have antagonistic motion vectors, according to the sub- CA criterion, as illustrated in FIG. 13. Indeed, the faces 130, 133 thus detected are liable to turn over because their vertices 131, 132 are positioned on different objects, one of the two objects obscuring its neighbor.
• an edge fusion (in English "edge collapse") is then practiced between the two neighboring vertices 131, 132 having an antagonistic movement. This results in the disappearance of a triangle 130, 133, reflecting the disappearance of part of an object.
• the vertex obtained by merging the vertices 131 and 132 is placed on the edge initially connecting the vertices 131 and 132, for example in the middle, or at the position of one of the two vertices 131 and 132, as illustrated in Figure 13, where the merged vertex is located in the position initially occupied by the vertex 132. 3.3.2.
• N-manifold mesh A second method implemented according to the invention for managing the occultation phenomena of meshes consists in working on an n-manifold mesh, as illustrated in FIG. 14.
• An n-manifold mesh is defined as a triangular mesh in which an edge can be shared by n triangles, where n ⁇ 2.
• FIG. 14 presents an example of a 3-manifold mesh, in which the edge referenced 143 is common to the three triangles referenced 140, 141, and 142.
• an motion is first estimated. of the image, without taking into account the "relevant" marker described above.
• the triangles associated with this zone then become carriers of a marker "overlapped” (or in French “overlapped”).
• the zones of the mesh carrying the "overlapped" marker mark the discoveries or the overlaps of meshes: they therefore correspond to occulted objects.
• the method implemented in the particular embodiment described in this paragraph consists in temporarily removing the triangles carrying the "overlapped" marker, while keeping them in memory, so as to be able to manage their possible reappearance on the image.
• n-manifold mesh advantageously makes it possible to conserve the photometric and / or colorimetric information associated with areas of the mesh which may disappear or appear repeatedly during the video sequence considered.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=8852529

Family Applications (1)

Country Status (4)

Families Citing this family (27)

Citations (1)

Family Cites Families (7)

• 2000
  • 2000-07-13 FR FR0009273A patent/FR2811791B1/fr not_active Expired - Fee Related
• 2001
  • 2001-07-03 EP EP01951762A patent/EP1299859A1/de not_active Withdrawn
  • 2001-07-03 US US10/332,684 patent/US7502413B2/en not_active Expired - Fee Related
  • 2001-07-03 WO PCT/FR2001/002131 patent/WO2002007099A1/fr active Application Filing

Patent Citations (1)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events