GB2561824A - Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model - Google Patents

Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model Download PDF

Info

Publication number
GB2561824A
GB2561824A GB1706177.1A GB201706177A GB2561824A GB 2561824 A GB2561824 A GB 2561824A GB 201706177 A GB201706177 A GB 201706177A GB 2561824 A GB2561824 A GB 2561824A
Authority
GB
United Kingdom
Prior art keywords
vertex
predictor
vertices
predictors
color
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB1706177.1A
Other versions
GB2561824B (en
GB201706177D0 (en
Inventor
Laroche Guillaume
Onno Patrice
Gisquet Christophe
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Priority to GB1706177.1A priority Critical patent/GB2561824B/en
Publication of GB201706177D0 publication Critical patent/GB201706177D0/en
Publication of GB2561824A publication Critical patent/GB2561824A/en
Application granted granted Critical
Publication of GB2561824B publication Critical patent/GB2561824B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • G06T9/001Model-based coding, e.g. wire frame
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • G06T9/004Predictors, e.g. intraframe, interframe coding

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Encoding and decoding geometry data of a 3D object where encoding vertex positions includes predicting the position of a current vertex with a position predictor selected, based on a coding-cost-based criterion, from a set of position predictors 1502, 1510, and encoding a residue between the current vertex position and the selected position predictor. The set of position predictors is defined based on a value representing the regularity of a vertex position distribution in the 3D model 1508. Also disclosed and claimed is the encoding or decoding of colour attribute data associated to vertices of a 3D model or mesh by predicting the colour attribute associated to a current vertex by choosing a predictor from a set of predictors, based on a coding-cost-based criterion. The set of predictors is determined based on a value representing the regularity of a vertex position distribution in the 3D model. The regularity or irregularity of the 3D object grid can be evaluated either locally or globally by evaluating the predictors used to predict a test set of vertices 1506. A parametric predictor may be used in case of irregularity 1502, while parallelogram or E-parallelogram predictors are advantageously used in case of regularity 1510.

Description

(54) Title of the Invention: Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model
Abstract Title: Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model
Encoding and decoding geometry data of a 3D object where encoding vertex positions includes predicting the position of a current vertex with a position predictor selected, based on a coding-cost-based criterion, from a set of position predictors 1502, 1510, and encoding a residue between the current vertex position and the selected position predictor. The set of position predictors is defined based on a value representing the regularity of a vertex position distribution in the 3D model 1508. Also disclosed and claimed is the encoding or decoding of colour attribute data associated to vertices of a 3D model or mesh by predicting the colour attribute associated to a current vertex by choosing a predictor from a set of predictors, based on a coding-cost-based criterion. The set of predictors is determined based on a value representing the regularity of a vertex position distribution in the 3D model. The regularity or irregularity of the 3D object grid can be evaluated either locally or globally by evaluating the predictors used to predict a test set of vertices 1506. A parametric predictor may be used in case of irregularity 1502, while parallelogram or Eparallelogram predictors are advantageously used in case of regularity 1510.
1500
Figure GB2561824A_D0001
1/17
Isolated
Figure GB2561824A_D0002
Fig. 1
2/17
CN
CN
CN
Figure GB2561824A_D0003
3/17
Figure GB2561824A_D0004
4/17
Figure GB2561824A_D0005
Fig. 4
Figure GB2561824A_D0006
5/17
Figure GB2561824A_D0007
6/17
Figure GB2561824A_D0008
(Ο best[0] = best[1] ?
7/17
Figure GB2561824A_D0009
8/17
Figure GB2561824A_D0010
Fig. 8
9/17
Figure GB2561824A_D0011
2C-B
Figure GB2561824A_D0012
Figure GB2561824A_D0013
Fig. 9 Fig. 10
10/17
1101 1102 1103
Figure GB2561824A_D0014
Fig. 11
11/17
Figure GB2561824A_D0015
Fig. 12
Figure GB2561824A_D0016
12/17
Figure GB2561824A_D0017
δ^ co
Fig. 13
13/17
Current vertex
Figure GB2561824A_D0018
Figure GB2561824A_D0019
14/17
Μ
Figure GB2561824A_D0020
15/17
Current vertex
Figure GB2561824A_D0021
16/17
Figure GB2561824A_D0022
Fig. 20 -±- Fig. 21
2108-^ Predict the color c of vertex v with one a predictor of the predictors set £Z X
Figure GB2561824A_D0023
17/17
Figure GB2561824A_D0024
Fig.22
Application No. GB 1706177.1
RTM
Date :16 October 2017
Intellectual
Property
Office
The following terms are registered trade marks and should be read as such wherever they occur in this document:
“IEEE” (page 9)
Intellectual Property Office is an operating name of the Patent Office www.gov.uk/ipo
ENCODING AND DECODING OF GEOMETRY DATA IN A 3D MODEL BASED ON EVALUATED REGULARITY OF THE MODEL
FIELD OF THE INVENTION
The present invention relates generally to the encoding of three-dimensional (3D) objects, and more specifically to methods and devices for encoding, into a bitstream, 3D models made of vertices or points, including mesh models and points clouds. The present invention focuses on the encoding of geometry data describing the vertex positions of the 3D model in a coordinate system, and/or color data associated with each vertex.
BACKGROUND OF THE INVENTION
3D models may be made of a cloud of points. However, more common 3D models are 3D mesh models composed of polygons, e.g. triangles, between connected points, also denoted vertices.
Typically, 3D mesh models are represented by three types of data: connectivity data, geometry data and property data, while the points clouds are deprived of connectivity data. Connectivity data describe the adjacency relationship between the vertices composing the triangles, while geometry data specify vertex locations or positions in a coordinate system, and property data specify attributes of the vertices such as the normal vector, material reflectance, texture mapping and color value. Most 3D compression algorithms compress connectivity data and geometry data separately.
One known compression algorithm to encode the connectivity data is the Scalable Complexity 3D Mesh Compression-Triangle-Fan (SC3DMC-TFAN) approach of the MPEG Group, for instance as described in publication “A Triangle-FAN-based approach for low complexity 3D mesh compression (Mamou K., Zaharia T., Preteux F., IEEE International Conference on Image Processing, 2009).
SC3DMC-TFAN includes traversing successively each vertex constituting the 3D mesh model; and for each vertex successively traversed, performing a triangle-based fan (TFAN) analysis centered on the traversed vertex (i.e. one vertex of the triangles considered in the fan is the traversed vertex) to discover any new triangle not yet discovered when traversing the previous vertices, and determining (TFAN) connectivity data between vertices based on the newly discovered triangle or triangles. The T-FAN analysis makes it possible to obtain trianglebased fan portions, each made of a contiguous series of newly discovered triangle or triangles.
The fan portions and other data are encoded as connectivity data.
Most efficient known algorithms to encode the geometry data, i.e. point or vertex positions, are based on connectivity-based geometry prediction.
The main idea of this approach method is to use the connectivity between the vertices (i.e. the connectivity data) to generate a geometry predictor. One known method of the connectivity-based geometry prediction is the parallelogram predictor described for instance in publication “Triangle mesh compression (Touma C. and Gotsman C., Proceedings ofthe 1998 Graphics Interface Conference, p.26-34, 1998).
Improved parallelogram-based predictors have been developed, for instance through combinations of two or more parallelogram predictors. This is discussed in the following publications: “Rate-distortion optimized compression and view-dependent transmission of 3-D normal meshes (Sim J.-Y. et al., 2005, IEEE Transactions on Circuits and Systems for Video Technology 15, 7, 854-868), “Geometry prediction for high degree polygons (Isenburg M. et al., In Proceedings of the Spring Conference on Computer Graphics, 147-152), “FreeLence coding with free valences (Kalberer F. et al., 2005. Computer Graphics Forum 24, 3, 469-478) and “Multi-Way Geometry Encoding” (Cohen-Or D. et al., 2002, Technical report, School of Computer Science, Tel-Aviv University).
The parallelogram predictor or the average or combination between parallelogram predictors usually gives the best results for the geometry prediction. However, they appear to be efficient for very regular 3D meshes and small 3D objects only. On the other hand, when the 3D objects are large and/or when the 3D mesh becomes very irregular, the parallelogram predictor efficiency drastically reduces.
In a similar way, efficient algorithms to encode the geometry data of points clouds involve prediction using predictors built from already-encoded points.
There is thus a need to improve the encoding ofthe geometry data.
SUMMARY OF INVENTION
The inventors have already proposed, in pending applications GB 1610607.2 and GB 1610610.6 filed on June 17, 2016, a competitive-based scheme to predict geometry data and new position predictors, in order to address both regular and irregular meshes with the same set of position predictors.
The inventors have stressed that a very efficient set of position predictors is made of a so-called E-parallelogram predictor and of a parametric predictor.
However, the improvement in coding efficiency when using this set of position predictors remains low for regular or semi-regular meshes. This is mainly because the conventional parallelogram predictor is still efficient. Adaptations are thus required in order to improve the position prediction ofthe 3D models, including 3D mesh models and points clouds.
In this context, embodiments ofthe invention provide a method for encoding, into a bitstream, a three-dimensional model made of vertices or points, the method comprising a step of encoding geometry data describing the positions of vertices in the 3D model, wherein encoding the geometry data of a current vertex includes:
predicting the geometry data of the current vertex with a position predictor selected, based on a coding-cost-based criterion, from a set of one or more position predictors, and encoding a residue between the geometry data of the current vertex and the selected position predictor, the method further comprising:
determining a value representative of regularity of a vertex position distribution in the 3D model, and setting the set of one or more position predictors used to select the position predictor, based on the determined regularity value.
At decoding side, a method is proposed for decoding, from a bitstream, a threedimensional model made of vertices or points, the method comprising a step of decoding geometry data describing the positions of vertices in the 3D model, wherein decoding the geometry data of a current vertex includes:
selecting, based on the bitstream, a position predictor from a set of one or more position predictors used to predict the geometry data of the current vertex;
obtaining, from the bitstream, a position residue for the current vertex position to decode;and computing a decoded position of the current vertex by adding the selected position predictor to the obtained position residue, the method further comprising:
determining a value representative of regularity of a vertex position distribution in the 3D model, and setting the set of one or more position predictors used to select the position predictor, based on the determined regularity value.
The regularity value thus helps classifying the current 3D structure.
This approach makes it possible to obtain an efficient trade-off between processing complexity and coding efficiency. Indeed, by setting the set of predictors (for instance by switching between two [or more] sets) according to an evaluated regularity/irregularity of the 3D model, it is possible to keep few predictors in the set (to keep low processing complexity) that are both efficient to encode the vertex positions (to improve coding gain compared when one or more predictors in the set are never used because not adapted to the regularity/irregularity).
In first embodiments, the regularity value is determined based on the position predictors used to predict the geometry data of a test set of vertices of the 3D model that are encoded or decoded before the geometry data of the current vertex is predicted. For instance, the regularity value includes a ratio of use of a reference predictor, to predict the geometry data of the vertices of the test set.
In second embodiments related to 3D mesh models made of connected vertices, the regularity value includes a valence value for the center vertex around which a TFAN analysis is performed, corresponding to a number of vertices connected to the center vertex.
Furthermore, as the property data can be encoded using the same encoding schemes as for the geometry and/or connectivity data, the proposed competitive-based scheme to predict geometry data and the new position predictors may be improved for use to vertex color encoding.
Thus, third embodiments of the invention propose a method for encoding, into a bitstream, or for decoding, from a bitstream, a three-dimensional model made of vertices, the method comprising a step of encoding or decoding color data describing a color attribute associated with each vertex in the 3D model, wherein encoding or decoding the color data of a current vertex includes:
predicting the color data of the current vertex using a color predictor selected from a set of color predictors, the set of one or more color predictors being determined based on value representative of regularity of a vertex position distribution in the 3D model.
Better prediction of color data is thus obtained.
Correspondingly, there are provided devices for encoding into a bitstream or decoding from a bitstream, a three-dimensional model made of vertices, the device comprising at least one microprocessor configured for carrying out a step of respectively encoding or decoding geometry data describing the positions of vertices in the 3D model.
Encoding the geometry data of a current vertex includes:
predicting the geometry data of the current vertex with a position predictor selected, based on a coding-cost-based criterion, from a set of one or more position predictors, and encoding a residue between the geometry data of the current vertex and the selected position predictor.
Decoding the geometry data of a current vertex includes:
selecting, based on the bitstream, a position predictor from a set of one or more position predictors used to predict the geometry data of the current vertex;
obtaining, from the bitstream, a position residue for the current vertex position to decode;and computing a decoded position of the current vertex by adding the selected position predictor to the obtained position residue.
In the invention, the devices are configured for:
determining a value representative of regularity of a vertex position distribution in the 3D model, and setting the set of one or more position predictors used to select the position predictor, based on the determined regularity value.
Also, there are provided devices for encoding into a bitstream or decoding from a bitstream, a three-dimensional model made of vertices, the device comprising at least one microprocessor configured for carrying out a step of encoding or decoding color data describing a color attribute associated with each vertex in the 3D model, wherein encoding or decoding the color data of a current vertex includes:
predicting the color data ofthe current vertex using a color predictor selected from a set of color predictors, the set of one or more color predictors being determined based on value representative of regularity of a vertex position distribution in the 3D model.
Optional features of these embodiments are defined in the appended claims with reference to methods. Of course, same features can be transposed into system features dedicated to any device according to the embodiments ofthe invention.
Another aspect of the invention relates to a non-transitory computer-readable medium storing a program which, when executed by a microprocessor or computer system in a device, causes the device to perform any method as defined above.
The non-transitory computer-readable medium may have features and advantages that are analogous to those set out above and below in relation to the methods and devices.
At least parts of the methods according to the invention may be computer implemented. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a circuit, module or system. Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.
Since the present invention can be implemented in software, the present invention can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a hard disk drive, a magnetic tape device or a solid state memory device and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.
BRIEF DESCRIPTION OF THE DRAWINGS
Further advantages of the present invention will become apparent to those skilled in the art upon examination of the drawings and detailed description. Embodiments of the invention will now be described, by way of example only, and with reference to the following drawings.
Figure 1 illustrates a 3D mesh model is a representation of a tri-dimensional object or scene;
Figure 2 illustrates general steps of a 3D mesh encoding process, in particular in the context of Scalable Complexity 3D Mesh Compression (SC3DMC);
Figure 3 illustrates general steps of the corresponding bitstream decoding process, into a 3D mesh model, in the context of SC3DMC;
Figures 4 illustrate parallel-based predictors according to the prior art;
Figure 5 illustrates exemplary texture mapping data;
Figure 6 illustrates an exemplary encoding process implementing predictor competition;
Figure 7 illustrates the decoding process corresponding to the encoding process of
Figure 6;
Figure 8 shows meshes having increasing irregularities;
Figure 9 illustrates a proposed 2C-B predictor;
Figure 10 illustrates a proposed C+ predictor;
Figure 11 illustrates, using a flowchart, general steps for determining the C+ predictor of Figure 10;
Figure 12 illustrates a proposed parametric predictor;
Figure 13 illustrates, using a flowchart, general steps for generating the parametric predictor;
Figure 14 illustrates a variation of the parametric predictor of Figure 12;
Figure 15 illustrates, using a flowchart, first embodiments adapting the set of predictors based on the regularity/irregularity of the 3D model;
Figure 16 illustrates the position of the parallelogram predictor and the current vertex for some regular 3D structures when the valence at the center vertex is changed;
Figure 17 illustrates an adaptation of the process of Figure 15 to provide on-the-fly adjustments of the set of position predictors, according to second embodiments of the invention;
Figure 18 illustrates the impact of 3D model regularity on the distances between neighboring and connected vertices;
Figure 19 illustrates, using a flowchart, an implementation using a geometric parameter to evaluate 3D model regularity and then adjust the set of position parameters;
Figure 20 illustrates another impact of 3D model regularity on the distances between neighboring and connected vertices;
Figure 21 illustrates, using a flowchart, exemplary prediction of color data according to third embodiments of the invention; and
Figure 22 shows a schematic representation a device in accordance with embodiments of the present invention.
DETAILED DESCRIPTION
The invention will now be described by means of specific non-limiting exemplary embodiments and by reference to the figures.
The present invention relates generally to the encoding of three-dimensional (3D) objects into a bitstream, including clouds of points and 3D mesh models made of a plurality of points, for instance connected vertices forming triangles. The present invention particularly seeks to provide improvements to the coding of the geometry data that describes the locations or positions of the vertices within the space.
Embodiments also provide new encoding scheme to encode the color data associated with the vertices.
While the description below mainly makes reference to 3D mesh model, embodiments of the invention may also apply to 3D clouds of points when these embodiments do not rely on connectivity between vertices.
A 3D mesh model is a representation of a tri-dimensional object or scene. As shown in Figure 1, it comprises vertices, edges connecting vertices and faces defined by edges. Isolated vertices, one of which is shown in the Figure, does not belong to any edge or face. Two isolated vertices may be connected one to the other without belonging to a face.
Although various face geometries can be used, for instance polygons such as quadrilaterals, a substantial part of the 3D mesh models is based on triangle-shaped faces. The present description focuses on the encoding of triangle-faced 3D mesh models for ease of illustration.
The relationship between the various vertices as represented by the faces or edges is called connectivity, while the positions of the vertices form what is called geometry. Connectivity data and geometry data are the main data to describe a 3D mesh model, and may be supplemented with other property data including a color per vertex (other property data may include a normal per vertex or per face, a texture per face, etc.).
The geometry data are usually provided as a list of 3D vertex positions.
Most 3D compression algorithms compress connectivity data and geometry data separately. The coding order of geometry data is determined by the underlying connectivity coding. 3D mesh property data are usually compressed by a method similar to geometry compression. This is the case of the color data.
Figure 2 illustrates general steps of a 3D mesh encoding process, in particular in the context of Scalable Complexity 3D Mesh Compression (SC3DMC).
Connectivity data (defining the mesh connectivity), geometry data and property data are coded separately. Each of them 212 to 217 is extracted from the original 3D mesh model 201.
These data, except the connectivity data, can be quantized 223 to 227. The quantization is based on the number of bits needed to represent the data. The quantized geometry and property data are predicted and/or encoded directly, e.g. with respective entropy encoders, which may be arithmetic encoders 233 to 237.
Also the encoder compresses, i.e. encodes, the connectivity data at block 222. This is usually based on a well-known triangle-based fan (TFAN) approach. However, alternative approaches to the triangle-based fan (TFAN) approach also exist.
The current standard offers a plurality of coding methods and prediction methods for the quantized geometry and property data.
With respect to the geometry data, a widely-used method is the so-called “connectivity-based geometry prediction”, which provides efficient prediction. The main idea of this method is to use the connectivity between vertices (i.e. the connectivity data) to generate a geometry predictor. One known method of the connectivity-based geometry prediction is the parallelogram predictor described for instance in publication “Triangle mesh compression (Touma C. and Gotsman C., Proceedings ofthe 1998 Graphics Interface Conference, p.26-34, 1998).
The use of the mesh connectivity between the vertices to encode the geometry and/or property data is shown in the Figure by the hash arrow. While the Figure shows a hash arrow from block 222 (i.e. from connectivity data being encoded), the hash arrow may come from the connectivity data 212.
Next to the encoding, all the obtained sub-streams 242 to 247 (each representing one of the connectivity/geometry/property data) are combined together to form the output bitstream 251, for instance through a mere concatenation of sub-streams.
Figure 3 illustrates general steps of the corresponding bitstream decoding process, into a 3D mesh model, in the context of SC3DMC. The Figure is self-explicit since it produces the inverse process as the one of Figure 2.
For instance, the bitstream 301 is extracted and split into subparts 302 to 307 corresponding to each coded data from the connectivity, geometry and property data. Each of them is entropy decoded and decompressed or inverse predicted (blocks 312 to 317). As mentioned for the encoding process, the prediction can be based on the decoded connectivity data 332.
Next, the decoded geometry and property data are dequantized at blocks 323 to 327 and inserted in the 3D object file 351 together with the connectivity data 332.
Obtained 3D object file 351 is the same as original 3D mesh model 201 except for the losses due to the quantization/dequantization processes.
Embodiments of the present invention mainly focus on the encoding and/or decoding ofthe geometry data 213/303.
The encoding of connectivity data includes generating mesh connectivity data from the data ofthe original textured 3D mesh model 201 and then encoding each set of connectivity data thus generated. Of course, for 3D models made of points clouds deprived of connectivity data, such encoding is not performed.
As mentioned above, each vertex constituting the textured 3D mesh model is successively traversed, thereby defining a connectivity or “encoding” order, to perform a TFAN analysis centered on the currently-traversed vertex to discover new triangles. The newly discovered triangles form one or more fan portions, each made of a contiguous series of new triangle or triangles. The connectivity data seek to describe these triangle-based fan portions, in order to be encoded and transmitted in the bitstream to the decoder, for the latter to be able to reconstruct the mesh connectivity between the vertices ofthe 3D mesh.
As far as the geometry data are concerned, it may be noted that they usually form a bigger portion than the connectivity data, in the bitstream. Efficient prediction and coding thereof is thus sought.
The basic information of the geometry data in a raw file is 3D positions of points, including vertices forming the above-mentioned triangles, preferably reordered to match the connectivity order. Some points may be isolated, meaning they are not vertices of a triangle.
As mentioned above, the first step for the compression or encoding of the geometry data 213 is a quantization step 222. This step quantizes all coordinates (usually three position components: abscissa, ordinate and applicate) of the vertex positions.
This step is necessary because, in input files or in the memory of computers, the vertex coordinates are often represented by three IEEE 32-bit floating-point numbers. However, the 32-bit precision provided by such a representation is not required in most of the applications.
Quantization can thus significantly reduce the amount of geometry data (in terms of bits) to encode, without limited quality losses (usually not perceptible by a human being).
Conventional scalar quantization consists in transforming the floating-point number positions into integer positions. A mesh bounding box encompassing the 3D mesh, usually the smallest box (e.g. a rectangular cuboid) including the 3D mesh, is first determined, and then partitioned into a 3D grid. The number of cells per axis depends on the maximum integer that can be coded with the number of quantization bits. The size of each cell can either be uniform or non-uniform.
Each vertex of the mesh is approximated with a reference position of the cell to which it belongs, e.g. of the cell center. The integer position is then composed of the three index coordinates of the cell reference position.
Most of the geometry encoders use a uniform scalar quantization. The number of quantization bits usually ranges from 8 to 16. Of course higher numbers may be considered for higher precision. The mesh geometry is consequently slightly altered contrary to the mesh connectivity.
In SC3DMC, the quantization of vertex position is based on the range of each component (X, Y, Z) as described in the following pseudo code:
for i = {X, Y, Z}
V’[i] = (V[i]-Min[i])*(1 «nQBits -1)/(Max[i]-Min[i]) +0.5f where V[i] the original component 1' of vertex position V, V’[i] is the quantized component Ί', and Min[i] and Max[i] are respectively the minimum and the maximum values of all V[i] (i.e. define the range of the mesh bounding box along the i-axis). Operator X « Y means a left shift of the number X by the number of digit Y in base 2. It corresponds to Y integer divisions by 2 (i.e. division by 2Y).
This quantization means that the value range Max[i]-Min[i] for each coordinate is represented according to a certain number of bits given by the quantization parameter nQBits.
But with this quantization, the components of a vertex do not have the same precision given the different value ranges for the position components within the mesh.
To obtain the same precision for all components of each vertex, the quantization must use the same divisor (Max[i]-Min[i]) for each component. In a proposed implementation, the quantization of SC3DMC is changed to take into account the biggest range of the mesh bounding box defined as follows: MaxRange = MAX(Max[X]-Min[X], Max[Y]-Min[Y], Max[Z]Min[Z]).
Thus, for i = {X, Y, Z}
V’[i] = (V[i]-Min[i])*(1 «nQBits -1)/(MaxRange) +0.5f
Next to the quantization, the position of a current vertex to encode is predicted using the geometry data (i.e. position) of the already-encoded neighbors, i.e. the other vertices connected to the current vertex that have already been encoded.
If the prediction is accurate, the prediction error is small, so it can later be efficiently entropy coded. Otherwise, the vertex position can be encoded without prediction.
Various approaches have been proposed in the prior art.
A first approach (“differential”) uses differential prediction according to which the position of the current vertex is predicted by the position of the immediately previous vertex. The difference between these two values, also known as residue, representing the vector between the two positions, is then encoded. This differential method, described in “Geometry compression (Deering M., 1995, In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, pp.13-20) is available in the SC3DMC standard, in addition to a conventional no prediction method.
An improvement of the differential prediction is described in “Single resolution compression of arbitrary triangular meshes with properties (Bajaj C. et al., 1999, Computational Geometry 14, 1-3, pp. 167-186) which provides a second-order predictor that encodes the differences between consecutive delta predictions.
A second approach is the topological surgery algorithm described in “Geometric compression through topological surgery (Taubin G. et al., 1998, ACM Transactions on Graphics 17, 2, pp.84-115). This approach uses linear prediction: the position of the current vertex is predicted by a linear combination of the K immediately preceding vertices in the vertex spanning tree. The K parameters of the linear function minimize the mean square prediction error over the mesh. Their values are stored in the compressed file to be available to the decoder.
A third approach (“adaptive”) is the adaptive geometry prediction method of the SC3DMC standard. In this approach, each component of the current vertex is predicted by one from among the seven last encoded vertices or is not predicted. The best prediction, in terms of coding cost, from among these eight possible predictions is transmitted.
A fourth approach for geometry prediction is the above-mentioned connectivity based geometry prediction, which uses the mesh connectivity between vertices (connectivity data 212) to define predictors for the geometry data.
Besides valence-driven connectivity encoding, “Triangle mesh compression (Touma C. and Gotsman C., Proceedings of the 1998 Graphics Interface Conference, p.26-34, 1998) introduced parallelogram prediction. In the case of Figure 4a, a predictor is searched for the next vertex connected to vertices A and C (i.e. forming a triangle with them). A piece of knowledge is triangle ABC.
The parallelogram-based prediction introduces a new vertex P as shown in Figure 4a to form a triangle from the edge AC, such that the ABCP form a parallelogram. Vertex P is thus used as a predictor for the current vertex to encode.
Experimental results show that triangle mesh geometry can be compressed at about 8.5 bpvwith an 8-bit quantization, using the parallelogram-based prediction.
An improvement of the parallelogram-based prediction is shown in Figure 4b. It is a dual parallelogram prediction described in publication “Rate-distortion optimized compression and view-dependent transmission of 3-D normal meshes (Sim J.-Y. et al., 2005, IEEE Transactions on Circuits and Systems for Video Technology 15, 7, pp.854-868). It uses the average position given by two parallelogram predictions whenever possible, one from the parallelogram prediction of Figure 4a, the other from a parallelogram prediction based on another triangle having either A or C as vertex. The additional vertex already known for this other triangle is referenced E.
Figure 4b shows the generated predictor P based on the average of virtual vertices Pt and P2. The dual parallelogram can be used in about 75% of cases. It results in slight improvements over parallelogram prediction.
Publication “Geometry prediction for high degree polygons (Isenburg M. et al., In Proceedings of the Spring Conference on Computer Graphics, pp. 147-152) proposes a generalization of the parallelogram prediction for arbitrary polygons. Missing vertex positions of a polygon are predicted with weights computed for polygons of different degrees from different known vertices. These weights come from a Fourier decomposition of polygons, where the highest frequencies are set to zero to ensure that the polygons are nicely shaped.
In publication “FreeLence coding with free valences (Kalberer F. et al., 2005, Computer Graphics Forum 24, 3, pp.469-478), a proposed FreeLence coder uses a (weighted) combination of three parallelogram predictors to encode the geometry data of triangular meshes.
Two predictors are standard ones such as with dual parallelogram prediction described above (Pt and P2), and an additional predictor (P3) is created by forming a parallelogram with the central vertex (A in Figure 4c) and the two outer vertices (C and E in Figure 4c).
Another publication “Multi-Way Geometry Encoding (Cohen-Or D. et al., 2002, Technical Report, School of Computer Science, Tel-Aviv University) introduces multiway prediction. The principle is to predict the position of new vertices with as many parallelogram predictors as possible built from already-encoded vertices.
As described above, the connectivity-based geometry prediction methods are mainly based on a parallelogram predictor or on a weighted combination of several parallelogram predictors. Success of the various parallelogram-based predictions is due to its efficiency because it processes geometry data that are generally present for very simple mesh structures and because it is a second order predictor.
Figure 5 illustrates general steps of the encoding of geometry data in the context of connectivity-based geometry prediction of the SC3DMC software. This algorithm may thus be implemented in step 233 after the connectivity data have been encoded (222).
As briefly introduced above, the conventional encoding of connectivity data involves traversing successively each vertex constituting the 3D mesh model, usually according to a vertex order, e.g. a spiral order; and for each vertex successively traversed (thus defining an encoding order), performing a triangle-based fan (T-FAN) analysis centered on the traversed vertex (i.e. one vertex of the triangles considered in the fan is the traversed vertex) to discover any new triangle not yet discovered when traversing the previous vertices. The discovered new triangles for the traversed vertex are grouped into one or more fan portions, each being made of a contiguous series of newly discovered triangle or triangles. A fan portion may be up to an entire fan, if it is closed.
Given such knowledge from the connectivity encoding, the encoding of the geometry data includes a first part (steps 501-519) to encode the geometry data of the vertices forming each fan portion, and a second part (steps 520-523) to encode the geometry data of isolated vertices in the mesh.
A first loop (501) goes through each fan portion. If the size (in number of vertices) of the current fan portion is strictly superior to 2 (test 502), vertex ‘a’ is set equal to the first vertex of this fan portion according to an encoding order. This is step 503. Otherwise the current fan portion is made of one or two isolated vertices. This is because a fan portion contains at least one vertex or at least one triangle (the size in number of vertices cannot be equal to 2 by definition of a triangular mesh). Isolated vertices are encoded in the second part from step 520.
If the first vertex ‘a’ of the current fan portion has not yet been encoded (test 504), it is not predicted, meaning that the value to encode, namely the residual below, for this vertex is set equal to the current position value of this vertex (for each position component X, Y and Z). This is step 505. Note that the first vertex of a fan portion is usually already coded if it is not the very first fan portion of a 3D object.
Thus if the 3D mesh model contains a single 3D object made of connected vertices, only the starting vertex is not predicted. However, if the 3D mesh model contains several independent (i.e. not connected) 3D objects, the starting vertex of each 3D object is not predicted.
Next (if ‘a’ is already encoded or next to step 505), vertex b is set equal to the second vertex of the current fan portion at step 506. Again, if the second vertex ‘b’ of the current fan portion has not yet been encoded (test 507), it is predicted by the previous vertex, namely by ‘a’. It means that the residual for this second vertex is set equal to its position value minus the position value of the first vertex ‘a’. This is step 508.
Next (if ‘b’ is already encoded or next to step 508), vertex c is set equal to the third vertex of the current fan portion at step 509.
Note that the vertices a, b and c form the first triangle of the current fan portion (having vertex ‘a’ as center vertex).
If ‘c’ has already been encoded, the process goes to step 514 to encode each other vertex of the current fan portion.
Otherwise, if there is a triangle already encoded which contains the vertices a and b (test 511), this triangle is used to generate a connectivity-based predictor to predict third vertex c: the residual for this third vertex ‘c’ is set equal to its position value minus the position value of the connectivity-based predictor. This is step 512.
As an example, the connectivity-based predictor is the parallelogram predictor introduced above.
If no such triangle exists, third vertex c is predicted by second vertex ‘b’ at step 513: the residual for this third vertex ‘c’ is set equal to its position value minus the position value of the second vertex ‘b’.
Next to steps 512 and 513, all other vertices of the current fan portion are successively considered through steps 514, 515 and the loop from 518.
If the vertex has not already been encoded (test 516), it is predicted by a connectivity-based predictor, e.g. a parallelogram predictor, at step 517.
Once all the vertices of the current fan portion have been processed, the next fan portion is considered at step 519.
Once all the fan portions have been analyzed, the algorithm goes through all vertices of the 3D encoded object (step 520) in order to encode the isolated vertices.
This is done by checking whether the vertex currently considered is already encoded (test 521), in which case the next vertex is considered.
Otherwise, the residual for the vertex currently considered (and thus isolated vertex) is set to the position value of this current vertex. It means that the isolated vertices are not predicted.
Although not represented in the Figures, the corresponding decoding process is straightforward from the encoding process. It consists in changing the word encoded by decoded in Figure 5 and in adding the predictor to the residue retrieved from the bitstream, instead of subtracting the connectivity-based predictor from the vertex position value. For instance, when the encoder performs the following operation “Residual[c] = c - predictor, the decoder correspondingly performs an inverse operation “c = Residual[c] + predictor”.
Note that the prediction is made for the three position components (X, Y, Z) at the same time.
Regardless of the predictors used, it remains that the residue obtained for each vertex (for any non-predicted vertex, the residue is the position value of the vertex) is entropy coded.
Note that the same geometry prediction scheme can be used to encode other property data (except connectivity data) according to the SC3DMC standard.
The above encoding scheme of 3D mesh geometry data, in particular in the context of SC3DMC-TFAN, is not satisfactory. Various improvements are proposed to obtain better coding efficiency of the geometry data.
One proposed improvement to the SC3DMC encoding of 3D geometry data introduces competition between position predictors to offer better use of predictors, with a view of improving prediction while limiting the amount of bits to add in the bitstream to signal the predictors. It is described now through embodiments which are directed to a method for encoding, into a bitstream, a three-dimensional model (e.g. points cloud or mesh model) made of vertices (e.g. connected vertices forming through triangles), the method comprising a step of encoding geometry data describing the positions of the vertices in the 3D model, wherein the encoding includes predicting the geometry data of at least one current vertex with a position predictor selected, based on a coding-cost-based criterion, from a set made of two or more position predictors, and encoding a residue between the geometry data and the position predictor.
Figures 6 and 7 illustrate exemplary encoding and corresponding decoding processes implementing the predictor competition. In these examples, the set of position predictors is made of two position predictors, namely predl and pred2.
Referring to the encoding process of Figure 6, a first part, steps 601 to 611 provides a coding-cost-based competition between the position predictors of the set (no matter they are two or more). A second and optional part of the process, namely steps 612 to 619, regards an optimization to avoid signaling which position predictor is used. This is to save bits in the bitstream, and thus reduce bitrate for the 3D (mesh) model.
Indeed, at least one predictor index is usually set, in memory of the encoding device, to identify a position predictor selected between two position predictors of the set.
The process of Figure 6 can be inserted in one or more of steps 508, 512, 513 and 517 of Figure 5 to select a position predictor with a view of predicting a current vertex position.
While the illustrated process is based on two position predictors predl (601) and pred2 (602), it can be extended to N position predictors without difficulties.
A current vertex whose vertex position is to be predicted is denoted ‘d’ in the
Figure.
The process starts at step 603 where the two position predictors are compared. If they are equal, it is similar to having a single position predictor.
Thus the residue for ‘d’, residual[d], is set equal to “d - pred1” (or “d - pred2”) at step 604. In addition, the predictor index used to identify the position predictor selected for vertex d is set equal to -1 at step 605 meaning that it will not be inserted in the bitstream. In other words, the predictor index is not added to the bitstream if the two or more position predictors are equal. This is to save signaling bits when useless, since whatever the value of the index, the same value of position predictor is used.
In a slight variant, the equality of test 603 may not be strict, so that inserting bits for signaling the predictor index is avoided if the two or more position predictors are close enough given a distance threshold.
If the position predictors are different, a residue is computed for each position predictor available at step 606, here res1 and res2 are computed from predl and pred2, as a difference between the position value of d and the position value of the predictor.
Their respective rate or coding costs are compared at step 607.
The cost of a residue can be, for example, the number of bits needed to encode the current residue, or the absolute value of the residue or any other function useful to compare two values. One may note that the real rate for both residues is the optimal cost function. However it may be complex for some applications. The cost function used here shall be the same at encoder and decoder side.
If the coding cost of the residue res1 associated with the first predictor, predl, is better or equivalent to the cost of the second residue res2, the predictor index is set equal to 0 at step 610 to indicate the first predictor, predl, is used. The residue for vertex d is thus set equal to “d - predl ” at step 611.
Otherwise, the predictor index is set equal to 1 at step 608 to indicate the second predictor, pred2, is used. The residue for vertex d is thus set equal to “d - pred2” at step 609.
As mentioned above, the process can end (step 620) at this stage for some implementations that require low processing complexity.
However, to increase coding efficiency, the second and optional part of the process may be implemented to remove useless predictor indexes. It requires additional comparisons.
The principle of the following steps is to remove predictor indexes which are not needed. This is because, according to the residual value and the position predictors available in the set, the decoder can guess or infer the selected predictor. To achieve that, the process may further comprises:
determining whether the position predictor can be inferred from the residue with certainty or not; and in case of positive determining, not adding, in the bitstream, a predictor index identifying the selected position predictor.
In particular, the determining step may include:
computing reconstructed vertex positions from the residue and from each of the position predictors. This is step 612 below;
determining whether the coding-cost-based selection of a position predictor from the set of position predictors selects the same position predictor for all the reconstructed vertex positions, or not. This is steps 613 to 619 below.
As shown in the Figure, the encoder derives the two possible candidates candl and cand2 (more generally as many candidates as the number of position predictors in the set) based on the current residue and both predictors predl and pred 2, at step 612.
Next, the best position predictors for both candidates candl and cand2 are determined at steps 613 to 618, using the same coding-cost selection scheme as used in step 607 (of course, the decoder will use the same scheme also).
The costs associated with the residue obtained from candl and with a prediction by predl and by pred2 respectively are compared at step 613. The best predictor index between predl and pred2 (more generally between the position predictors of the set) is set in variable best[0] at step 614 (for pred2) or step 615 (for predl).
In the same way, the costs associated with the residue obtained from cand2 and with a prediction by predl and by pred2 respectively are compared at step 616. The best predictor index between predl and pred2 (more generally between the position predictors of the set) is set in variable best[1] at step 618 (for pred2) or step 617 (for predl).
Next, if variables best[0] and best[1] are equal (test 619), it means that the position predictor can be inferred with certainty from the residue. Thus the predictor index for vertex d does not need to be transmitted, in which case variable PredlDX[d] is set equal to -1 at step 605 mentioned above. Variable PredlDX[ ], added to the bitstream, is used to indicate, to the decoder, whether the predictor index is indicated in the bitstream or can be directly inferred without being transmitted.
The proposed process reduces significantly the amount of predictor indexes which are transmitted. Consequently the bitstream is reduced compared to a conventional competitive method.
As some predictor indexes may not be added to the bitstream, the number of such predictor indexes may become different from the number of encoded vertices. Preferably, a number of predictor indexes identifying the position predictors selected for the vertices and added in the bitstream is specified within the bitstream, for instance before the encoding of the list of predictor indexes, in order for the decoder to be able to correctly decode the predictor indexes.
Note that the equality test of step 603 is not necessary and thus step 603 can be removed at the expense of an increase of process complexity. Yet, several computations of residual cost estimation at encoder and decoder side are avoided with the equality test of step 603.
In a variant, the equality test may be softened by not restricting it to a strict equality. For example, the equality test can consider that two values are equal given a delta or margin equal to + or - N. This may apply to the evaluations 613 and 616 of coding costs, for instance when the cost function is the absolute value. In such situation, if, for a reconstructed vertex position candl or cand2, the coding costs associated with respectively each of the position predictors of the set are similar given a cost margin (+/- N), all the position predictors of the set are considered to be selectable for the corresponding coding-cost-based selection step. This is to consider that the concerned test 613 or 616 does not limit the choice of predl or pred2. Thus, the other test 616 or 613 will determine the inferred position predictor to be used, such that it is not signaled in the bitstream.
As an example N is set equal to 1.
Although the above description is based on “monolithic” predictors, i.e. predictors provided at vertex level (a single position predictor of the set of predictors is used to predict all position component values of the current vertex), the position prediction can be made independently per each position component of the current vertex. In other words, one (component) predictor is used for each position component value of the vertex position. In such configuration, the above mechanism to remove some useless predictor indexes from the bitstream is even more efficient.
The amount of predictors is very important for the coding efficiency of a competitive method. To avoid too many bits to be used to signal the position predictors, in particular when a predictor is used per position component (X, Y or Z), the set of position predictors is preferably made of two position predictors per position component.
The corresponding decoding process is shown in Figure 7. Due to prediction, the decoding process obtains, from the bitstream, a residue for a current vertex;
obtains a position predictor for the residue; and adds the position predictor to the residue to obtain the geometry data of the current vertex.
The process only corresponds to the second part of the encoding process, because the first part is only for the encoder to determine which predictor to use.
The decoding implementation of the second part of the encoding process takes place during the obtaining of the position predictor and includes for the decoder the steps of:
determining whether a position predictor can be inferred from the residue with certainty or not; and in case of positive determining, using the inferred position predictor for the step of adding (to reconstruct the geometry data of the vertex); otherwise, retrieving, from the bitstream, a predictor index identifying, in a set of position predictors, the position predictor to be used for the step of adding.
Thus, a predictor index is retrieved from the bitstream only if the predictor can be deduced with certainty by the decoder.
As shown in Figure 7, the same process as the one provided in the second part of the encoding process is implemented. In other words, steps 701-703 are identical to steps 601603 and steps 705-712 are identical to steps 612-619.
If both predictors predl (701) and pred2 (702) are equal (test 703), the predictor index i is equal to 0 at step 704, meaning that predl is selected. The current position value of vertex d is reconstructed at step 717 by the following formula: “Residue[d] + predl” corresponding to the adding step mentioned above where Residue[d] is the decoded residue for vertex d.
Otherwise, the two possible candidates candl and cand2 are derived at step 705 from the retrieved residue and each of the two predictors. The coding costs of the residues obtained from candl by prediction (candl - predl and candl - pred2) are compared at step 706 and the best predictor index between these predictors is set in variable best[0] (step 707 or 708). In the same way, the best predictor for the second candidate cand2 (step 709) is determined and the best predictor index is set in variable best[1] (step 710 or 711).
If variables best[0] and best[1] are equal (test 712), the predictor index i is equal to the value best[0] (or best[1]) at step 716.
Otherwise, the predictor index 715 is extracted from the bitstream at step 713 in particular from the decoded list of predictor indexes 714 (or it needs to be decoded according to the kind of possible implementation).
Net to step 715 or 716, the current position value V[d] of vertex d is reconstructed at step 717 by the following formula:
if i=0, V[d]=Residue[d] + predl otherwise, V[d]=Residue[d] + pred2.
An improvement to the SC3DMC encoding or decoding process of 3D geometry data regards the use of coding contexts, in particular to take advantage of the entropy existing at vertex position component level when the geometry data are made of two or more position components. To that end, data related to the two or more position components for the vertices, for instance the two or more component values of a vertex position or of a residue between a vertex position and a position predictor, are encoded or decoded using two or more respective independent entropy encoders or decoders. This sharply contrasts with SC3DMC standard in which a single entropy encoder is used for the whole vertex.
In the same way, the predictor indexes may be encoded with separate and independent entropy encoders. The predictor indexes values are encoded according to their related component. In other words, where each component value of a point position is predicted using a corresponding position component predictor, indexes identifying the two or more position component predictors corresponding to the two or more component values of the point position may be encoded or decoded using the two or more respective independent entropy encoders or decoders.
Consequently, above variable PredlDX[] is divided into an appropriate number of independent variables, namely three in the above example with three position components.
Due to the independency between the components, index sizes may vary from one component to the other. So the size of each variable is preferably transmitted. In other words, index sizes defining the respective sizes of the indexes identifying the two or more position component predictors are included in the bitstream.
One advantage of coding contexts for the predictor indexes is still a coding efficiency improvement. This is because the predictor indexes selection can be very different from one component to the other, as the vertices residue component values are also very different one from the other.
Also the size of the encoded codewords may be adapted. In SC3DMC, the size of the codewords is fixed to 8 bits. Thus embodiments propose that the component values of the residue are encoded using codewords or decoded from codewords having different bit sizes from one position component to the other. The bit size of each position component of the codewords may then be included in the bitstream to be transmitted to the decoder. So the bit sizes are transmitted.
An advantage of adapting the codeword size (bit size) at component level is a coding efficiency improvement. This is because the statistics and the possible residue values for each residual component depend on the topology structures and the quantization parameters.
Turning now to the composition of the set of position predictors, which plays an important role to have an efficient competitive scheme, the prior art proposes a list based on the encoding order of the vertices, which usually includes a fixed predictor (SC3DMC), namely the parallelogram predictor or its improved versions (dual or weighted combination).
The parallelogram predictor and its improved versions (dual or weighted combination) are very efficient predictors for quite regular mesh topologies as depicted in Figure 8a (illustrated for 3D mesh models). However, in practice it is less efficient when the 3D model structure becomes less regular, for instance as depicted in Figure 8b, and can become inefficient for very irregular 3D model structures as depicted in Figure 8c. While these figures illustrate 3D mesh models, the same observation can be made with respect to points clouds, being noted that a 3D model regularity can be assessed with respect to a regularity of the distribution of vertex positions within the 3D model.
Other position predictors are proposed that are well adapted to less regular to irregular 3D models. They may supplement the parallelogram predictor in a set of position predictors or be used without the parallelogram predictor to introduce competition between several position predictors. Of course, these other position predictors can also be used independently one to the other, without competition.
In particular, such position predictors may be used at the encoder or at the decoder.
When based on the connectivity data, geometry data prediction can rely on the TFAN algorithm of the SC3DMC standard. In that case, vertex A as depicted in Figure 4a is the first vertex of the current fan portion and thus defines the center vertex in common for all the triangles of the fan portion, i.e. the vertex around which other vertices directly connected define the outer vertices of the triangles of the fan portion.
Vertex B refers to the last but one previously processed/considered vertex (or a vertex connected to A and C) and vertex C is the last processed vertex. Thus, new vertex of the fan portion for which position is to encode (or decode) is vertex D.
Notation A, B, C, D is kept below for ease of explanation.
New position predictors encompass one or more of a C+ predictor, a parametric predictor, a 2C-B predictor and an E-parallelogram predictor.
The C+ predictor corresponds to a point, each position component value (or coordinate) of which being one position component value selected, based on a selection criterion between the corresponding position component values of three vertices already encoded/decoded.
The parametric predictor corresponds to a point, the value of a first position component of which being the position component value, having the same component axis, of an intersection point between two first planes respectively perpendicular to the two other axes at estimates of corresponding second and third position components of the current vertex D, and a third plane including three vertices preceding the current vertex according to an encoding order.
The 2C-B and E-parallelogram predictors rely more strictly on 3D mesh connectivity, and more particularly depend on a fan portion describing connectivity between vertices around a center vertex , the fan portion being made of successive outer vertices that define a series of adjacent triangles having the center vertex in common, the current vertex being a current one of the successive outer vertices and defining, with already-encoded or already-decoded vertices, a next triangle adjacent to a previous triangle.
The 2C-B predictor corresponds to a point which is the symmetric, with respect to the outer vertex of the previous triangle belonging to the adjacent edge, of the other outer vertex of the previous triangle.
The E-parallelogram predictor corresponds to a point which forms a parallelogram, with the center vertex in common to the adjacent triangles, the outer vertex of the previous triangle belonging to the adjacent edge and an outer vertex of another triangle not adjacent to the previous triangle and having the center vertex in common.
Thus, four new main position predictors are proposed.
A representation of the 2C-B predictor defined above is shown in Figures 9a and 9b. The first vertex used as point of symmetry is C, while the second vertex whose symmetric is generated is B. Thus, the 2C-B predictor is obtained by the formula 2 * V[C] - V[B], where V[x] is the position of vertex x and * is the multiplication sign.
The 2C-B predictor is a second order predictor as the parallelogram predictor. As shown in particular in Figure 9b, it can be much more efficient than the conventional parallelogram predictor when the triangle mesh forms squares.
The C+ predictor is shown in Figure 10. The C+ predictor may thus be defined by one or more position component values of vertex C and one or more positions component values of common vertex A.
Using the connectivity data (fan portion and so on), the three vertices already considered from which the C+ predictor is built include the three vertices, A, B and C defining the previous triangle.
The C+ predictor is an extension of predictor C (i.e. prediction by vertex C), in which its value is switched at component level to one or more component values of vertex A.
The three vertices already considered from which the C+ predictor is built are preferably the last three vertices A, B, C considered when considering the current vertex D.
As shown in Figure 10, if the value of vertex C for the current component (X, Y or Z) is between vertices A and B, the corresponding component (X, Y or Z) of the C+ predictor is equal to the current component value of vertex A; otherwise it is set equal to the current component value of vertex C.
In other words, the selection criterion used to select a position component value for the C+ predictor is based on a relative positioning of the corresponding position component values of the three vertices (A,B,C) already considered. More precisely, the selecting step based on the selection criterion includes determining whether the corresponding position component value of the outer vertex (C) of the previous triangle belonging to the adjacent edge (AC) is between the corresponding position component value of the traversed vertex (A) in common to the adjacent triangles and the corresponding position component value of the other outer vertex (B) of the previous triangle, and in case of positive determining, selecting the corresponding position component value of the traversed vertex (A) as the corresponding position component value of the C+ predictor; otherwise selecting the corresponding position component value of the outer vertex (C) belonging to the adjacent edge (AC) as the corresponding position component value of the C+ predictor.
Preferably, this approach is made for each of the position components X, Y, Z.
Figure 11 illustrates, using a flowchart, general steps for determining the C+ predictor given the knowledge of A, B and C (1101, 1102, 1103). Such knowledge is well known because it is, for example, used for the computation of the conventional parallelogram predictor.
The process includes, for each position component T (1104), determining whether the component value of vertex C for this position component, i.e. value V[c][i], is between the corresponding components values of vertices A and B, or not. This is test 1105.
In case of positive determination, the corresponding component value of the C+ predictor, i.e. pred[d][i], is set equal to the corresponding component value of vertex A, i.e. V[a][i], at step 1107; otherwise it is set equal to the corresponding component value of vertex C, i.e. V[c][i], at step 1106.
The next position component can be considered through step 1108 looping back to step 1104.
One advantage of using the predictor C+ is an improvement of coding efficiency compared to coding efficiency of predictor C or predictor A.
Turning now to the E-parallelogram predictor, it corresponds to a point which forms, with the traversed vertex (A) in common to the adjacent triangles, the outer vertex (C) of the previous triangle belonging to the adjacent edge and an outer vertex (E) of another triangle not adjacent to the previous triangle and having the traversed vertex in common, a parallelogram.
It may correspond to point P3 introduced above with respect to Figure 4c. Using the connectivity data, the current vertex to encode/decode, namely vertex D, is a current one of the successive outer vertices (forming the fan portion) and defines, with already-encoded or already-decoded vertices, two contiguous triangles respectively adjacent to a previous triangle (ABC in the Figure) and to a following triangle (AEF in the Figure). These two triangles thus defines the last two triangles of the current fan portion. Point P3 thus form a parallelogram with the traversed vertex (A) and the two outer vertices (C, E) belonging to the adjacent edges (AC and AE) between a respective one of the two contiguous triangles and one of the previous and following triangles.
A similar vertex E may be used in situations where vertex D does not define the last two triangles ofthe current fan portion.
Thus, more generally, the outer vertex of the other triangle for the E-parallelogram predictor may be the first already-encoded or already-decoded outer vertex of the fan portion, according to a scanning order ofthe triangles in the fan portion. The scanning order is the order used to successively consider each triangle of the fan turning around center vertex A.
Usually the fan portion ends with an ending triangle that is adjacent to the first triangle of a fan portion turning around center vertex A. If the fan portion has an ending triangle adjacent to a first triangle having the center vertex in common, the outer vertex of the other triangle may also be the outer vertex of the first triangle belonging to the edge adjacent to the ending triangle.
In the prior art, the P3 point is used within a combination of several points (generated by parallelograms) to obtain another position predictor. In sharp contrast, the inventors have noticed that the particular P3 point provides good results in some situations. Thus, it is proposed to use this particular P3 point as a standalone position predictor.
The E-parallelogram predictor or P3 point can be obtained by the following formula:
Pred[d] = V[c] - V[a] + V[e],
One may note that the E-parallelogram predictor is usually generated from vertices that are closer, in term of connectivity, to vertex D, than the conventional parallelogram predictor. For instance, vertices A, C and E may all be directly connected to vertex D, while the conventional parallelogram uses vertex E3 which is not directly connected to vertex D. It results that better prediction can be obtained, and thus improved coding efficiency.
Also, as vertex E used as an anchoring/reference point to draw the parallelogram is usually a vertex on which the fan portion should end, it provides a good estimate (predictor) to locate any vertex between the last triangle processed and the end of the fan portion, i.e. the current vertex.
As far as the parametric predictor is concerned, it is formed by a point representing an intersection point between various planes. Since, as explained below, this position predictor can be generated using parametric equations representing planes, it is why it is called below “parametric predictor”.
The principle of the parametric predictor is to determine the equations of the planes of a triangle neighboring current vertex D to encode/decode, and to use the equations to generate a predictor at least for one position component of vertex D.
Figure 12 illustrates this principle ofthe parametric predictor.
Assuming the first component value (XD) and the second component value (YD) of vertex D are known, the Zpp component of the parametric predictor is the corresponding position component ofthe intersection point between the two planes respectively perpendicular to the two other axes (X-axis and Y-axis) at estimates of corresponding position components ofthe current vertex, for instance the values ofthe corresponding position components ofthe current vertex, i.e. XD and YD, and the plane ofthe previously processed triangle (ABC), i.e. the plane including three vertices preceding the current vertex D according to an encoding/decoding order, the three vertices of the first plane include for instance the three vertices defining the previous triangle (ABC), if the connectivity data can be used.
The value Zpp is then used to predict the value of the corresponding position component of current vertex D. It means that the values of the two other position components of the parametric predictor are set to the values of the corresponding position components of a position predictor different from a parametric predictor. The conventional parallelogram predictor may be used to set these two other position components. In variants, one ofthe 2C-B predictor, the C+ predictor and the E-parallelogram predictor can be used.
Of course, it is not necessarily the Z component that is predicted. The first position component of the parametric predictor (thus the position component to predict) is one of abscissa along an X-axis, ordinate along an Y-axis and applicate along a Z-axis. In that case, the two other position components are thus used to generate the two perpendicular planes.
A choice of which position component is the first position component of the parametric predictor may be performed by the encoder. Thus the encoder preferably determines, based on a coding cost criterion, the first position component of the parametric component (i.e. the position component to be predicted) from available position components including abscissa along an X-axis, ordinate along an Y-axis and applicate along a Z-axis.
In one embodiment, the encoder may test the three possibilities, i.e. may predict separately the values of the three position components of current vertex D by three respective parametric predictors, the parametric predictors having each a different position component built from an intersection point between two first planes perpendicular to the two other position component axes and the third plane. In other words, to test component X of current vertex D, a first parametric predictor is generated using two planes perpendicular to Y-axis and Z-axis and using the plane of the previous triangle; to test component Y, a second parametric predictor is generated using two planes perpendicular to X-axis and Z-axis and using the plane of the previous triangle; and to test component Z, a third parametric predictor is generated using two planes perpendicular to X-axis and Y-axis and using the plane of the previous triangle;
The encoder next evaluates a cost for each of these predictions; and compares the evaluated costs to select the best one (in terms of coding efficiency) of the position components.
A drawback of this approach is that three computations are required for each current vertex, which is quite complex in terms of processing.
As a consequence, another embodiment uses the same approach but tests only a restricted subset of vertices before applying parametric predictors to the vertices of the 3D model. The tests on the restricted subset make it possible to evaluate what should be the best position component for parametric prediction.
This approach is advantageously less complex in terms of processing, in particular for large 3D meshes.
In variants to testing points, the encoder can select the position component which has the maximum range of the bounding box nesting the 3D model. In other words, the encoder obtains a bounding box encompassing the 3D model and selects the position component having the highest range in the bounding box as the first position component of the parametric predictor.
This approach is efficient because on one hand, for this specific position component, the residue should be larger than other position components when predicted by conventional predictors. And, on the second hand, it is worth having the position component with the largest rate (thus largest residues) to be predicted by the efficient parametric predictor.
Figure 13 illustrates, using a flowchart, general steps for generating the first position component value, e.g. Zpp, of the parametric predictor for current vertex D=(Xd,Yd,Zd). Of course, the same reasoning may apply for position component XD or YD (to have Xpp or Ypp)
Vertices A(Xa, Ya, Za) (1301) B(Xb, Yb, Zb) (1302), C(Xc, Yc, Zc) (1303) are original knowledge (as for conventional parallelogram predictor).
Based on this knowledge, the coefficients of the parametric equation of the plane formed by vertices A, B, C are obtained by following formula at step 1304:
a = (Yb-Ya)*(Zc-Za) - (Zb-Za)*(Yc-Ya); β = (Zb-Za)*(Xc-Xa) - (Xb-Xa)*(Zc-Za); γ = (Xb-Xa)*(Yc-Ya) - (Yb-Ya)*(Xc-Xa); δ = -ot*Xa - 3*Ya - y*Za;
When the position component ZD of current vertex D (XD, YD, ZD) (1306) is predicted at encoder side by the parametric predictor or its residual is added to the parametric predictor PP at decoder side, its component value Zpp (Pred[d][2] representing the predictor for component Z of current vertex D in Figure 13) is obtained by the following equation formula at step 1305:
Zpp = - (a * XD +β * YD + δ)/γ (XD, YD, Zpp) is thus a parametric predictor for current vertex D, but only for the third position component ZD.
One may note that this formula is generated by considering that each vertex (X, Y, Z) in the plane forming by the vertices A, B and C solves the following equation:
ot*X + β*Υ + γ*Ζ + δ =0.
It cannot be excluded that the obtained Zpp be negative. However, the abovementioned quantization ofthe vertex positions necessarily provides positive position component values. Consequently, it is proposed that if the position component value of the intersection point having the same component axis is negative (the two others XD, YD are necessarily positive), the first position component value of the parametric predictor is set based on a reference position predictor.
For instance, as shown in the Figure, when Zpp generated is negative (test 1307), it is changed into another predictor value at step 1308, for example a by-default value from reference predictor RP 1309.
The parametric predictor is particularly well adapted to the prediction of vertices in very irregular mesh structures as shown in Figure 8c. It also provides significant coding improvements for poorly regular mesh structures as shown in Figure 8b.
While the above illustrates the use of parametric predictors for the prediction of a single position component, some embodiments may provide that two or three position components are predicted using such parametric predictors.
For example, the value of a second position component ofthe parametric predictor is the position component value, having the same component axis, of a point representing an intersection point between two first planes respectively perpendicular to the two other axes at estimates of corresponding first and third position components ofthe current vertex, and a third plane including three points preceding the current vertex according to an encoding order.
However, for the second position component, it is not possible to use the two other position component values of current vertex D to generate the parametric predictor because both are not available at decoder side (the first one should be obtained by the parametric parameter relying on the values of the two others). So the position component value not available is preferably replaced by another predictor value. For instance, the estimate of the first position component of the current vertex is a value of the corresponding first position component of a reference position predictor.
The three position component values of the current vertex can be predicted using the parametric predictor. In that case, the value of the third position component of the parametric predictor is the position component value, having the same component axis, of a point representing an intersection point between two first planes respectively perpendicular to the two other axes at estimates of corresponding first and second position components of the current vertex, and a third plane including three points preceding the current vertex according to an encoding order.
Always due to non-availability of the first and second position components of the current point (not known by the decoder at this stage), the estimates of the first and second position components of the current vertex are values of the corresponding first and second position components of the reference position predictor.
For instance, the reference predictor used in replacement of the missing position components may be the conventional parallelogram predictor, i.e. a point forming a parallelogram with at least two vertices (A, C) already encoded and directly connected to the current point.
Assuming the reference predictor is noted P=(P[0], P[1], P[2]), the parametric predictor PP (Xpp, Ypp, Zpp) can be obtained by applying the following formulas:
Χρρ = -(β*Ρ[1] + γ*Ρ[2]+δ)/α
Ypp = -(ot * XD + γ * P[2] + δ)/β
Zpp = -(a * XD + β * YD + δ)/γ
In this embodiment, Zpp is the only position component generated by the two other true values of current vertex D. So, prediction of position component Z should have a better prediction on average than the two other position components and position component Y should have a better prediction on average than position component X (which depends on two component values of the reference predictor).
Of course the formulas can be changed to modify the order of the position components, in order to vary which position component would have better prediction. For example, position components X and Y can be switched, or the following order: Y, Z, X can be used. In the latter case the formulas are the following:
Ypp = -(α * Ρ[0] + γ * Ρ[2] + δ)/β
Ζρρ = -(α * Ρ[0] + β * YD + δ)/γ
Χρρ = -(β * YD + γ * ZD + δ)/α
One may note that the parametric predictor needs some division operations. Consequently, it is worth using floating operations to compute this predictor.
However, it may not be acceptable for some applications due to additional complexity. Thus, in one embodiment, all operations to generate the parametric predictor are integer operations.
In one embodiment, when the quantization parameter (number of bits per position component) is less than a threshold, the parametric predictor is generated using integer operations; otherwise using floating operations. This is because, when the quantization is high, the precision with integer operations is enough to obtain good results.
In the explanation above, the third plane is generated by the triangle previously processed (i.e. ABC). In variants using connectivity data in which the current vertex defines, with already-encoded or already-decoded vertices, two contiguous triangles respectively adjacent to a previous triangle (ABC) and to a following triangle (AEF), the three points of the third plane may include the two outer vertices (B, C) of the previous triangle and the outer vertex (E) of the following triangle belonging to the edge adjacent (AE) to one of the two contiguous triangles.
It means that one of the vertices of the previous triangle (A,B,C) is changed by vertex E. An illustration of this is shown in Figure 14.
More generally, vertex E may be any vertex E as defined above for the Eparallelogram predictor. In other words, vertex E is an outer vertex of another triangle not adjacent to the previous triangle (ABC) and having the traversed vertex (A) in common. It may be the first already-encoded or already-decoded outer vertex of the fan portion, according to a scanning order of the triangles in the fan portion and/or the outer vertex of the first triangle belonging to the edge adjacent to the ending triangle.
In a preferred embodiment, vertex A is replaced by vertex E (and not B as in the parallelogram with E). This is because for the parametric predictor, it is expected that a larger area should give a better parametric predictor.
Indeed, usually the average valence in a mesh is around six. Thus, triangle EBC (shaded triangle in Figure 14b) should give a larger area on average than triangle ABC (dotted triangle in Figure 14a). As a consequence, triangle EBC can be taken into account to generate the third plane and consequently the parametric predictor.
As indicated above, preference is given to providing a competition between several position predictors. It means that the position predictor is selected, based on a coding-costbased criterion, from a set made of two or more position predictors, which preferably includes two or more of the 2B-C predictor, the C+ predictor, the E-parallelogram predictor and the parametric predictor. The parallelogram predictor may also be involved in the set of position predictors.
In a preferred embodiment, a parametric predictor whose values of the second and third position components are set based on corresponding position component values of the C+ predictor (in addition to the first position component built from the intersection point) is used in the competition.
For instance, the set for competition includes, as a first position predictor, the Eparallelogram predictor, and includes, as a second position predictor, this parametric predictor, The set can be restricted to these two position predictors.
When using this set of position predictors, substantial improvements in the coding of irregular meshes (Figure 8c) or more generally irregular 3D models can be obtained. However, the improvements in coding efficiency may sometimes remain low for regular or semiregular meshes or 3D models (Figures 8a and 8b). The inventors believe it is mainly because the conventional parallelogram predictor is already very efficient due to the regular structure where each neighboring triangle seems to form a parallelogram with a current triangle. In particular, good predictors for regular or semi-regular 3D models seem to include the Eparallelogram predictor and the C+ predictor.
Adaptations are now proposed to further improve the position prediction according to the regularity of the 3D models, when a competition between predictors is implemented.
The adaptations thus impact any method for encoding, into a bitstream, a threedimensional model made of vertices (or “points”), the method comprising a step of encoding geometry data describing the positions of vertices in the 3D model, wherein encoding the geometry data of a current vertex includes:
predicting the geometry data of the current vertex with a position predictor selected, based on a coding-cost-based criterion, from a set of one or more position predictors, and encoding a residue between the geometry data of the current vertex and the selected position predictor.
It may also impact a corresponding decoding method for decoding, from a bitstream, a three-dimensional model made of vertices, the method comprising a step of decoding geometry data describing the positions of vertices in the 3D model, wherein decoding the geometry data of a current vertex includes:
selecting, based on the bitstream, a position predictor from a set of one or more position predictors and used to predict the geometry data of the current vertex;
obtaining, from the bitstream, a position residue for the current vertex position to decode;and computing a decoded position of the current vertex by adding the selected position predictor to the obtained position residue.
Proposed adaptations require determining a value representative of regularity (or irregularity) of a vertex position distribution in the 3D model (i.e. a mesh regularity/irregularity in the case of a 3D mesh model), and setting the set of position predictors used to select the position predictor, based on the determined regularity value.
The regularity value thus helps classifying the current 3D model structure.
Various criteria may be used to evaluate the regularity/irregularity of the 3D model, be it a local evaluation (in the vicinity of the current vertex for which the position is to be predicted) or an overall evaluation (over a substantial part or more of the 3D model).
This approach makes it possible to obtain an efficient trade-off between processing complexity and coding efficiency. Indeed, by setting the set of predictors (for instance by switching between two [or more] sets) according to an evaluated regularity/irregularity, it is possible to keep few predictors in the set (to keep low processing complexity) that are both efficient to encode the vertex positions (to improve coding gain compared when one or more predictors in the set are never used because not adapted to the regularity/irregularity of the 3D model).
Below, various exemplary criteria are proposed which determine the regularity value based respectively on the position predictors used to predict the geometry data of a test set of vertices of the 3D model that are encoded (or decoded for the decoding method) before the geometry data of the current vertex is predicted. This is an overall regularity estimate of the 3D model over the already-encoded or already-decoded vertices. Embodiments of this approach are described below with reference to Figure 15; and on connectivity data (in the case of a 3D mesh structure) related to vertices in the vicinity of the current vertex describing the connectivity between the vertices in the 3D mesh model. This is a local mesh regularity/irregularity estimate in the vicinity of the current vertex. Embodiments of this approach are described below with reference to Figures 16 to 19. As shown in one illustrative embodiment described below with reference to Figure 17, the two criteria above may be combined; and on the geometry data related to vertices in the vicinity of the current vertex. This is another local regularity/irregularity estimate in the vicinity of the current vertex. A geometric parameter can be evaluated using the geometry data to represent the regularity of the 3D model. It may be based on distances between vertices as suggested in Figure 19 described below (in particular test 1916 suggests an exemplary but not limiting geometric parameter that could be used or extended to a higher number of distances to be compared); for instance based on the distances between couples of vertices in the vicinity of the current vertex; for example, based on the distances between the current vertex and each of a plurality of vertices in its vicinity. As an example, the geometric parameter to be used as regularity value may be a variation or deviation of such calculated distances.
For ease of explanation, the description below is made with respect to 3D mesh models made of connected vertices. One skilled in the art can directly extend part or all of these explanations to points clouds where no connectivity data is required.
Figure 15 illustrates, using a flowchart, first embodiments adapting the set of predictors based on the regularity/irregularity of the mesh. In these embodiments, a ratio of use of a reference predictor, to predict the geometry data of the vertices of the test set is evaluated to determine if the mesh structure is a regular mesh structure or an irregular mesh structure. In the exemplary process of the Figure, the parametric predictor is chosen as reference predictor. This is because this particular predictor is highly suitable to position prediction in irregular mesh structures. The higher the parametric predictor use ratio, the more irregular the mesh structure.
Preferably, the values of the second and third position components of the parametric predictor are C+ predicting components, each C+ predicting component value being one position component value selected, based on a selection criterion, between the corresponding position component values of three vertices already encoded or decoded. As indicated above, the choice of parametric position component between X, Y and Z components is meaningless.
The process of Figure 15 may be implemented at one or more or each prediction step in the process of Figure 5, for instance steps 508, 512, 513, 517.
In this process, each vertex of the 3D mesh model is successively considered through the loop 1511-1503. The order for consideration is preferably the connectivity-based order, i.e. the order used for the encoding.
The first steps, 1500-1502, of the process are to define a first set of one or more position predictors, PredSetl, each time a new 3D object is processed (several independent 3D objects may belong to the same RAW file).
At step 1500, a usage counter is initialized to 0. The counter is used to count how many times the reference predictor, here the parametric predictor, is used as a predictor of the vertex positions.
1502 illustrates the first set PredSetl suitable for irregular meshes. Thus, the first set include the reference predictor here the parametric predictor. In the example, the first set is made of two predictors, also including the E-parallelogram predictor. These predictors of PredSetl are predictors obtained from the geometry data of already-encoded vertices as defined above. Preferably they are used individually at position component level, meaning one predictor per component is enabled for the competitive based scheme used at step 1505 below. In a variant, they may be used as vertex-level predictors (or “monolithic” predictors), meaning that a single position predictor of the set is determined to predict all position component values of a vertex currently considered.
At step 1501, PredSetl is used as the by-default set for the current mesh.
For each vertex of the object (1503), the predictor set is derived at step 1504 meaning the E-parallelogram predictor is calculated given the positions of already-encoded vertices (A,C,E) and the parametric predictor (including the two C+ predicting components) is also calculated based on the positions of the other vertices (A,B,C).
A coding-cost-based competition is evaluated to select the best predictor or components of the predictors to predict the position of current vertex v (step 1505).
In case the reference predictor, here the parametric predictor, is used (test 1506), the usage counter is incremented (step 1507). Thus, as the vertices are successively considered, the usage counter may increase, from which a ratio of use can be easily determined.
Otherwise, or next to step 1507, it is evaluates whether or not a sufficiently large set of vertices (“test set”) has been considered to estimate the ratio of use of the reference predictor. In the illustrative example, the test set of already-encoded or already-decoded vertices is made of the N first connected vertices composing the 3D mesh model, according to the encoding order. For instance N=1000 vertices.
One should note that, although this example envisions considering the N first vertices, alternative may contemplate using an N-wide sliding window of vertices, as long as the first set of predictors PresSetl is used. This includes keeping up-to-date the usage counter to mirror the use of the reference predictor within the sliding window only.
Still at step 1507, if the number of considered vertices has reached the value N, the ratio of use of the reference predictor is evaluated and compared to a first threshold th1, in particular to determine whether the usage ratio is less than the first threshold.
An exemplary first ratio threshold is th1 =0.3. Note that a variant of step 1507 may directly compare the usage counter to N * th 1.
In case of a negative outcome at test 1507, the next vertex in the 3D mesh is considered (step 1511 looping back to step 1503).
Otherwise, when the mesh regularity/irregularity value indicates a reference predictor use ratio less than the first threshold, the process switches at step 1509 from the first set PredSetl of one or more position predictors used to predict the geometry data of the vertices of the test set, to a second set PredSet2 (shown in 1510) of one or more position predictors to predict the geometry data of the current vertex, and preferably to predict the geometry data of several, even all, the vertices subsequent to the current vertex in the 3D mesh model according to an encoding order.
The second set PredSet2 is more suitable (than PredSetl) for position prediction within regular to semi-regular mesh structures.
This second set PredSet2 does not include the reference predictor. It preferably includes the E-parallelogram predictor and the C+ predictor. Preferably, they are monolithic predictors, i.e. they operate at vertex level (a single position predictor of PredSet2 is determined to predict all position component values of a vertex currently considered).
In one specific embodiment, the second set PredSet2 may be restricted to a single position predictor. This is mainly to avoid competition in case of regular to semi-regular meshes, thereby reducing processing complexity.
The above-illustrated process distinguishes between irregular mesh structures and regular to semi-regular mesh structures, thanks to the first threshold. Enhanced embodiments may envision further distinguishing between regular mesh structures and semi-regular mesh structures, thereby adjusting the set of predictors to be used and improving coding efficiency.
In these enhanced embodiments, the same principle as above is applied, but with a stricter usage ratio threshold.
For instance, if the mesh regularity/irregularity value indicates a reference predictor use ratio less than a second threshold th2, the second threshold th2 being lower than the first threshold th1 (thereby indicating a more regular mesh structure), the process switches to a third set PredSet3 of one or more position predictors to predict the current vertex.
Predictors even more adapted to regular meshes can be used. For instance, a conventional parallelogram predictor may be provided in PredSet2 (for semi-regular meshes) where the E-parallelogram predictor may be provided in PredSet3 (for regular meshes).
In some embodiments, the position predictors of PredSet2 and PredSet3 are the same. However, while the predictors of PredSet3 are vertex-level predictors (monolithic), each position predictor of the second set PredSet2 may be a component-level predictor, meaning that, for each position component of the current vertex, a position predictor of the second set is determined to predict the value of said position component.
The illustrated first embodiments of Figure 15 thus switch from one set of position predictors to the other, depending on evaluated regularity of the 3D mesh structures. As the used sets of position predictors are more adapted to the specific regularity of the meshes, prediction is made more efficient, thereby improving coding efficiency.
One directly understands that this process can be implemented at encoding side or decoding side. The sole difference is that the mesh regularity/irregularity value is evaluated using already-encoded vertices in case of the encoder, while it is evaluated using alreadydecoded vertices in case of the decoder.
Turning now to second embodiments wherein the mesh regularity value is based on connectivity data related to vertices in the vicinity of the current vertex describing the connectivity between the vertices in the 3D mesh model, it is made reference to Figures 16 to 19.
As apparent from below, the second embodiments evaluate a local regularity/irregularity of the mesh, and then adapt the predictor or set of predictors accordingly. In other words, the second embodiments provide dynamic adapting of the predictor or set of predictors, even on-the-fly adapting. One may thus contemplate using the first embodiments above to switch between general sets of position predictors, and using the second embodiments below to adjust the selected set of position predictors.
The second embodiments are based on the following observations. As indicated above, the parallelogram predictor and the E-parallelogram predictor are quite efficient for regular mesh structures as shown in Figure 8a.
As illustrated in Figure 16a, the parallelogram predictor sounds to be ideal for position prediction when the valence for vertex A (corresponding to the number of vertices connected to A) is 6.
However, when the valence is smaller as illustrated in Figure 16b, the parallelogram predictor is no longer an efficient position predictor even if the structure is regular. In the same way, when the valence is higher than 6 as illustrated in Figure 16c, the parallelogram predictor is also no longer efficient; the E-parallelogram predictor is in the case of the Figure more efficient.
The local-based adjustment of the set of predictors is advantageously suitable for 3D objects that comprise both pieces with irregular structures and pieces with regular structures.
As proposed below, the second embodiments thus propose a mesh regularity value that includes a valence value for the center vertex, corresponding to a number of vertices connected to the center vertex. It is recalled that the center vertex is defined given the TFAN analysis: a fan portion describing connectivity between vertices around a center vertex to which the current vertex is connected is obtained, the fan portion being made of successive outer vertices that define a series of adjacent triangles having the center vertex in common, the current vertex being a current one of the successive outer vertices and defining, with alreadyencoded vertices, a next triangle adjacent to a previous triangle.
In a variant that provides a vertex-by-vertex adjustment, the valence value for the current vertex may be used as mesh irregularity value driving the selection of the set of position predictors. More generally, the valence value for any vertex in the vicinity of the current vertex (for example any vertex connected to the current vertex) may be used instead.
Figure 17 illustrates an adaptation of the process of Figure 15 to provide on-the-fly adjustments of the position predictors in PredSet2, i.e. the position predictors suitable for regular to semi-regular mesh structures. Depending on the valence of vertex A (the same may be performed based on the valence of vertex D to be encoded, since all the connectivity data are already known by the encoder or decoder when encoding the geometry data), PredSet2 may be slightly adjusted to encode/predict the positions of outer vertices of the fan portion centered on A.
Boxes 1700-1711 respectively correspond to boxes 1500-1511.
Additional process is provided through boxes 1712-1714 following step 1703.
In case the set of position predictors for current vertex v is already PredSet2 (test 1712), i.e. the mesh structure has already been evaluated as being regular or semi-regular on overall, the valence of vertex A (or D) is checked against value 6 (test 1713).
If this valence is equal to 6, the second predictor, Predl, in PredSet2 is set equal to the parallelogram predictor at step 1714 for the current vertex, in replacement of the C+ predictor. Thus, PredSet2 includes a parallelogram predictor only if the valence value is 6, which corresponds to a point forming a parallelogram with the three vertices of the previous triangle.
PredSet2 is thus made of the E-parallelogram predictor and the parallelogram predictor, meaning that if vertex E does not exist, PredSet2 is made of the parallelogram predictor only. Of course, regardless of whether E exist or not, PredSet2 may always be made of the parallelogram predictor only when valence = 6.
In the negative of test 1712 or 1713, the set of predictors (as defined through steps 1700-1711, i.e. as in Figure 15) is used, meaning for instance that the set includes the Eparallelogram predictor (PredSetl and PredSet2), the C+ predictor (PredSet2) and/or the parametric predictor with two C+ predicting components (PredSetl).
Note that the adjustment of step 1714 is made only on PredSet2 used for the current vertex, meaning that the by-default configuration of PredSet2 (as defined in 1710) is reused for the subsequent vertices when entering the new steps 1712-1714.
Based on the observation made above with reference to Figure 16c, if the valence value is 8, the set of one or more position predictors is made of the E-parallelogram predictor only.
Variants of these described second embodiments may not involve the switching between PredSetl and PredSet2 based on a reference predictor use ratio. It means steps 1706-1710 and test 1712 are not implemented. Only one set of position predictors is used (PredSetl - 1702) which may be adjusted (1714) based on the valence value (1713).
To refine the valence-based evaluation of mesh regularity, some implementations of the second embodiments may further comprise, if the valence value is 6, determining at least one geometric parameter based on the geometry data of the center vertex and two or more already-encoded or already-decoded outer vertices, and may comprise including the parallelogram predictor in the set of one or more position predictors depending on the determined geometry parameter.
An exemplary geometric parameter is illustrated with reference to Figures 18. In this example, the geometric parameter includes an estimate of a difference between two distances, one distance representing a distance between the center vertex and the outer vertex of the previous triangle belonging to the adjacent edge and the other distance representing a distance between the center vertex and the vertex symmetrical to said outer vertex with respect to the center vertex, given the connectivity between the vertices. In other words, as there are six connections arriving to the center vertex (in case the valence at vertex A is 6), the symmetrical vertex to the current vertex is the one corresponding to the third connection from the connection of the current vertex.
As shown in Figure 18a, if edges connected to center vertex A have similar sizes, there is a high probability that the mesh structure is regular. On the other hand, as shown in Figure 18b, if same edges have substantial different lengths, there are chances that the 3D mesh has an irregular structure.
Figure 19 illustrates, using a flowchart, an implementation using such a geometric parameter to evaluate mesh regularity and then adjust the set of position parameters. Note that the proposed figure does not implement the switching between PredSetl and PredSet2 (i.e. boxes corresponding to 1706-1710 are not provided). However, it could involve the switching mechanism in variants.
Again, boxes 1900-1905, 1911 respectively correspond to boxes 1500-1505, 1511.
Upon considering each vertex of the 3D mesh model, it is first evaluated whether the valence (of the center vertex A or current vertex D) is 6 (test 1913) in which case the geometric parameter must be determined.
First, vertex G is determined if possible, meaning that G has already been encoded or decoded (test 1915). Vertex G is defined as being the symmetrical to the last outer vertex C processed with respect to center vertex A.
If this it is not the case, the same process as in Figure 17 is applied, meaning the parallelogram predictor is used as Predl in the set of predictors (step 1914 identical to step 1714).
On the other hand, if G is available, the geometric parameter is determined. Distance [AC] between center vertex A and outer vertex C of the previous triangle (ABC) belonging to the adjacent edge is computed, as well as the distance [AG] between center vertex A and vertex G., during test 1916.
Several approaches to calculate these distances may be used. In one approach, the distance is the square root of the sum of square differences between each component of both points.
In an exemplary implementation, if the absolute difference between the distance [AC] and [AG] multiplied by 10 is strictly inferior to distance [AC], it is considered that the distances [AC] and [AG] are close enough. Such closeness is thus evaluated during test 1916.
If the distances are close enough, it is considered that the local mesh is regular, in which case PredSetl is adjusted in order to contain the parallelogram predictor (step 1914).
Otherwise (distances are different), the mesh structure is considered as being irregular, in which case PredSetl is not modified and is used to derive the predictors (step 1904).
In a variant to the estimate of distance difference, the geometric parameter may include an estimate of a deviation between the geometric angles at the center vertex within triangles made of already-encoded or already-decoded vertices in the fan portion. In other words, the angles of the triangles connected to vertex A are determined and compared. If these angles are similar, it means that the structure is regular in which case step 1914 can be executed; otherwise, the structure is irregular.
In the above description of the second embodiments, it is proposed to add the parallelogram predictor in the set of predictors in case of evaluated regular mesh structure. As proposed, the parallelogram predictor replaces previous Predl predictor in the set.
In one specific embodiment, the replacement may be limited to one component of Predl: one component of Predl is replaced by a parallelogram component predictor. Preferably, the replaced component of Predl is the parametric predicting component (illustrated above with reference to Figure 12) if any. This is because the parametric predictor is much more efficient for irregular structures than for regular structure.
Turning now to the encoding (or decoding) of property data such as color data associated with the vertices, it is recalled here that they may be encoded (decoded) using the same schemes as those used for geometry data.
The colors are represented in RGB in the PLY file of WRL file (raw file types), using float values. Usually, a quantization process convers the float color components into 8-bit (or more) color components. The RGB color components are then coded with SC3DMC.
In SC3DMC, six encoding schemes for color data are proposed, five of them being prediction-based:
- encoding with no prediction: each color component of the color data associated with a current vertex is encoded without prediction;
- differential encoding: each color per vertex is predicted by the last encoded color per vertex;
- circular encoding: it is based on the differential encoding, wherein the predicted value is modified if it exceeds a maximum size;
- adaptive encoding: each color component of the color data associated with a current vertex is predicted by one of the seven last encoded colors per vertex or is not predicted. The index of the best predictor is transmitted;
- XOR encoding: the encoded color component values are the result of an XOR operation between the color components of the current vertex and of the previous vertex;
- fast parallelogram encoding: the classical parallelogram prediction at color level.
Experimentally when the connectivity is TFAN, the best prediction method seems to be the circular encoding method.
The new encoding/decoding schemes for the geometry data may be transposed to the color data, with some adaptations. In particular according to third embodiments of the invention, the color data of the current vertex may be predicted using a color predictor selected from a set of color predictors, the set of one or more color predictors being determined based on a value representative of regularity of a vertex position distribution in the 3D model.
In the specific case of a 3D mesh model, the one or more color predictors can be determined based on the color data associated with the one or more other vertices of the 3D mesh model connected to the current vertex. Thus, the color data of each current vertex may be predicted based on color data associated with one or more other vertices of the 3D mesh model connected to the current vertex.
As previously proposed for the encoding/decoding of geometry data, a competition between color predictors determined from connected vertices can be implemented. In other words, encoding the color data may include:
selecting the color predictor from a set of one or more color predictors based on a coding-cost-based criterion, and encoding a residue between the color data of the current vertex and the selected color predictor.
From decoding perspective, decoding the color data may include:
selecting, based on the bitstream, a color predictor from the set of one or more color predictors used to predict the color data of the current vertex;
obtaining, from the bitstream, a color residue for the current vertex color data to decode;and computing a decoded color data for the current vertex by adding the selected color predictor to the obtained color residue.
In one embodiment, the set of color predictors preferably contains the colors of neighboring and connected vertices or combinations of these neighboring and connected vertices. Also, projection predictors as parallelogram predictors or parametric predictors are excluded. This advantageously provides a set of color predictors adapted to the colors prediction and consequently improves coding efficiency.
Due to the mere nature of the colors per vertex, each color predictor of the set is a vertex-level predictor (or “monolithic”), meaning that a single color predictor of the set is determined that predicts all color component values associated with the current vertex.
This is because the colors have more correlations than the vertices positions: when a color is correlated to the colors of its neighboring and connected vertices, all three components are correlated in the same fashion.
In addition, one can easily understand that the colors may have higher correlation than the geometry data (vertex positions): indeed, the vertices of a 3D mesh model have naturally different positions one from the other (any duplicate vertices are generally removed/merged to obtain only one vertex position in order to improve coding efficiency), while the colors of various vertices may be the same, which is often the case for neighboring and connected vertices.
The set of color predictors may be limited to two color predictors, as done for geometry data, in order to limit the processing cost for competition assessment.
In one embodiment, the set of one or more color predictors may include the colors of vertices C and E as shown in Figure 20, and not the color of center vertex A. Given the TFAN analysis already introduced above, the set thus includes the color data associated with the outer vertex C of the previous triangle belonging to the adjacent edge and includes the color data associated with an outer vertex E of another triangle not adjacent to the previous triangle and having the center vertex in common.
It is worth selecting the colors of vertices C and E instead of A and E or of C and A because the set made of the colors of C and E offers a better diversity for the general case where the current vertex is closer to C and/or E than to A or at substantially the same distance (Figure 20a).
More rare cases are when the current vertex the color of which is to be predicted is significantly closer to A (Figure 20b). To take into account this specific configuration that may occur, embodiments provide that a similarity indicator, used as a (mesh) regularity value, is determined, based on geometry data describing the positions of the vertices in the 3D mesh model. The similarity indicator is representative of a similarity between, on one hand, a distance between the center vertex A and the current vertex, and on the other hand, a distance between the current vertex and the outer vertex C of the previous triangle and/or a distance between the current vertex and the outer vertex E of the other triangle.
In that case, if the similarity indicator indicates a low distance between the current and center vertices compared to the distances with the outer vertices (e.g. AD < 0.7 * d, where D is the current vertex and d is the distance with one outer vertex or a mean distance with the outer vertices), the set of one or more color predictors is set to include the color data associated with the center vertex A. Preferably, only the color of center vertex A is used as a color predictor in that case.
Otherwise, if the similarity indicator indicates a high similarity between the distances or a high distance between the current and center vertices compared to the distances with the outer vertices (AD > 0.7 * d), the set of one or more color predictors is set to include color predictors determined only from the color data associated with vertices C and E.
Similar to what has been already introduced above for geometry data, the encoding/decoding of the color data, in particular the set of color predictors, may also be adjusted based on the mesh regularity. For instance, the value representative of (mesh) regularity of the 3D model in the vicinity of the current vertex may be determined; and the set of color predictors to encode or decode the current vertex may be set based on the determined (mesh) regularity value.
The (mesh) regularity value may be based on the encoding or decoding of geometry data describing the positions of the vertices in the 3D model and/or on connectivity data related to vertices in the vicinity of the current vertex describing the connectivity between the vertices in the 3D model, and/or on the geometry data related to vertices in the vicinity of the current vertex (e.g. standard deviation of between-vertices distances calculated in such vicinity).
For instance, if the mesh structure is evaluated to be irregular, the set of color predictors may be set to the colors of vertices C and E.
On the other hand, if the mesh structure is evaluated to be regular or semi-regular, the set of color predictors may contain the average (mean value) between the colors of vertices C and E and may contain the color of vertex C. In one embodiment, the set for regular or semiregular meshes may be restricted to the sole mean value. This is because the mean/average predictor between C and E is also efficient when it is used without competition, especially for regular mesh structures. For these regular structures, C and E are at an equivalent distance to the current vertex. The average predictor is a good summary of these two color predictors.
Figure 21 illustrates, using a flowchart, an implementation of this approach where depending on the mesh regularity value, either a first set of color predictors or a second and different set of color predictors is defined as the set to encode or decode the color data of the current vertex.
The criterion used in the example to set to the mesh regularity value, thereby defining the set of color predictors to be used, is based on the encoding or decoding of geometry data, in particular on whether the encoding or decoding of the geometry data of the current vertex determines separately one position predictor for each position component of the geometry data (component-level prediction), or not (vertex-level prediction).
The process goes through each vertex (2101,2109) to predict each color (2108).
If the set of position predictors used to predict the geometry position of the vertex v current are component-level predictors (test 2102), it may mean the 3D mesh structure in the vicinity of current vertex is quite regular (see for instance the situation in Figure 15 or the embodiments involving three set of predictors). In that case, the set of color predictors is set equal (step 2106) to PredSet2 including the mean value between C and E and including C (2105).
Otherwise, the 3D mesh structure is identified as being irregular, in which case PredSetl made of the colors of C and E (2103) is selected as the set of color predictors (step 2104).
The color predictors are then derived (step 2107) by determining which vertices are vertices C and E for the current vertex, and by determining their corresponding colors. Next, they are used to predict the color of the current vertex (step 2108) before considering the next vertex (step 2109).
Please note that Figure 19 is an example adapted to both local and global geometry predictors set selection. Again, this may be implemented based on a test set of (e.g. 1000) vertices, in which case the selected predictors set is then used for the colors of all the other vertices in the mesh.
In one embodiment, the RGB colors are preferably converted into YUV colors before they are predicted using a set of YUV color predictors. The inverse conversion is applied by the decoder after the inverse decoding.
Preferably, the RGB to YUV conversion and the inverse conversion are lossless conversion. In that case, the bit-depth for YUV color can be increased by 2 bits compared to the bit-depth of the RGB colors.
In other words, if the RGB colors are represented using 8 bits per component, the YUV colors are represented using 10 bits per component. The following YCoCg conversion can be applied:
offset = 2nQbits+1
U = R-B
T=B + (U»1)
V=G-T
Y=T + (V» 1)
U = U + offset
V = V + offset where nQbits is the number of bits for a RGB color components; the symbol » is the shifted operator; R,G,B,Y,U,V are the color values of the components conventionally designated with the same letter; T is a temporary variable.
This embodiment advantageously reduces the entropy of the RGB input signal. This is because, thanks to this conversion, the signal is mostly contained in Y component.
Consequently, coding efficiency is improved.
Figure 22 schematically illustrates a device 2200 to implement at least one embodiment of the present invention. The device 2200 may preferably be a device such as a micro-computer, a workstation or a light portable device. The device 2200 comprises a communication bus 2213 to which there are preferably connected:
• a central processing unit 2211, such as a microprocessor, denoted CPU;
• a read only memory 2207, denoted ROM, for storing computer programs for implementing the invention;
• a random access memory 2212, denoted RAM, for storing the executable code of methods according to embodiments of the invention as well as the registers adapted to record variables and parameters necessary for implementing methods according to embodiments of the invention; and • at least one communication interface 2202 connected to a communication network 2201 over which data may be transmitted.
Optionally, the device 2200 may also include the following components:
• a data storage means 2204 such as a hard disk, for storing computer programs for implementing methods according to one or more embodiments of the invention;
• a disk drive 2205 for a disk 2206, the disk drive being adapted to read data from the disk 2206 or to write data onto said disk;
• a screen 2209 for displaying data and/or serving as a graphical interface with the user, by means of a keyboard 2210 or any other pointing means.
The device 2200 may be optionally connected to various peripherals, such as for example a digital camera 2208, each being connected to an input/output card (not shown) so as to supply data to the device 2200.
Preferably the communication bus provides communication and interoperability between the various elements included in the device 2200 or connected to it. The representation of the bus is not limiting and in particular the central processing unit is operable to communicate instructions to any element of the device 2200 directly or by means of another element of the device 2200.
The disk 2206 may optionally be replaced by any information medium such as for example a compact disk (CD-ROM), rewritable or not, a ZIP disk, a USB key or a memory card and, in general terms, by an information storage means that can be read by a microcomputer or by a microprocessor, integrated or not into the apparatus, possibly removable and adapted to store one or more programs whose execution enables a method according to the invention to be implemented.
The executable code may optionally be stored either in read only memory 2207, on the hard disk 2204 or on a removable digital medium such as for example a disk 2206 as described previously. According to an optional variant, the executable code of the programs can be received by means of the communication network 2201, via the interface 2202, in order to be stored in one of the storage means of the device 2200, such as the hard disk 2204, before being executed.
The central processing unit 2211 is preferably adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to the invention, which instructions are stored in one of the aforementioned storage means. On powering up, the program or programs that are stored in a non-volatile memory, for example on the hard disk 2204 or in the read only memory 2207, are transferred into the random access memory 2212, which then contains the executable code of the program or programs, as well as registers for storing the variables and parameters necessary for implementing the invention.
In a preferred embodiment, the device is a programmable apparatus which uses software to implement the invention. However, alternatively, the present invention may be implemented in hardware (for example, in the form of an Application Specific Integrated Circuit or ASIC).
Although the present invention has been described hereinabove with reference to specific embodiments, the present invention is not limited to the specific embodiments, and modifications will be apparent to a skilled person in the art which lie within the scope of the present invention.
Many further modifications and variations will suggest themselves to those versed in the art upon making reference to the foregoing illustrative embodiments, which are given by way of example only and which are not intended to limit the scope of the invention, that being determined solely by the appended claims. In particular the different features from different embodiments may be interchanged, where appropriate.
In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used.

Claims (40)

1. A method for encoding, into a bitstream, a three-dimensional model made of vertices, the method comprising a step of encoding geometry data describing the positions of vertices in the 3D model, wherein encoding the geometry data of a current vertex includes:
predicting the geometry data of the current vertex with a position predictor selected, based on a coding-cost-based criterion, from a set of one or more position predictors, and encoding a residue between the geometry data of the current vertex and the selected position predictor, the method further comprising:
determining a value representative of regularity of a vertex position distribution in the 3D model, and setting the set of one or more position predictors used to select the position predictor, based on the determined regularity value.
2. A method for decoding, from a bitstream, a three-dimensional model made of vertices, the method comprising a step of decoding geometry data describing the positions of vertices in the 3D model, wherein decoding the geometry data of a current vertex includes:
selecting, based on the bitstream, a position predictor from a set of one or more position predictors used to predict the geometry data of the current vertex;
obtaining, from the bitstream, a position residue for the current vertex position to decode;and computing a decoded position of the current vertex by adding the selected position predictor to the obtained position residue, the method further comprising:
determining a value representative of regularity of a vertex position distribution in the 3D model, and setting the set of one or more position predictors used to select the position predictor, based on the determined regularity value.
3. The method of Claim 1 or 2, wherein the regularity value is determined based on the position predictors used to predict the geometry data of a test set of vertices of the 3D model that are encoded or decoded before the geometry data of the current vertex is predicted.
4. The method of Claim 3, wherein the regularity value includes a ratio of use of a reference predictor, to predict the geometry data of the vertices of the test set.
5. The method of Claim 4, further comprising, when the regularity value indicates a reference predictor use ratio less than a first threshold, switching from a first set of one or more position predictors used to predict the geometry data of the vertices of the test set, to a second set of one or more position predictors to predict the geometry data of the current vertex.
6. The method of Claim 5, wherein each position predictor ofthe second set is a vertex-level predictor, meaning that a single position predictor ofthe second set is determined to predict all position component values ofthe current vertex.
7. The method of Claim 5, further comprising obtaining a fan portion describing connectivity between vertices around a center vertex to which the current vertex is connected, the fan portion being made of successive outer vertices that define a series of adjacent triangles having the center vertex in common, the current vertex being a current one of the successive outer vertices and defining, with already-encoded or already-decoded vertices, a next triangle adjacent to a previous triangle, wherein the second set includes an E-parallelogram predictor corresponding to a point which forms a parallelogram with the center vertex in common to the adjacent triangles, with the outer vertex of the previous triangle belonging to the adjacent edge and with an outer vertex of another triangle not adjacent to the previous triangle and having the center vertex in common.
8. The method of Claim 7, wherein the second set further includes a C+ predictor corresponding to a point, each position component value of which being one position component value selected, based on a selection criterion, between the corresponding position component values of three vertices already encoded or decoded.
9. The method of Claim 5, wherein the reference predictor is a parametric predictor, the value of a first position component of which being the position component value, having the same component axis, of a point representing an intersection point between two first planes respectively perpendicular to the two other axes at estimates of corresponding second and third position components ofthe current vertex, and a third plane including three vertices preceding the current vertex according to an encoding order.
10. The method of Claim 3, wherein the test set of already-encoded or alreadydecoded vertices is made of the N first vertices composing a 3D model, according to an encoding order.
11. The method of Claim 1 or 2, wherein the regularity value is determined based on the geometry data related to vertices in the vicinity ofthe current vertex, or the 3D model is a 3D mesh model made of connected vertices and the regularity value is determined based on connectivity data related to vertices in the vicinity ofthe current vertex describing the connectivity between the vertices in the 3D mesh model.
12. The method of Claim 11, wherein the regularity value is determined based on distances between the current vertex and each of a plurality of vertices in its vicinity, for example the regularity value represents a variation or deviation ofthe distances.
13. The method of Claim 11, wherein the regularity value includes a valence value for the current vertex, corresponding to a number of vertices connected to the current vertex.
14. The method of Claim 11, further comprising obtaining a fan portion describing connectivity between vertices around a center vertex to which the current vertex is connected, the fan portion being made of successive outer vertices that define a series of adjacent triangles having the center vertex in common, the current vertex being a current one of the successive outer vertices and defining, with already-encoded vertices, a next triangle adjacent to a previous triangle, wherein the regularity value includes a valence value for the center vertex, corresponding to a number of vertices connected to the center vertex.
15. The method of Claim 13 or 14, wherein the set of one or more position predictors to predict the current vertex includes a parallelogram predictor only if the valence value is 6, which corresponds to a point forming a parallelogram with the three vertices of the previous triangle.
16. The method of Claim 15, further comprising, if the valence value is 6, determining at least one geometric parameter based on the geometry data of the center vertex and two or more already-encoded or already-decoded outer vertices, and including the parallelogram predictor in the set of one or more position predictors depending on the determined geometry parameter.
17. The method of Claim 16, wherein the geometric parameter includes an estimate of a difference between two distances, one distance representing a distance between the center vertex and the outer vertex of the previous triangle belonging to the adjacent edge and the other distance representing a distance between the center vertex and the vertex symmetrical to said outer vertex with respect to the center vertex, given the connectivity between the vertices.
18. The method of Claim 16, wherein the geometric parameter includes an estimate of a deviation between the geometric angles at the center vertex within triangles made of already-encoded or already-decoded vertices in the fan portion.
19. The method of Claim 15, wherein the set of one or more position predictors including the parallelogram predictor is made of the parallelogram predictor only.
20. The method of Claim 13 or 14, wherein if the valence value is different from 6, the set of one or more position predictors includes a C+ predictor corresponding to a point, each position component value of which being one position component value selected, based on a selection criterion, between the corresponding position component values of three vertices already encoded.
21. The method of Claim 13 or 14, wherein the set of one or more position predictors includes an E-parallelogram predictor corresponding to a point which forms a parallelogram with the center vertex in common to the adjacent triangles, with the outer vertex of the previous triangle belonging to the adjacent edge and with an outer vertex of another triangle not adjacent to the previous triangle and having the center vertex in common.
22. The method of Claim 21, wherein if the valence value is 8, the set of one or more position predictors is made of the E-parallelogram predictor only.
23. The method of Claim 11, wherein the set of one or more position predictors is further set based on a second regularity value determined based on the position predictors used to predict the geometry data of a test set of vertices of the 3D mesh model that are encoded or decoded before the geometry data of the current vertex is predicted.
24. A method for encoding, into a bitstream, or for decoding, from a bitstream, a three-dimensional model made of vertices, the method comprising a step of encoding or decoding color data describing a color attribute associated with each vertex in the 3D model, wherein encoding or decoding the color data of a current vertex includes:
predicting the color data of the current vertex using a color predictor selected from a set of color predictors, the set of one or more color predictors being determined based on a value representative of regularity of a vertex position distribution in the 3D model.
25. (cas du mesh) The method of Claim 24, wherein the 3D model is a mesh model made of connected vertices, and the one or more color predictors are determined based on the color data associated with the one or more other vertices of the 3D mesh model connected to the current vertex.
26. The method of Claim 24, wherein encoding the color data includes: selecting the color predictor from the set of one or more color predictors based on a coding-cost-based criterion, and encoding a residue between the color data of the current vertex and the selected color predictor.
27. The method of Claim 24, wherein decoding the color data includes: selecting, based on the bitstream, a color predictor from the set of one or more color predictors used to predict the color data of the current vertex;
obtaining, from the bitstream, a color residue for the current vertex color data to decode;and computing a decoded color data for the current vertex by adding the selected color predictor to the obtained color residue.
28. The method of Claim 26 or 27, wherein each color predictor of the set is a vertex-level predictor, meaning that a single color predictor of the set is determined that predicts all color component values associated with the current vertex.
29. The method of Claim 26 or 27, further comprising obtaining a fan portion describing connectivity between vertices around a center vertex to which the current vertex is connected, the fan portion being made of successive outer vertices that define a series of adjacent triangles having the center vertex in common, the current vertex associated with the color data to encode or decode being a current one of the successive outer vertices and defining, with vertices having their color data already encoded or decoded, a next triangle adjacent to a previous triangle, wherein the set of one or more color predictors includes the color data associated with the outer vertex of the previous triangle belonging to the adjacent edge and includes the color data associated with an outer vertex of another triangle not adjacent to the previous triangle and having the center vertex in common.
30. The method of Claim 26 or 27, further comprising:
obtaining a fan portion describing connectivity between vertices around a center vertex to which the current vertex is connected, the fan portion being made of successive outer vertices that define a series of adjacent triangles having the center vertex in common, the current vertex associated with the color data to encode or decode being a current one of the successive outer vertices, determining, based on geometry data describing the positions of the vertices in the 3D model, an indicator representative of a similarity between, on one hand, a distance between the center vertex and the current vertex, and on the other hand, a distance between the current vertex and the outer vertex of the previous triangle and/or a distance between the current vertex and the outer vertex of the other triangle, wherein the set of one or more color predictors includes the color data associated with the center vertex if the similarity indicator indicates a low distance between the current and center vertices compared to the distances with the outer vertices, and the set of one or more color predictors includes color predictors determined only from the color data associated with the outer vertex of the previous triangle belonging to the adjacent edge and the color data associated with an outer vertex of another triangle not adjacent to the previous triangle and having the center vertex in common, if the similarity indicator indicates a high similarity between the distances.
31. The method of Claim 30, wherein the set of one or more color predictors including the color data associated with the center vertex is made of the color data associated with the center vertex only.
32. The method of Claim 26 or 27, further comprising :
determining the value representative of regularity of the 3D model in the vicinity of the current vertex, and setting the set of color predictors to encode or decode the current vertex, based on the determined regularity value.
33. The method of Claim 32, wherein when the encoding or decoding of the geometry data of the current vertex determines separately one position predictor for each position component of the geometry data, the regularity value is set to a value driving the setting of a first set of color predictors as the set to encode or decode the color data of the current vertex; otherwise the regularity value is set to another value driving the setting of a second and different set of color predictors as the set to encode or decode the color data of the current vertex.
34. The method of Claim 33, further comprising obtaining a fan portion describing connectivity between vertices around a center vertex to which the current vertex is connected, the fan portion being made of successive outer vertices that define a series of adjacent triangles having the center vertex in common, the current vertex associated with the color data to encode or decode being a current one of the successive outer vertices, wherein the first set of color predictors includes a mean value between the color data associated with the outer vertex of the previous triangle belonging to the adjacent edge and the color data associated with an outer vertex of another triangle not adjacent to the previous triangle and having the center vertex in common.
35. The method of Claim 34, wherein the first set is made of the mean value only or the first set is made of the mean value and of the color data associated with the outer vertex of the previous triangle belonging to the adjacent edge.
36. The method of Claim 34, wherein the second set of color predictors is made of the color data associated with the outer vertex of the previous triangle belonging to the adjacent edge and of the color data associated with an outer vertex of another triangle not adjacent to the previous triangle and having the center vertex in common.
37. A non-transitory computer-readable medium storing a program which, when executed by a microprocessor or computer system in a device, causes the device to perform the method of Claim 1,2 or 24.
38. A device for encoding, into a bitstream, a three-dimensional model made of vertices, the device comprising at least one microprocessor configured for carrying out a step of encoding geometry data describing the positions of vertices in the 3D model, wherein encoding the geometry data of a current vertex includes:
predicting the geometry data of the current vertex with a position predictor selected, based on a coding-cost-based criterion, from a set of one or more position predictors, and encoding a residue between the geometry data of the current vertex and the selected position predictor, the method further comprising:
determining a value representative of regularity of the 3D model, and setting the set of one or more position predictors used to select the position predictor, based on the determined regularity value.
39. A device for decoding, from a bitstream, a three-dimensional model made of vertices, the device comprising at least one microprocessor configured for carrying out a step of decoding geometry data describing the positions of vertices in the 3D model, wherein decoding the geometry data of a current vertex includes:
selecting, based on the bitstream, a position predictor from a set of one or more position predictors used to predict the geometry data of the current vertex;
obtaining, from the bitstream, a position residue for the current vertex position to decode;and
5 computing a decoded position of the current vertex by adding the selected position predictor to the obtained position residue, the method further comprising:
determining a value representative of regularity of a vertex position distribution in the 3D model, and
10 setting the set of one or more position predictors used to select the position predictor, based on the determined regularity value.
40. A device for encoding, into a bitstream, or for decoding, from a bitstream, a three-dimensional model made of vertices, the device comprising at least one microprocessor configured for carrying out a step of encoding or decoding color data describing a color attribute
15 associated with each vertex in the 3D model, wherein encoding or decoding the color data of a current vertex includes:
predicting the color data of the current vertex using a color predictor selected from a set of color predictors, the set of one or more color predictors being determined based on value representative of regularity of a vertex position distribution in the 3D model.
Intellectual
Property
Office
Application No: GB1706177.1
GB1706177.1A 2017-04-19 2017-04-19 Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model Active GB2561824B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
GB1706177.1A GB2561824B (en) 2017-04-19 2017-04-19 Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB1706177.1A GB2561824B (en) 2017-04-19 2017-04-19 Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model

Publications (3)

Publication Number Publication Date
GB201706177D0 GB201706177D0 (en) 2017-05-31
GB2561824A true GB2561824A (en) 2018-10-31
GB2561824B GB2561824B (en) 2020-04-01

Family

ID=58744382

Family Applications (1)

Application Number Title Priority Date Filing Date
GB1706177.1A Active GB2561824B (en) 2017-04-19 2017-04-19 Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model

Country Status (1)

Country Link
GB (1) GB2561824B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020123252A1 (en) * 2018-12-11 2020-06-18 Google Llc Compression of data representing triangular mesh attributes using machine learning
WO2021253372A1 (en) * 2020-06-19 2021-12-23 深圳华大智造科技有限公司 High-compatibility pcr-free library building and sequencing method
WO2022161495A1 (en) * 2021-01-29 2022-08-04 Genesail Biotech (shanghai) Co., Ltd. Recombinant sars-cov-2 vaccine
WO2022213569A1 (en) * 2021-04-09 2022-10-13 Beijing Xiaomi Mobile Software Co., Ltd. Method and apparatus of encoding/decoding point cloud geometry data captured by a spinning sensors head

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11166048B2 (en) * 2018-10-02 2021-11-02 Tencent America LLC Method and apparatus for video coding

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000077740A1 (en) * 1999-06-14 2000-12-21 Sun Microsystems, Inc. Geometric compression of three-dimensional graphics
US20080205779A1 (en) * 2007-02-23 2008-08-28 International Business Machines Corporation Selective predictor and selective predictive encoding for two-dimensional geometry compression

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000077740A1 (en) * 1999-06-14 2000-12-21 Sun Microsystems, Inc. Geometric compression of three-dimensional graphics
US20080205779A1 (en) * 2007-02-23 2008-08-28 International Business Machines Corporation Selective predictor and selective predictive encoding for two-dimensional geometry compression

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020123252A1 (en) * 2018-12-11 2020-06-18 Google Llc Compression of data representing triangular mesh attributes using machine learning
US11631218B2 (en) 2018-12-11 2023-04-18 Google Llc Efficient compression of data representing triangular mesh attributes
WO2021253372A1 (en) * 2020-06-19 2021-12-23 深圳华大智造科技有限公司 High-compatibility pcr-free library building and sequencing method
WO2022161495A1 (en) * 2021-01-29 2022-08-04 Genesail Biotech (shanghai) Co., Ltd. Recombinant sars-cov-2 vaccine
WO2022213569A1 (en) * 2021-04-09 2022-10-13 Beijing Xiaomi Mobile Software Co., Ltd. Method and apparatus of encoding/decoding point cloud geometry data captured by a spinning sensors head

Also Published As

Publication number Publication date
GB2561824B (en) 2020-04-01
GB201706177D0 (en) 2017-05-31

Similar Documents

Publication Publication Date Title
CN110996098B (en) Method and device for processing point cloud data
US10535162B2 (en) Attribute mapping to encode and decode 3D models
US11252441B2 (en) Hierarchical point cloud compression
CN110708560B (en) Point cloud data processing method and device
GB2561824A (en) Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model
JP5512704B2 (en) 3D mesh model encoding method and apparatus, and encoded 3D mesh model decoding method and apparatus
JP5932051B2 (en) Predictive position decoding
KR100819597B1 (en) Multi-mode alpha image processing
CN114600163A (en) TRISOUP node size per slice
GB2560319A (en) Improved encoding and decoding of texture mapping data in textured 3D mesh models
WO2013067674A1 (en) Predictive position encoding
WO2020123469A1 (en) Hierarchical tree attribute coding by median points in point cloud coding
Vasa et al. Exploiting connectivity to improve the tangential part of geometry prediction
US11983905B2 (en) Methods for level partition of point cloud, and decoder
US20160316204A1 (en) Method and system for image compression using image block characteristics
KR20140037897A (en) System and method for encoding and decoding a bitstream for a 3d model having repetitive structure
GB2551387A (en) Improved encoding and decoding of geometry data in 3D mesh models
KR101986282B1 (en) Method and apparatus for repetitive structure discovery based 3d model compression
GB2551389A (en) New predictors to encode or decode geometry data in 3D objects
WO2022120594A1 (en) Point cloud encoding method, point cloud decoding method, encoder, decoder, and computer storage medium
JP2024507046A (en) Point cloud encoding and decoding methods, encoders, decoders, and computer storage media
EP4244813B1 (en) Devices and methods for scalable coding for point cloud compression
Bidgoli et al. A geometry-aware compression of 3D mesh texture with random access
WO2022131946A2 (en) Devices and methods for spatial quantization for point cloud compression
WO2024123569A1 (en) Geometry point cloud coding method, encoder and decoder