EP4677855A1 - Adaptive quantization of displacement vectors in v-dmc and use thereof - Google Patents

Abstract

In an encoding process of V-DMC (2200), an apparatus performs the following: setting (2205) a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video; computing (2210) displacement vectors for each LOD defined by the subdivision iteration count; generating (2215) such displacement vectors for multiple input mesh frames; quantizing (2220) the generated displacement vectors; signaling (2265) the quantization information used to generate the quantized displacement vectors; packing the quantized displacement vectors in displacement video frame(s); and coding displacement video frame(s) as a displacement bitstream that is part of bitstream for the V-DMC. A decoding apparatus receives (2305) the bitstreams, unpacks (2325) the displacement quantization information, decodes (2335) displacement vector components based at least on unpacked displacement quantization information, wherein the decoded displacement vector components comprise components for at least a first axis, and uses (2360) the decoded displacement vector components to form reconstructed data.

Description

DESCRIPTION

Adaptive Quantization of Displacement Vectors in V-DMC and Use Thereof

TECHNICAL FIELD

[0001] Exemplary embodiments herein relate generally to video encoding and decoding and, more specifically, relates to adaptive quantization of displacement values in V- DMC.

BACKGROUND

[0002] Video encoding is a process of converting video files, which can be a sequence of individual images, to digital files so that they are not saved as individual images but as fluid videos. Video is encoded into a bitstream, that bitstream is then received and decoded by a decoder. A codec is able to perform both encoding and decoding.

[0003] There are multiple different coding schemes available. For instance, examples herein relate to a new standardization activity referred to as Video-based Dynamic Mesh Coding (V-DMC) ISO/IEC 23090-29, which is a new application of the Visual Volumetric Video Coding (V3C) standard family ISO/IEC 23090-5. In this new scheme, there is an area where quantization could be useful, but the scheme currently does not support quantization in this area.

BRIEF SUMMARY

[0004] This section is intended to include examples and is not intended to be limiting.

[0005] In an exemplary embodiment, a method is disclosed that includes, as part of an encoding process of video using video-based dynamic mesh coding, performing the following: setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video; computing displacement vectors for each level of detail defined by the subdivision iteration count; generating such displacement vectors for a plurality of input mesh frames; quantizing the generated displacement vectors; signaling at least the quantization information used to generate the quantized displacement vectors; packing the quantized displacement vectors in one or more displacement video frames; and coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the video-based dynamic mesh coding.

[0006] An additional exemplary embodiment includes a computer program, comprising code for performing the method of the previous paragraph, when the computer program is run on a processor. The computer program according to this paragraph, wherein the computer program is a computer program product comprising a computer-readable medium bearing computer program code embodied therein for use with a computer. Another example is the computer program according to this paragraph, wherein the program is directly loadable into an internal memory of the computer.

[0007] An exemplary apparatus includes one or more processors and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform: as part of an encoding process of video using video-based dynamic mesh coding, performing the following: setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video; computing displacement vectors for each level of detail defined by the subdivision iteration count; generating such displacement vectors for a plurality of input mesh frames; quantizing the generated displacement vectors; signaling at least the quantization information used to generate the quantized displacement vectors; packing the quantized displacement vectors in one or more displacement video frames; and coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the videobased dynamic mesh coding.

[0008] An exemplary computer program product includes a computer-readable storage medium bearing instructions that, when executed by an apparatus, cause the apparatus to perform at least the following: as part of an encoding process of video using video-based dynamic mesh coding, performing the following: setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video; computing displacement vectors for each level of detail defined by the subdivision iteration count; generating such displacement vectors for a plurality of input mesh frames; quantizing the generated displacement vectors; signaling at least the quantization information used to generate the quantized displacement vectors; packing the quantized displacement vectors in one or more displacement video frames; and coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the videobased dynamic mesh coding.

[0009] In another exemplary embodiment, an apparatus comprises means for performing: as part of an encoding process of video using video-based dynamic mesh coding, performing the following: setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video; computing displacement vectors for each level of detail defined by the subdivision iteration count; generating such displacement vectors for a plurality of input mesh frames; quantizing the generated displacement vectors; signaling at least the quantization information used to generate the quantized displacement vectors; packing the quantized displacement vectors in one or more displacement video frames; and coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the video-based dynamic mesh coding.

[0010] In an exemplary embodiment, a method is disclosed that includes as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following: receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream; unpacking the displacement quantization information received in the bitstream; decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame.

[0011] An additional exemplary embodiment includes a computer program, comprising instructions for performing the method of the previous paragraph, when the computer program is run on an apparatus. The computer program according to this paragraph, wherein the computer program is a computer program product comprising a computer-readable medium bearing the instructions embodied therein for use with the apparatus. Another example is the computer program according to this paragraph, wherein the program is directly loadable into an internal memory of the apparatus.

[0012] An exemplary apparatus includes one or more processors and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform: as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following: receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream; unpacking the displacement quantization information received in the bitstream; decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame.

[0013] An exemplary computer program product includes a computer-readable storage medium bearing instructions that, when executed by an apparatus, cause the apparatus to perform at least the following: as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following: receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream; unpacking the displacement quantization information received in the bitstream; decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame. [0014] In another exemplary embodiment, an apparatus comprises means for performing: as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following: receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream; unpacking the displacement quantization information received in the bitstream; decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame.

BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In the attached drawings:

[0016] FIG. 1, split over FIGS. 1A and IB, is an example of volumetric media conversion at an encoder in FIG. 1 A, and reconstruction at a decoder side in FIG. IB, where the 3D media is converted to a series of 2D representations: occupancy, geometry, and attributes, and additional atlas information is also included in the bitstream to enable inverse reconstruction;

[0017] FIG. 2 illustrates an example of block to patch mapping in accordance with ISO/IEC 23090-5;

[0018] FIG. 3A provides an example of an atlas coordinate system, FIG. 3B provides an example of a local 3D patch coordinate system, and FIG. 3C provides an example of a final target 3D coordinate system in accordance with ISO/IEC 23090-5;

[0019] FIG. 4 is an illustration of elements of a mesh;

[0020] FIG. 5 illustrates how one subdivision step of a triangle into four triangles is performed by connecting mid points of the initial triangle edges;

[0021] FIG. 6 illustrates multi-resolution analysis of a mesh;

[0022] FIG. 7 is a block diagram of an encoding process;

[0023] FIG. 8 is a block diagram illustrating pre-processing steps at the encoder; [0024] FIG. 9 is a block diagram illustrating an intra frame encoder scheme;

[0025] FIG. 10 is a block diagram illustrating an inter frame encoder scheme;

[0026] FIG. 11 is a block diagram illustrating a decoder scheme composed of a decoder module that demuxes (demultiplexes) and decodes all sub-streams and a post-processing module that reconstructs the dynamic mesh sequence;

[0027] FIGS. 12 and 13 are block diagrams illustrating the decoding process in INTRA and INTER modes, respectively;

[0028] FIG. 14 is a block diagram used to illustrate a base mesh encoder in the V- DMC encoder;

[0029] FIG. 15 is a block diagram illustrating the base mesh encoder of the V-DMC encoder of FIG. 14;

[0030] FIG. 16 is a block diagram of a base mesh decoder;

[0031] FIG. 17 illustrates an overview of base mesh data substream structure;

[0032] FIG. 18 illustrates segmentation of a mesh into sub-meshes;

[0033] FIG. 19 presents an example with two submeshes;

[0034] FIG. 20A is an example of LODs 0 through 4 for a single video frame of a displacement map and FIG. 20B is an example of packing displacement into a single frame (X- component of the displacement vectors);

[0035] FIG. 21 is a graph of distributions of displacements;

[0036] FIG. 22 is a flow diagram of encoding adaptive quantization of displacement values in V-DMC, and signaling of those quantized values;

[0037] FIG. 23 is a flow diagram of decoding adaptive quantization of displacement values in V-DMC;

[0038] FIG. 24 is an example of a block diagram of an apparatus suitable for implementing any of the encoding or decoding operations or block diagrams described herein;

[0039] FIG. 25 is a block diagram illustrating a system in accordance with an example. DETAILED DESCRIPTION OF THE DRAWINGS

[0040] Abbreviations that may be found in the specification and/or the drawing figures are defined below, at the end of the detailed description section.

[0041] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments. All of the embodiments described in this Detailed Description are exemplary embodiments provided to enable persons skilled in the art to make or use the invention and not to limit the scope of the invention which is defined by the claims.

[0042] When more than one drawing reference numeral, word, or acronym is used within this description with and in general as used within this description, the “/” may be interpreted as “or”, “and”, or “both”.

[0043] As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises”, “comprising”, “has”, “having”, “includes” and/or “including”, when used herein, specify the presence of stated features, elements, and/or components etc., but do not preclude the presence or addition of one or more other features, elements, components and/ or combinations thereof.

[0044] Any flow diagram (such as FIGS. 22 and 23) or signaling diagram herein is considered to be a logic flow diagram, and illustrates the operation of an exemplary method, results of execution of computer program instructions embodied on a computer readable memory, functions performed by logic implemented in hardware, and/or interconnected means for performing functions in accordance with an exemplary embodiment. Block diagrams (such as FIGS. 1 A, IB, 7-16, 24, and 25) also illustrate the operation of an exemplary method, results of execution of computer program instructions embodied on a computer readable memory, functions performed by logic implemented in hardware, and/or interconnected means for performing functions in accordance with an exemplary embodiment. [0045] Examples herein relate to a new standardization activity referred to as Videobased Dynamic Mesh Coding (V-DMC) ISO/IEC 23090-29, which is a new application of the Visual Volumetric Video Coding (V3C) standard family ISO/IEC 23090-5.

[0046] Before proceeding with description of embodiments, an overview of the technical area is presented. The following part of the description is separated into sections using non-limiting section headings.

[0047] Volumetric video data

[0048] Volumetric video data represents a three-dimensional scene or object and can be used as input for AR, VR and MR applications. Such data describes geometry (shape, size, position in 3D-space) and respective attributes (e.g., color, opacity, reflectance, and the like), plus any possible temporal transformations of the geometry and attributes at given time instances (like frames in 2D video). Volumetric video is either generated from 3D models, i.e., CGI, or captured from real-world scenes using a variety of capture solutions, e.g., multi-camera, laser scan, combination of video and dedicated depth sensors, and more. Also, a combination of CGI and real-world data is possible. Typical representation formats for such volumetric data are triangle meshes, point clouds, or voxels. Temporal information about the scene can be included in the form of individual capture instances, i.e., “frames” in 2D video, or other means, e.g., position of an object as a function of time.

[0049] Because volumetric video describes a 3D scene (or object), such data can be viewed from any viewpoint. Therefore, volumetric video is an important format for any AR, VR, or MR applications, especially for providing 6DOF viewing capabilities.

[0050] Increasing computational resources and advances in 3D data acquisition devices have enabled reconstruction of highly detailed volumetric video representations of natural scenes. Infrared, lasers, time-of-flight and structured light are all examples of devices that can be used to construct 3D video data. Representation of the 3D data depends on how the 3D data is used. Dense Voxel arrays have been used to represent volumetric medical data. In 3D graphics, polygonal meshes are extensively used. Point clouds on the other hand are well suited for applications such as capturing real world 3D scenes where the topology is not necessarily a 2D manifold. Another way to represent 3D data is coding this 3D data as set of texture and depth map as is the case in the multi-view plus depth. Closely related to the techniques used in multiview plus depth is the use of elevation maps, and multi-level surface maps.

[0051] MPEG visual volumetric video-based coding (V3C)

[0052] In the following, selected excerpts from ISO/IEC 23090-5 Visual Volumetric Video-based Coding and Video-based Point Cloud Compression 2nd Edition standard are related.

[0053] Visual volumetric video, a sequence of visual volumetric frames, if uncompressed, may be represented by a large amount of data, which can be costly in terms of storage and transmission. This has led to the need for a high coding efficiency standard for the compression of visual volumetric data.

[0054] V3C specification enables the encoding and decoding processes of a variety of volumetric media by using video and image coding technologies. This is achieved through first a conversion of such media from their corresponding 3D representation to multiple 2D representations, also referred to as V3C components, before coding such information. Such representations may include occupancy, geometry, and attribute components. The occupancy component can inform a V3C decoding and/or rendering system of which samples in the 2D components are associated with data in the final 3D representation. The geometry component contains information about the precise location of 3D data in space, while attribute components can provide additional properties, e.g., texture or material information, of such 3D data. An example is shown in FIG. 1, split over FIGS. 1A and IB, which illustrates volumetric media conversion at (FIG. 1 A) encoder and reconstruction at (FIG. IB) decoder side. The 3D media (from the volumetric capture at a viewpoint 10 of a scene 15, which includes a human being 20) is converted (see FIG. 1A) via projection to a series of 2D representations: occupancy, geometry, and attributes. Additional atlas information is also included in the bitstream to enable inverse reconstruction. See ISO/IEC 23090-5.

[0055] Additional information that allows associating all these subcomponents and that enables the inverse reconstruction, from a 2D representation back to a 3D representation is also included in a special component, referred to in this document as the atlas (see the atlas information). An atlas comprises multiple elements, referred to as patches. Each patch identifies a region in all available 2D components and contains information necessary to perform the appropriate inverse projection of this region back to the 3D space. The shape of such regions is determined through a 2D bounding box associated with each patch as well as their coding order. The shape of these regions is also further refined after the consideration of the occupancy information. The atlas information, occupancy, geometry, and attribute 2D representations are coded into a V3C bitstream.

[0056] For decoding (see FIG. IB), the received V3C bitstream is separated into its components: atlas information; occupancy, geometry, and attribute 2D representations. A 3D reconstruction is performed to reconstruct the scene 15-1 created looking at the viewpoint 10-1 with a “reconstructed” human being 20- 1. The “- 1 ” are used to indicate that these are reconstructions of the original.

[0057] For more detail about atlases, these are partitioned into patch packing blocks of equal size. The 2D bounding boxes of patches and their coding order determine the mapping between the blocks of the atlas image and the patch indices. FIG. 2 shows an example of block- to-patch mapping with four projected patches (patchldx of 0, 1, 2, and 3) onto an atlas when asps _patch_precedence_order _flag is equal to 0 (zero). The asps frame height and asps_frame_width are shown, as are the AtlasPatch2dPosX[2] and the AtlasPatch2dSixeX[2] for the abscissa (X axis) and AtlasPatch2dPosY[2], AtlasPatch2dSixeY[2], and PatchPackingBlockSize on the ordinate (Y axis). Projected points are represented with dark grey. The area that does not contain any projected points is represented with light grey. Patch packing blocks are represented with dashed lines. The number inside each patch packing block represents the patch index of the patch to which it is mapped.

[0058] Axes orientations are specified for internal operations. For instance, the origin of the atlas coordinates is located on the top-left comer of the atlas frame. For the reconstruction step, an intermediate axes definition for a local 3D patch coordinate system is used. The 3D local patch coordinate system is then converted to the final target 3D coordinate system using appropriate transformation steps.

[0059] FIG. 3 A shows an example of a single patch packed onto an atlas image. This patch is then converted to a local 3D patch coordinate system (U, V, D) defined by the projection plane with origin O’, tangent (U), bi-tangent (V), and normal (D) axes. This patch starts at position (AtlasPatch2DPosX, AtlasPatch2DPosY), and the patch has sizes in X and Y of AtlasPatch2DSizeX and AtlasPatch2SSizeY, respectively. For an orthographic projection, the projection plane is equal to the sides of an axis-aligned 3D bounding box (Patch 3D bounding box), as shown in FIG. 3B. The location of the bounding box in the 3D model coordinate system, defined by a left-handed system with axes (X, Y, Z), can be obtained by adding offsets TilePatch3dOffsetU, TilePatch3DOffsetV, and TilePatch3DOffsetD, as illustrated in FIG. 3C.

[0060] V3C High Level Syntax

[0061] Coded V3C video components are referred to in this document as video bitstreams, while an atlas component is referred to as the atlas bitstream. Video bitstreams and atlas bitstreams may be further split into smaller units, referred to here as video and atlas subbitstreams, respectively, and may be interleaved together, after the addition of appropriate delimiters, to construct a V3C bitstream.

[0062] V3C patch information is contained in atlas bitstream, atlas_sub_bitstream(), which contains a sequence ofNAL units. ANAL unit is specified to format data and provide header information in a manner appropriate for conveyance on a variety of communication channels or storage media. All data are contained in NAL units, each of which contains an integer number of bytes. A NAL unit specifies a generic format for use in both packet-oriented and bitstream systems. The format ofNAL units for both packet-oriented transport and sample streams is identical except that, in the sample stream format specified in Annex D of ISO/IEC 23090-5, each NAL unit can be preceded by an additional element that specifies the size of the NAL unit.

[0063] NAL units in atlas bitstream can be divided to atlas coding layer (ACL) and non-atlas coding layer (non- ACL) units. The former is dedicated to carry patch data while the latter is to carry data necessary to properly parse the ACL units or any additional auxiliary data.

[0064] In the nal_unit_header() syntax, nal unit type specifies the type of the RBSP data structure contained in the NAL unit as specified in Table 4 of ISO/IEC 23090-5. nal_layer_id specifies the identifier of the layer to which an ACL NAL unit belongs or the identifier of a layer to which a non-ACL NAL unit applies. The value of nal layer id shall be in the range of 0 to 62, inclusive. The value of 63 may be specified in the future by ISO/IEC. Decoders conforming to a profile specified in Annex A of ISO/IEC 23090-5 shall ignore (i.e., remove from the bitstream and discard) all NAL units with values of nal layer id not equal to 0 (zero). [0065] V3C extension mechanisms

[0066] While designing V3C specification, it was envisaged that amendments or new editions can be created in the future. In order to ensure that the first implementations of V3C decoders are compatible with any future extension, a number of fields for future extensions to parameter sets were reserved. [0067] For example, the second edition of V3C introduced an extension in VPS related to MIV and packed video component. The first V3C unit in a V3C sequence is a V3C Parameter Set (VPS) that provides information about profile and level of the sequence as well mapping of video components to codec. See the following table.

[0068] Rendering and meshes

[0069] A polygon mesh is a collection of vertices, edges and faces that defines the shape of a polyhedral object in 3D computer graphics and solid modelling. The faces usually consist of triangles (triangle mesh), quadrilaterals (quads), or other simple convex polygons (n- gons), since this simplifies rendering, but may also be more generally composed of concave polygons, or even polygons with holes.

[0070] Objects created with polygon meshes are represented by different types of elements. These include vertices, edges, faces, polygons and surfaces as shown in FIG. 4. This figure is from en.wikipedia.org/wiki/Polygon_mesh.

[0071] Polygon meshes are defined by the following elements.

[0072] Vertex: A position in 3D space defined as (x,y,z) along with other information such as color (r,g,b), normal vector and texture coordinates.

[0073] Edge: A connection between two vertices.

[0074] Face: A closed set of edges, in which a triangle face has three edges, and a quad face has four edges. A polygon is a coplanar set of faces. In systems that support multisided faces, polygons and faces are equivalent. Mathematically a polygonal mesh may be considered an unstructured grid, or undirected graph, with additional properties of geometry, shape and topology.

[0075] Surfaces: or smoothing groups, are useful, but not required to group smooth regions.

[0076] Groups: Some mesh formats contain groups, which define separate elements of the mesh, and are useful for determining separate sub-objects for skeletal animation or separate actors for non-skeletal animation.

[0077] Materials: defined to allow different portions of the mesh to use different shaders when rendered. [0078] UV coordinates: Most mesh formats also support some form of UV coordinates, which are a separate 2D representation of the mesh "unfolded" to show what portion of a 2-dimensional texture map to apply to different polygons of the mesh. It is also possible for meshes to contain other such vertex attribute information such as color, tangent vectors, weight maps to control animation, etc. (sometimes also called channels).

[0079] Generic mesh compression

[0080] Mesh data may be compressed directly without projecting it into 2D-planes, like in V-PCC based mesh coding. In fact, the anchor for V-PCC mesh compression call for proposals (CfP) utilizes off-the shelf mesh compression technology, Draco (google. github.io/draco/) (an open-source library for compressing and decompressing 3D geometric meshes and point clouds), for compressing mesh data excluding textures. Draco is used to compress vertex positions in 3D, connectivity data (faces) as well as UV coordinates. Additional per- vertex attributes may be also compressed using draco. The actual UV texture may be compressed using traditional video compression technologies, such as H.265 or H.264.

[0081] Draco uses the edgebreaker algorithm at its core to compress 3D mesh information. It offers a good balance between simplicity and efficiency, and is part of Khronos (a group providing royalty-free open standards for 3D graphics, Virtual and Augmented Reality, Parallel Computing, Machine Learning, and Vision Processing) endorsed extensions for the glTF specification (a royalty-free specification for the efficient transmission and loading of 3D scenes and models by engines and applications). The main idea of the algorithm is to traverse mesh triangles in a deterministic way so that each new triangle is encoded next to an already encoded triangle. This enables prediction of vertex specific information from the previously encoded data by simply adding delta to the previous data. Edgebreaker utilizes symbols to signal how each new triangle is connected to the previously encoded part of the mesh. Connecting triangles in such a way results on average in 1 to 2 bits per triangle when combined with existing binary encoding techniques.

[0082] ISO/IEC 23090-29: V-DMC

[0083] The V-DMC standardization work started after the completion of the call for proposal (CfP) issued by MPEG 3DG (ISO/IEC SC29 WG 2) on integration of MESH compression into the V3C family of standards (ISO/IEC 23090-5). The retained technology after the CfP result analysis is based on multiresolution mesh analysis and coding. This approach includes the following:

[0084] 1) generating a base mesh that is a simplified (low resolution) mesh approximation of the original mesh, called base mesh (this is done for all frames of the dynamic mesh sequence) m ,

[0085] 2) performing several mesh subdivision iterative steps (e.g., each triangle is converted into four triangles by connecting the triangle edge midpoints as illustrated on FIG. 5, which illustrates how one subdivision step of a triangle into four triangles is performed by connecting mid points of the initial triangle edges) on the generated base mesh, generating other approximation meshes mⁿt where n stands for the number of iterations with mi = m°i,

[0086] 3) defining displacement vectors di, also named error vectors, for each vertex of each mesh approximation mⁿt with n > 0, noted d”i

[0087] 4) for each subdivision level, the deformed mesh, obtained by m”i + cfi, i.e., by adding the displacement vectors to the subdivided mesh vertices, generates the best approximation of the original mesh at that resolution, given the base mesh and prior subdivision levels;

[0088] 5) the displacement vectors may undergo a lazy wavelet transform prior to compression;

[0089] 6) the attribute map of the original mesh is transferred to the deformed mesh at the highest resolution (i.e., subdivision level) such that texture coordinates are obtained for the deformed mesh and a new attribute map is generated.

[0090] This scheme is illustrated in FIG. 6, which illustrates multi-resolution analysis of a mesh. A base mesh (left) undergoes a first step of subdivision and error vectors are added to each vertex (illustrated by small arrows), after a series of iterative subdivision and displacements, the highest resolution mesh is generated (right). The connectivity of the highest resolution deformed mesh is generally different from the original mesh, however, the geometry of the deformed mesh is a good approximation of the original mesh geometry. [0091] The encoding process can be separated into two main modules: the preprocessing module and the actual encoder module as illustrated on FIG. 7, which is a block diagram of an encoding process 700. This may also be considered to a block diagram of an encoder 730. The encoder 730 is composed of a pre-processing module t710 hat generates a base mesh m(i) and the displacement vectors d(i), given the input mesh sequence M(i) and its attribute maps d(i). The encoder module 720 generates the compressed bitstream by ingesting the inputs and outputs of the pre-processing module 710.

[0092] The pre-processing includes mainly three steps: (mesh) decimation (reducing the original mesh resolution to produce a base mesh, forming a decimated mesh dm(i) uv-atlas isocharting (creating a parameterization of the base mesh, illustrated as atlas parameterization, and forming a parametrized decimated mesh pm(i)) and the subdivision surface fitting as illustrated on FIG. 8, which illustrates pre-processing steps at the encoder 730.

[0093] The encoder is illustrated on FIGS. 9 and 10 for the INTRA 720-1 and INTER 720-2 case respectively. In the latter, the base mesh connectivity of the first frame of a group of frames is imposed to the subsequent frame’s base meshes to improve compression performance.

[0094] FIG. 9 shows the encoder process 900 for INTRA frame encoding. It is noted that the encoder process 900 may also be considered to be a block diagram of elements in an encoder module 720-1. Inputs to this module are the base mesh (that is an approximation of the input mesh but that contains fewer faces and vertices), the patch information related to the input base mesh, the displacements, the static/ dynamic input mesh frame and the attribute map. Outputs of this module comprise a compressed bitstream 930 that contains a V3C extended signaling sub-bitstream 920- 1 including patch data information, a compressed base mesh substream 920-2, a compressed displacement video component substream 920-3, and a compressed attribute video component sub-bitstream 920-4. Examples herein may be implemented to form part of the compressed displacement video component substream 920-3. The module takes the input base mesh and first quantize its data in the Quantization module, which can be dynamically tuned by a Control Module. The quantized base mesh is then encoded with the static mesh encoder module, which outputs a compressed base mesh sub-bitstream that is muxed (multiplexed) in the output bitstream. The encoded base mesh is decoded in the Static Mesh Decoder module that generates a reconstructed quantized base mesh. The Update Displacements module takes as input the reconstructed quantizes base mesh, the pristine base mesh and the input displacements to generate new updated displacements that are remapped to the reconstructed base mesh data in order to avoid precision errors due to the static mesh encoding and decoding process. The updated displacements are filtered with a wavelet transform in the Wavelet Transform module (that also takes as input the reconstructed base mesh) and then quantized in the Quantization module. The quantized wavelet coefficients produced from the updated displacements are then packed into a video component in the Image Packing module. This video component is then encoded with a 2D video encoder such as HEVC, WC, etc., in the Video Encoder module, and the output compressed displacement video component sub-bitstream is muxed along with the V3C signaling information sub-bitstream into the output compressed bitstream. Then the compressed displacement video component is first decoded and reconstructed and then unpacked into encoded and quantized wavelet coefficients in the Image Unpacking module. These wavelet coefficients are then unquantized in the inverse quantization module and reconstructed with the inverse wavelet transform module that generates reconstructed displacements. The reconstructed base mesh is unquantized in the inverse quantization module and the unquantized base mesh is combined with the reconstructed displacements in the Reconstruct Deformed Mesh module to obtain the reconstructed deformed mesh. This reconstructed deformed mesh is then fed into the Attribute Transfer module together with the Attribute map produced by the pre-processing and the input static/dynamic mesh frame. The output of the Attribute Transfer module is an updated attribute map that now corresponds to the reconstructed deformed mesh frame. The updated attribute map is then padded, undergoes color conversion and is encoded as a video component with a 2D video codec such as HEVC or WC, in the Padding, Color Conversion and Video encoder modules respectively. The output compressed attribute map bitstream is multiplexed into the encoder output bitstream.

[0095] The inter encoding process 1000 is illustrated by FIG. 10, is similar to the intra encoding process, similar to the intra case, but with the base mesh connectivity being constrained for all frames of a group of frames. It is noted that the encoder process 1000 may also be considered to be a block diagram of elements in an encoder module 720-2. A motion encoder is used to efficiently encode displacements between base meshes compared to the base mesh of the first frame of the group of frames. More detail about the changes are as follows. The reconstructed reference base mesh is an input of the inter coding process. A new module called Motion Encoder takes as input the quantized input base mesh and the reconstructed quantized reference base mesh to produce compressed motion information encoded as a compressed motion bitstream, which is multiplexed into the encoder output compressed bitstream. All other modules and processes are similar to the intra encoding case.

[0096] The compressed bitstream generated by the encoder multiplexes:

[0097] 1) A sub-bitstream with the encoded base mesh using a static mesh codec.

[0098] 2) A sub-bitstream with the encoded motion data using an animation codec for base meshes in case INTER coding is enabled.

[0099] 3) A sub-bitstream with the wavelet coefficients of the displacement vectors packed in an image and encoded using a video codec.

[00100] 4) A sub-bitstream with the attribute map encoded using a video codec.

[00101] 5) A sub-bitstream that contains all metadata required to decode and reconstruct the mesh sequence based on the aforementioned sub-bitstreams. The signaling of the metadata is based on the V3C syntax and includes necessary extensions that are specific to meshes.

[00102] The decoding process 1100 is illustrated on FIG. 11, and may be implemented by a decoder 1130. The decoding process 1100 may be considered to be a decoder 1130 too. In the decoding module 1120, first the compressed bitstream is demultiplexed into sub-bitstreams that are reconstructed, i.e., metadata metadata(i) , reconstructed base mesh m ’(i), reconstructed displacements d’(i), and the reconstructed attribute map data A ’(i). The reconstruction of the mesh sequence is performed based on that data in the post-processing module, 1110 which forms the reconstructed mesh sequence M”(i) and its corresponding reconstructed attribute map data

[00103] FIGS. 12 and 13 illustrate the decoding process in INTRA and INTER modes respectively, via corresponding block diagrams. [00104] The intra frame decoding process 1200 (of FIG. 12) includes the following modules and processes as part of the decoder module 1120-1. First, the input compressed bitstream is de-multiplexed into V3C extended atlas data information (or patch information) 1220-1, a compressed static mesh bitstream 1220-2, a compressed displacement video component 1220-3, and a compressed attribute map bitstream 1220-4, respectively. Examples herein can be implemented via the compressed displacement video component 1220-3. The static mesh decoding module converts the compressed static mesh bitstream into a reconstructed quantized static mesh, which represents a base mesh. This reconstructed quantized base mesh undergoes inverse quantization in the inverse quantization module to produce a decoded reconstructed base mesh. The compressed displacement video component bitstream is decoded in the video decoding module to generate a reconstructed displacement video component. This displacement video component is unpacked into reconstructed quantized wavelet coefficients in the image unpacking module. Reconstructed quantized wavelet coefficients are inverse quantized in the inverse quantization module and then undergo an inverse wavelet transform in the inverse wavelet transform module, that produces decoded displacement vectors. The reconstruct deformed mesh module takes into account the patch information and takes as input the decoded reconstructed base mesh and decoded displacement vectors to produce the output decoded mesh frame. The compressed attribute map video component is decoded, and possibly undergoes color conversion to produce a decoded attribute map frame that corresponds to the decoded mesh frame.

[00105] The inter decoding process 1300 (in FIG. 13), which is performed by the decoding module 1120-2, is similar to the intra decoding process module with the following changes. The decoder also demultiplexes a compressed information bitstream. A decoded reference base mesh is taken as input of a motion decoder module together with the compressed motion information sub-bitstream. This decoded reference base mesh is selected from a buffer of previously decoded base mesh frames (by the intra decoder process for the first frame of a group of frames). The reconstruction of base mesh module takes the decoded reference base mesh and the decoded motion information as input to produce a decoded reconstructed quantized base mesh. All other processes are similar to the intra decoding process. [00106] The signaling of the metadata and substreams produced by the encoder and ingested by the decoder was proposed as an extension of V3C in the technical submission to the dynamic mesh coding CfP, and should be considered as purely indicative for the moment. It is as follows and mainly consists in additional V3C unit header syntax, additional V3C unit payload syntax, and Mesh Intra patch data unit.

[00107] A number of tables having V3C data in them are presented below.

[00108] V3C unit header syntax is as follows. It is noted that some of these use “lod” or “LOD”, which are level of detail, which is a term used to describe several levels of wavelet transform, each level having increased amount of details in reconstructed data. [00109] V3C unit payload syntax is indicated below.

[00110] Mesh Intra patch data unit information appears below.

proposed in J. Kim, A. Tourapis, K. Mammou, D. Podborsky, D. Flynn, Support of the V3C framework in V-DMC-TM, input document m60363, WG 07 MPEG 3D Graphics Coding and Haptics Coding, July 2022, and is as follows.

[00112] Base meshes are the output of the base mesh substream decoder.

[00113] A submesh is a set of vertices, their connectivity and the associated attributes which can be decoded completely independently in a mesh frame. Each base mesh can have one or more submeshes.

[00114] Resampled base meshes are the output of the mesh subdivision process. The inputs to the process are the base meshes (or sets of submeshes) as well the information from the atlas data substream on how to subdivide/resample the meshes (submeshes).

[00115] A displacement video is the output of the displacement decoder. The inputs to the process is the decoded geometry video as well the information from the atlas data substream on how to interpret/process this video. The displacement video contains displacement values to be added to the corresponding vertices.

[00116] A facegroupld is one of the attribute types assigned to each triangle face of the resampled base meshes. Facegroupld can be compared with the ids of the subparts in a patch to determine the corresponding facegroups to the patch. If facegrould is not conveyed through the base mesh substream decoder, it is derived by the information in the atlas data substream.

[00117] V3C unit

[00118] Compressed base meshes are signaled in a new substream, named as the Base Mesh data substream (unit type V3C_MD). As with other v3c units, the unit type, and its associated v3c parameter set id and atlas id are signaled in the v3c_unit_header(), illustrated below.

[00119] V3c parameter set extension

[00120] A new extension needs to be introduced in the v3c_parameter_set syntax structure to handle V-DMC. Several new parameters are introduced in this extension including the following: [00121] 1) vps_ext_mesh_data_facegroup_id_attribute_present _flag equals 1 indicates that one of the attribute types present in the base mesh data stream is the facegroup Id.

[00122] 2) vps ext mesh data attribute count indicates the number of total attributes in the base mesh including both the attributes signalled through the base mesh data substream and the attributes signalled in the video sub streams (using ai attribute count). When vps_ext_mesh_data_facegroup_id_attribute _present_flag equals 1 , it shall be greater or equal to ai_attribute_count+l. This can be constrained by profile/levels.

[00123] 3) The types of attributes that are signalled through the base mesh substream and not through the video substreams are signalled are signaled as vps_ext_mesh_attribute_type data types. When vps_ext_mesh_data_facegroup_id_attribute_present_flag equals 1, one of the vps_ext_mesh_attribute_type must be a facegroup_id.

[00124] 4) vps_ext_mesh_data_substream_codec_id indicates the identifier of the codec used to compress the base mesh data. This codec may be identified through the profiles a component codec mapping SEI message, or through means outside this document. [00125] 5) vps_ext_attribute_frame_width[i] and vps_ext_attribute_frame_height[i] indicate the corresponding with and height of the video data corresponding to the i-th attribute among the attributes signaled in the video substreams.

[00126] Atlas sequence parameter set extension

[00127] The information contained in this extension can be overwritten by the same information in the AFPS extension or the patch data units. The following parameters are introduced:

[00128] 1) asps_vmc_ext_prevent_geometry_video_conversion_flag prevents the outputs of the geometry video substream decoder from being converted. When the flag is true, the outputs are used as they are without any conversion process from Annex B in ISO/IEC 23090-5 V3C. When the flag is true, the size of geometry video shall be same as nominal video sizes indicated in the bitstream.

[00129] 2) asps vmc ext jorevent attribute video conversion flag prevents the outputs of attribute video substream decoder from being converted. When the flag is true, the outputs are used as they are without any conversion process from Annex B in ISO/IEC 23090-5 V3C. When the flag is true, the size of attribute video shall be same as nominal video sizes indicated in the bitstream.

[00130] 3) asps vmc ext subdivision method and asps vmc ext subdivision iteration count signal information about the subdivision method.

[00131] 4) asps vmc ext transform index indicates the transform applied to the displacement. The transform index can indicate any transform is not applied. When the transform is LINEAR LIFTING, the necessary parameters are signaled as vmc_lifting_transform_parameters.

[00132] 5) asps vmc ext joatch mapping method indicates how to map a subpart of a submesh to a patch.

[00133] a) When asps vmc ext patch mapping method is equal to 0, all the triangles in the corresponding submesh are associated with the current patch. In this case, there is only one patch associated with the submesh.

[00134] b) When asps vmc ext patch mapping method is equal to 1 , the subpart ids are explicitly signaled in the mesh patch data unit to indicate the associated subparts.

[00135] c) In other cases, the triangle faces in the corresponding submesh are divided into subparts by the method indicated by asps_vmc_ext_patch_mapping_method.

[00136] 6) asps vmc ext tjunction removing method indicates the method to remove t-junctions created by different subdivision methods or by different subdivision iterations of two triangles sharing an edge.

[00137] 7) asps vmc ext num attribute indicates the total number of attributes that the corresponding mesh carries. Its value shall be less or equal to vps ext mesh data attribute count.

[00138] 8) asps vmc ext attribute type is the type of the i-th attribute and it shall be one of ai_attribute_type_ids or vps_ext_mesh_attribute_types.

[00139] 9) asps_vmc_ext_direct_atrribute_projection_enabled_flag indicates that the

2d locations where attributes are projected are explicitly signaled in the mesh patch data units. Therefore, the projection id and orientation index in V3C V-PCC ISO/IEC 23090-5:2021 can be also used as in ISO/IEC 23090-5:2021.

[00140] Atlas Frame Parameter set extension

[00141] 1) afps_vmc_ext_single_submesh_in_frame_flag indicates there is only one submesh for the mesh frame

[00142] 2) When afps_vmc_ext_overriden_flag in afps_vmc_extension() is true, the subdivision method, displacement coordinate system, transform index, transform parameters, and attribute transform parameters can be signaled again and the information ioverrides the one signaled in asps_vmc_extension().

[00143] 3) afps_vmc_ext_single_attribute_tile_in_frame_flag indicates there is only one tile for each attribute signaled in the video streams.

[00144] 4) afps_ext_vmc_attribute_tile_information() contains the tile information for the attributes signaled through the video substreams.

[00145] Atlas Tile Header

[00146] A tile can be associated with one or more submeshes whose id is ath submesh id.

[00147] Patch data unit

[00148] As with the V-PCC Patch data units, Mesh patch data units are signaled in the Atlas data substream. Mesh Intra patch data unit, Mesh Inter patch data unit, Mesh Merge patch data unit, and Mesh Skip patch data unit can be used. [00149] 1) mdu submesh id indicate which submesh the patch is associated with among those indicated in the atlas tile header.

[00150] 2) mdu vertex count minus 1 and mdu triangle count minus 1 indicate the number of vertices and triangles associated with the current patch. [00151] When asps_vmc_ext_patch_mapping_method is not 0, the syntax elements mdu num subparts and mdu subpart id are signaled. When asps_vmc_ext_patch_mapping_method is 1, the associated triangle faces are the union of the triangle faces whose facegroupld is equal to mdu subpart id.

[00152] When mdu patch paramctcrs cnablc flag is true, the subdivision method, displacement coordinate system, transform index, transform parameters, and attribute transform parameters can be signaled again and the information overrides the corresponding information signaled in in asps_vmc_extension().

[00153] The signaling of the base mesh substream is also under investigation and is tentatively proposed in A. Tourapis, J. Kim, D. Podborski, K. Mammou, Base mesh data substream format for V-DMC, WG 07 MPEG 3D Graphics Coding and Haptics Coding, input document m60362, July 2022, as follows. FIGS. 14-16 illustrate these, where: FIG. 14 is a block diagram used to illustrate a base mesh encoder 1410 in the V-DMC encoder 720-3; FIG. 15 is a block diagram illustrating the base mesh encoder 1410 of the V-DMC encoder 720-3 of FIG. 14; and FIG. 16 is a block diagram of a base mesh decoder 1120-3. The output bitstream 1430 in FIG. 14 includes the following: the base-mesh bitstream 1420-1; the displacements bitstream 1420-2; the attribute bitstream 1420-3; and the V3C metadata bitstream 1420-4. The examples herein can be implemented to add to the displacements bitstream 1420-2.

[00154] One of the key features of the current V-DMC specification design is the support for a base mesh signal that can be encoded using any currently or future specified static mesh codec. For example, such information could be coded using Draco 3D Graphics Compression. This representation could provide the basis for applying other decoded information to reconstruct the output mesh frame within the context of V-DMC. [00155] Furthermore, for coding dynamic mesh frames, it is highly desirable to be able to exploit any temporal correlation that may exist with previously coded base mesh frames. In the original design (see FIGS. 14, 15, and 16) (see K. Mammou, J. Kim, A. Tourapis, D. Podborski, K. Kolarov, “[V-CG] Apple’s Dynamic Mesh Coding CfP Response,” ISO/IEC JTCl/SC29/WG7/m59281, April 2022), this was accomplished by encoding a mesh motion field instead of directly encoding the base mesh, and using this information and a previously encoded base mesh to reconstruct the base mesh of the current frame. This approach could be seen as the equivalent of inter prediction in video coding.

[00156] It is highly desirable also to associate all coded base mesh frames or motion fields with information that could help determine their decoding output order as well as their referencing relationships. It is possible, for example, that better coding efficiency could be achieved if the coding order of all frames does not follow the display order or by using as reference for generating a motion field for frame N an arbitrary previously coded motion field or base mesh instead of the immediately previous coded one. Also highly desirable is the ability to instantly detect random access points and independently decode multiple sub-meshes that together can form a single mesh, much like subpictures in video compression.

[00157] For all the above reasons, a new Base Mesh Substream format is introduced. This new format is very similar to a video coding format such as HE VC or the atlas subbitstream used in V3C, with the base mesh sub-bitstream also constructed using NAL units. High Level Syntax (HLS) structures such as base mesh sequence parameter sets, base mesh frame parameter sets, submesh layer are also specified. An overview of this bitstream with its different subcomponents is shown in FIG. 17.

[00158] One of the desirable features of this design is the ability to segment a mesh into multiple smaller partitions, referred to in this document as submeshes (see FIG. 18, which illustrates segmentation of a mesh 1810 into sub-meshes 1820-1 and 1820-2). These submeshes 1820 can be decoded completely independently, which can help with partial decoding and spatial random access. Although it may not be a requirement for all applications, some applications may require that the segmentation in submeshes remains consistent and fixed in time. The submeshes do not need to use the same coding type, i.e., for one frame one submesh may use intra coding while for another inter coding could be used at the same decoding instance, but it is commonly a requirement that the same coding order is used and the same references are available for all submeshes corresponding at a particular time instance. Such restrictions can help guarantee proper random-access capabilities for the entire stream. An example where two submeshes are used is shown in FIG. 19.

[00159] NAL unit syntax

[00160] As discussed earlier, the new bitstream is also based on NAL units, and it is similar to those of the atlas substream in V3C. The syntax is provided below.

[00161] General NAL unit syntax

[00162] NAL unit header syntax

[00163] NAL unit semantics

[00164] This section contains some of the semantics that correspond to the above syntax structures. More details would be provided for syntax elements that have not been defined in complete detail.

[00165] 1. General NAL unit semantics

[00166] NumBytesInNalUnit specifies the size of the NAL unit in bytes. This value is required for decoding of the NAL unit. Some form of demarcation of NAL unit boundaries is necessary to enable inference of NumBytesInNalUnit. One such demarcation method is specified in Annex TBD for the sample stream format. Other methods of demarcation can be specified outside this document. [00167] NOTE 1 - The mesh coding layer (MCL) is specified to efficiently represent the content of the mesh data. The NAL is specified to format that data and provide header information in a manner appropriate for conveyance on a variety of communication channels or storage media. All data are contained in NAL units, each of which contains an integer number of bytes. A NAL unit specifies a generic format for use in both packet-oriented and bitstream systems. The format of NAL units for both packet-oriented transport and sample streams is identical except that in the sample stream format specified in an annex that is to be determined, each NAL unit can be preceded by an additional element that specifies the size of the NAL unit.

[00168] rbsp_byte[ i ] is the i-th byte of an RBSP. An RBSP is specified as an ordered sequence of bytes as follows:

[00169] The RBSP contains a string of data bits (SODB) as follows:

[00170] - If the SODB is empty (i.e., zero bits in length), the RBSP is also empty.

[00171] - Otherwise, the RBSP contains the SODB as follows:

[00172] l)The first byte of the RBSP contains the first (most significant, left-most) eight bits of the SODB; the next byte of the RBSP contains the next eight bits of the SODB, etc., until fewer than eight bits of the SODB remain.

[00173] 2) The rbsp_trailing_bits( ) syntax structure is present after the SODB as follows:

[00174] i) The first (most significant, left-most) bits of the final RBSP byte contain the remaining bits of the SODB (if any).

[00175] ii) The next bit consists of a single bit equal to 1 (i.e., rbsp stop one bit).

[00176] iii) When the rbsp stop one bit is not the last bit of a byte-aligned byte, one or more bits equal to 0 (i.e., instances of rbsp alignment zero bit) are present to result in byte alignment.

[00177] Syntax structures having these RBSP properties are denoted in the syntax tables using an "_rbsp" suffix. These structures are carried within NAL units as the content of the rbsp_byte[ i ] data bytes. [00178] 2. NAL unit header semantics

[00179] Similar NAL unit types, as for the atlas case, were defined for the base mesh enabling similar functionalities for random access and segmentation of the mesh. Unlike the atlas that is split into tiles, in this document we define the concept of a sub-mesh and define specific nal units that correspond to coded mesh data. In addition, NAL units that can include metadata such as SEI messages are also defined.

[00180] In particular, the base mesh NAL unit types supported are specified as follows: [00181] Raw byte sequence payloads, trailing bits, and byte alignment syntax

[00182] 1. Base mesh sequence parameter set RBSP syntax

[00183] As with similar bitstreams, the primary syntax structure that is defined for a base mesh bitstream is a sequence parameter set. This syntax structure contains basic information about the bitstream, identifying features for the codecs supported for either the intra coded and inter coded meshes, as well as information about references.

[00184] 1.1. General base mesh sequence parameter set RBSP syntax

[00185] bmsps lo g2_max_mesh_frame_order_cnt_lsb_minus4 , bmsps_max_dec_mesh_frame_buffering_minus 1 , bmsps_long_term_ref_mesh_frames_flag, bmsps_num_ref_mesh_frame_lists_in_bmsps, bmesh_ref_list_struct( i ) are equivalent to those in ASPS.

[00186] bmsps intra mesh codec id indicates the static mesh codec used to encode the base meshes in this base mesh substream. It could be associated with a specific mesh or motion mesh codec through the profiles specified in the corresponding specification, or could be explicitly indicated with an SEI message as is done in the V3C specification for the video subbitstreams.

[00187] bmsps_intra_mesh_data_size_precision_bytes_minusl (+1) specifies the precision, in bytes, of the size of the coded mesh data.

[00188] bmsps_inter_mesh_codec_present flag indicates if a specific codec indicated by bmsps_inter_mesh_codec_id is used to encode the inter predicted submeshes.

[00189] bmsps_inter_mesh_data_size_precision_bytes_minusl(+l) specifies the precision, in bytes, of the size of the inter predicted mesh data. This precision is signaled considering the size of the coded mesh data and the inter predicted mesh data (e.g., motion field) can be significantly different.

[00190] bmsps facegroup segmentation method indicates how facegroups could be derived for a mesh. A facegroup is a set of triangle faces in a submesh. Each triangle face is associated with a Facegroupld indicating the facegroup it belongs to. When bmsps facegroup segmentation method is 0, then Facegroupld is present directly in the coded submesh. Other values indicate that the facegroup can be derived using different methodologies based on the characteristics of the stream. For example, value 1 means that there is no Facegroupld associated with any face. A value 2 means that all faces are identified with a single ID, a 3 that facegroups are identified based on the connected component method, while a value of 4 indicates that each individual face has its own unique ID. Currently ue(v) is used to indicate bmsps facegroup segmentation method, but fixed length coding or partitioning to more elements could have been used instead. [00191] 1.2. Base Mesh Profile, tier, and level syntax

[00192] bmptl extended sub profilc flag providing support for sub profiles can be quite useful for further restricting the base mesh profiles depending on usage and applications. [00193] 1.3. Base mesh frame parameter set RBSP syntax [00194] The base mesh frame parameter set has the frame level information such as number of submeshes in the frames corresponding to one mfh mesh frm order cnt lsb. A submesh is coded in one mesh_data_submesh_layer() and is independently decodable from other submeshes. In the case of inter frame prediction, a submesh can refer only to the submeshes with the same smh_id in its associated reference frames. The mechanism is equivalent to what is specified in section 8.3.6.2.2 in V3C.

[00195] The mechanism is equivalent to the Atlas frame tile information syntax

(83.6.2.2 in V3C). [00196] 1.4. Base mesh submesh layer rbsp syntax

[00197] 1.4.1. bmesh submesh layer rbsp syntax

[00198] A bmesh submesh layer contains a submesh information. One or more bmesh submesh layer rbsp can correspond to one mesh frame indicated by mfh mesh frm order cnt lsb.

[00199] 1.4.2. submesh header syntax

[00200] The mechanism is equivalent to the atlas tile header (8.3.6.11 in [3])

[00201] smh id is the id of the current submesh contained in the mesh data submesh data.

[00202] smh ype indicates how the mesh is coded. If smh ype is I SUBMESH, the mesh data is coded with the indicated static mesh codec. If smh type is P SUBMESH, inter prediction is used to code the mesh data.

[00203] 1.4.3. Submesh data unit

[00204] smdu intra_sub_mesh_unit( unitSize ) contains a sub mesh unit stream of size unitSize, in bytes, as an ordered stream of bytes or bits within which the locations of unit boundaries are identifiable from patterns in the data. The format of such sub mesh unit stream is identified by a 4CC (four-character code) as defined by bmptl_profile_codec_group_idc or by a component codec mapping SEI message.

[00205] smdu_inter_sub_mesh_unit( unitSize ) contains a sub mesh unit stream of size unitSize, in bytes, as an ordered stream of bytes or bits within which the locations of unit boundaries are identifiable from patterns in the data. The format of such sub mesh unit stream is identified by a 4CC code as defined by bmptl_profile_codec_group_idc or by a component codec mapping SEI message.

[00206] The current basis for the V-DMC test model iteratively subdivides the base mesh and calculates displacement vectors for every generated vertex at each iteration. These displacement vectors can be filtered by a wavelet transform to improve their compression. These vectors are necessary to reconstruct a deformed mesh that provides higher fidelity than the reconstructed base mesh. Following a spatial scalability philosophy, several scales or levels of details (LODs) are defined based on iterative “midpoint” subdivision, and displacements are computed for each vertex of the mesh at each iteration.

[00207] The displacements are quantized and packed in a YUV444 or YUV400 frame and encoded losslessly using traditional 2D video codecs.

[00208] FIG. 20A illustrates a displacement video frame of a displacement map and shows LODs 0, 1, 2, and 3. In more detail, the vertical axis is the Y-coordinate of a single displacement video frame, which will be encoded by video encoder. So, in general, displacements are a form of mapping representation of the 3D points as a ID vector, by traversing of those points in a predefined manner. After that, this ID vector is packed into a 2D displacement video frame, which is presented at FIG. 20A. In FIG. 20B, the vertical axis is the frequency, or the number of times Y that a displacement has the corresponding X value. In other words, it is a histogram. As seen in FIG. 20B, the signal characteristics of the displacement maps corresponding to the LODs in FIG. 20A do not correspond at all to natural images for which video codecs are optimized, rather they appear like distributed noise, for which traditional video codec encoding tools are not designed. Thus, compression is inefficient. [00209] Still, video codecs provide the essential benefit that they are available on hardware and allow for efficient decoding for example on mobile devices, where power consumption is a key factor.

[00210] One of possible solutions could be enabling of quantization in the video codec. But a video codec has no information about the amount of LODs, nor about initial distribution of values per LOD. And it reduces effectiveness of lossy quantization in video codec. That is, the absence of information on structure of data prevents video encoder from effectively using lossy mode (e.g., DCT + quantization).

[00211] For example, FIG. 21 demonstrates distribution of displacement coefficients per LOD. For various sequences and even various frames of the same sequence, this distribution of coefficients will be different. Therefore, identical quantization parameters (QPs) will generate different quality of reconstruction per LOD and per frame. Regularizing the distribution is needed to improve coding quality.

[00212] The examples herein address these and other issues. In examples herein, methods and signaling for adaptive quantization of displacement values are disclosed. With this approach, improved coding efficiency is achieved and more flexible adjustment of the reconstructed mesh geometry is also achieved. Furthermore, it is suitable for lossy coding of displacement values, thanks to the regularization of displacement’s distortion and more accurate reconstruction of displacement values.

[00213] Example embodiments include the following:

[00214] 1) A method of adaptive quantization of V-DMC displacement vectors.

[00215] 2) A method of estimation of parameters for adaptive quantization (offsets and scales).

[00216] 3) Signaling of new parameters for adaptive quantization.

[00217] Encoder embodiments

[00218] This section is described in part through reference to FIG. 22, which is a flow diagram of encoding adaptive quantization of displacement values in V-DMC, and signaling of those quantized values. It is assumed that FIG. 22 is performed by an encoder in an apparatus such as that shown in FIG. 24. The encoder performs the blocks as (part of) an encoding process 2200.

[00219] In block 2205, the encoder (e.g., preprocessor) sets a target subdivision iteration count to reach a comparable target face count as in the original mesh frame. The encoder (e.g., preprocessor) computes in block 2210 the displacement vectors (also referred to as coefficients in the following) for each LOD defined by the subdivision iteration count. This encoder may generate (block 2215) such displacement vectors for every input mesh frame (see discussion above about this generation).

[00220] The encoder quantizes the generated displacement vectors in block 2220. In block 2223, the generated displacement vectors are quantized at least by performing shifting the generated displacement vectors by an offset and/or (“/”) scaling the generated displacement vectors by a scale factor. In one embodiment, the generated displacement vectors are quantized with the following formula, see block 2225:

QuantizedDisplacement = (Displacement - Offset[LOD_ID]) / (Quantizer * Scale[LOD_ID]), where Displacement is a value of a generated displacement vector, “Quantizer” is defined via the encoder parameters (e.g., lifting_quantization_parameters_*) (see block 2227, where the asterisk indicates a set of parameters)), the offset (e.g., “Offset”) represents offset for all coefficients in this LOD, and Scale is a single multiplier for all values in certain wavelet band (for this LOD ID). The ID of each LOD can be unique and LOD O...LOD N represents a full set of details. The Quantizer is a scaling factor (i.e., a number) that is currently used in the V-DMC techniques.

[00221] Reference 2226 indicates blocks 2230-2260 may depend from either block 2223 or 2225.

[00222] The offset (e.g., “Offset”) is computed (block 2230) per LOD (having an ID of LOD ID) and, as previously stated, represents offset for all coefficients in this LOD. The scaling factor (e.g., “Scale”) is a single multiplier for all values in certain wavelet band (for this LOD ID). [00223] In one embodiment, the offset e.g., “Offset”, is computed as the most frequent value of all displacement values in a single LOD. (See distribution example at FIG. 21.) See block 2235.

[00224] In another embodiment, the offset e.g., “Offset” is computed as the mean value of all displacement values in a single LOD. See block 2240.

[00225] In another embodiment, the offset e.g., “Offset”, is computed as the median value of all displacement values in a single LOD. See block 2245.

[00226] It is noted that blocks 2235, 2240, and 2245 are considered to be alternatives. However, it is possible for one LOD that one would pick 2235, and then for another LOD one would pick 2240 for example. That is, individual blocks 2235, 2240, or 2245 could be picked

[00227] In one embodiment, the offset e.g., “Offset”, is quantized with higher accuracy (larger bit depth) than other displacement values and thus improves quality of reconstructed signal. See block 2250. That is, the accuracy may be tuned, e.g., per LOD. Signaling of this can be performed so the decoder extracts the accuracy information in metadata and therefore can determine the higher accuracy applied to the offset.

[00228] For an example mesh frame, optimal offsets have been computed for each LOD and each coordinate in the local coordinate system, using an 8-bit depth for Offset signaling:

[00229] LODDisplacementOffset[0] = (131, 129,128);

[00230] LODDisplacementOffset[l] = (224, 128,128);

[00231] LODDisplacementOffset[2] = (159, 128,128); and

[00232] LODDisplacementOffset[3] = (135, 128,128).

[00233] The signaling of these values is explicated later in the document. As it can be observed from the values, the offset may significantly differ from the mean value 128, especially for LODs 1 and 2, whereas the offset for LODs 0 and 3 are not that different than the mean value 128. This enables reducing the range of displacement values to be encoded and more accurately quantize them.

[00234] In one embodiment, “Scale” is computed based on magnitude of coefficients between a current and a previous LOD. See block 2255. [00235] For an example mesh frame, the observed magnitudes of displacement values are as follows:

[00236] LODO: 1806

[00237] LODI: 9488

[00238] LOD2: 26231

[00239] LOD3: 74395

[00240] The current V-DMC Test Model scales by a factor two between each LOD and is not accurate because the test model is not adaptive. Signaling an adaptive scale further enables refinement of the quantization accuracy of displacement values.

[00241] In one embodiment, “Scale” is computed by taking into account “Offset” values, if those are available. See block 2260. For instance, the Offset could be applied before computation of the Scale.

[00242] Signaling embodiments

[00243] In block 2265, the V-DMC displacement adaptive quantization information is signaled, e.g., to a computer system having a decoder.

[00244] In one embodiment, the V-DMC displacement adaptive quantization information is signaled in or along the bitstream. See block 2270.

[00245] In one embodiment, the information is signaled in the atlas sequence parameter set and in the lifting transform parameters. See block 2275. The approach is not limited to the linear lifting transform and could be signaled in a similar manner, mutatis, mutandis, for other lifting transforms if they are included in the V-DMC specifications.

[00246] Possible signaling tables include the following.

[00247] Vmc_transform_lifting_quantization_parameters_offset_bitdepth_minusl indicates the bit depth of Vmc_transform_lifting_quantization_parameters_offset_x, Vmc_transform_lifting_quantization_parameters_offset_y and Vmc_transform_lifting_quantization_parameters_offset_z.

Vmc_transform_lifting_quantization_parameters_offset_bitdepth_minusl shall be in the range of 0 to 31, inclusive. [00248] asps_vmc_ext_ adaptive_quantization_type indicates that adaptive quantization mode is enabled, according to the following table:

[00249] asps_vmc_ext_ adaptive quantization flags indicates which parts of adaptive quantization mode are enabled, according to the following table:

[00250] While the above is one example, there are many others. For instance, vmc_ext_ adaptive quantization flags could be in bits the following:

[00251] 0000 means no quantization is used;

[00252] 0001 means uniform quantization is used (so not adaptive);

[00253] **1* means adaptive offset is used (* means whatever value);

[00254] *1** means adaptive scaling is used;

[00255] 1*** means signed adaptive offset is used. [00256] So, 111 * means the adaptive offsets AND adaptive scaling AND adaptive signed offset are all used. The bitwise flags enable one to combine the different adaptive offsets/scale factors in a single 4 bits set of flags.

[00257] vmc transform lifting quantization _parameters_offset_x[attributelndex][ Itplndex ] [lodld] indicates the offset of the displacement vector X-component of, for attributeindex, Itplndex and LOD index lodld.

[00258] vmc transform lifting quantization _parameters_offset_y[attributelndex][ Itplndex ] [lodld] indicates the offset of the displacement vector Y-component of, for attributeindex, Itplndex and LOD index lodld.

[00259] vmc transform lifting quantization _parameters_offset_z[attributelndex][ Itplndex ] [lodld] indicates the offset of the displacement vector Z-component of, for attributeindex, Itplndex and LOD index lodld.

[00260] vmc transform lifting quantization _parameters_x[attributelndex][ Itplndex ] [lodld] indicates the quantization parameter for the displacement vector X-component for attributeindex, letplndex and LOD index lodld.

[00261] vmc transform lifting quantization _parameters_y[attributelndex][ Itplndex ] [lodld] indicates the quantization parameter for the displacement vector Y -component for attributeindex, letplndex and LOD index lodld.

[00262] vmc transform lifting quantization _parameters_z[attributelndex][ Itplndex ] [lodld] indicates the quantization parameter for the displacement vector Z-component for attributeindex, letplndex and LOD index lodld.

[00263] vmc_transform_log2_lifting_lod_inverseScale_x[attributeIndex][ Itplndex] [lodld] indicates the scale of the displacement vector X-component for attributeindex, Itplndex and LOD index lodid, relatively to the previous scale.

[00264] vmc_transform_log2_lifting_lod_inverseScale_y[attributeIndex][ Itplndex] [lodld] indicates the scale of the displacement vector Y-component for attributeindex, Itplndex and LOD index lodid, relatively to the previous scale. [00265] vmc_transform_log2_lifting_lod_inverseScale_z[attributeIndex][ Itplndex] [lodld] indicates the scale of the displacement vector Z-component for attributeindex, Itplndex and LOD index lodid, relatively to the previous scale.

[00266] In another embodiment, the displacement vectors are encoded with only one component (ID) and only vmc_transform_log2_lifting_lod_inverseScale, vmc transform lifting quantization _parameters and vmc transform lifting quantization _parameters_offset are encoded along the bitstream.

[00267] In another embodiment, instead of signaling bitwise flags for vmc_ext_ adaptive quantization flags, an index table is used to signal vmc ext adaptive quantization modes.

[00268] It is noted that pre-computing the target subdivision iteration count, as in block 2205, may be performed in the preprocessing module 710 of FIG. 7. Adapting this value may be performed in the control module of FIGS. 9, 10, and 14. The adapting may also be performed in the mode decision of FIG. 15. Computing the displacements (they are precomputed in the preprocessing but need to be recomputed in the encoder based on the reconstructed base mesh) may happen in the Update Displacements of all these figures. Quantizing the generated displacement vectors may be performed in the Quantization module after the Wavelet Transform in all these figures. Signaling the quantized generated displacements may be performed by the control module of these figures. Locations for the corresponding operations would be similar in the decoders described herein.

[00269] Decoder Embodiments

[00270] This section is described by reference in part to FIG. 23, which is a flow diagram of decoding adaptive quantization of displacement values in V-DMC. This is assumed to be performed by a decoder implanted in an apparatus such as that shown in FIG. 24. This is performed as (part of) a decoding process 2300.

[00271] In block 2305, the decoder receives the signaled V-DMC displacement adaptive quantization information. As previously described in FIG. 22 (see blocks 2270 and 2275), this information may be signaled in or alongside the bitstream (block 2310) or in the atlas sequence parameter set and in the lifting transform parameters (block 2315). [00272] In one embodiment, the decoder decodes one or more (e.g., all) provided bitstreams (block 2320), including unpacking the V-DMC displacement adaptive quantization information (block 2325) and may perform the unpacking based on the signaled displacement packing parameters (block 2330).

[00273] The following pseudo-code represents one possible decoding embodiment, for the displacement vector x-component without loss of generality, covering both cases where displacements are encoded in ID or 3D. See block 2335. Note that the y-components and z- components may be performed similarly if 3D is used.

[00274] DispFrame is an array indexed by vertex indices, obtained by decoding and unpacking the received displacement component video frame. See block 2340.

[00275] The decoder derives the number of vertices of each, e.g., individual, LOD and stores, e.g., in the following variables: lodVertexCount[lod!ndex]. See block 2345.

[00276] In block 2350, for the vertices of the LODs in the array, the decoder determines the displacement vector x-components based on corresponding one or both of scale factors or offsets for vertices. It is noted that the algorithm below uses both scale factors and offsets, but these could be applied independently.

[00277] Furthermore, much of decoding is performed using symmetric versions of the encoding operations. In other words, encoding and decoding often involve symmetry and “mirror image” operations. Thus, in block 2353, encoding operations are performed as decoding versions of the same encoding operations by applied for decoding. That is, this block performs “inverse” versions of operations in the encoding (e.g., blocks 2223, 2225, 2227, 2230, 2250, 2255, and 2260). For example, the equation at 2225 of FIG. 22 can be modified as follows. The equation at the decoder side should be the following:

Displacement = function (QuantizedDisplacement, Offset[LOD_ID], Quantizer, Scale[LOD_ID]), where QuantizedDisplacement is the quantized displacement, and function (•) uses the indicated parameters to output the displacement. The other elements of the equation are the same as previously described.

[00278] A refinement of the equation includes the following. Displacement = quantizedDisplacement * Scale[LOD_ID] * Quantizer +

Offset[LOD_ID], where the function (•) has been replaced by one possible function.

[00279] Setting without loss of generality the

Vmc_transform_lifting_quantization_parameters_offset_bitdepth_minusl to 7, in pseudocode, an algorithm for this includes the following:

[00280] lodCount = asps vmc ext subdivision iteration count;

[00281] for (i=0; i <lodCount; i++) {

[00282] lodQp_x[i] = vmc transform lifting quantization j)arameters_x[attributelndex][ Itplndex ][i];

[00283] lodQpOffset_x[i] = vmc transform lifting quantization

_parameters_offset_x[attributelndex][ Itplndex ][i];

[00284] }

[00285]

[00286] for (i=0; i <lodCount-l; i++) {

[00287] lod!nverseScale[i] = vmc_transform_log2_lifting_lod_inverseScale_x[attributeIndex] [ Itplndex ] [i] ;

[00288] }

[00289] bitDepthPosition = bmsps_geometry_3d_bit_depth_minusl + 1;

[00290]

[00291] for (i=0; i < lodCount; i++) {

[00292] qp = lodQP_x[i];

[00293] if(qp==0){

[00294] iscale[i] = 0.0;

[00295] }

[00296] else {

[00297] iscale[i] = pow(0.5, 16 - bitDepthPosition + (4 - qp) / 6.0);

[00298] }

[00299] ilodScale[i] = lod!nverseScale[i]; [00300] }

[00301]

[00302] for (it = 0, vcountStart = 0; it < lodCount; it++) {

[00303] vcountEnd =lodVertexCount[it];

[00304] offset_x[it] = (lodQpOffset_x[it] - 128) / 128.0;

[00305] if(it){

[00306] iscale_x[it] = ilodScale[it] / 256.0;

[00307] }

[00308] for (v = vcountStart; v < vcountEnd; v++) {

[00309] d = dispFrame[v];

[00310] d = d * iscale_x[it] + offset_x[it];

[00311] }

[00312] vcountStart = vcountEnd:

[00313] }

[00314] It is noted for the pseudocode of “d = d * iscale_x[it] + offset_x[it]”, that offset could be calculated and applied separately for positive and negative values of displacement. In this case, there could be two adaptive offsets, such as offset_x[it][0] and offset_x[i][l], where the first is added to positive “d” values and the second is added to negative “d” values. Computation of offset could be the same, e.g., mean, median or most frequent value, just computed separately for positive and negative values of displacement. Pseudocode for implementation on the decoder (with similar code used on the encoder) could be the following:

[00315] for (v = vcountStart; v < vcountEnd; v++) {

[00316] d = dispFrame[v] * iscale_x[it] + offset_x[it];

[00317] if (d>0) d += signed_offset_x[0][it];

[00318] if (d<0) d -= signed_offset_x[l][it];

[00319] }

[00320] vcountStart = vcountEnd:

[00321] } [00322] One possible signaling implementation indicating the above adaptive signed offset should be used is via extension adaptive flags with "ADAPTIVE_SIGNED_OFFSET", such as part of vmc_ext_ adaptive quantization flags from above, or via an extra flag.

[00323] In block 2355, if 3D, the decoder performs decoding of displacement vectors (block 2335) for y and z axes. Note that if 3D is used, the decoding of displacement vectors in the x, y, and z axes could be performed serially, in parallel, or some combination of these.

[00324] In block 2360, the decoded displacement vector components are used to form reconstructed data for LODs in the received displacement component video frame.

[00325] Turning to FIG. 24, this figure is an example of a block diagram of an apparatus 180 suitable for implementing any of the encoding or decoding operations or block diagrams described herein. The apparatus 180 could be a computer system (e.g., desktop, laptop, server), a wireless device such as a smartphone, a camera system, a television or projector, or any other apparatus that takes video or displays video.

[00326] The apparatus 180 includes circuitry comprising one or more processors 120, one or more memories 125, one or more transceivers 130, one or more network (N/W) interface(s) (I/F(s)) 155 and user interface (UI) circuitry and elements 157, interconnected through one or more buses 127. Apparatus that implement the encoding or decoding herein may not have all of the circuitry, or may have additional circuitry, not described here. FIG. 24 is presented merely as an example of one possible implementation.

[00327] Each of the one or more transceivers 130 includes a receiver, Rx, 132 and a transmitter, Tx, 133. The one or more buses 127 may be address, data, and/or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like. The one or more transceivers 130 are connected to one or more antennas 105, which could be one of the antennas 128 (from UE 110) or antennas 158 (from base station 170), and may communicate using wireless link 111.

[00328] The one or more memories 125 include computer program code 123. The apparatus 180 includes a control module 140, which may implement an encoder 730, a decoder 1130 or a codec (implementing both encoder 730 and decoder 1130). It is further noted that any of the encoding processes described such as the encoding process 700 may be performed by the control module 140. Similarly, any of the decoding processes described such as the decoding process 1100 may be performed by the control module 140. The control module 140 comprises one of or both parts 140-1 and/or 140-2, which may be implemented in a number of ways. The control module 140 may be implemented in hardware as control module 140-1, such as being implemented as part of the one or more processors 120. The control module 140-1 may be implemented also as an integrated circuit or through other hardware such as a programmable gate array. In another example, the control module 140 may be implemented as control module 140- 2, which is implemented as computer program code (having corresponding instructions) 123 and is executed by the one or more processors 120. For instance, the one or more memories 125 store instructions that, when executed by the one or more processors 120, cause the apparatus 180 to perform one or more of the operations as described herein. Furthermore, the one or more processors 120, one or more memories 125, and example algorithms (e.g., as flowcharts and/or signaling diagrams), encoded as instructions, programs, or code, are means for causing performance of the operations described herein.

[00329] The network interface(s) (N/W I/F(s)) 155 are wired interfaces communicating using link(s) 156, which could be fiber optic or other wired interfaces. The link(s) 156 could be the link(s) 131 and/or 176 from FIG.1A. The link(s) 131 and/or 176 from FIG.1A could also be implements using transceiver(s) 130 and corresponding wireless link(s) 111. The apparatus could include only wireless transceiver(s) 130, only N/W I/Fs 155, or both wireless transceiver(s) 130 and N/W I/Fs 155.

[00330] The apparatus 180 may or may not include UI circuitry and elements 157. These could include a display such as a touchscreen, speakers, or interface elements such as for headsets. For instance, a UE 110 of a smartphone would typically include at least a touchscreen and speakers. The UI circuitry and elements 157 may also include circuity to communicate with external UI elements (not shown) such as displays, keyboards, mice, headsets, and the like.

[00331] The computer readable memories 125 may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, flash memory, firmware, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The computer readable memories 125 may be means for performing storage functions. The processors 120 may be of any type suitable to the local technical environment, and may include one or more of general-purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on a multi-core processor architecture, as nonlimiting examples. The processors 120 may be means for performing functions, such as controlling the apparatus 180, and other functions as described herein.

[00332] Turning to FIG. 25, this figure is a block diagram illustrating a system 2500 in accordance with an example. In the example, the encoder 730 is used to encode video from the scene 15 (as shown in FIG. 1 A), and the encoder 730 is implemented in a transmitting apparatus 180- 1. The encoder produces a bitstream 2510 that is received by the receiving apparatus 180-2, which implements a decoder 1130. The encoder 730 sends the bitstream 2510, comprising the following: displacement vector quantization information 2550; indications 2520, which comprise in this example: indication that quantization mode is enabled 2520-1 (see mode ADAPTIVE in vmc_ext_ adaptive_quantization_type) and one or more of: indication of adaptive quantization; 2520-2 (see ADAPTIVE OFFSET in vmc_ext_ adaptive quantization flags); indication of adaptive scale 2520-3 (see ADAPTIVE SCALE in vmc_ext_ adaptive quantization flags); or indication of adaptive signed offset 2520-4. The decoder 1130 forms the video for the scene 15- 1 (see FIG. IB), and the receiving apparatus 180-2 would present this to the user, e.g., via a smartphone, television, or projector among many other options.

[00333] Without in any way limiting the scope, interpretation, or application of the claims appearing below, a technical effect and advantage of one or more of the example embodiments disclosed herein is reduced bitrate requirements. Another technical effect and advantage of one or more of the example embodiments disclosed herein is improved encoding and decoding performance. Another technical effect and advantage of one or more of the example embodiments disclosed herein is hardware decoding simplified (fewer frames to decode).

[00334] The following are additional examples.

[00335] Example 1. A method, comprising: [00336] as part of an encoding process of video using video-based dynamic mesh coding, performing the following:

[00337] setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video;

[00338] computing displacement vectors for each level of detail defined by the subdivision iteration count;

[00339] generating such displacement vectors for a plurality of input mesh frames;

[00340] quantizing the generated displacement vectors;

[00341] signaling at least the quantization information used to generate the quantized displacement vectors;

[00342] packing the quantized displacement vectors in one or more displacement video frames; and

[00343] coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the video-based dynamic mesh coding.

[00344] Example 2. The method according to example 1, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors by performing one or both of shifting the generated displacement vectors by an offset or scaling the generated displacement vectors by a scale factor.

[00345] Example 3. The method according to example 2, wherein, for individual ones of the levels of detail, the offset is computed via one of the following:

[00346] as a most frequent value of all displacement values in a corresponding level of detail;

[00347] as a mean value of all displacement values in a corresponding level of detail; or

[00348] as a median value of all displacement values in a corresponding level of detail.

[00349] Example 4. The method according to any one of examples 2 to 3, wherein the offset is computed separately for positive and negative values of displacement.

[00350] Example 5. The method according to example 4, wherein signaling at least the quantized generated displacement vectors further comprises signaling an indication of adaptive signed offset indicating the offset is to be computed separately for positive and negative values of displacement.

[00351] Example 6. The method according to any one of examples 2 to 5, wherein the offset is quantized with higher accuracy than other displacement values, and signaling comprises sending signaling indicating the offset is quantized with higher accuracy than other displacement values.

[00352] Example 7. The method according to any one of examples 2 to 6, wherein the scale factor is computed based on magnitude of displacement vectors between a current and a previous level of detail.

[00353] Example 8. The method according to any one of examples 2 to 6, wherein the scale factor is computed by taking into account values of the offset.

[00354] Example 9. The method according to any one of examples 2 to 8, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors using the following equation:

[00355] QuantizedDisplacement = (Displacement - Offset[LOD_ID]) / (Quantizer * Scale[LOD_ID]),

[00356] where QuantizedDisplacement is a quantized version of a generated displacement vector, Displacement is a value of a generated displacement vector, Quantizer is defined via encoder parameters, offset is the offset that is computed per level of detail and represents offset for all coefficients in the corresponding level of detail, the corresponding level of detail having an identification of LOD ID, and Scale is the scale factor that is a single multiplier for all values in a certain wavelet band.

[00357] Example 10. The method according to example 9, wherein the Quantizer is determined via a set of lifting quantization parameters.

[00358] Example 11. The method according to any one of examples 1 to 10, wherein signaling the quantized generated displacement vectors information comprises signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in or along a bitstream signaled for the video-based dynamic mesh coding. [00359] Example 12. The method according to any one of examples 1 to 10, wherein signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

[00360] Example 13. The method according to any one of examples 2 to 12, wherein one or both of the offset or the scale factor are set differently per level of detail.

[00361] Example 14. A method, comprising:

[00362] as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following:

[00363] receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream;

[00364] unpacking the displacement quantization information received in the bitstream;

[00365] decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and

[00366] using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame.

[00367] Example 15. The method of example 14, decoding displacement vector components based on unpacked displacement quantization information for a received displacement component video frame comprises:

[00368] obtaining an array indexed by vertex indices, by decoding and unpacking the received displacement component video frame;

[00369] deriving a number of vertices of individual levels of detail;

[00370] for the number of vertices of the levels of detail in the array, determining the displacement vector components based on corresponding one or both of scale factors or offsets for vertices. [00371] Example 16. The method according to example 15, wherein the offset is computed separately for positive and negative values of displacement.

[00372] Example 17. The method according to example 16, wherein receiving at least displacement quantization information further comprises receiving an indication of adaptive signed offset indicating the offset is to be computed separately for positive and negative values of displacement.

[00373] Example 18. The method according to any one of examples 15 to 17, wherein the offset is quantized with higher accuracy than other displacement values, and the method comprises receiving signaling indicating the offset is quantized with higher accuracy than other displacement values, and wherein decoding displacement vector components applies the higher accuracy of quantization to the offset.

[00374] Example 19. The method according to any one of examples 15 to 18, wherein the scale factor is computed based on magnitude of displacement vectors between a current and a previous level of detail.

[00375] Example 20. The method according to any one of examples 15 to 18, wherein the scale factor is computed by taking into account values of the offset.

[00376] Example 21. The method according to any one of examples 15 to 20, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors using the following equation:

[00377] Displacement = function (quantizedDisplacement, offset[LOD_ID], Quantizer, Scale[LOD_ID]),

[00378] where QuantizedDisplacement is a quantized version of a generated displacement vector, Displacement is a value of a generated displacement vector, Quantizer is defined via encoder parameters, offset is the offset that is computed per level of detail and represents offset for all coefficients in the corresponding level of detail, the corresponding level of detail having an identification of LOD ID, and Scale is the scaling factor that is a single multiplier for all values in a certain wavelet band.

[00379] Example 22. The method according to example 21, wherein the function (•) is the following: quantizedDisplacement * Scale[LOD_ID] * Quantizer + Offset[LOD_ID], [00380] Example 23. The method according to example 21 or 22, wherein the Quantizer is determined via a set of lifting quantization parameters.

[00381] Example 24. The method according to any one of examples 14 to 23, wherein receiving the displacement quantization information comprises receiving the displacement quantization information in or along the bitstream signaled for the video-based dynamic mesh coding.

[00382] Example 25. The method according to any one of examples 14 to 23, wherein receiving at least displacement adaptive quantization information comprises receiving the displacement adaptive quantization information in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

[00383] Example 26. The method according to any one of examples 15 to 25, wherein one or both of the offset or the scale factor are set differently per level of detail.

[00384] Example 27. The method according to any one of examples 14 to 26, wherein the decoding displacement vector components and using the decoded displacement vector components are performed for first, second, and third axes.

[00385] Example 28. A computer program, comprising instructions for performing the methods of any of examples 1 to 27, when the computer program is run on an apparatus.

[00386] Example 29. The computer program according to example 28, wherein the computer program is a computer program product comprising a computer-readable medium bearing instructions embodied therein for use with the apparatus.

[00387] Example 30. The computer program according to example 28, wherein the computer program is directly loadable into an internal memory of the apparatus.

[00388] Example 31. n apparatus, comprising means for performing:

[00389] as part of an encoding process of video using video-based dynamic mesh coding, performing the following:

[00390] setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video;

[00391] computing displacement vectors for each level of detail defined by the subdivision iteration count; [00392] generating such displacement vectors for a plurality of input mesh frames;

[00393] quantizing the generated displacement vectors;

[00394] signaling at least the quantization information used to generate the quantized displacement vectors;

[00395] packing the quantized displacement vectors in one or more displacement video frames; and

[00396] coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the video-based dynamic mesh coding.

[00397] Example 32. The apparatus according to example 31, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors by performing one or both of shifting the generated displacement vectors by an offset or scaling the generated displacement vectors by a scale factor.

[00398] Example 33. The apparatus according to example 32, wherein, for individual ones of the levels of detail, the offset is computed via one of the following:

[00399] as a most frequent value of all displacement values in a corresponding level of detail;

[00400] as a mean value of all displacement values in a corresponding level of detail; or

[00401] as a median value of all displacement values in a corresponding level of detail.

[00402] Example 34. The apparatus according to any one of examples 32 to 33, wherein the offset is computed separately for positive and negative values of displacement.

[00403] Example 35. The apparatus according to example 34, wherein signaling at least the quantized generated displacement vectors further comprises signaling an indication of adaptive signed offset indicating the offset is to be computed separately for positive and negative values of displacement.

[00404] Example 36. The apparatus according to any one of examples 32 to 35, wherein the offset is quantized with higher accuracy than other displacement values, and signaling comprises sending signaling indicating the offset is quantized with higher accuracy than other displacement values. [00405] Example 37. The apparatus according to any one of examples 32 to 36, wherein the scale factor is computed based on magnitude of displacement vectors between a current and a previous level of detail.

[00406] Example 38. The apparatus according to any one of examples 32 to 36, wherein the scale factor is computed by taking into account values of the offset.

[00407] Example 39. The apparatus according to any one of examples 32 to 38, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors using the following equation:

[00408] QuantizedDisplacement = (Displacement - Offset[LOD_ID]) / (Quantizer * Scale[LOD_ID]),

[00409] where QuantizedDisplacement is a quantized version of a generated displacement vector, Displacement is a value of a generated displacement vector, Quantizer is defined via encoder parameters, offset is the offset that is computed per level of detail and represents offset for all coefficients in the corresponding level of detail, the corresponding level of detail having an identification of LOD ID, and Scale is the scale factor that is a single multiplier for all values in a certain wavelet band.

[00410] Example 40. The apparatus according to example 39, wherein the Quantizer is determined via a set of lifting quantization parameters.

[00411] Example 41. The apparatus according to any one of examples 31 to 40, wherein signaling the quantized generated displacement vectors information comprises signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in or along a bitstream signaled for the video-based dynamic mesh coding.

[00412] Example 42. The apparatus according to any one of examples 31 to 40, wherein signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

[00413] Example 43. The apparatus according to any one of examples 32 to 43, wherein one or both of the offset or the scale factor are set differently per level of detail.

[00414] Example 44. An apparatus, comprising means for performing: [00415] as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following:

[00416] receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream;

[00417] unpacking the displacement quantization information received in the bitstream;

[00418] decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and

[00419] using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame.

[00420] Example 45. The apparatus of example 44, decoding displacement vector components based on unpacked displacement quantization information for a received displacement component video frame comprises:

[00421] obtaining an array indexed by vertex indices, by decoding and unpacking the received displacement component video frame;

[00422] deriving a number of vertices of individual levels of detail;

[00423] for the number of vertices of the levels of detail in the array, determining the displacement vector components based on corresponding one or both of scale factors or offsets for vertices.

[00424] Example 46. The apparatus according to example 45, wherein the offset is computed separately for positive and negative values of displacement.

[00425] Example 47. The apparatus according to example 46, wherein receiving at least displacement quantization information further comprises receiving an indication of adaptive signed offset indicating the offset is to be computed separately for positive and negative values of displacement. [00426] Example 48. The apparatus according to any one of examples 45 to 47, wherein the offset is quantized with higher accuracy than other displacement values, and the apparatus comprises receiving signaling indicating the offset is quantized with higher accuracy than other displacement values, and wherein decoding displacement vector components applies the higher accuracy of quantization to the offset.

[00427] Example 49. The apparatus according to any one of examples 45 to 48, wherein the scale factor is computed based on magnitude of displacement vectors between a current and a previous level of detail.

[00428] Example 50. The apparatus according to any one of examples 45 to 48, wherein the scale factor is computed by taking into account values of the offset.

[00429] Example 51. The apparatus according to any one of examples 45 to 50, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors using the following equation:

[00430] Displacement = function (quantizedDisplacement, offset[LOD_ID], Quantizer, Scale[LOD_ID]),

[00431] where QuantizedDisplacement is a quantized version of a generated displacement vector, Displacement is a value of a generated displacement vector, Quantizer is defined via encoder parameters, offset is the offset that is computed per level of detail and represents offset for all coefficients in the corresponding level of detail, the corresponding level of detail having an identification of LOD ID, and Scale is the scaling factor that is a single multiplier for all values in a certain wavelet band.

[00432] Example 52. The apparatus according to example 51, wherein the function ( • ) is the following: quantizedDisplacement * Scale[LOD_ID] * Quantizer + Offset[LOD_ID],

[00433] Example 53. The apparatus according to example 51 or 52, wherein the Quantizer is determined via a set of lifting quantization parameters.

[00434] Example 54. The apparatus according to any one of examples 44 to 53, wherein receiving the displacement quantization information comprises receiving the displacement quantization information in or along the bitstream signaled for the video-based dynamic mesh coding. [00435] Example 55. The apparatus according to any one of examples 44 to 53, wherein receiving at least displacement adaptive quantization information comprises receiving the displacement adaptive quantization information in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

[00436] Example 56. The apparatus according to any one of examples 45 to 55, wherein one or both of the offset or the scale factor are set differently per level of detail.

[00437] Example 57. The apparatus according to any one of examples 44 to 56, wherein the decoding displacement vector components and using the decoded displacement vector components are performed for first, second, and third axes.

[00438] Example 58. The apparatus according to any preceding apparatus example, wherein the means comprises:

[00439] at least one processor; and

[00440] at least one memory storing instructions that, when executed by at least one processor, cause the performance of the apparatus.

[00441] Example 59. An apparatus, comprising:

[00442] one or more processors; and

[00443] one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform:

[00444] as part of an encoding process of video using video-based dynamic mesh coding, performing the following:

[00445] setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video;

[00446] computing displacement vectors for each level of detail defined by the subdivision iteration count;

[00447] generating such displacement vectors for a plurality of input mesh frames;

[00448] quantizing the generated displacement vectors;

[00449] signaling at least the quantization information used to generate the quantized displacement vectors; [00450] packing the quantized displacement vectors in one or more displacement video frames; and

[00451] coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the video-based dynamic mesh coding.

[00452] Example 60. The apparatus according to example 59, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors by performing one or both of shifting the generated displacement vectors by an offset or scaling the generated displacement vectors by a scale factor.

[00453] Example 61. The apparatus according to example 60, wherein, for individual ones of the levels of detail, the offset is computed via one of the following:

[00454] as a most frequent value of all displacement values in a corresponding level of detail;

[00455] as a mean value of all displacement values in a corresponding level of detail; or

[00456] as a median value of all displacement values in a corresponding level of detail.

[00457] Example 62. The apparatus according to any one of examples 60 to 61, wherein the offset is computed separately for positive and negative values of displacement.

[00458] Example 63. The apparatus according to example 62, wherein signaling at least the quantized generated displacement vectors further comprises signaling an indication of adaptive signed offset indicating the offset is to be computed separately for positive and negative values of displacement.

[00459] Example 64. The apparatus according to any one of examples 60 to 63, wherein the offset is quantized with higher accuracy than other displacement values, and signaling comprises sending signaling indicating the offset is quantized with higher accuracy than other displacement values.

[00460] Example 65. The apparatus according to any one of examples 60 to 64, wherein the scale factor is computed based on magnitude of displacement vectors between a current and a previous level of detail. [00461] Example 66. The apparatus according to any one of examples 60 to 64, wherein the scale factor is computed by taking into account values of the offset.

[00462] Example 67. The apparatus according to any one of examples 60 to 66, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors using the following equation:

[00463] QuantizedDisplacement = (Displacement - Offset[LOD_ID]) / (Quantizer * Scale[LOD_ID]),

[00464] where QuantizedDisplacement is a quantized version of a generated displacement vector, Displacement is a value of a generated displacement vector, Quantizer is defined via encoder parameters, offset is the offset that is computed per level of detail and represents offset for all coefficients in the corresponding level of detail, the corresponding level of detail having an identification of LOD ID, and Scale is the scale factor that is a single multiplier for all values in a certain wavelet band.

[00465] Example 68. The apparatus according to example 67, wherein the Quantizer is determined via a set of lifting quantization parameters.

[00466] Example 69. The apparatus according to any one of examples 59 to 68, wherein signaling the quantized generated displacement vectors information comprises signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in or along a bitstream signaled for the video-based dynamic mesh coding.

[00467] Example 70. The apparatus according to any one of examples 59 to 68, wherein signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

[00468] Example 71. The apparatus according to any one of examples 60 to 70, wherein one or both of the offset or the scale factor are set differently per level of detail.

[00469] Example 72. An apparatus, comprising:

[00470] as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following: [00471] receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream;

[00472] unpacking the displacement quantization information received in the bitstream;

[00473] decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and

[00474] using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame.

[00475] Example 73. The apparatus according to example 72, decoding displacement vector components based on unpacked displacement quantization information for a received displacement component video frame comprises:

[00476] obtaining an array indexed by vertex indices, by decoding and unpacking the received displacement component video frame;

[00477] deriving a number of vertices of individual levels of detail;

[00478] for the number of vertices of the levels of detail in the array, determining the displacement vector components based on corresponding one or both of scale factors or offsets for vertices.

[00479] Example 74. The apparatus according to example 73, wherein the offset is computed separately for positive and negative values of displacement.

[00480] Example 75. The apparatus according to example 74, wherein receiving at least displacement quantization information further comprises receiving an indication of adaptive signed offset indicating the offset is to be computed separately for positive and negative values of displacement.

[00481] Example 76. The apparatus according to any one of examples 73 to 75, wherein the offset is quantized with higher accuracy than other displacement values, and the apparatus comprises receiving signaling indicating the offset is quantized with higher accuracy than other displacement values, and wherein decoding displacement vector components applies the higher accuracy of quantization to the offset.

[00482] Example 77. The apparatus according to any one of examples 73 to 76, wherein the scale factor is computed based on magnitude of displacement vectors between a current and a previous level of detail.

[00483] Example 78. The apparatus according to any one of examples 73 to 76, wherein the scale factor is computed by taking into account values of the offset.

[00484] Example 79. The apparatus of any one of examples 73 to 78, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors using the following equation:

[00485] Displacement = function (quantizedDisplacement, offset[LOD_ID], Quantizer, Scale[LOD_ID]),

[00486] where QuantizedDisplacement is a quantized version of a generated displacement vector, Displacement is a value of a generated displacement vector, Quantizer is defined via encoder parameters, offset is the offset that is computed per level of detail and represents offset for all coefficients in the corresponding level of detail, the corresponding level of detail having an identification of LOD ID, and Scale is the scaling factor that is a single multiplier for all values in a certain wavelet band.

[00487] Example 80. The apparatus according to example 79, wherein the function ( • ) is the following: quantizedDisplacement * Scale[LOD_ID] * Quantizer + Offset[LOD_ID],

[00488] Example 81. The apparatus according to example 79 or80, wherein the Quantizer is determined via a set of lifting quantization parameters.

[00489] Example 82. The apparatus according to any one of examples 72 to 81, wherein receiving the displacement quantization information comprises receiving the displacement quantization information in or along the bitstream signaled for the video-based dynamic mesh coding.

[00490] Example 83. The apparatus according to any one of examples 72 to 81, wherein receiving at least displacement adaptive quantization information comprises receiving the displacement adaptive quantization information in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

[00491] Example 84. The apparatus according to any one of examples 73 to 83, wherein one or both of the offset or the scale factor are set differently per level of detail.

[00492] Example 85. The apparatus according to any one of examples 72 to 84, wherein the decoding displacement vector components and using the decoded displacement vector components are performed for first, second, and third axes.

[00493] As used in this application, the term “circuitry” may refer to one or more or all of the following:

[00494] (a) hardware-only circuit implementations (such as implementations in only analog and/or digital circuitry) and

[00495] (b) combinations of hardware circuits and software, such as (as applicable): (i) a combination of analog and/or digital hardware circuit(s) with software/firmware and (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and

[00496] (c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[00497] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and/or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[00498] Embodiments herein may be implemented in software (executed by one or more processors), hardware (e.g., an application specific integrated circuit), or a combination of software and hardware. In an example embodiment, the software (e.g., application logic, an instruction set) is maintained on any one of various conventional computer-readable media. In the context of this document, a “computer-readable medium” may be any media or means that can contain, store, communicate, propagate or transport the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer, with one example of a computer described and depicted, e.g., in FIG. 25. A computer-readable medium may comprise a computer-readable storage medium (e.g., memories 125 or other device) that may be any media or means that can contain, store, and/or transport the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer. A computer-readable storage medium does not comprise propagating signals, and therefore may be considered to be non-transitory. The term “non-transitory”, as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM, random access memory, versus ROM, read-only memory).

[00499] If desired, the different functions discussed herein may be performed in a different order and/or concurrently with each other. Furthermore, if desired, one or more of the above-described functions may be optional or may be combined.

[00500] Although various aspects of the invention are set out in the independent claims, other aspects of the invention comprise other combinations of features from the described embodiments and/or the dependent claims with the features of the independent claims, and not solely the combinations explicitly set out in the claims.

[00501] It is also noted herein that while the above describes example embodiments of the invention, these descriptions should not be viewed in a limiting sense. Rather, there are several variations and modifications which may be made without departing from the scope of the present invention as defined in the appended claims.

[00502] The following abbreviations that may be found in the specification and/or the drawing figures are defined as follows:

[00503] 2D two dimensional

[00504] 3D three dimensional

[00505] 6DOF six degrees of freedom [00506] ACL access coding layer

[00507] ASPS or asps atlas sequence parameter set

[00508] AR augmented reality

[00509] CfP call for proposal

[00510] CGI computer-generated imagery

[00511] HEVC high efficiency video coding

[00512] ID identification

[00513] LOD level of detail

[00514] LODs levels of detail

[00515] NAL network abstraction layer

[00516] MCL mesh coding layer

[00517] MIV MPEG (Motion Pictures Experts Group) immersive video

[00518] MR mixed reality

[00519] QP quantization parameter

[00520] RBSP raw byte sequence payload

[00521] ROI region of interest

[00522] SEI supplemental enhancement information

[00523] SODB string of data bits

[00524] V3C Visual Volumetric Video-based Coding

[00525] V-DMC video-based dynamic mesh coding

[00526] VPS V3C parameter set

[00527] VR virtual reality

Claims

What is claimed is:

1. A method, comprising: as part of an encoding process of video using video-based dynamic mesh coding, performing the following: setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video; computing displacement vectors for individual levels of detail defined by the target subdivision iteration count; generating such displacement vectors for a plurality of input mesh frames; quantizing the generated displacement vectors to form quantized displacement vectors; signaling at least quantization information used to generate the quantized displacement vectors; packing the quantized displacement vectors in one or more displacement video frames; and coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the video-based dynamic mesh coding.

2. A method, comprising: as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following: receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream; unpacking the displacement quantization information received in the bitstream; decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame.

3. An apparatus, comprising means for performing: as part of an encoding process of video using video-based dynamic mesh coding, performing the following: setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video; computing displacement vectors for individual levels of detail defined by the target subdivision iteration count; generating such displacement vectors for a plurality of input mesh frames; quantizing the generated displacement vectors to form quantized displacement vectors; signaling at least quantization information used to generate the quantized displacement vectors; packing the quantized displacement vectors in one or more displacement video frames; and coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the video-based dynamic mesh coding.

4. The apparatus according to claim 3, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors by performing one or both of shifting the generated displacement vectors by an offset or scaling the generated displacement vectors by a scale factor.

5. The apparatus according to claim 4, wherein the offset is computed separately for positive and negative values of displacement, and wherein signaling at least the quantized generated displacement vectors further comprises signaling an indication of adaptive signed offset indicating the offset is to be computed separately for positive and negative values of displacement.

6. The apparatus according to any one of claims 4 to 5, wherein the offset is quantized with higher accuracy than other displacement values, and signaling comprises sending signaling indicating the offset is quantized with higher accuracy than other displacement values.

7. The apparatus according to any one of claims 4 to 6, wherein the scale factor is computed based on one of the following: magnitude of displacement vectors between a current and a previous level of detail; or by taking into account values of the offset.

8. The apparatus according to any one of claims 3 to 7, wherein signaling the quantization information used to generate the quantized generated displacement vectors comprises performing one of the following: signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in or along a bitstream signaled for the video-based dynamic mesh coding; or signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

9. The apparatus according to any one of claims 4 to 8, wherein one or both of the offset or the scale factor are set differently per level of detail.

10. An apparatus, comprising means for performing: as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following: receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream; unpacking the displacement quantization information received in the bitstream; decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame.

11. The apparatus according to claim 10, wherein decoding displacement vector components based on unpacked displacement quantization information for a received displacement component video frame comprises: obtaining an array indexed by vertex indices, by decoding and unpacking the received displacement component video frame; deriving a number of vertices of individual levels of detail; for the number of vertices of the levels of detail in the array, determining the displacement vector components based on corresponding one or both of scale factors or offsets for vertices.

12. The apparatus according to claim 11, wherein individual offsets are computed separately for positive and negative values of displacement, and wherein receiving at least displacement quantization information further comprises receiving an indication of adaptive signed offset indicating the adaptive signed offset is to be computed separately for positive and negative values of displacement.

13. The apparatus according to any one of claims 11 to 12, wherein the individual offsets are quantized with higher accuracy than other displacement values, and the apparatus comprises receiving signaling indicating the offsets are quantized with higher accuracy than other displacement values, and wherein decoding displacement vector components applies the higher accuracy of quantization to the offsets.

14. The apparatus according to any one of claims 11 to 13, wherein an individual scale factor is computed based on one of the following: magnitude of displacement vectors between a current and a previous level of detail; or by taking into account values of the offsets.

15. The apparatus according to any one of claims 10 to 14, wherein receiving the displacement quantization information comprises performing one of the following: receiving the displacement quantization information in or along the bitstream signaled for the video-based dynamic mesh coding; or receiving the displacement adaptive quantization information in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

16. The apparatus according to any one of claims 10 to 15, wherein the decoding displacement vector components and using the decoded displacement vector components are performed for first, second, and third axes.

17. An apparatus, comprising: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform: as part of an encoding process of video using video-based dynamic mesh coding, performing the following: setting a target subdivision iteration count to reach a comparable target face count as in an original mesh frame corresponding to the dynamic mesh coding of the video; computing displacement vectors for individual levels of detail defined by the target subdivision iteration count; generating such displacement vectors for a plurality of input mesh frames; quantizing the generated displacement vectors to form quantized displacement vectors; signaling at least quantization information used to generate the quantized displacement vectors; packing the quantized displacement vectors in one or more displacement video frames; and coding the one or more displacement video frames as a displacement bitstream that is part of bitstream for the video-based dynamic mesh coding.

18. The apparatus according to claim 17, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors by performing one or both of shifting the generated displacement vectors by an offset or scaling the generated displacement vectors by a scale factor.

19. The apparatus according to claim 18, wherein the offset is computed separately for positive and negative values of displacement, and wherein signaling at least the quantized generated displacement vectors further comprises signaling an indication of adaptive signed offset indicating the offset is to be computed separately for positive and negative values of displacement.

20. The apparatus according to claim 18, wherein the offset is quantized with higher accuracy than other displacement values, and signaling comprises sending signaling indicating the offset is quantized with higher accuracy than other displacement values.

21. The apparatus according to claim 18, wherein the scale factor is computed based on one of the following: magnitude of displacement vectors between a current and a previous level of detail; or by taking into account values of the offset.

22. The apparatus according to claim 18, wherein quantizing the generated displacement vectors quantizes the generated displacement vectors using the following equation: Displacement = function (quantizedDisplacement, offset[LOD_ID], Quantizer,

Scale[LOD_ID]), where QuantizedDisplacement is a quantized version of a generated displacement vector, Displacement is a value of a generated displacement vector, Quantizer is defined via encoder parameters, offset is the offset that is computed per level of detail and represents offset for all coefficients in the corresponding level of detail, the corresponding level of detail having an identification of LOD ID, and Scale is the scaling factor that is a single multiplier for all values in a certain wavelet band.

23. The apparatus according to claim 17, wherein signaling the quantization information used to generate the quantized generated displacement vectors comprises signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in or along a bitstream signaled for the video-based dynamic mesh coding.

24. The apparatus according to claim 17, wherein signaling one or both of an adaptive quantization offset or adaptive quantization scale factor in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

25. The apparatus according to claim 18, wherein one or both of the offset or the scale factor are set differently per level of detail.

26. An apparatus, comprising: as part of a decoding process of a bitstream that used video-based dynamic mesh coding to encode video, performing the following: receiving at least displacement quantization information signaled in the bitstream and one or more displacement video frames in a displacement bitstream that is part of the bitstream; unpacking the displacement quantization information received in the bitstream; decoding displacement vector components from one or more displacement video frames in the displacement bitstream based at least on unpacked displacement quantization information for a received displacement component video frame, wherein the decoded displacement vector components comprise components for at least a first axis; and using the decoded displacement vector components to form reconstructed data for levels of detail in the received displacement component video frame.

27. The apparatus according to claim 26, decoding displacement vector components based on unpacked displacement quantization information for a received displacement component video frame comprises: obtaining an array indexed by vertex indices, by decoding and unpacking the received displacement component video frame; deriving a number of vertices of individual levels of detail; for the number of vertices of the levels of detail in the array, determining the displacement vector components based on corresponding one or both of scale factors or offsets for vertices.

28. The apparatus according to claim 27, wherein individual offsets are computed separately for positive and negative values of displacement, and receiving at least displacement quantization information further comprises receiving an indication of adaptive signed offset indicating the adaptive signed offset is to be computed separately for positive and negative values of displacement.

29. The apparatus according to claim 27, wherein the individual offsets are quantized with higher accuracy than other displacement values, and the apparatus comprises receiving signaling indicating the offsets are quantized with higher accuracy than other displacement values, and wherein decoding displacement vector components applies the higher accuracy of quantization to the offsets.

30. The apparatus according to claim 27, wherein an individual scale factor is computed based on magnitude of displacement vectors between a current and a previous level of detail.

31. The apparatus according to claim 27, wherein an individual scale factor is computed by taking into account values of the offsets.

32. The apparatus according to claim 26, wherein receiving the displacement quantization information comprises receiving the displacement quantization information in or along the bitstream signaled for the video-based dynamic mesh coding.

33. The apparatus according to claim 26, wherein receiving at least displacement adaptive quantization information comprises receiving the displacement adaptive quantization information in an atlas sequence parameter set and in lifting transform parameters for the video-based dynamic mesh coding.

34. The apparatus according to claim 26, wherein the decoding displacement vector components and using the decoded displacement vector components are performed for first, second, and third axes.