WO2023180843A1 - Génération de correctif pour codage maillé dynamique - Google Patents

Génération de correctif pour codage maillé dynamique Download PDF

Info

Publication number
WO2023180843A1
WO2023180843A1 PCT/IB2023/052106 IB2023052106W WO2023180843A1 WO 2023180843 A1 WO2023180843 A1 WO 2023180843A1 IB 2023052106 W IB2023052106 W IB 2023052106W WO 2023180843 A1 WO2023180843 A1 WO 2023180843A1
Authority
WO
WIPO (PCT)
Prior art keywords
triangle
triangles
checking
points
depth
Prior art date
Application number
PCT/IB2023/052106
Other languages
English (en)
Inventor
Danillo GRAZIOSI
Alexandre ZAGHETTO
Ali Tabatabai
Original Assignee
Sony Group Corporation
Sony Corporation Of America
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/987,846 external-priority patent/US20230306684A1/en
Application filed by Sony Group Corporation, Sony Corporation Of America filed Critical Sony Group Corporation
Priority to CN202380013355.1A priority Critical patent/CN117897957A/zh
Publication of WO2023180843A1 publication Critical patent/WO2023180843A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

  • the present invention relates to three dimensional graphics. More specifically, the present invention relates to coding of three dimensional graphics.
  • volumetric content such as point clouds
  • V3C visual volumetric video-based compression
  • MPEG had issued a call for proposal (CfP) for compression of point clouds.
  • CfP call for proposal
  • MPEG is considering two different technologies for point cloud compression: 3D native coding technology (based on octree and similar coding methods), or 3D to 2D projection, followed by traditional video coding.
  • 3D native coding technology based on octree and similar coding methods
  • 3D to 2D projection followed by traditional video coding.
  • TMC2 test model software
  • This method has proven to be more efficient than native 3D coding, and is able to achieve competitive bitrates at acceptable quality.
  • 3D point clouds of the projection-based method also known as the video-based method, or V-PCC
  • the standard is expected to include in future versions further 3D data, such as 3D meshes.
  • current version of the standard is only suitable for the transmission of an unconnected set of points, so there is nomechanism to send the connectivity of points, as it is required in 3D mesh compression.
  • V-PCC V-PCC
  • a mesh compression approach like TFAN or Edgebreaker.
  • the limitation of this method is that the original mesh has to be dense, so that the point cloud generated from the vertices is not sparse and can be efficiently encoded after projection.
  • the order of the vertices affect the coding of connectivity, and different method to reorganize the mesh connectivity have been proposed.
  • An alternative way to encode a sparse mesh is to use the RAW patch data to encode the vertices position in 3D.
  • RAW patches encode (x,y,z) directly
  • all the vertices are encoded as RAW data
  • the connectivity is encoded by a similar mesh compression method, as mentioned before.
  • the vertices may be sent in any preferred order, so the order generated from connectivity encoding can be used.
  • the method can encode sparse point clouds, however, RAW patches are not efficient to encode 3D data, and further data such as the attributes of the triangle faces may be missing from this approach.
  • An architecture includes new blocks that transform mesh connectivity to enable lossy compression.
  • a method generates surface patches from meshes and preserves the overall aspect of the object, while generating patches for efficient encoding using the V3C standard.
  • Triangles are joined together into connected components, and conditional projection of each triangle to the surface is performed to enforce connectivity at the projected surface.
  • a new depth filtering algorithm is able to be used to improve compression performance.
  • the new blocks added for connectivity compression and the techniques introduced to ensure 2D connectivity to allow for UV texture map transmission and to filter depth values and avoid high- frequency edges in the depth image are described herein.
  • a method of patch generation comprises computing face normals of triangles, computing face adjacency of the triangles, computing face neighbors of the triangles, implementing initial segmentation including classifying the normals of the triangles, performing refinement including locating and smoothing outliers in the classified normals and performing patch segmentation of the triangles.
  • Computing face adjacency of the triangles and computing face neighbors of the triangles includes generating lists of adjacent triangles and neighboring triangles. Generating lists of the adjacent triangles and the neighboring triangles includes factoring in a texture map boundary. Factoring in the texture map boundary involves excluding a triangle that is on an opposite side of the boundary of a current triangle.
  • Performing patch segmentation comprises: removing degenerates, checking for vertex occlusion, checking for half-edges, checking for surface occlusion, depth filtering, depth range and connectivity analysis, and rasterizing a triangle.
  • Removing degenerate triangles includes removing triangles that are lines or points by determining an area between the points is smaller than a threshold
  • checking for vertex occlusion includes determining whether a vertex of the triangle is occluded by an already projected triangle
  • checking for half-edges includes determining if the triangle shares an edge that is already shared by two projected triangles
  • checking for surface occlusion includes determining if an area between points of the triangle are occluded or would occlude the already projected triangle
  • depth filtering determines depth differentials of neighboring points and points of the triangle are smaller than a depth threshold
  • depth range and connectivity analysis includes determining whether the triangle touches at least one edge or vertex of the already projected triangles and projecting and rasterizing the triangle.
  • an apparatus comprises a non-transitory memory for storing an application, the application for: computing face normals of triangles, computing face adjacency of the triangles, computing face neighbors of the triangles, implementing initial segmentation including classifying the normals of the triangles, performing refinement including locating and smoothing outliers in the classified normals and performing patch segmentation of the triangles and a processor coupled to the memory, the processor configured for processing the application.
  • Computing face adjacency of the triangles and computing face neighbors of the triangles includes generating lists of adjacent triangles and neighboring triangles. Generating lists of the adjacent triangles and the neighboring triangles includes factoring in a texture map boundary.
  • Performing patch segmentation comprises: removing degenerates, checking for vertex occlusion, checking for half-edges, checking for surface occlusion, depth filtering, depth range and connectivity analysis, and rasterizing a triangle.
  • Removing degenerate triangles includes removing triangles that are lines or points by determining an area between the points is smaller than a threshold
  • checking for vertex occlusion includes determining whether a vertex of the triangle is occluded by an already projected triangle
  • checking for half-edges includes determining if the triangle shares an edge that is already shared by two projected triangles
  • checking for surface occlusion includes determining if an area between points of the triangle are occluded or would occlude the already projected triangle
  • depth filtering determines depth differentials of neighboring points and points of the triangle are smaller than a depth threshold
  • depth range and connectivity analysis includes determining whether the triangle touches at least one edge or vertex of the already projected triangles and projecting and rasterizing the triangle. The triangle is added to a missing triangles list when a criterion is not met.
  • a method of patch segmentation comprises removing degenerate information, checking for vertex occlusion of a triangle, checking for half-edges of the triangle, checking for surface occlusion of the triangle, depth filtering of the triangle, performing depth range and connectivity analysis of the triangle and projecting and rasterizing the triangle onto a canvas.
  • Removing degenerate triangles includes removing triangles that are lines or points by determining an area between the points is smaller than a threshold.
  • Checking for vertex occlusion includes determining whether a vertex of the triangle is occluded by an already projected triangle.
  • Checking for half-edges includes determining if the triangle shares an edge that is already shared by two projected triangles.
  • Checking for surface occlusion includes determining if an area between points of the triangle are occluded or would occlude the already projected triangle.
  • Depth filtering determines depth differentials of neighboring points and points of the triangle are smaller than a depth threshold.
  • Depth range and connectivity analysis includes determining whether the triangle touches at least one edge or vertex of the already projected triangles. The triangle is added to a missing triangles list when a criterion is not met.
  • Figure 1 illustrates a flowchart of a method of implementing dynamic mesh coding according to some embodiments.
  • Figure 2 illustrates a flowchart of a method of implementing patch generation according to some embodiments.
  • Figure 3 illustrates a diagram of computing a normal according to some embodiments.
  • Figure 4 illustrates a diagram of face adjacency and face neighbor analysis according to some embodiments.
  • Figure 5 illustrates a diagram of patch generation according to some embodiments.
  • Figure 6 illustrates a flowchart and diagram of patch segmentation using conditional rasterization according to some embodiments.
  • Figure 7 illustrates a diagram of depth filtering according to some embodiments.
  • Figure 8 illustrates a block diagram of an exemplary computing device configured to implement the patch generation for dynamic mesh coding method according to some embodiments.
  • An architecture includes new blocks that transform mesh connectivity to enable lossy compression.
  • a method generates surface patches from meshes and preserves the overall aspect of the object, while generating patches for efficient encoding using the V3C standard.
  • Triangles are joined together into connected components, and conditional projection of each triangle to the surface is performed to enforce connectivity at the projected surface.
  • a new depth filtering algorithm is able to be used to improve compression performance.
  • the new blocks added for connectivity compression and the techniques introduced to ensure 2D connectivity to allow for UV texture map transmission and to filter depth values and avoid high- frequency edges in the depth image are described herein.
  • the architecture described herein is related to U.S. Patent Application Serial No.
  • Figure 1 illustrates a flowchart of a method of implementing dynamic mesh coding according to some embodiments.
  • An input mesh 100 is received or acquired.
  • the input mesh is downloaded (e.g., from a network device) or acquired/captured by a device (e.g., camera or autonomous vehicle).
  • a device e.g., camera or autonomous vehicle.
  • Mesh voxelization 102 is performed.
  • the mesh is able to have positions of vertices in floating point, so these positions are converted to the integer space.
  • V-PCC and V3C assume a voxelized point cloud.
  • 3D mesh simplification 104 is performed. 3D mesh simplification 104 is done before patch generation. 3D mesh simplification 104 reduces the number of triangles for projection, generates patches with low resolution information at the boundaries, and is able to recover the high-resolution information from geometry images, but boundaries are still low resolution.
  • Patch generation 106 includes: normal calculation 150, adjacency calculation 152, neighbor calculation 154, initial segmentation 156, refinement 158, and patch segmentation 160.
  • Normal calculation 150 is calculating the normals of each triangle (e.g., cross product of the triangle’s edges).
  • Adjacency calculation 152 involves calculating each triangle’s adjacency (e.g., which triangles in the mesh neighbor or touch the current triangle or other triangles).
  • Neighbor calculation 154 includes calculating neighboring triangles.
  • Initial segmentation 156 includes classifying the normal according to the orientation.
  • a normal of a triangle is able to point up, down, left, right, front, or back, and is able to be classified based on the directi on/orientation.
  • the triangles are color-coded based on the orientation of their normals (e.g., all of the triangles with a normal pointing up are colored green).
  • Refinement 158 involves locating outliers (e.g., a single red triangle surrounded by blue triangles) and smoothing out the outliers (e.g., changing the single red triangle to match its neighbors which are blue). The refinement 158 is performed by analyzing the neighbors and smoothing the orientation (e.g., adjusting the orientation of the normal).
  • patch segmentation 160 is performed which involves segmenting the patches for a specific classification of triangles (e.g., based on orientation). Segmentation is able to include patch projection. With the segmentation, the vertices and connectivity are shown on the patches. For example, the body and face in this example are separate segmentations since there are triangles with different classifications separating the two.
  • the segmentation is rasterized (e.g., sampling points on the surface including the distance of the points to generate a geometry image and the attributes of the surface). The rasterized mesh surface is very similar to a V3C image.
  • Patch generation results in segmented patches or a rasterized mesh surface and vertices locations and connectivity.
  • the segmented patches or rasterized mesh surface are utilized in patch packing, in the step 108.
  • Patch packing 108 is able to include packing the patches in an atlas or 2D canvas.
  • 2D mesh simplification 114 is performed after patch generation. 2D mesh simplification 114 reduces only the triangles inside a patch (boundary triangles are not reduced), generates patches with high-resolution information at the boundaries, and is able to recover high-resolution information from geometry images, including the boundary region, which was preserved in the patch.
  • texture image generation 110 From texture image generation 110 is image padding 116 and then the padded image goes to a video codec 122 for encoding.
  • a texture image stores the texture information.
  • Image padding 116 provides spacing between patches.
  • the video codec 122 is able to perform any appropriate encoding scheme.
  • image padding 118 From geometry image generation 112 is image padding 118 and then the padded image goes to a video codec 124 for encoding.
  • a geometry image stores the geometry information.
  • Image padding 118 provides spacing between patches.
  • the video codec 124 is able to perform any appropriate encoding scheme.
  • connectivity encoding 120 From 2D mesh simplification 114 is connectivity encoding 120 and then connectivity information goes to mesh zippering 126 and an atlas codec 128 for encoding.
  • the mesh zippering 126 also receives information from the video codec 124.
  • Connectivity encoding 120 includes encoding the connectivity information of the points.
  • Mesh zippering 126 is an implementation to remove any gaps between triangles or patches.
  • a multiplexer 130 outputs the V3C bitstream.
  • the V3C bitstream enables point cloud reconstruction and/or mesh construction.
  • the point cloud and/or the mesh are able to be extracted from the V3C bitstream, which provides significant flexibility.
  • fewer or additional steps are implemented.
  • the order of the steps is modified.
  • Figure 2 illustrates a flowchart of a method of implementing patch generation according to some embodiments.
  • Patch segmentation generates connected components 200.
  • Connected components 200 are triangles that are neighbors and/or adjacent and have the same classification (e.g., same color in Figure 5).
  • a conditional rasterization 202 is performed, meaning a triangle is projected into a patch only if it meets specified criteria.
  • the rasterization criteria (e.g., if the triangle is going to be projected or not), includes removing degenerates 204, checking vertex occlusion 206, checking half-edge 208, checking surface occlusion 210, depth filtering 212, depth range and connectivity 214, and rasterizing the triangle 216.
  • Removing degenerate triangles 204 includes removing triangles that are lines (e.g., where two of the points overlap, thus instead of a triangle, a line exists). A line or a point is removed, and only triangles that have at least a certain area (e.g., above a threshold) are retained.
  • Checking vertex occlusion 206 includes determining whether a vertex of the triangle is occluded. If the vertex of the triangle is occluded by (or would occlude) triangles that have already been projected, then the projection is not allowed, and the triangle is removed from the list and goes to the missed triangle list.
  • Checking half-edges 208 determines if the triangle to be projected shares an edge that is already shared by two projected triangles. If already shared, then the projection is not allowed, and the triangle is removed from the list and goes to the missed triangle list.
  • Depth filtering 212 checks to makes sure the depth differentials of the triangles of the neighboring points are smaller than a threshold (e.g., points of the current triangle are compared with points of neighboring triangles). If the depth differentials are higher than the threshold, then the triangle is not projected.
  • a threshold e.g., points of the current triangle are compared with points of neighboring triangles
  • Depth range and connectivity 214 includes determining whether the triangle to be projected touches at least one of the already projected triangles (e.g., a shared edge or vertex). If there is no connectivity between triangle to be projected and the already projected triangles, then the projection is not allowed, and the triangle is removed from the list and goes to the missed triangle list. The depth range is also checked.
  • the already projected triangles e.g., a shared edge or vertex
  • the triangle is acceptable to be projected (e.g., on to a canvas), and the triangle is rasterized 216 (e.g., generating the points on the surface and generating the geometry image and the texture image to encode the connectivity on the patch).
  • Figure 3 illustrates a diagram of computing a normal according to some embodiments.
  • the normal 300 is the cross product of the two vectors (e.g., AB and AC).
  • Image 302 illustrates an exemplary image with face normals.
  • Figure 4 illustrates a diagram of face adjacency and face neighbor analysis according to some embodiments.
  • Triangle 400 has many triangles that are adjacent and neighboring. Adjacent triangles are those triangles that share at least one vertex with the specified triangle. For example, Triangles 1-15 ⁇ T 1 ,...,T 15 ⁇ all share a vertex with Triangle 400. Neighboring triangles share an edge with a triangle. For example, Triangles 4, 10 and 14 ⁇ T 4 , T 10 and T 14 ⁇ share an edge with Triangle 400. Lists of adjacent triangles and neighboring triangles are able to be kept for each triangle. A flag is also able to be included to preserve boundaries.
  • Three dimensional (3D) triangles are mapped onto a two dimensional (2D) surface.
  • the surface of the mesh is separated (e.g., into different groupings or sections).
  • the texture map boundary 402 separates one part or grouping 404 on a texture map and a second part or grouping 406 on the texture map.
  • two triangles may be next to each other (e.g., neighbors) in a 3D space, they may be separated (e.g., not neighbors) in the 2D space since they are in different groupings in the 2D space.
  • the texture map boundary 402 is factored in.
  • the list of adjacent triangles is ⁇ T 6 ,...,T 14 ⁇
  • the list of neighboring triangles is ⁇ T 10 , T 14 ⁇ , for Triangle 400. Therefore, when the patches are projected, the borders are maintained, which preserves the regional texture scenes, so they are available on the decoder side.
  • FIG. 5 illustrates a diagram of patch generation according to some embodiments.
  • patch generation involves normal calculation, adjacency calculation, initial segmentation (or normal categorization) and segmentation refinement (or category refinement).
  • Calculating a normal of each triangle involves the cross-product between triangle edges.
  • Adjacency calculation is determining if triangles share a vertex, and if so, the triangles are neighbors.
  • Initial segmentation and segmentation refinement are performed the same as V-PCC by analyzing the orientation of the normals, classifying the orientations of the normals (e.g., up, down, left, right, forward, backward), and determining if an orientation of a normal is classified differently than neighboring normals that are all classified the same (e.g., a first patch is classified as up, while most or all of the neighboring patches are classified as forward), then changing the classification of the normal of the patch to match the neighbors’ orientations (e.g., first patch classification is changed to forward).
  • orientations of the normals e.g., up, down, left, right, forward, backward
  • patch generation is implemented which segments the mesh into patches.
  • Patch generation also generates 1) a rasterized mesh surface and 2) vertices location and connectivity information.
  • the rasterized mesh surface is a set of points which go through the V3C image or a V-PCC image generation and is encoded as a V3C image or a V-PCC image.
  • the vertices location and connectivity information is received for base-mesh coding.
  • Patch generation described herein is similar to patch generation in V-PCC. However, instead of calculating the normal per point, the normal per triangle is calculated. The normal per triangle is calculated using the cross-product between the edges to determine a normal vector. Then, the triangles are categorized according to the normals.
  • the normals are divided into n (e.g., 6) categories such as front, back, top, down, left and right.
  • the normals are indicated in different colors to show the initial segmentation.
  • Figure 5 shows the different colors in grayscale such as the black and light gray as different colors indicating different normals.
  • top surfaces e.g., the top of the person’s head, the top of the ball and the top of the sneakers
  • top surfaces are one color (e.g., green)
  • a first side of the person/ball is very dark, representing another color (e.g., red)
  • the bottom of the ball is another color (e.g., purple)
  • the front of the person and ball which is mostly light gray, represents another color (e.g., cyan).
  • the main direction is able to be found.
  • a smoothing/refinement process is able to be implemented. For example, if the number of neighboring triangles above a threshold are all blue, then this triangle also be classified as blue, even if there was an anomaly which initially indicated the triangle was red.
  • Connected components of triangles are generated to identify which of the triangles have the same color (e.g., triangles with the same category sharing at least one vertex).
  • Connectivity information describes how the points are connected in 3D. These connections together generate triangles (to be more specific, 3 distinct connections that share 3 points), which consequently generate surfaces (described by a collection of triangles). Although triangles are described herein, other geometric shapes are also allowed (e.g., rectangles).
  • Colors are able to be used to encode the connectivity by identifying the triangles with different colors. Each triangle, identified by three connections, is coded with a unique color.
  • FIG. 6 illustrates a flowchart and diagram of patch segmentation using conditional rasterization according to some embodiments.
  • Patch segmentation generates connected components 200.
  • Connected components 200 are triangles that are neighbors and/or adjacent and have the same classification (e.g., same color in Figure 5).
  • a conditional rasterization 202 is performed, meaning a triangle is projected into a patch only if meets specified criteria. This ensures that the triangles are projected consistently. If a triangle does not meet the specified criteria, the triangle is considered a missed triangle, and a missed triangle goes back into a list and is re-tried with a different neighbor or adjacent triangle, and the process of analysis with the specified criteria is repeated.
  • the rasterization criteria (e.g., if the triangle is going to be projected or not), includes removing degenerates 204, checking vertex occlusion 206, checking half-edge 208, checking surface occlusion 210, depth filtering 212, depth range and connectivity 214, and rasterizing the triangle 216.
  • Removing degenerate triangles 204 includes removing triangles that are lines (e.g., where two of the points overlap, thus instead of a triangle, a line exists). A line or a point is removed, and only triangles that have at least a certain area (e.g., above a threshold) are retained.
  • Checking vertex occlusion 206 includes determining whether a vertex of the triangle is occluded. If the vertex of the triangle is occluded by (or would occlude) triangles that have already been projected, then the projection is not allowed, and the triangle is removed from the list and goes to the missed triangle list.
  • Checking half-edges 208 determines if the triangle to be projected shares an edge that is already shared by two projected triangles. If already shared, then the projection is not allowed, and the triangle is removed from the list and goes to the missed triangle list.
  • Depth filtering 212 checks to makes sure the depth differentials of the triangles of the neighboring points are smaller than a threshold (e.g., points of the current triangle are compared with points of neighboring triangles). If the depth differentials are higher than the threshold, then the triangle is not projected.
  • a threshold e.g., points of the current triangle are compared with points of neighboring triangles.
  • Depth range and connectivity 214 includes determining whether the triangle to be projected touches at least one of the already projected triangles (e.g., a shared edge or vertex). If there is no connectivity between triangle to be projected and the already projected triangles, then the projection is not allowed, and the triangle is removed from the list and goes to the missed triangle list. The depth range is also checked.
  • the already projected triangles e.g., a shared edge or vertex
  • the triangle is acceptable to be projected (e.g., on to a canvas), and the triangle is rasterized 216 (e.g., generating the points on the surface and generating the geometry image and the texture image to encode the connectivity on the patch).
  • Figure 7 illustrates a diagram of depth filtering according to some embodiments.
  • Depth filtering checks to makes sure the depth differentials of the triangles of the neighboring points are smaller than a threshold. If the depth differentials are higher than the threshold, then the triangle is not projected. For example, a depth of a point and depths of neighboring points are analyzed. However, only neighboring points from other triangles are analyzed. In the Figure, points labeled T are from the triangle, so they are not analyzed. The depths from points of Triangle A (labeled T A ) and Triangle B (labeled T B ) are analyzed.
  • the depth of point T is compared with (e.g., calculate the difference between) the depth of T A in the lower left square of the grid, the depth of T A in the middle left square of the grid, the depth of T A in the upper left square of the grid, the depth of T B in the middle upper square of the grid, and the depth of T B in the upper right square of the grid. If the difference between the depth of the point and any of the other points compared is greater than a threshold, then Triangle 700 is not projected. By performing depth filtering, large gradients are avoided. Although a 3x3 filter is shown, a larger filter is able to be implemented.
  • FIG. 8 illustrates a block diagram of an exemplary computing device configured to implement the patch generation for dynamic mesh coding method according to some embodiments.
  • the computing device 800 is able to be used to acquire, store, compute, process, communicate and/or display information such as images and videos including 3D content.
  • the computing device 800 is able to implement any of the encoding/decoding aspects.
  • a hardware structure suitable for implementing the computing device 800 includes a network interface 802, a memory 804, a processor 806, I/O device(s) 808, a bus 810 and a storage device 812.
  • the choice of processor is not critical as long as a suitable processor with sufficient speed is chosen.
  • the memory 804 is able to be any conventional computer memory known in the art.
  • the storage device 812 is able to include a hard drive, CDROM, CDRW, DVD, DVDRW, High Definition disc/drive, ultra-HD drive, flash memory card or any other storage device.
  • the computing device 800 is able to include one or more network interfaces 802. An example of a network interface includes a network card connected to an Ethernet or other type of LAN.
  • the I/O device(s) 808 are able to include one or more of the following: keyboard, mouse, monitor, screen, printer, modem, touchscreen, button interface and other devices.
  • Patch generation for dynamic mesh coding application(s) 830 used to implement the patch generation for dynamic mesh coding implementation are likely to be stored in the storage device 812 and memory 804 and processed as applications are typically processed. More or fewer components shown in Figure 8 are able to be included in the computing device 800.
  • patch generation for dynamic mesh coding hardware 820 is included.
  • the computing device 800 in Figure 8 includes applications 830 and hardware 820 for the patch generation for dynamic mesh coding implementation, the patch generation for dynamic mesh coding method is able to be implemented on a computing device in hardware, firmware, software or any combination thereof.
  • the patch generation for dynamic mesh coding applications 830 are programmed in a memory and executed using a processor.
  • the patch generation for dynamic mesh coding hardware 820 is programmed hardware logic including gates specifically designed to implement the patch generation for dynamic mesh coding method.
  • the patch generation for dynamic mesh coding application(s) 830 include several applications and/or modules.
  • modules include one or more sub-modules as well. In some embodiments, fewer or additional modules are able to be included.
  • suitable computing devices include a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, a smart phone, a portable music player, a tablet computer, a mobile device, a video player, a video disc writer/player (e.g., DVD writer/player, high definition disc writer/player, ultra high definition disc writer/player), a television, a home entertainment system, an augmented reality device, a virtual reality device, smart jewelry (e.g., smart watch), a vehicle (e.g., a self-driving vehicle) or any other suitable computing device.
  • a personal computer e.g., a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console
  • a device acquires or receives 3D content (e.g., point cloud content).
  • 3D content e.g., point cloud content.
  • the patch generation for dynamic mesh coding method is able to be implemented with user assistance or automatically without user involvement.
  • the patch generation for dynamic mesh coding method enables more efficient and more accurate 3D content encoding compared to previous implementations.
  • a method of patch generation comprising: computing face normals of triangles; computing face adjacency of the triangles; computing face neighbors of the triangles; implementing initial segmentation including classifying the normals of the triangles; performing refinement including locating and smoothing outliers in the classified normals; and performing patch segmentation of the triangles.
  • performing patch segmentation comprises: removing degenerates, checking for vertex occlusion, checking for half-edges, checking for surface occlusion, depth filtering, depth range and connectivity analysis, and rasterizing a triangle.
  • removing degenerate triangles includes removing triangles that are lines or points by determining an area between the points is smaller than a threshold; checking for vertex occlusion includes determining whether a vertex of the triangle is occluded by an already projected triangle; checking for half-edges includes determining if the triangle shares an edge that is already shared by two projected triangles; checking for surface occlusion includes determining if an area between points of the triangle are occluded or would occlude the already projected triangle; depth filtering determines depth differentials of neighboring points and points of the triangle are smaller than a depth threshold; depth range and connectivity analysis includes determining whether the triangle touches at least one edge or vertex of the already projected triangles; and projecting and rasterizing the triangle.
  • An apparatus comprising: a non-transitory memory for storing an application, the application for: computing face normals of triangles; computing face adjacency of the triangles; computing face neighbors of the triangles; implementing initial segmentation including classifying the normals of the triangles; performing refinement including locating and smoothing outliers in the classified normals; and performing patch segmentation of the triangles; and a processor coupled to the memory, the processor configured for processing the application.
  • computing face adjacency of the triangles and computing face neighbors of the triangles includes generating lists of adjacent triangles and neighboring triangles.
  • performing patch segmentation comprises: removing degenerates, checking for vertex occlusion, checking for half-edges, checking for surface occlusion, depth filtering, depth range and connectivity analysis, and rasterizing a triangle.
  • removing degenerate triangles includes removing triangles that are lines or points by determining an area between the points is smaller than a threshold; checking for vertex occlusion includes determining whether a vertex of the triangle is occluded by an already projected triangle; checking for half-edges includes determining if the triangle shares an edge that is already shared by two projected triangles; checking for surface occlusion includes determining if an area between points of the triangle are occluded or would occlude the already projected triangle; depth filtering determines depth differentials of neighboring points and points of the triangle are smaller than a depth threshold; depth range and connectivity analysis includes determining whether the triangle touches at least one edge or vertex of the already projected triangles; and projecting and rasterizing the triangle.
  • a method of patch segmentation comprising: removing degenerate information; checking for vertex occlusion of a triangle; checking for half-edges of the triangle; checking for surface occlusion of the triangle; depth filtering of the triangle; performing depth range and connectivity analysis of the triangle; and projecting and rasterizing the triangle onto a canvas.
  • removing degenerate triangles includes removing triangles that are lines or points by determining an area between the points is smaller than a threshold.
  • checking for vertex occlusion includes determining whether a vertex of the triangle is occluded by an already projected triangle.
  • checking for half-edges includes determining if the triangle shares an edge that is already shared by two projected triangles. 19. The method of clause 15 wherein checking for surface occlusion includes determining if an area between points of the triangle are occluded or would occlude the already projected triangle. 20. The method of clause 15 wherein depth filtering determines depth differentials of neighboring points and points of the triangle are smaller than a depth threshold.
  • depth range and connectivity analysis includes determining whether the triangle touches at least one edge or vertex of the already proj ected triangles.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Generation (AREA)

Abstract

L'invention concerne une architecture comprend de nouveaux blocs qui transforment la connectivité de maillage pour permettre une compression avec perte. L'invention concerne également un procédé qui génère des correctifs de surface à partir de maillages et préserve l'aspect global de l'objet, tout en générant des correctifs pour un codage efficace à l'aide de la norme V3C. Des triangles sont joints en composants connectés, et une projection conditionnelle de chaque triangle par rapport à la surface est effectuée pour appliquer une connectivité au niveau de la surface projetée. Un nouvel algorithme de filtrage de profondeur peut être utilisé pour améliorer les performances de compression. La présente invention concerne en outre les nouveaux blocs ajoutés pour la compression de connectivité et les techniques présentées pour assurer une connectivité 2D, afin de permettre une transmission de carte de texture UV et de filtrer des valeurs de profondeur et éviter des bords haute fréquence dans l'image de profondeur.
PCT/IB2023/052106 2022-03-25 2023-03-07 Génération de correctif pour codage maillé dynamique WO2023180843A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202380013355.1A CN117897957A (zh) 2022-03-25 2023-03-07 用于动态网格编码的面元生成

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202263269910P 2022-03-25 2022-03-25
US63/269,910 2022-03-25
US17/987,846 2022-11-15
US17/987,846 US20230306684A1 (en) 2022-03-25 2022-11-15 Patch generation for dynamic mesh coding

Publications (1)

Publication Number Publication Date
WO2023180843A1 true WO2023180843A1 (fr) 2023-09-28

Family

ID=85724807

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2023/052106 WO2023180843A1 (fr) 2022-03-25 2023-03-07 Génération de correctif pour codage maillé dynamique

Country Status (1)

Country Link
WO (1) WO2023180843A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019077199A1 (fr) * 2017-10-18 2019-04-25 Nokia Technologies Oy Appareil, procédé, et programme d'ordinateur pour vidéo volumétrique
US20210174551A1 (en) * 2019-12-10 2021-06-10 Sony Corporation Mesh compression via point cloud representation
US20210295566A1 (en) * 2020-03-18 2021-09-23 Sony Corporation Projection-based mesh compression

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019077199A1 (fr) * 2017-10-18 2019-04-25 Nokia Technologies Oy Appareil, procédé, et programme d'ordinateur pour vidéo volumétrique
US20210174551A1 (en) * 2019-12-10 2021-06-10 Sony Corporation Mesh compression via point cloud representation
US20210295566A1 (en) * 2020-03-18 2021-09-23 Sony Corporation Projection-based mesh compression

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DANILLO GRAZIOSI (SONY) ET AL: "[V-PCC][EE2.6-related] Mesh Geometry Smoothing Filter", no. m55374, 13 October 2020 (2020-10-13), XP030291886, Retrieved from the Internet <URL:https://dms.mpeg.expert/doc_end_user/documents/132_OnLine/wg11/m55374-v1-m55374_mesh_geometry_smoothing.zip m55374_mesh_geometry_smoothing.pdf> [retrieved on 20201013] *
DANILLO GRAZIOSI (SONY) ET AL: "[V-PCC][EE2.6-related] Triangle Patch Data", no. m55370, 7 October 2020 (2020-10-07), XP030292893, Retrieved from the Internet <URL:https://dms.mpeg.expert/doc_end_user/documents/132_OnLine/wg11/m55370-v1-m55370_triangle_patch_data.zip m55370_triangle_patch_data.pdf> [retrieved on 20201007] *

Similar Documents

Publication Publication Date Title
US11348285B2 (en) Mesh compression via point cloud representation
US11373339B2 (en) Projection-based mesh compression
US11836953B2 (en) Video based mesh compression
US20200296401A1 (en) Method and Apparatus of Patch Segmentation for Video-based Point Cloud Coding
JP7371691B2 (ja) ホモグラフィ変換を使用した点群符号化
US11908169B2 (en) Dense mesh compression
Alface et al. V3c-based coding of dynamic meshes
US20230306684A1 (en) Patch generation for dynamic mesh coding
WO2023180843A1 (fr) Génération de correctif pour codage maillé dynamique
US20230306643A1 (en) Mesh patch simplification
WO2023180842A1 (fr) Simplification de patch de maillage
US20230306683A1 (en) Mesh patch sub-division
US20230306687A1 (en) Mesh zippering
CN113557745B (zh) 点云几何结构填充
WO2023180841A1 (fr) Sous-division d&#39;éléments de maillage
WO2023180844A1 (fr) Zippering de maillage
WO2024074961A1 (fr) Orthoatlas : génération de carte de texture pour mailles dynamiques à l&#39;aide de projections orthographiques
CN117897731A (zh) 网格拉链扣紧

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23712606

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202380013355.1

Country of ref document: CN