WO2025002019A1 - 三维网格编码方法、解码方法、装置、编码端及解码端 - Google Patents

三维网格编码方法、解码方法、装置、编码端及解码端 Download PDF

Info

Publication number
WO2025002019A1
WO2025002019A1 PCT/CN2024/100835 CN2024100835W WO2025002019A1 WO 2025002019 A1 WO2025002019 A1 WO 2025002019A1 CN 2024100835 W CN2024100835 W CN 2024100835W WO 2025002019 A1 WO2025002019 A1 WO 2025002019A1
Authority
WO
WIPO (PCT)
Prior art keywords
grid
mesh
reconstructed
region
dimensional
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/100835
Other languages
English (en)
French (fr)
Inventor
邹文杰
张伟
杨付正
吕卓逸
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Publication of WO2025002019A1 publication Critical patent/WO2025002019A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present application belongs to the field of communication technology, and specifically relates to a three-dimensional grid encoding method, decoding method, device, encoding end and decoding end.
  • Three-dimensional models have become a new generation of digital media after audio, images, and videos. Compared with traditional multimedia such as images and videos, they have stronger interactivity and realism.
  • Three-dimensional mesh is a commonly used representation of three-dimensional models.
  • Three-dimensional meshes are composed of three basic elements: vertices, edges, and faces. Among them, vertices are the most basic elements in three-dimensional meshes, which define a position in a three-dimensional space; edges are line segments connecting two vertices in a three-dimensional mesh; faces are polygons formed by closed paths of the edges of three-dimensional meshes.
  • the three-dimensional mesh is mapped from three-dimensional space to a two-dimensional plane area through mesh parameterization.
  • This mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with the vertices of the three-dimensional mesh.
  • UV coordinates or texture coordinates which are associated with the vertices of the three-dimensional mesh.
  • texture coordinates of the three-dimensional mesh need to be encoded and stored or transmitted accordingly to achieve the reconstruction of the texture coordinates of the three-dimensional mesh at the decoding end. Since the code stream of the texture coordinates often occupies most of the code stream of the three-dimensional mesh, this will not only lead to low encoding efficiency, but also occupy storage resources or transmission resources.
  • the embodiments of the present application provide a three-dimensional grid encoding method, decoding method, device, encoding end and decoding end, which can improve the encoding efficiency of the three-dimensional grid and reduce the occupation of storage resources or transmission resources by the code stream of the three-dimensional grid.
  • a three-dimensional grid encoding method comprising:
  • the encoding end compresses and encodes the first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded;
  • the encoding end performs three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid
  • the encoder performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates;
  • the encoding end converts the first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is the texture map corresponding to the three-dimensional mesh to be encoded;
  • the encoding end encodes the second texture map to obtain a texture map bit stream.
  • a three-dimensional grid encoding device comprising:
  • a first encoding module configured to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded;
  • a first reconstruction module used for performing three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid
  • a first processing module configured to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates;
  • a conversion module configured to convert a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded;
  • the second encoding module is used to encode the second texture map to obtain a texture map code stream.
  • a three-dimensional grid decoding method comprising:
  • the decoding end obtains the grid code stream and the texture map code stream
  • the decoding end decodes the grid code stream to obtain a third grid, and decodes the texture map code stream to obtain a third texture map;
  • the decoding end performs mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.
  • a three-dimensional grid decoding device comprising:
  • a first acquisition module is used to acquire a grid code stream and a texture map code stream
  • a decoding module used for decoding the grid code stream to obtain a third grid, and decoding the texture map code stream to obtain a third texture map;
  • the second processing module is used to perform mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.
  • an encoding end which includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.
  • a coding end comprising a processor and a communication interface, wherein the processor is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded; reconstruct a three-dimensional grid based on the grid code stream to obtain a first reconstructed grid; perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, wherein the second reconstructed grid is a first reconstructed grid including texture coordinates; convert a first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional grid to be encoded; encode the second texture map to obtain a texture map code stream.
  • a decoding end which includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the third aspect are implemented.
  • a decoding end comprising a processor and a communication interface, wherein the processor is used to obtain a grid code stream and a texture map code stream; decode the grid code stream to obtain a third grid, and decode the texture map code stream; The third texture map is obtained by row decoding; and the third grid is subjected to grid parameterization processing to obtain a fourth grid, wherein the fourth grid is a grid including texture coordinates.
  • a three-dimensional mesh encoding and decoding system comprising: an encoding end and a decoding end, wherein the encoding end can be used to execute the steps of the three-dimensional mesh encoding method as described in the first aspect, and the decoding end can be used to execute the steps of the three-dimensional mesh decoding method as described in the third aspect.
  • a readable storage medium on which a program or instruction is stored.
  • the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the third aspect are implemented.
  • a chip comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the third aspect.
  • a computer program/program product is provided, wherein the computer program/program product is stored in a storage medium, and the computer program/program product is executed by at least one processor to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the third aspect.
  • the encoding end compresses and encodes the first mesh to obtain a mesh code stream, wherein the first mesh is a mesh determined according to the three-dimensional mesh to be encoded; the encoding end reconstructs the three-dimensional mesh based on the mesh code stream to obtain a first reconstructed mesh; the encoding end performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is a first reconstructed mesh including texture coordinates; the encoding end converts a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded; the encoding end encodes the second texture map to obtain a texture map code stream, that is, in the embodiment of the present application, the encoding end only generates a mesh code stream and a texture map code stream of the three-dimensional mesh, but does not generate
  • the mesh parameterization processing of the first reconstructed mesh is performed, which can not only reduce the amount of data processing, but also ensure the accuracy of the texture coordinates of the first reconstructed mesh, compared with the mesh parameterization processing of the first mesh in the prior art.
  • FIG1 is a block diagram of a wireless communication system to which an embodiment of the present application can be applied;
  • FIG2 is a schematic diagram of a mesh parameterization process based on an orthogonal projection algorithm provided in an embodiment of the present application
  • FIG3 is a flow chart of a three-dimensional grid coding method provided in an embodiment of the present application.
  • FIG4 is a schematic diagram of basic grid compression provided by an embodiment of the present application.
  • FIG5 is a second schematic diagram of a mesh parameterization process based on an orthogonal projection algorithm provided in an embodiment of the present application
  • FIG6a is one of the schematic diagrams before the connected regions with overlapping regions are split according to an embodiment of the present application
  • FIG6b is one of the schematic diagrams of the connected regions with overlapping regions after being split according to an embodiment of the present application
  • FIG6c is a second schematic diagram of the splitting of the connected regions with overlapping regions provided in the embodiment of the present application.
  • FIG. 7a is a third schematic diagram of a mesh parameterization process based on an orthogonal projection algorithm provided in an embodiment of the present application.
  • FIG7b is a schematic diagram of extracting connected regions provided in an embodiment of the present application.
  • FIG8 is a schematic diagram of a grid simplification provided in an embodiment of the present application.
  • FIG9 is a schematic diagram of a mesh subdivision deformation provided in an embodiment of the present application.
  • FIG10 is a schematic diagram of a grid subdivision provided in an embodiment of the present application.
  • FIG11 is a schematic diagram of a three-dimensional grid coding framework provided in an embodiment of the present application.
  • FIG12 is a flow chart of a three-dimensional grid decoding method provided in an embodiment of the present application.
  • FIG13 is a schematic diagram of basic grid decoding provided by an embodiment of the present application.
  • FIG14 is a schematic diagram of a three-dimensional grid decoding framework provided in an embodiment of the present application.
  • FIG15 is a structural diagram of a three-dimensional grid encoding device provided in an embodiment of the present application.
  • FIG16 is a structural diagram of a three-dimensional grid decoding device provided in an embodiment of the present application.
  • FIG17 is a structural diagram of a communication device provided in an embodiment of the present application.
  • FIG18 is a structural diagram of an encoding end provided in an embodiment of the present application.
  • FIG. 19 is a structural diagram of a decoding end provided in an embodiment of the present application.
  • first, second, etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by “first” and “second” are generally of one type, and the number of objects is not limited, for example, the first object can be one or more.
  • “or” in the present application represents at least one of the connected objects.
  • “A or B” covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B.
  • the character "/" generally indicates that the objects associated with each other are in an "or” relationship.
  • indication in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication).
  • a direct indication can be understood as the sender explicitly informing the receiver of specific information, operations to be performed, or request results in the sent indication;
  • an indirect indication can be understood as the receiver determining the corresponding information according to the indication sent by the sender, or making a judgment and determining the operation to be performed or the request result according to the judgment result.
  • LTE Long Term Evolution
  • LTE-A Long Term Evolution
  • CDMA Code Division Multiple Access
  • TDMA Time Division Multiple Access
  • FDMA Frequency Division Multiple Access
  • OFDMA Orthogonal Frequency Division Multiple Access
  • SC-FDMA Single-carrier Frequency-Division Multiple Access
  • NR New Radio
  • 6G 6th Generation
  • FIG1 shows a block diagram of a wireless communication system applicable to the embodiment of the present application.
  • the wireless communication system includes a terminal 11 and a network side device 12 .
  • the terminal 11 can be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer), a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (Ultra-mobile Personal Computer, UMPC), a mobile Internet device (Mobile Internet Device, MID), an augmented reality (Augmented Reality, AR), a virtual reality (Virtual Reality, VR) device, a robot, a wearable device (Wearable Device), a flight vehicle (flight vehicle), a vehicle user equipment (VUE), a shipborne equipment, a pedestrian terminal (Pedestrian User Equipment, PUE), a smart home (home appliances with wireless communication functions, such as refrigerators, televisions, washing machines or furniture, etc.), a game console, a personal computer (Personal Computer, PC
  • Wearable devices include: smart watches, smart bracelets, smart headphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc.
  • the vehicle-mounted device can also be called a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiment of the present application.
  • the network side device 12 may include an access network device or a core network device, wherein the access network device may also be called a radio access network (Radio Access Network, RAN) device, a radio access network function or a radio access network unit.
  • the access network device may include a base station, a wireless local area network (Wireless Local Area Network, WLAN) access point (Access Point, AP) or a wireless fidelity (Wireless Fidelity, WiFi) node, etc.
  • WLAN wireless Local Area Network
  • AP Access Point
  • WiFi wireless Fidelity
  • the base station can be called Node B (Node B, NB), Evolved Node B (Evolved Node B, eNB), the next generation Node B (the next generation Node B, gNB), New Radio Node B (New Radio Node B, NR Node B), access point, Relay Base Station (Relay Base Station, RBS), Serving Base Station (Serving Base Station, SBS), Base Transceiver Station (Base Transceiver Station, BTS), radio base station, radio transceiver, base
  • the base station is not limited to specific technical terms as long as the same technical effect is achieved. It should be noted that in the embodiments of the present application, only the base station in the NR system is taken as an example for introduction, and the specific type of the base station is not limited.
  • the core network equipment may include but is not limited to at least one of the following: core network node, core network function, mobility management entity (Mobility Management Entity, MME), access mobility management function (Access and Mobility Management Function, AMF), session management function (Session Management Function, SMF), user plane function (User Plane Function, UPF), policy control function (Policy Control Function, PCF), policy and charging rules function Energy unit (Policy and Charging Rules Function, PCRF), edge application service discovery function (Edge Application Server Discovery Function, EASDF), unified data management (Unified Data Management, UDM), unified data storage (Unified Data Repository, UDR), home user server (Home Subscriber Server, HSS), centralized network configuration (CNC), network storage function (Network Repository Function, NRF), network exposure function (Network Exposure Function, NEF), local NEF (Local NEF, or L-NEF), binding support function (Binding Support Function, BSF), application function (Application Function, AF), etc.
  • MME Mobility
  • the encoding end of the embodiment of the present application may be an electronic device or a component of an electronic device such as a terminal or a network side device or a server.
  • the decoding end of the embodiment of the present application may also be an electronic device or a component of an electronic device such as a terminal or a network side device or a server.
  • the encoding end and the decoding end of the embodiment of the present application may be located in the same electronic device or in different electronic devices.
  • Three-dimensional models have become a new generation of digital media after audio, images, and videos.
  • Three-dimensional grids are a commonly used way to represent three-dimensional models. Compared with traditional multimedia such as images and videos, three-dimensional grids have stronger interactivity and realism, making them more and more widely used in various fields such as commerce, manufacturing, construction, education, medicine, entertainment, art, and military.
  • a 3D mesh can be considered to be composed of three basic elements: vertices, edges, and faces. Vertices are the most basic elements in a mesh, and they define a position in a 3D space. Edges are line segments connecting two vertices in a mesh. Faces can be considered as polygons formed by closed paths of edges. For a triangular mesh, each face is a triangle.
  • the information contained in a three-dimensional mesh is usually divided into three categories: geometric information, connection information, and attribute information.
  • geometric information refers to the position of each vertex of the three-dimensional mesh in three-dimensional space.
  • Connection information describes the association between elements in the three-dimensional mesh, that is, the connection relationship between vertices.
  • Attribute information is an optional information that can associate attributes to corresponding mesh elements (such as vertex color, normal vector, etc. can be associated with mesh vertices).
  • mesh parameterization can be used to map the three-dimensional mesh from three-dimensional space to a two-dimensional plane area. This mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with mesh vertices. This two-dimensional mapping can be used to represent high-resolution attribute information, such as texture, normal vector, etc.
  • 3D meshes In almost all application fields that use 3D meshes (such as computational simulation, entertainment, medical imaging, digital cultural relics, computer design, e-commerce, etc.), as people's demand for the visual effects of 3D models increases, 3D models are becoming more and more complex, and the accuracy of 3D models is also increasing. Therefore, the amount of data required to represent 3D meshes is also increasing accordingly.
  • 3D mesh compression can be regarded as a way to solve the above problems. It can reduce the size of model data and is beneficial to 3D mesh. Format processing, storage and transmission.
  • VDMC Video-based Dynamic Mesh Coding
  • V3C Visual Volumetric Video-based Coding
  • the V3C standard provides a general method for compressing 3D models, which can be presented in the form of point clouds, meshes or panoramic videos. Making the compression method of 3D meshes compatible with this standard will help the promotion and applicability of the method. Therefore, it is of great significance to optimize the 3D mesh encoding and decoding method in VDMC and combine the optimization method with the V3C standard.
  • An optional optimization method is to optimize the displacement coding method.
  • the displacement is obtained by calculating the distance between the reconstructed mesh and the original mesh vertex, and its purpose is to improve the quality of the mesh.
  • the displacement is encoded using a video encoder, and providing multiple options for the displacement encoding method will help improve the encoding performance.
  • OrthoAtlas is a projection-based mesh parameterization method that generates texture coordinates for a 3D mesh through orthogonal projection.
  • the main process is shown in Figure 2 and includes the following steps:
  • triangle adjacency can be defined as sharing a single vertex or sharing an edge
  • the normal vector and area of a triangle can be derived from the projected triangle (later used to calculate the distortion after projection).
  • the triangles are clustered, and the area formed by the same type of triangles is called a connected area. There are two methods to select the starting point of the clustering algorithm:
  • the first method is to obtain the projection direction based on the statistical results of all face normal vectors. Then the triangular face whose normal vector is closest to the projection direction can be selected as the starting point of the clustering algorithm and it has not yet been added to the connected area.
  • the second method is to select the first available face as the starting point of the clustering algorithm (that is, the face has not yet been added to the connected area), and determine whether to add the adjacent face to the connected area according to the criteria;
  • the criteria include:
  • the angle between the normal vector of the last face added to the connected region and the normal vector of the last face added to the connected region is less than the threshold
  • the area is less than a threshold of the total area
  • the number of triangles is less than the threshold.
  • the connected regions are sorted in a specific order.
  • the sorting criteria can be based on the number of triangles contained and the proximity between the average normal vector of the connected region and the corresponding projection direction.
  • COST(M i +M j ) L 2 (M i +M j ) + ⁇ PERIMETER(M i +M j ), for all areas (COST(M i +M j ) ⁇ COST(M i ) + COST(M j )), the area that minimizes the cost function is selected, the taken area is merged into the area and the sorting of the connected areas is updated;
  • L 2 () represents the stretching degree of the connected area after projection
  • PERIMETER() represents the perimeter of the above-mentioned connected area
  • ⁇ represents the weight coefficient.
  • the stretching function in the cost function measures the degree of distortion after projection, which is obtained by calculating the area of the triangle after projection in each direction.
  • Detecting overlapping areas means determining whether the currently projected surface overlaps with the projected area. If an overlap occurs, the current surface is removed from the connected area, and the surface and other removed surfaces form a new connected area. Finally, in the removed connected area, check whether there is a disconnected area, and split the disconnected area into multiple connected areas.
  • FIG. 3 is a flowchart of a three-dimensional grid encoding method provided in an embodiment of the present application.
  • the method can be executed by an encoding end, as shown in FIG. 3, and includes the following steps:
  • Step 301 The encoding end compresses and encodes a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded.
  • the first grid may be a three-dimensional grid to be encoded, or may be a grid obtained after processing the three-dimensional grid to be encoded, for example, a grid obtained after grid simplification processing of the three-dimensional grid to be encoded.
  • the grid code stream is also a code stream obtained after compression encoding of the first grid.
  • the three-dimensional grid to be encoded may also be referred to as an input grid of the encoding end
  • the first grid may also be referred to as a basic grid
  • the grid code stream may also be referred to as a basic grid code stream.
  • the compression encoding of the first grid may be performed by, for example, adopting compression modes such as intra-frame mode, inter-frame mode or skip mode.
  • compression modes such as intra-frame mode, inter-frame mode or skip mode.
  • the first mesh in intra-frame mode, is encoded by a static mesh encoder. In this compression mode, the first mesh needs to contain geometric coordinates, connection relationships, and attribute information associated with vertices.
  • the first mesh In inter-frame mode, the first mesh first uses rate-distortion optimization (RDO) to decide whether to use skip mode. If skip mode is used, the reference mesh in the mesh buffer is directly selected as the first mesh of the current frame. If skip mode is not used, the motion vector of the vertex is calculated based on the reference frame in the mesh buffer and the current frame, and the corresponding motion vector is encoded by the motion vector encoder. It should be noted that after encoding the first mesh, the encoded mesh needs to be reconstructed, that is, the first mesh needs to be reconstructed so as to be provided to subsequent modules for processing.
  • RDO rate-distortion optimization
  • Step 302 The encoding end reconstructs a three-dimensional grid based on the grid code stream to obtain a first reconstructed grid.
  • a three-dimensional mesh reconstruction may be performed based on the mesh code stream.
  • the three-dimensional mesh reconstruction may be achieved by decoding the mesh code stream.
  • corresponding decoding can be performed based on the compression mode of the above-mentioned grid code stream.
  • the compression mode of the above-mentioned grid code stream is the intra-frame mode
  • the above-mentioned grid code stream can be decoded based on the static grid decoder to obtain a reconstructed basic grid, that is, a first reconstructed grid
  • the motion vector of the above-mentioned grid code stream can be decoded, and the corresponding basic grid can be reconstructed according to the reference frame in the grid buffer
  • the compression mode of the above-mentioned basic grid is the skip mode
  • the corresponding reference frame in the grid buffer can be used as the basic grid of the current frame.
  • Step 303 The encoding end performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates.
  • the mesh parameterization process is used to generate corresponding texture coordinates for the first reconstructed mesh.
  • the mesh parameterization process can be performed on the first reconstructed mesh based on an Isochart algorithm, an orthogonal projection algorithm, or the like.
  • Step 304 The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map.
  • the first texture map may also be referred to as an input texture map, which may be a texture map corresponding to the three-dimensional mesh to be encoded.
  • an optional texture map conversion method may include the following steps:
  • Step a1 Calculate the texture coordinates of each pixel on the texture map to be generated (ie, the second texture map).
  • Step a2 for each texture coordinate, determine whether the texture coordinate is within a certain face of the second reconstructed mesh.
  • Step a3 If the texture coordinate does not belong to any surface of the second reconstructed mesh, the pixel is marked as an empty pixel, which can be filled with a filling algorithm.
  • Step a4 If the texture coordinate belongs to a face of the second reconstructed mesh, perform the following steps:
  • Step a41 marking the pixel as filled
  • Step a42 calculating the center of gravity coordinates of the texture in the current surface according to the texture coordinates
  • Step a43 Map the two-dimensional texture coordinates to three-dimensional geometric coordinates according to the barycentric coordinates and the corresponding surface. That is, the point on the second reconstructed grid mapped to the texture coordinate;
  • Step a44 searching for the point closest to the three-dimensional coordinate on the input original grid (i.e. the three-dimensional grid to be encoded);
  • Step a45 calculating the center of gravity coordinates of the three-dimensional coordinates according to the surface where the three-dimensional coordinates are located and mapping them to two dimensions to calculate the texture coordinates;
  • Step a46 sampling the input original texture map (i.e., the first texture map) by using the texture coordinates to obtain the value of the corresponding pixel position;
  • Step a47 assigning the value to the corresponding pixel on the texture map to be generated (ie, the second texture map mentioned above).
  • Step 305 The encoding end encodes the second texture map to obtain a texture map bit stream.
  • a video encoder may be used to encode the second texture map to obtain a texture map bit stream.
  • the encoder when the encoder obtains the grid code stream and the texture map code stream, the encoder can store or transmit the grid code stream and the texture map code stream. It should also be noted that, when the encoder does not generate a code stream of the texture coordinates of the grid, the decoder can perform a grid parameterization process on the grid obtained by decoding the grid code stream to generate the texture coordinates of the grid, thereby ensuring that the decoder can normally apply the decoded grid.
  • the encoding end compresses and encodes the first mesh to obtain a mesh code stream, wherein the first mesh is a mesh determined according to the three-dimensional mesh to be encoded; the encoding end reconstructs the three-dimensional mesh based on the mesh code stream to obtain a first reconstructed mesh; the encoding end performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is a first reconstructed mesh including texture coordinates; the encoding end converts a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded; the encoding end encodes the second texture map to obtain a texture map code stream, that is, in the embodiment of the present application, the encoding end only generates a mesh code stream and a texture map code stream of
  • mesh parameterization is performed on the first reconstructed mesh, which can not only reduce the amount of data processing, but also ensure the accuracy of the texture coordinates of the first reconstructed mesh, compared with mesh parameterization of the first mesh in the prior art.
  • the encoding end performs grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, including:
  • the encoding end uses an orthogonal projection algorithm to perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid.
  • the present embodiment uses an orthogonal projection algorithm to perform grid parameterization processing on the first reconstructed grid, which has a low implementation complexity and is easy to apply to the decoding end.
  • the encoding end uses an orthogonal projection algorithm to perform grid parameterization processing on the first reconstructed grid to obtain To the second reconstructed mesh, include:
  • the encoding end divides the first reconstructed grid into at least two connected regions, wherein each of the connected regions includes at least one face of the first reconstructed grid, and faces having the same projection face and being connected in three-dimensional space belong to the same connected region;
  • the encoding end merges the at least two connected regions to obtain at least one first connected region
  • the encoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each of the first connected regions to detect overlapping regions, and splitting the first connected regions with overlapping regions into at least two non-overlapping connected regions;
  • the encoding end merges the connected regions obtained after the first processing to obtain at least one second connected region
  • the encoding end arranges the projection area of the at least one second connected area in two dimensions to obtain a second reconstructed grid.
  • the encoder uses an orthogonal projection algorithm to perform mesh parameterization processing on the first reconstructed mesh, which includes the following processes in sequence: initial partitioning, merging partitions, detecting overlapping areas, simple area merging, and two-dimensional arrangement.
  • initial partitioning that is, the above-mentioned encoder divides the first reconstructed mesh into at least two connected areas.
  • merging partitions that is, the encoder merges the at least two connected areas to obtain at least one first connected area.
  • the above-mentioned detection of overlapping areas that is, the encoder performs a first process on the at least one first connected area respectively.
  • the above-mentioned simple area merging that is, the encoder merges the connected areas obtained after the first process, to obtain at least one second connected area.
  • the above-mentioned two-dimensional arrangement that is, the encoder arranges the projection areas of the at least one second connected area in two dimensions to obtain a second reconstructed mesh.
  • the following takes the face of the first reconstructed mesh as a triangular face as an example to illustrate the above-mentioned various processes.
  • the above initial partitioning may include the following process:
  • the normal vector is normalized to obtain the unit normal vector of the current triangle.
  • the normal vector is normalized to obtain the unit normal vector of the current triangle.
  • the projection surface of each triangular face is determined according to the pre-set projection surface. This process is to calculate the dot product of the normal vector of the current triangular face and the normal vector of each pre-set projection surface, and take the projection surface corresponding to the largest product point as the projection surface of the current triangular face. Then, the triangular faces in the first reconstructed mesh that belong to the same projection surface and are connected in three-dimensional space are divided into the same connected area. Through this process, the first reconstructed mesh can be split into multiple connected areas.
  • the above-mentioned pre-set projection surface can be reasonably set according to actual needs.
  • the above-mentioned pre-set projection surface can be the 6 faces of a cube.
  • the above merging partitions may include the following process:
  • the connected areas obtained by the first reconstructed mesh division are sorted.
  • the connected areas can be sorted according to at least one of the number of triangular faces contained in each connected area and the degree of proximity between the average normal vector of each connected area and the corresponding projection direction (i.e., the size of the dot product of the two vectors). For example, first sort the connected areas in ascending order according to the number of triangular faces contained in each connected area; for multiple connected areas with the same number of triangular faces, they can be sorted according to the degree of proximity between the average normal vector of each connected area and the corresponding projection direction, and the connected areas with higher proximity are ranked higher.
  • the average normal vector of the above-mentioned connected area can refer to the average value of the normal vectors of all triangular faces contained in the above-mentioned connected area.
  • COST(M i +M j ) L 2 (M i +M j ) + ⁇ *PERIMETER(M i +M j ), and all candidate areas whose cost function after the merger is less than the sum of the cost functions of the two areas before the merger (that is, COST(M i +M j ) ⁇ COST(M i ) + COST(M j )) are obtained.
  • the candidate area that minimizes the cost function is selected, the current connected area is merged with the candidate area, the projection surface of the merged connected area is updated, and the sorting of the connected areas is updated, and iteration is performed until all connected areas are traversed.
  • the adjacency of two connected regions can be understood as sharing at least one edge between the two connected regions.
  • the above Mi and Mj represent the first connected region in the connected region sorting, and the other represents the candidate region of the first connected region.
  • COST(M i +M j ) represents the cost function of the connected region after the above Mi and M j are merged.
  • L 2 (M i +M j ) represents the degree of stretching after the projection of the connected region after the above Mi and M j are merged.
  • PERIMETER(M i +M j ) represents the perimeter of the connected region after the above Mi and M j are merged.
  • ⁇ represents the weight coefficient.
  • COST(M i ) represents the cost function of the above Mi.
  • COST(M j ) represents the cost function of the above M j .
  • the stretching degree of the above connected region after projection can be determined in the following manner:
  • ⁇ p,p 2 ,p 3 > represents the area of the triangle pp 2 p 3
  • ⁇ p,p 3 ,p 1 > represents the area of the triangle pp 3 p 1
  • ⁇ p,p 1 ,p 2 > represents the area of the triangle pp 1 p 2
  • p represents the coordinates of any point on the triangle p 1 p 2 p 3. Since the mapping is affine, its partial derivative is a constant on (s, t), and its calculation process is shown in formula (2) and formula (3):
  • ⁇ max and ⁇ min represent the maximum and minimum lengths of a unit length vector when it is mapped from a two-dimensional texture domain to a three-dimensional surface, i.e., the maximum and minimum local "stretching".
  • the stretching measure on the triangular face T is defined as shown in formula (6):
  • A′(T i ) is the surface area of the triangular face T i in three-dimensional space.
  • the cost function of this example is defined by L 2 (M i ) and PERIMETER (M i ), and the two variables are weighed by the ⁇ parameter. After the merging partition step, the number of partitions in the connected area will be reduced compared to the initial partition.
  • each connected area can be projected.
  • the projection process it is necessary to prevent the projection area of the current triangle from blocking the projected area.
  • the above detection of overlapping areas may include the following process:
  • the overlap of the projection area can be divided into three situations, namely: one or more vertices on the projection area of the current triangular face are within the projection area of the projected triangular face of the current connected area; the vertices of the projection area of the projected triangular face of the current connected area are within the projection area of the current triangular face; one or more edges of the projection area of the current triangular face intersect with the edges of the projection area of the projected triangular face of the current connected area.
  • the projection of the current triangle overlaps with the projection area of the projected triangle of the current connected area, the projection of the current triangle is skipped first, and the next triangle of the current connected area is traversed. Then, after a round of traversal, it is determined whether there are unprojected triangles in the current connected area, and these unprojected triangles are projected as new connected areas and it is determined whether there is overlap in the projection area, and it is iterated until the final projection area does not overlap.
  • the connected area with overlapping areas can be split into multiple non-overlapping connected areas.
  • a connected area with overlapping areas i.e., the connected area shown in Figure 6a
  • two non-overlapping connected areas i.e., the two connected areas shown in Figures 6b and 6c.
  • the process of projecting the triangular surface of the connected area is also the process of obtaining the initial texture coordinates of the vertices of the triangular surface.
  • the above simple region merging can be to merge some connected regions with fewer faces with their adjacent connected regions, for example, The connected areas whose number is less than a preset value are merged with the connected areas connected to them, wherein the preset value can be reasonably set according to actual needs, for example, 4, 3, or 2. It can be understood that in the process of simple area merging, it is necessary to ensure that there is no overlap between the projection areas of each surface of the merged connected area.
  • the result of the parameterized final partition of the first reconstructed mesh can be obtained, and these partitions are arranged in a two-dimensional image according to certain rules to obtain the final texture coordinates of all vertices, that is, the final parameterized result of the first reconstructed mesh.
  • the above two-dimensional arrangement may include the following process:
  • the scaling ratio of the entire first reconstructed mesh vertices is calculated so that they can be arranged in a two-dimensional image of a specified size. Then, the width and height occupied by each connected area after projection are calculated respectively. Next, the image occupied by the connected area is rasterized, that is, the pixel value of the area occupied by the triangle face is set to 1, and the pixel value of the rest of the area is set to 0.
  • each connected region try to arrange it on the final image, that is, for the current connected region to be arranged, traverse each pixel block on the two-dimensional image as the upper left corner of the connected region, and then try different placement directions. If it is found that the connected region can be placed in the area with the current pixel as the upper left corner (that is, the occupied area of the current connected region on the two-dimensional image does not occupy other placed areas), then all pixels in the area are set to occupied and the texture coordinates of all points in the current connected region are adjusted (including rotation, scaling and translation), and then the position of the upper left corner of the current connected region on the two-dimensional image and the width and height of the current connected region are recorded; if it is still not successfully placed after traversing all pixels, the current connected region is scaled, and then the scaled connected region is re-tried to be arranged on the two-dimensional image. If the current connected region still cannot be successfully placed after scaling, the entire grid is scaled and then rearranged.
  • the final parameterization result of the first reconstructed mesh that is, the texture coordinates of the first reconstructed mesh
  • the texture coordinates of the first reconstructed mesh can be obtained, and accordingly the first reconstructed mesh including the texture coordinates is obtained.
  • a step of extracting connected areas may be included, as shown in FIG7a.
  • a connected area may be split into multiple connected areas, and there may be some mutually disconnected faces or areas in these split connected areas, as shown in FIG6c.
  • the above-mentioned extraction of connected areas is to extract mutually disconnected areas in a connected area, and each is projected as an independent connected area, as shown in FIG7b.
  • the above-mentioned simple area merging can be a simple area merging of the connected areas obtained after extracting the connected areas.
  • the encoding end after the encoding end performs the first processing on the at least one first connected region respectively, it merges the connected regions obtained after the first processing again, which can further reduce the number of connected regions and further improve the encoding efficiency.
  • the encoding end performs a merging process on the connected regions obtained after the first process to obtain at least one second connected region, including:
  • the encoder end obtains a candidate region set for a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region among the connected regions obtained after the first processing, and the third connected region is any connected region among the connected regions obtained after the first processing;
  • the encoder obtains at least one target candidate region of the merged candidate region set, wherein when the third connected region is merged into the target candidate region, the projection region of the third connected region is adjacent to the target candidate region.
  • the projected areas of the regions do not overlap;
  • the encoder merges the third connected region into one of the at least one target candidate region.
  • the target candidate region is a candidate region whose projection area does not overlap with the projection area of the third connected region after being merged with the third connected region.
  • the projection surface of the target candidate region and the third connected region may be the projection surface corresponding to the target candidate region, or may be the projection surface corresponding to the merged connected region, etc.
  • the above merging of the third connected region into one of the at least one target candidate region may, for example, merge the third connected region into any target candidate region, or merge the third connected region into the target candidate region with the largest number of faces.
  • the third connected region may be a connected region obtained after the first processing and having a number of faces less than a preset value, or a connected region obtained by splitting connected regions with overlapping regions.
  • the encoder traverses each candidate area in the merged candidate area list, tries to merge the current connected area into the candidate area, and then detects whether there will be overlap after projection.
  • the projection surface can be the projection surface corresponding to the current merged candidate area.
  • the overlap detection process here is the same as the above-mentioned overlapping area detection process. If the projection area after projection overlaps, the current candidate area is deleted from the merged candidate area list. In this way, the remaining candidate areas in the merged candidate area list are areas that can be merged with the current area and will not overlap after projection, that is, the above-mentioned target candidate areas.
  • This embodiment merges the candidate areas whose projection areas after merging do not overlap with the projection area of the third connected area from the connected areas adjacent to the third connected area. This ensures that the projection areas of the connected areas after merging do not overlap, thereby ensuring the accuracy of the texture coordinates of the obtained mesh.
  • the encoder merges the third connected region into one of the at least one target candidate region, including:
  • the encoder merges the third connected region into the first target candidate region
  • the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.
  • the target candidate area with the smallest angle between the normal vectors of the projection plane and the third connected area that is, the target candidate area with the closest normal vectors to the projection plane and the third connected area, can ensure that the distortion after projection is minimized.
  • each target candidate area can be traversed, and the target candidate area whose normal vector is closest to the projection surface and the third connected area can be selected as the first target candidate area to ensure that the distortion after projection is minimized; when there are multiple target candidate areas whose normal vector is closest to the projection surface and the third connected area, the target candidate area with the largest number of faces can be selected from the multiple target candidate areas as the first target candidate area, which is helpful in reducing the number of connected areas.
  • merging the third connected region into the first target candidate region means adding all faces in the third connected region to the first target candidate region, deleting the original third connected region, and further obtaining texture coordinates of newly added vertices in the merged connected region.
  • the first grid is determined according to a simplified grid
  • the simplified grid is a grid obtained by performing grid simplification processing on the three-dimensional grid to be encoded
  • the number of vertices of the simplified mesh is smaller than the number of vertices of the three-dimensional mesh to be encoded, or the number of faces of the simplified mesh is smaller than the number of faces of the three-dimensional mesh to be encoded.
  • the first mesh is determined according to the simplified mesh.
  • the first mesh may be the simplified mesh, or the first mesh may be a mesh obtained by displacing each vertex of the simplified mesh.
  • the above-mentioned mesh simplification process can be understood as simplifying the three-dimensional mesh to be encoded into a mesh with relatively fewer points and faces, and maintaining the shape of the original mesh as much as possible.
  • the vertices at both ends of the edge of the three-dimensional mesh to be encoded are merged into one vertex and the connection between the two vertices is deleted.
  • the entire mesh repeats this process according to certain rules to reduce the number of faces and vertices of the mesh to the target value, wherein the above target value can be determined according to the number required for lossy encoding.
  • a certain error metric can be selected to optimize the simplified result. For example, the sum of the equation coefficients of all adjacent faces of the vertex can be selected as the error metric of the vertex, that is, the error of the vertex is measured by the sum of the equation coefficients of all adjacent faces of the vertex.
  • the error of the corresponding edge is the sum of the errors of the two vertices on the edge, that is, the error generated by the merging of an edge is the sum of the distances from the merged vertex to the original two vertices of the edge.
  • iterative mesh simplification begins, for example, calculating the errors of the vertices of the three-dimensional mesh to be encoded, thereby obtaining the errors of each edge of the three-dimensional mesh to be encoded.
  • each edge is arranged from small to large in error, and the edge with the smallest error is selected for merging each time.
  • the position of the merged vertex is calculated, and the errors of all edges related to the merged vertex are updated. That is, the order of edge arrangement is updated to ensure that each iteration is based on the global error metric.
  • the faces of the 3D mesh to be encoded are simplified to the number required for lossy encoding through iteration.
  • This embodiment performs mesh simplification processing on the three-dimensional mesh to be encoded to obtain a simplified mesh, and compresses and encodes the first mesh determined by the simplified mesh. Since the number of faces or vertices of the mesh is reduced after the mesh simplification processing, this can not only improve the efficiency of mesh encoding, but also further reduce the transmission resources or storage resources required for the mesh code stream.
  • the method further comprises:
  • the encoding end subdivides and deforms the simplified grid to obtain a second grid
  • the encoding end converts the first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, including:
  • the encoding end calculates the displacement of each vertex of the second mesh according to the second reconstructed mesh and the second mesh;
  • the encoding end encodes the displacement of each vertex of the second mesh to obtain a displacement code stream
  • the encoding end performs three-dimensional grid reconstruction according to the first reconstruction grid and the displacement code stream to obtain a third reconstruction grid;
  • the encoding end converts the first texture map according to the three-dimensional grid to be encoded and the third reconstructed grid to obtain a second texture map.
  • the basic idea of subdivision deformation can be shown in FIG9 , where the input 2D curve (represented by a 2D polyline), called the original curve, is first downsampled to generate a basic curve/polyline, called a simplified curve. Then, a subdivision curve can be generated based on the polyline of the simplified curve.
  • the polyline of the subdivision curve is deformed to obtain a deformed curve that is closer to the original curve, that is, a displacement vector is calculated for each vertex of the subdivision curve, as shown by the arrow in FIG9 , so that the shape of the subdivision deformed curve is as close to the shape of the original curve as possible; and then each vertex can be moved correspondingly according to the displacement vector of each vertex of the subdivision curve to obtain a subdivision deformed curve with a shape close to the original curve.
  • the simplified mesh may be subdivided to obtain a subdivided mesh, wherein the specific implementation method of the subdivision may be arbitrarily selected, and one optional implementation method is the midpoint subdivision method, where the faces included in the simplified mesh are triangular faces, for example, each triangular face may be subdivided into four sub-triangular faces in each subdivision iteration, as shown in FIG10.
  • a new vertex is introduced in the middle of each edge of the simplified mesh, and the subdivision of the geometric information and the attribute information is performed independently, because the connection relationship between the geometric information and the attribute information is usually different.
  • the position Pos(v 12 ) of the midpoint v 12 of the newly introduced edge (v 1 ,v 2 ) can be calculated using formula (8):
  • Pos(v 1 ) and Pos(v 2 ) are the geometric coordinates of vertices v 1 and v 2.
  • the texture coordinates of the midpoint v 12 can be calculated in the same way as the geometric coordinates, which will not be described in detail here.
  • the search can be accelerated by using a data structure such as a k-dimensional tree (kdTree).
  • kdTree k-dimensional tree
  • the displacement of the geometric coordinates of each vertex of the subdivided mesh is obtained by calculating the distance between the geometric coordinates of each vertex on the subdivided mesh and its nearest neighbor point on the simplified mesh. Then, the corresponding vertex can be moved based on the displacement of the geometric coordinates of each vertex of the subdivided mesh to obtain a subdivided deformed mesh, that is, the second mesh mentioned above.
  • the encoder can calculate the displacement of each vertex of the subdivided deformed mesh based on the subdivided deformed mesh and the second reconstructed mesh. For example, the second reconstructed mesh can be subdivided to determine the correspondence between each vertex of the subdivided second reconstructed mesh and each vertex of the subdivided deformed mesh, and the displacement of each vertex can be calculated according to each vertex of the subdivided second reconstructed mesh and each vertex of the corresponding subdivided deformed mesh. Then, the encoder can encode the calculated displacement. In this embodiment, the above encoding method of the calculated displacement is not limited.
  • the calculated displacement can be encoded by a video encoding method, which can specifically include the following steps: converting the coordinate system where the displacement is located, that is, converting the coordinate system of the displacement of each vertex to a coordinate system constructed by the normal vector of the vertex corresponding to the displacement and two components tangent to the normal vector, and then transforming the displacement, such as wavelet transform, quantizing the transformed coefficients, and arranging them in the image in the order of scanning, and applying video encoding to the image; or, the calculated displacement can be directly encoded by entropy coding.
  • the video encoding method may include the following steps:
  • the generated displacements are in the same global coordinate system as the input mesh.
  • An optional optimization method is to transform them into a local coordinate system, where the local coordinate system of each vertex is defined by the vertex normal vector on the subdivided deformed mesh.
  • the advantage of this approach is that the normal component of the displacement has a more significant impact on the quality of the reconstructed mesh than the two tangential components, so a larger quantization parameter can be set for the tangential component.
  • wavelet transform For the quantized displacement, wavelet transform can be applied thereto.
  • An optional wavelet transform such as linear wavelet transform is shown below:
  • v is the newly inserted midpoint on the edge (v 1 ,v 2 ), Signal(v), Signal(v 1 ) and Signal(v 2 ) are the displacements corresponding to vertices v, v 1 and v 2 respectively, and “ ⁇ ” means updating the content on the left side of the arrow by the content on the right side of the arrow.
  • v * is the set of all adjacent vertices of vertex v.
  • disp[v] represents the wavelet coefficient of the displacement of the vth vertex
  • d[k] represents the kth coefficient of its displacement
  • floor represents rounding down
  • bitDepthPosition represents the bit depth of the current grid position
  • qp[k] represents the quantization parameter of the kth coefficient.
  • lodScale[k] represents the coefficient of the quantization parameter of the current subdivision level.
  • the position of the corresponding NxM pixel block on the image is calculated according to the Morton order.
  • the arrangement method is not limited here, that is, other arrangement schemes can be used, such as zigzag order, raster
  • the encoder can explicitly specify the corresponding arrangement scheme in the code stream.
  • the video encoder After arranging the wavelet coefficients on the two-dimensional image, the video encoder can be used to encode it directly.
  • the solution of the present application is independent of which video encoding technology is used, and the type of video encoder can be explicitly specified in the bit stream.
  • the video encoding here can use a lossless method, and there is no need to reconstruct the displacement at the encoding end.
  • the above-mentioned calculated displacement is directly compressed and encoded using the existing entropy coding.
  • the calculated displacement can be compressed and encoded using Context Adaptive Binary Arithmetic Coding (CABAC).
  • CABAC Context Adaptive Binary Arithmetic Coding
  • the present embodiment can also directly use entropy coding for the coefficients after wavelet transformation. Corresponding quantization can be performed before entropy coding.
  • the encoding end can reconstruct the displacement based on the displacement code stream, and perform three-dimensional mesh reconstruction based on the reconstructed displacement and the first reconstructed mesh to obtain a third reconstructed mesh.
  • the first reconstructed mesh can be meshed subdivided to obtain a subdivided first reconstructed mesh, and then the vertices of the first reconstructed mesh corresponding to the displacement of each reconstructed vertex can be moved to obtain a third reconstructed mesh, which is also a reconstructed subdivided deformed mesh.
  • the first texture map can be converted according to the three-dimensional mesh to be encoded and the third reconstructed mesh to obtain a second texture map.
  • the process of converting the first texture map according to the three-dimensional mesh to be encoded and the third reconstructed mesh can refer to the relevant description of the aforementioned texture map conversion method, which will not be repeated here.
  • the simplified mesh is subdivided and deformed to obtain a second mesh
  • the displacement of each vertex of the second mesh is calculated based on the second reconstructed mesh and the second mesh
  • the displacement of each vertex of the second mesh is encoded to obtain a displacement code stream
  • the three-dimensional mesh is reconstructed based on the first reconstructed mesh and the displacement code stream to obtain a third reconstructed mesh, thereby ensuring the quality of the reconstructed mesh.
  • the method further comprises:
  • the encoding end determines the displacement of the geometric coordinates of each vertex of the simplified mesh according to the displacement of the geometric coordinates of each vertex of the subdivided mesh; the encoding end moves each vertex of the simplified mesh according to the displacement of the geometric coordinates of each vertex of the simplified mesh to obtain a first mesh.
  • the displacement of the geometric coordinates of each vertex of the subdivided mesh can be referred to the relevant description of the above embodiment, which will not be repeated here.
  • the method further comprises:
  • the encoding end performs a first operation on first encoding information, wherein the first encoding information includes the grid code stream and the texture map code stream, and the first operation includes at least one of the following: storage, transmission.
  • the first coding information further includes at least one of the following:
  • the first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.
  • the above-mentioned first encoding information when the encoding end encodes to obtain the above-mentioned displacement code stream, the above-mentioned first encoding information also includes the displacement code stream, and when the above-mentioned displacement code stream is not encoded to obtain the above-mentioned displacement code stream, the above-mentioned first encoding information does not include the displacement code stream.
  • the above information related to grid coding for example, the intra-frame coding flag, used to indicate whether the current frame needs to be constructed according to the reference frame grid; the reference frame list, used to indicate the index of the reference frame required for the current frame, and the reference frame list can be applied to the reconstruction of the first grid; the static grid encoder type, etc.
  • the above information related to texture map coding for example, the video encoder type.
  • the above information related to grid subdivision and deformation for example, the subdivision deformation flag, used to indicate whether the reconstructed grid needs to be subdivided and deformed; the subdivision method, that is, the method of grid subdivision when reconstructing the subdivided deformed grid.
  • the above information related to grid displacement for example, the encoding method of displacement; the displacement transformation method; the wavelet coefficient arrangement method, etc.
  • the above information related to grid parameterization processing for example, the algorithm used for grid parameterization processing (for example, Isochart algorithm, orthogonal projection algorithm, etc.).
  • the decoding end decodes the grid code stream according to the information related to the grid coding, decodes the texture map code stream according to the information related to the texture map coding, subdivides and deforms the decoded grid according to the information related to the grid subdivision and deformation, performs grid parameterization on the decoded grid according to the information related to the grid parameterization processing, and decodes the displacement code stream according to the information related to the grid displacement. This is conducive to the decoding end to decode the encoded grid more accurately.
  • the encoding end performs mesh simplification processing on the input mesh to obtain a simplified mesh, and subdivides and deforms the simplified mesh to obtain a base mesh (i.e., a first mesh) and a subdivided deformed mesh (i.e., a second mesh).
  • the base mesh is compressed and encoded to obtain a mesh code stream.
  • the base mesh is reconstructed based on the mesh code stream to obtain a reconstructed base mesh (i.e., a first reconstructed mesh).
  • the first reconstructed mesh is mesh parameterized to obtain a first reconstructed mesh (i.e., a second reconstructed mesh) containing texture coordinates.
  • the displacement of each vertex of the subdivided deformed mesh is calculated based on the subdivided deformed mesh and the second reconstructed mesh.
  • the displacement may include the displacement of the geometric coordinates and the displacement of the texture coordinates.
  • the displacement of each vertex calculated is transformed, quantized, and the like, and the processed displacement is encoded to obtain a displacement code stream.
  • the encoding end performs displacement reconstruction based on the displacement code stream to obtain a reconstructed location, and reconstructs a subdivision deformed grid (i.e., a third reconstructed grid) based on the reconstructed displacement and the first reconstructed grid, and converts the input texture map (i.e., the first texture map) based on the reconstructed subdivision deformed grid and the input grid to obtain a converted texture map (i.e., the second texture map), and uses a video encoder to encode the converted texture map to obtain a texture map code stream, and then the encoding end can store or transmit the obtained grid code stream, texture map code stream and displacement code stream.
  • a subdivision deformed grid i.e., a third reconstructed grid
  • FIG. 12 is a flowchart of a three-dimensional grid decoding method provided in an embodiment of the present application. The method can be executed by a decoding end, as shown in FIG. 12 , and includes the following steps:
  • Step 1201 The decoding end obtains a grid code stream and a texture map code stream.
  • the mesh code stream and texture map code stream are generated by the encoder according to the three-dimensional mesh coding method.
  • the decoder can receive the mesh code stream and texture map code stream from the encoder.
  • Step 1202 The decoding end decodes the grid code stream to obtain a third grid, and decodes the texture map code stream to obtain a third texture map.
  • the third grid is a grid obtained after the grid code stream is decoded.
  • the third grid can also be called a reconstructed basic grid.
  • the third texture map is a texture map obtained after the texture code stream is decoded.
  • the decoding end should use a decoding method corresponding to the grid coding method of the encoding end to decode the grid code stream. For example, if the encoding end uses intra-frame mode encoding, the decoding end also uses intra-frame mode decoding; if the encoding end uses inter-frame mode encoding, the decoding end also uses inter-frame mode decoding; if the encoding end uses skip mode encoding, the decoding end also uses skip mode decoding.
  • the decoding end also uses intra-frame mode decoding.
  • a static grid decoder is used for decoding; if it is an inter-frame mode, the corresponding motion vector is decoded, and the corresponding basic grid is reconstructed according to the reference frame in the grid buffer; if it is a skip mode, the corresponding reference frame in the grid buffer is directly used as the basic grid of the current frame.
  • the decoding end should use a decoding method corresponding to the texture map encoding method of the encoding end to decode the texture map bitstream. For example, if the encoding end uses a video encoder, the decoding end uses the corresponding video encoder to decode the texture map bitstream. In some optional embodiments, the decoding end can also perform color space conversion on the texture map obtained by decoding to obtain an image format consistent with the texture map input by the encoding end.
  • Step 1203 The decoding end performs mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.
  • the decoding end can use the same mesh parameterization processing method as the encoding end to perform mesh parameterization processing on the third mesh to obtain a third mesh containing texture coordinates, that is, the fourth mesh. Then, the decoding end can apply the third texture map and the fourth mesh.
  • a mesh obtained by decoding a mesh code stream at a decoding end is subjected to mesh parameterization processing to obtain the texture coordinates of the mesh. This can reduce the storage resources or transmission resources occupied by the code stream of the three-dimensional mesh while ensuring the texture coordinates of the mesh that can be obtained by decoding.
  • the decoding end performs grid parameterization processing on the third grid to obtain a fourth grid, including:
  • the decoding end performs grid parameterization processing on the third grid by using an orthogonal projection algorithm to obtain a fourth grid.
  • the decoding end performs grid parameterization processing on the third grid using an orthogonal projection algorithm to obtain a fourth grid, including:
  • the decoding end divides the third grid into at least two connected regions, wherein each of the connected regions includes at least one face of the third grid, and faces having the same projection face and being connected in three-dimensional space belong to the same connected region;
  • the decoding end merges the at least two connected regions to obtain at least one first connected region
  • the decoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each of the first connected regions to detect overlapping regions, and splitting the first connected regions with overlapping regions into at least two non-overlapping connected regions;
  • the decoding end merges the connected regions obtained after the first processing to obtain at least one second connected region
  • the decoding end arranges the projection area of the at least one second connected area in two dimensions to obtain a fourth grid.
  • the decoding end merges the connected regions obtained after the first processing to obtain at least one
  • the second connected region includes:
  • the decoding end obtains a candidate region set of a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region in the connected regions obtained after the first processing, and the third connected region is any connected region in the connected regions obtained after the first processing;
  • the decoding end obtains at least one target candidate region of the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection region of the third connected region does not overlap with a projection region of the target candidate region;
  • the decoding end merges the third connected region into one of the at least one target candidate region.
  • the decoding end merges the third connected region into one of the at least one target candidate region, comprising:
  • the decoding end merges the third connected region into the first target candidate region
  • the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.
  • the method further comprises:
  • the decoding end obtains the displacement code stream
  • the decoding end subdivides and deforms the fourth grid to obtain a fifth grid
  • the decoding end reconstructs a three-dimensional grid according to the fifth grid and the displacement obtained by decoding the displacement code stream to obtain a fourth reconstructed grid.
  • the decoder when the decoder obtains the displacement code stream, it decodes the displacement code stream. For example, if the encoder compresses the displacement by video encoding, the decoder decodes it by a corresponding video decoder and restores it from the two-dimensional image in the corresponding order according to the arrangement scheme. Then, it performs inverse transformation, inverse quantization and other operations to restore the displacement consistent with the encoder. If entropy encoding is used, it can be directly entropy decoded.
  • the subdivided deformed mesh is reconstructed based on the two parts.
  • subdivmesh[i].v[k] is the geometric coordinates of the kth vertex after the base mesh is subdivided in the current frame (index is i)
  • displacement[k] is the displacement corresponding to the kth vertex
  • deformedmesh[i].v[k] is the geometric coordinates of the kth vertex after subdivision and deformation in the current frame.
  • the method further comprises:
  • the decoding end obtains first auxiliary information, wherein the first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.
  • the decoding end when it obtains the first auxiliary information, it can perform decoding based on the first auxiliary information. For example, the decoding end decodes the grid code stream according to information related to grid encoding, decodes the texture map code stream according to information related to texture map encoding, subdivides and deforms the decoded grid according to information related to grid subdivision and deformation, performs grid parameterization on the decoded grid according to information related to grid parameterization, and decodes the displacement code stream according to information related to grid displacement. This is conducive to the decoding end to decode the encoded grid more accurately.
  • the above-mentioned first auxiliary information can be encoded into an independent code stream, or it can be carried in the above-mentioned grid code stream, texture map code stream and displacement code stream.
  • information related to grid coding is carried in the above-mentioned grid code stream
  • information related to texture map coding is carried in the above-mentioned texture map code stream
  • information related to grid displacement is carried in the displacement code stream, etc.
  • the decoding end obtains the decoding of each part of the code stream and reconstructs it. After the basic grid is reconstructed, the grid parameterization processing is performed to obtain the corresponding basic grid containing texture coordinates. The basic grid is subdivided and the displacement obtained by decoding is applied to obtain the subdivided deformed grid reconstructed by the decoding end, that is, the reconstructed grid finally output by the decoding end.
  • the 3D mesh coding method provided in the embodiment of the present application can be executed by a 3D mesh coding device, or a control module in the 3D mesh coding device for executing the 3D mesh coding method.
  • the 3D mesh coding device executing the 3D mesh coding method is taken as an example to illustrate the 3D mesh coding device provided in the embodiment of the present application.
  • FIG. 15 is a structural diagram of a three-dimensional grid encoding device provided in an embodiment of the present application.
  • the three-dimensional grid encoding device 1500 includes:
  • a first encoding module 1501 is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded;
  • a first reconstruction module 1502 is used to perform three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid
  • a first processing module 1503 is configured to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates;
  • a conversion module 1504 is used to convert the first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is the texture map corresponding to the three-dimensional mesh to be encoded;
  • the second encoding module 1505 is used to encode the second texture map to obtain a texture map bit stream.
  • the first processing module is specifically configured to:
  • An orthogonal projection algorithm is used to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh.
  • the first processing module is specifically configured to:
  • Performing a first process on each of the at least one first connected regions wherein the first process includes projecting each surface of each of the first connected regions to detect overlapping regions, and splitting the first connected regions with overlapping regions into at least two non-overlapping connected regions;
  • the projection area of the at least one second connected area is two-dimensionally arranged to obtain a second reconstructed grid.
  • the first processing module is specifically configured to:
  • the candidate region set includes at least one candidate region
  • the candidate region is a connected region adjacent to the third connected region in the connected regions obtained after the first processing
  • the third connected region is any connected region in the connected regions obtained after the first processing
  • the third connected region is merged into one of the at least one target candidate region.
  • the first processing module is specifically configured to:
  • the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.
  • the first grid is determined according to a simplified grid
  • the simplified grid is a grid obtained by performing grid simplification processing on the three-dimensional grid to be encoded
  • the number of vertices of the simplified mesh is smaller than the number of vertices of the three-dimensional mesh to be encoded, or the number of faces of the simplified mesh is smaller than the number of faces of the three-dimensional mesh to be encoded.
  • the device further comprises:
  • a first subdivision deformation module used for performing subdivision deformation on the simplified mesh to obtain a second mesh
  • the conversion module is specifically used for:
  • the first texture map is converted according to the three-dimensional grid to be encoded and the third reconstructed grid to obtain a second texture map.
  • the device further comprises:
  • the first execution module is used to perform a first operation on first coded information, wherein the first coded information includes the grid code stream and the texture map code stream, and the first operation includes at least one of the following: storage and transmission.
  • the first coding information further includes at least one of the following:
  • the first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.
  • the three-dimensional grid coding device in the embodiment of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip.
  • the electronic device can be a terminal or a network-side device, or it can be other devices other than a terminal or a network-side device.
  • the terminal can include but is not limited to the types of terminals 11 listed above
  • the network-side device can include but is not limited to the types of network-side devices 12 listed above.
  • Other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiment of the present application.
  • the three-dimensional grid coding device provided in the embodiment of the present application can implement each process implemented by the method embodiment of Figure 3 and achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • the 3D mesh decoding method provided in the embodiment of the present application can be executed by a 3D mesh decoding device, or a control module in the 3D mesh decoding device for executing the 3D mesh decoding method.
  • the 3D mesh decoding device executing the 3D mesh decoding method is taken as an example to illustrate the 3D mesh decoding device provided in the embodiment of the present application.
  • FIG. 16 is a structural diagram of a three-dimensional grid decoding device provided in an embodiment of the present application.
  • the three-dimensional grid decoding device 1600 includes:
  • the first acquisition module 1601 is used to acquire a grid code stream and a texture map code stream;
  • a decoding module 1602 configured to decode the mesh code stream to obtain a third mesh, and decode the texture map code stream to obtain a third texture map;
  • the second processing module 1603 is used to perform mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.
  • the second processing module is specifically configured to:
  • An orthogonal projection algorithm is used to perform grid parameterization processing on the third grid to obtain a fourth grid.
  • the second processing module is specifically configured to:
  • the first processing is performed on the at least one first connected area respectively, wherein the first processing includes projecting each surface of each first connected area to detect overlapping areas, and projecting the first connected areas with overlapping areas.
  • the region is split into at least two non-overlapping connected regions;
  • the projection area of the at least one second connected area is two-dimensionally arranged to obtain a fourth grid.
  • the second processing module is specifically configured to:
  • the candidate region set includes at least one candidate region
  • the candidate region is a connected region adjacent to the third connected region in the connected regions obtained after the first processing
  • the third connected region is any connected region in the connected regions obtained after the first processing
  • the third connected region is merged into one of the at least one target candidate region.
  • the second processing module is specifically configured to:
  • the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.
  • the device further comprises:
  • the second acquisition module is used to acquire the displacement code stream
  • a second subdivision deformation module used for performing subdivision deformation on the fourth mesh to obtain a fifth mesh
  • the second reconstruction module is used to perform three-dimensional grid reconstruction according to the fifth grid and the displacement obtained by decoding the displacement code stream to obtain a fourth reconstructed grid.
  • the device further comprises:
  • the third acquisition module is used to obtain first auxiliary information, wherein the first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.
  • the three-dimensional grid decoding device in the embodiment of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip.
  • the electronic device can be a network-side device or a terminal, or can be other devices other than network-side devices or terminals.
  • the network-side device can include but is not limited to the types of network-side devices 12 listed above
  • the terminal can include but is not limited to the types of terminals 11 listed above
  • other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiment of the present application.
  • the three-dimensional grid decoding device provided in the embodiment of the present application can implement each process implemented by the method embodiment of Figure 12 and achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • the embodiment of the present application further provides a communication device 1700, including a processor 1701 and
  • the memory 1702 stores a program or instruction that can be run on the processor 1701.
  • the program or instruction is executed by the processor 1701 to implement the various steps of the above-mentioned three-dimensional mesh encoding method embodiment, and can achieve the same technical effect.
  • the communication device 1700 is a decoding end
  • the program or instruction is executed by the processor 1701 to implement the various steps of the above-mentioned three-dimensional mesh decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it is not repeated here.
  • the embodiment of the present application also provides an encoding end, including a processor and a communication interface, the processor is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded; based on the grid code stream, a three-dimensional grid is reconstructed to obtain a first reconstructed grid; the first reconstructed grid is mesh parameterized to obtain a second reconstructed grid, wherein the second reconstructed grid is a first reconstructed grid including texture coordinates; according to the three-dimensional grid to be encoded and the second reconstructed grid, a first texture map is converted to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional grid to be encoded; the second texture map is encoded to obtain a texture map code stream.
  • the processor is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded
  • FIG. 18 is a schematic diagram of the hardware structure of an encoding end that implements an embodiment of the present application.
  • the encoding end 1800 includes but is not limited to: a radio frequency unit 1801, a network module 1802, an audio output unit 1803, an input unit 1804, a sensor 1805, a display unit 1806, a user input unit 1807, an interface unit 1808, a memory 1809 and at least some of the components of the processor 1810.
  • the encoding end 1800 may also include a power supply (such as a battery) for supplying power to each component, and the power supply may be logically connected to the processor 1810 through a power management system, so as to manage charging, discharging, and power consumption management through the power management system.
  • a power supply such as a battery
  • the encoding end structure shown in FIG18 does not constitute a limitation on the encoding end, and the encoding end may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently, which will not be described in detail here.
  • the input unit 1804 may include a graphics processing unit (GPU) 18041 and a microphone 18042, and the graphics processor 18041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode.
  • the display unit 1806 may include a display panel 18061, and the display panel 18061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc.
  • the user input unit 1807 includes a touch panel 18071 and at least one of other input devices 18072.
  • the touch panel 18071 is also called a touch screen.
  • the touch panel 18071 may include two parts: a touch detection device and a touch controller.
  • Other input devices 18072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
  • the RF unit 1801 can transmit the data to the processor 1810 for processing; in addition, the RF unit 1801 can send uplink data to the network side device.
  • the RF unit 1801 includes but is not limited to an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.
  • the memory 1809 may be used to store software programs or instructions and various data.
  • the memory 1809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, Applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc.
  • the memory 1809 may include a volatile memory or a non-volatile memory.
  • the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.
  • the volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM).
  • RAM random access memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • SDRAM synchronous dynamic random access memory
  • DDRSDRAM double data rate synchronous dynamic random access memory
  • ESDRAM enhanced synchronous dynamic random access memory
  • SLDRAM synchronous link dynamic random access memory
  • DRRAM direct memory bus random access memory
  • the processor 1810 may include one or more processing units; optionally, the processor 1810 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1810.
  • processor 1810 is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded; reconstruct a three-dimensional grid based on the grid code stream to obtain a first reconstructed grid; perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, wherein the second reconstructed grid is a first reconstructed grid including texture coordinates; convert a first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional grid to be encoded; encode the second texture map to obtain a texture map code stream.
  • the embodiment of the present application also provides a decoding end, including a processor and a communication interface, the processor is used to obtain a grid code stream and a texture map code stream; decode the grid code stream to obtain a third grid, and decode the texture map code stream to obtain a third texture map; perform grid parameterization processing on the third grid to obtain a fourth grid, wherein the fourth grid is a grid including texture coordinates.
  • This decoding end embodiment corresponds to the above-mentioned decoding end method embodiment, and each implementation process and implementation method of the above-mentioned method embodiment can be applied to this decoding end embodiment, and can achieve the same technical effect.
  • the embodiment of the present application also provides a decoding end.
  • the decoding end 1900 includes: an antenna 1901, a radio frequency device 1902, a baseband device 1903, a processor 1904 and a memory 1905.
  • the antenna 1901 is connected to the radio frequency device 1902.
  • the radio frequency device 1902 receives information through the antenna 1901 and sends the received information to the baseband device 1903 for processing.
  • the baseband device 1903 processes the information to be sent and sends it to the radio frequency device 1902.
  • the radio frequency device 1902 processes the received information and sends it out through the antenna 1901.
  • the method executed by the decoding end in the above embodiment can be implemented in the baseband device 1903, and the baseband device 1903 includes Baseband processor.
  • the baseband device 1903 may include, for example, at least one baseband board, on which multiple chips are arranged, as shown in Figure 19, one of which is, for example, a baseband processor, which is connected to the memory 1905 through a bus interface to call the program in the memory 1905 and execute the network device operations shown in the above method embodiment.
  • the decoding end may also include a network interface 1906, which may be, for example, a Common Public Radio Interface (CPRI).
  • CPRI Common Public Radio Interface
  • the decoding end 1900 of the embodiment of the present application also includes: instructions or programs stored in the memory 1905 and executable on the processor 1904.
  • the processor 1904 calls the instructions or programs in the memory 1905 to execute the methods executed by the modules shown in Figure 16 and achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored.
  • a program or instruction is stored.
  • the various processes of the above-mentioned three-dimensional grid coding method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
  • the processor is the processor in the terminal described in the above embodiment.
  • the readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
  • the readable storage medium may be a non-transient readable storage medium.
  • An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned three-dimensional grid coding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
  • the embodiment of the present application further provides a computer program/program product, which is stored in a storage medium.
  • the computer program/program product is executed by at least one processor to implement the various processes of the above-mentioned three-dimensional grid coding method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
  • An embodiment of the present application also provides a three-dimensional grid encoding and decoding system, including: an encoding end and a decoding end, the encoding end is used to execute the various processes as shown in Figure 3 and the various method embodiments described above, and the decoding end is used to execute the various processes as shown in Figure 12 and the various method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请公开了一种三维网格编码方法、解码方法、装置、编码端及解码端，属于通信技术领域，本申请实施例的三维网格编码方法包括：编码端对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；所述编码端基于所述网格码流进行三维网格重建，得到第一重建网格；所述编码端对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；所述编码端根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；所述编码端对所述第二纹理图进行编码，得到纹理图码流。

Description

三维网格编码方法、解码方法、装置、编码端及解码端
相关申请的交叉引用
本申请主张在2023年06月30日在中国提交的中国专利申请No.202310800556.8的优先权，其全部内容通过引用包含于此。
技术领域
本申请属于通信技术领域，具体涉及一种三维网格编码方法、解码方法、装置、编码端及解码端。
背景技术
三维模型成为继音频、图像、视频之后的新一代数字化媒体，与传统的图像、视频等多媒体相比，具有更强的交互性和逼真性。三维网格是一种常用的三维模型表示方式，三维网格由顶点、边和面这三种基本元素组成。其中，顶点是三维网格中最基本的元素，其定义了一个三维空间中的位置；边是连接三维网格中两个顶点的线段；面为由三维网格的边的闭合路径形成的多边形。此外，通过网格参数化将三维网格从三维空间映射到二维平面区域，这种映射关系通常由一组参数坐标描述，称为UV坐标或纹理坐标，与三维网格的顶点相关联。相关技术中，在对三维网格进行编码后传输或者存储的情况下，对应的需要对该三维网格的纹理坐标进行编码后存储或传输，以实现在解码端对该三维网格的纹理坐标的重建，由于纹理坐标的码流往往占用三维网格的码流的大部分，这样不仅会导致编码效率较低，还会对存储资源或传输资源的占用。
发明内容
本申请实施例提供一种三维网格编码方法、解码方法、装置、编码端及解码端，能够提高三维网格的编码效率，并可以减少三维网格的码流对存储资源或传输资源的占用。
第一方面，提供了一种三维网格编码方法，该方法包括：
编码端对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；
所述编码端基于所述网格码流进行三维网格重建，得到第一重建网格；
所述编码端对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；
所述编码端根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；
所述编码端对所述第二纹理图进行编码，得到纹理图码流。
第二方面，提供了一种三维网格编码装置，该装置包括：
第一编码模块，用于对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；
第一重建模块，用于基于所述网格码流进行三维网格重建，得到第一重建网格；
第一处理模块，用于对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；
转换模块，用于根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；
第二编码模块，用于对所述第二纹理图进行编码，得到纹理图码流。
第三方面，提供了一种三维网格解码方法，该方法包括：
解码端获取网格码流和纹理图码流；
所述解码端对所述网格码流进行解码得到第三网格，并对所述纹理图码流进行解码得到第三纹理图；
所述解码端对所述第三网格进行网格参数化处理，得到第四网格，其中，所述第四网格为包括纹理坐标的网格。
第四方面，提供了一种三维网格解码装置，该装置包括：
第一获取模块，用于获取网格码流和纹理图码流；
解码模块，用于对所述网格码流进行解码得到第三网格，并对所述纹理图码流进行解码得到第三纹理图；
第二处理模块，用于对所述第三网格进行网格参数化处理，得到第四网格，其中，所述第四网格为包括纹理坐标的网格。
第五方面，提供了一种编码端，该编码端包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第一方面所述的方法的步骤。
第六方面，提供了一种编码端，包括处理器及通信接口，其中，所述处理器用于对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；基于所述网格码流进行三维网格重建，得到第一重建网格；对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；对所述第二纹理图进行编码，得到纹理图码流。
第七方面，提供了一种解码端，该解码端包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第三方面所述的方法的步骤。
第八方面，提供了一种解码端，包括处理器及通信接口，其中，所述处理器用于获取网格码流和纹理图码流；对所述网格码流进行解码得到第三网格，并对所述纹理图码流进 行解码得到第三纹理图；对所述第三网格进行网格参数化处理，得到第四网格，其中，所述第四网格为包括纹理坐标的网格。
第九方面，提供了一种三维网格编解码系统，包括：编码端及解码端，所述编码端可用于执行如第一方面所述的三维网格编码方法的步骤，所述解码端可用于执行如第三方面所述的三维网格解码方法的步骤。
第十方面，提供了一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如第一方面所述的方法的步骤，或者实现如第三方面所述的方法的步骤。
第十一方面，提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如第一方面所述的方法的步骤，或实现如第三方面所述的方法的步骤。
第十二方面，提供了一种计算机程序/程序产品，所述计算机程序/程序产品被存储在存储介质中，所述计算机程序/程序产品被至少一个处理器执行以实现如第一方面所述的方法的步骤，或实现如第三方面所述的方法的步骤。
在本申请实施例中，编码端对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；所述编码端基于所述网格码流进行三维网格重建，得到第一重建网格；所述编码端对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；所述编码端根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；所述编码端对所述第二纹理图进行编码，得到纹理图码流，也即本申请实施例中编码端仅生成了三维网格的网格码流和纹理图码流，而未对应生成三维网格的纹理坐标的码流，这样不仅可以提高三维网格的编码效率，还可以减少三维网格的码流对存储资源或传输资源的占用。此外，本申请实施例中对第一重建网格进行网格参数化处理，相比于现有技术中对第一网格进行网格参数化处理，不仅可以减少数据处理量，还可以保证第一重建网格的纹理坐标的准确性。
附图说明
图1是本申请实施例可应用的一种无线通信系统的框图；
图2是本申请实施例提供的一种基于正交投影算法的网格参数化处理的示意图之一；
图3是本申请实施例提供的一种三维网格编码方法的流程图；
图4是本申请实施例提供的基础网格压缩的示意图；
图5是本申请实施例提供的一种基于正交投影算法的网格参数化处理的示意图之二；
图6a是本申请实施例提供的存在重叠区域的连通区域拆分前的示意图之一；
图6b是本申请实施例提供的存在重叠区域的连通区域拆分后的示意图之一；
图6c是本申请实施例提供的存在重叠区域的连通区域拆分后的示意图之二；
图7a是本申请实施例提供的一种基于正交投影算法的网格参数化处理的示意图之三；
图7b是本申请实施例提供的提取连通区域的示意图；
图8是本申请实施例提供的网格简化的示意图；
图9是本申请实施例提供的网格细分变形的示意图；
图10是本申请实施例提供的网格细分的示意图；
图11是本申请实施例提供的三维网格编码框架的示意图；
图12是本申请实施例提供的一种三维网格解码方法的流程图；
图13是本申请实施例提供的基础网格解码的示意图；
图14是本申请实施例提供的三维网格解码框架的示意图；
图15是本申请实施例提供的一种三维网格编码装置的结构图；
图16是本申请实施例提供的一种三维网格解码装置的结构图；
图17是本申请实施例提供的通信设备的结构图；
图18是本申请实施例提供的编码端的结构图；
图19是本申请实施例提供的解码端的结构图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员所获得的所有其他实施例，都属于本申请保护的范围。
本申请的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施，且“第一”、“第二”所区别的对象通常为一类，并不限定对象的个数，例如第一对象可以是一个，也可以是多个。此外，本申请中的“或”表示所连接对象的至少其中之一。例如“A或B”涵盖三种方案，即，方案一：包括A且不包括B；方案二：包括B且不包括A；方案三：既包括A又包括B。字符“/”一般表示前后关联对象是一种“或”的关系。
本申请的术语“指示”既可以是一个直接的指示(或者说显式的指示)，也可以是一个间接的指示(或者说隐含的指示)。其中，直接的指示可以理解为，发送方在发送的指示中明确告知了接收方具体的信息、需要执行的操作或请求结果等内容；间接的指示可以理解为，接收方根据发送方发送的指示确定对应的信息，或者进行判断并根据判断结果确定需要执行的操作或请求结果等。
值得指出的是，本申请实施例所描述的技术不限于长期演进型(Long Term Evolution，LTE)/LTE的演进(LTE-Advanced，LTE-A)系统，还可用于其他无线通信系统，诸如码分多址(Code Division Multiple Access，CDMA)、时分多址(Time Division Multiple Access，TDMA)、频分多址(Frequency Division Multiple Access，FDMA)、正交频分多址(Orthogonal  Frequency Division Multiple Access，OFDMA)、单载波频分多址(Single-carrier Frequency-Division Multiple Access，SC-FDMA)或其他系统。本申请实施例中的术语“系统”和“网络”常被可互换地使用，所描述的技术既可用于以上提及的系统和无线电技术，也可用于其他系统和无线电技术。以下描述出于示例目的描述了新空口(New Radio，NR)系统，并且在以下大部分描述中使用NR术语，但是这些技术也可应用于NR系统以外的系统，如第6代(6th Generation，6G)通信系统。
图1示出本申请实施例可应用的一种无线通信系统的框图。无线通信系统包括终端11和网络侧设备12。其中，终端11可以是手机、平板电脑(Tablet Personal Computer)、膝上型电脑(Laptop Computer)、笔记本电脑、个人数字助理(Personal Digital Assistant，PDA)、掌上电脑、上网本、超级移动个人计算机(Ultra-mobile Personal Computer，UMPC)、移动上网装置(Mobile Internet Device，MID)、增强现实(Augmented Reality，AR)、虚拟现实(Virtual Reality，VR)设备、机器人、可穿戴式设备(Wearable Device)、飞行器(flight vehicle)、车载设备(Vehicle User Equipment，VUE)、船载设备、行人终端(Pedestrian User Equipment，PUE)、智能家居(具有无线通信功能的家居设备，如冰箱、电视、洗衣机或者家具等)、游戏机、个人计算机(Personal Computer，PC)、柜员机或者自助机等终端侧设备。可穿戴式设备包括：智能手表、智能手环、智能耳机、智能眼镜、智能首饰(智能手镯、智能手链、智能戒指、智能项链、智能脚镯、智能脚链等)、智能腕带、智能服装等。其中，车载设备也可以称为车载终端、车载控制器、车载模块、车载部件、车载芯片或车载单元等。需要说明的是，在本申请实施例并不限定终端11的具体类型。网络侧设备12可以包括接入网设备或核心网设备，其中，接入网设备也可以称为无线接入网(Radio Access Network,RAN)设备、无线接入网功能或无线接入网单元。接入网设备可以包括基站、无线局域网(Wireless Local Area Network，WLAN)接入点(Access Point，AP)或无线保真(Wireless Fidelity，WiFi)节点等。其中，基站可被称为节点B(Node B，NB)、演进节点B(Evolved Node B，eNB)、下一代节点B(the next generation Node B，gNB)、新空口节点B(New Radio Node B，NR Node B)、接入点、中继站(Relay Base Station，RBS)、服务基站(Serving Base Station，SBS)、基收发机站(Base Transceiver Station，BTS)、无线电基站、无线电收发机、基本服务集(Basic Service Set，BSS)、扩展服务集(Extended Service Set，ESS)、家用B节点(home Node B，HNB)、家用演进型B节点(home evolved Node B)、发送接收点(Transmission Reception Point，TRP)或所述领域中其他某个合适的术语，只要达到相同的技术效果，所述基站不限于特定技术词汇，需要说明的是，在本申请实施例中仅以NR系统中的基站为例进行介绍，并不限定基站的具体类型。
核心网设备可以包含但不限于如下至少一项：核心网节点、核心网功能、移动管理实体(Mobility Management Entity，MME)、接入移动管理功能(Access and Mobility Management Function，AMF)、会话管理功能(Session Management Function，SMF)、用户平面功能(User Plane Function，UPF)、策略控制功能(Policy Control Function，PCF)、策略与计费规则功 能单元(Policy and Charging Rules Function，PCRF)、边缘应用服务发现功能(Edge Application Server Discovery Function，EASDF)、统一数据管理(Unified Data Management，UDM)、统一数据仓储(Unified Data Repository，UDR)、归属用户服务器(Home Subscriber Server，HSS)、集中式网络配置(Centralized network configuration，CNC)、网络存储功能(Network Repository Function，NRF)、网络开放功能(Network Exposure Function，NEF)、本地NEF(Local NEF，或L-NEF)、绑定支持功能(Binding Support Function，BSF)、应用功能(Application Function，AF)等。需要说明的是，在本申请实施例中仅以NR系统中的核心网设备为例进行介绍，并不限定核心网设备的具体类型。
需要说明的是，本申请实施例的编码端可以是终端或者网络侧设备或者服务器等电子设备或者电子设备的部件。本申请实施例的解码端也可以是终端或者网络侧设备或者服务器等电子设备或者电子设备的部件。此外，本申请实施例的编码端和解码端可以位于同一电子设备，也可以位于不同的电子设备。
为了方便理解，以下对本申请实施例涉及的一些内容进行说明：
一、三维网格
近年来，随着多媒体技术的迅速发展，相关研究成果迅速产业化，并成为人们生活中不可或缺的重要组成部分。三维模型成为继音频、图像、视频之后的新一代数字化媒体。三维网格是一种常用的三维模型表示方式。三维网格与传统的图像、视频等多媒体相比具有更强的交互性和逼真性，使其在商业、制造业、建筑业、教育、医学、娱乐、艺术、军事等各个领域都得到了越来越广泛的应用。
尽管目前有许多种三维网格的表示方法，但三角网格仍是目前最通用的表示方法。三维网格可以被视为由顶点、边和面这三种基本元素组成。顶点是网格中最基本的元素，它们定义了一个三维空间中的位置。边是连接网格中两个顶点的线段。面可以视为由边的闭合路径形成的多边形。对于三角网格来说，每一个面都是一个三角形。
三维网格中包含的信息通常分为几何信息，连接信息以及属性信息这三类。其中，几何信息即三维网格的每个顶点在三维空间中的位置。连接信息描述了三维网格中各元素之间的关联关系，即顶点之间的连接关系。属性信息是一种可选的信息，它可以将属性关联到对应的网格元素中去(如顶点颜色、法向量等可以与网格顶点相关联)。此外，还可以利用网格参数化将三维网格从三维空间映射到二维平面区域，这种映射关系通常由一组参数坐标描述，称为UV坐标或纹理坐标，与网格顶点相关联。该二维映射可以用来表示高分辨率的属性信息，如纹理、法向量等。
在几乎所有使用三维网格的应用领域(如计算模拟、娱乐、医学成像、数字化文物、计算机设计、电子商务等)，随着人们对三维模型在视觉效果上越来越高的需求，三维模型正变得越来越复杂，三维模型的精度也越来越高，因此表示三维网格所需要的数据量也相应的增大。以上问题就导致了三维网格的处理、可视化、传输和存储都越来越复杂。三维网格压缩可以视为解决上述问题的一个途径，它可以减少模型数据的大小，有利于三维网 格的处理、存储和传输。
近来，音视频编码压缩领域的国际标准化组织，动态图专家组(Moving Pictures Experts Group，MPEG)开始制定面向三维网格的压缩标准，基于视频的动态网格压缩(Video-based Dynamic Mesh Coding，VDMC)，该标准是基于已有的基于视频的可视体积内容编码(Visual Volumetric Video-based Coding，V3C)标准指定的，V3C标准提供了一种压缩三维模型的通用方法，三维模型可以是由点云、网格或全景视频等形式呈现的。将三维网格的压缩方法与该标准兼容有助于该方法的推广以及适用性。因此，对VDMC中的三维网格编解码方法进行优化并将优化方法与V3C标准结合具有重要意义。一种可选的优化方法是针对位移编码方式的优化，现有框架中位移是通过计算重建的网格与原始的网格顶点之间的距离得到的，它的目的是提高网格的质量。此外，现有的框架中位移使用视频编码器进行编码，为位移的编码方式提供多种选择有助于提高编码性能。
二、orthoAtlas
orthoAtlas是一种基于投影的网格参数化方法，它通过正交投影为三维网格生成纹理坐标，其主要过程如图2所示，包括如下步骤：
a)计算网格属性，包括每个面的邻接面以及每个面的面积和法向量；
b)根据法向量，确定每个面的投影平面；
c)开始根据投影平面将所有面进行聚类构成一个连通区域，首先选取聚类的起始面；
d)从起始面开始迭代，确定加入连通区域的面的邻接面是否可以加入连通区域；
e)每个连通区域迭代完成后，获得多个连通区域；
f)根据误差度量确定是否合并邻接的连通区域；
g)检测投影时是否存在重叠区域，将重叠的面拆除并重新生成连通区域；
h)将投影后的所有区域排列到一张二维图像中。
以下对上述主要过程进行描述：
(1)计算网格属性
计算网格一些属性，包括三角面的邻接表以及三角面的法向量和面积；三角形邻接可以定义为共享单个顶点或共享一条边；三角面的法向量和面积可以通过其投影后的三角面推导出来(后续用于计算投影后的失真)。
(2)聚类
将三角形进行聚类，同一类三角面构成的区域称为连通区域；聚类算法的起点选取有两种方法：
第一种方法、根据所有面法向量统计结果获取投影方向，则可以选择法向量与投影方向最接近的三角面作为聚类算法起点且其暂未添加到连通区域中；
第二种方法、选取第一个可用的面作为聚类算法起点(即该面暂未添加到连通区域中)，按照准则确定是否将邻接面添加到连通区域中；准则包括：
与该连通区域具有相同的投影方向；
与最后一个添加到连通区域的面的法向量的夹角小于阈值；
面积小于总面积的一个阈值；
三角面的数量小于阈值。
(3)合并
在聚类完成后，可能会产生较多小区域的分区，需要进行合并；合并时需要根据一个代价函数来判断两个分区是否可以合并，该函数由连通区域的周长和拉伸程度定义，详细步骤如下：
首先按特定顺序对各连通区域进行排序，排序准则可以按照包含的三角面数量以及连通区域的平均法向量与对应投影方向的接近程度；各连通区域的代价函数为COST(Mi)＝L2(Mi)+λPERIMETER(Mi)；
然后每次取排序后的第一个连通区域，获取与该连通区域邻接的连通区域构成候选列表，然后对于每个候选区域，首先计算两区域合并后的代价函数：COST(Mi+Mj)＝L2(Mi+Mj)+λPERIMETER(Mi+Mj)，对所有(COST(Mi+Mj)＜COST(Mi)+COST(Mj))的区域，选择其中使成本函数最小的区域，将取出的区域合并到该区域并更新连通区域的排序；其中，L2()表示连通区域投影后的拉伸程度，PERIMETER()表示上述连通区域的周长，λ表示权重系数。
代价函数中的拉伸程度函数衡量投影后的失真程度，通过计算各个方向投影后的三角形面积得到。
(4)检测重叠区域
最后，需要检查各连通区域中是否存在重叠的顶点或面，将重叠的区域从连通区域中拆分出来形成新的连通区域。检测重叠区域即判断当前投影的面是否与已投影的区域发生重叠，若发生重叠则将当前面从该连通区域中拆除，该面与其他拆除的面构成新的连通区域。最后，在拆除的连通区域中，检测是否存在不连通区域，将其中的不连通区域拆分为多个连通区域。
下面结合附图，通过一些实施例及其应用场景对本申请实施例提供的三维网格编码方法进行详细地说明。
请参见图3，图3是本申请实施例提供的一种三维网格编码方法的流程图，该方法可以由编码端执行，如图3所示，包括以下步骤：
步骤301、编码端对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格。
本实施例中，上述第一网格可以是待编码的三维网格，或者可以是对待编码的三维网格处理后所得到的网格，例如，对待编码的三维网格进行网格简化处理后所得到的网格。上述网格码流也即第一网格经压缩编码后得到的码流。需要说明的是，上述待编码的三维网格也可以称为编码端的输入网格，上述第一网格也可以称为基础网格，上述网格码流也可以称为基础网格码流。
上述对第一网格进行压缩编码，示例性的，可以采用帧内模式、帧间模式或跳过(skip)模式等压缩模式对第一网格进行压缩编码。以下结合图4对上述三种压缩模式进行举例说明。
如图4所示，在帧内模式下，通过静态网格编码器对第一网格进行编码，在该压缩模式下，第一网格需包含几何坐标、连接关系以及与顶点关联的属性信息。在帧间模式下，第一网格首先通过率失真权衡(Rate-distortion Optimization，RDO)决定是否采用skip模式，若采用skip模式，则直接选取网格缓冲区中的参考网格作为当前帧的第一网格。若未采用skip模式，则根据网格缓冲区中的参考帧与当前帧计算顶点的运动向量，并通过运动向量编码器编码相应的运动向量。需要说明的是，在对第一网格进行编码后，需要对编码后的网格进行重建，也即需要重建第一网格，从而提供给后续的模块进行处理。
步骤302、所述编码端基于所述网格码流进行三维网格重建，得到第一重建网格。
本实施例中，在对第一网格进行压缩编码得到网格码流后，可以基于该网格码流进行三维网格重建，示例性的，可以通过对上述网格码流进行解码以实现三维网格重建。
具体的，可以基于上述网格码流的压缩模式进行对应的解码，例如，在上述网格码流的压缩模式为帧内模式的情况下，则可以基于静态网格解码器对上述网格码流进行解码，得到重建的基础网格，即第一重建网格；在上述基础网格的压缩模式为帧间模式的情况下，则可以解码上述网格码流的运动向量，并根据网格缓冲区中的参考帧重建相应的基础网格；在上述基础网格的压缩模式为skip模式的情况下，则可以将网格缓冲区中对应的参考帧作为当前帧的基础网格。
步骤303、所述编码端对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格。
本实施例中，上述网格参数化处理用于为第一重建网格生成相应的纹理坐标。示例性的，可以基于Isochart算法、正交投影算法等对第一重建网格进行网格参数化处理。
步骤304、所述编码端根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图。
本实施例中，上述第一纹理图也可以称为输入纹理图，可以是与上述待编码的三维网格对应的纹理图。示例性的，一种可选的纹理图转换方式可以包括如下步骤：
步骤a1、计算待生成的纹理图(即第二纹理图)上每个像素的纹理坐标。
步骤a2、对于每个纹理坐标，判断该纹理坐标是否在第二重建网格的某个面内。
步骤a3、如果该纹理坐标不属于第二重建网格的任一个面，则将该像素标记为空像素，之后可以用填充算法进行填补。
步骤a4、如果该纹理坐标属于第二重建网格的一个面，则执行如下步骤：
步骤a41、将该像素标记为已填充；
步骤a42、根据该纹理坐标计算其在当前面中的重心坐标；
步骤a43、根据该重心坐标以及对应的面，将二维的纹理坐标映射到三维的几何坐标， 即映射到该纹理坐标对应的第二重建网格上的点；
步骤a44、在输入的原始网格(即待编码的三维网格)上寻找距该三维坐标最近的点；
步骤a45、将该三维坐标根据其所在面计算其重心坐标并将其映射到二维，计算其纹理坐标；
步骤a46、通过该纹理坐标在输入的原始纹理图(即上述第一纹理图)上进行采样，得到对应像素位置的值；
步骤a47、将该值赋值给待生成的纹理图(即上述第二纹理图)上的对应像素。
需要说明的是，对于空像素，可以使用已有的填充算法(如推拉(Push-Pull)算法)对这些空像素进行填充。
步骤305、所述编码端对所述第二纹理图进行编码，得到纹理图码流。
示例性的，可以采用视频编码器对第二纹理图进行编码，得到纹理图码流。
可以理解的是，编码端在得到网格码流和纹理图码流的情况下，可以对上述格码流和纹理图码流进行存储或传输等处理。还需要说明的是，在编码端未生成网格的纹理坐标的码流的情况下，解码端可以通过对上述网格码流解码得到的网格进行网格参数化处理以生成该网格的纹理坐标，进而保证解码端可以正常应用解码得到的网格。
本申请实施例提供的三维网格编码方法，编码端对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；所述编码端基于所述网格码流进行三维网格重建，得到第一重建网格；所述编码端对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；所述编码端根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；所述编码端对所述第二纹理图进行编码，得到纹理图码流，也即本申请实施例中编码端仅生成了三维网格的网格码流和纹理图码流，而未对应生成三维网格的纹理坐标的码流，这样不仅提高三维网格的编码效率，还可以减少三维网格的码流对存储资源或传输资源的占用。此外，本申请实施例中对第一重建网格进行网格参数化处理，相比于现有技术中对第一网格进行网格参数化处理，不仅可以减少数据处理量，还可以保证第一重建网格的纹理坐标的准确性。
可选地，所述编码端对所述第一重建网格进行网格参数化处理，得到第二重建网格，包括：
所述编码端采用正交投影算法对所述第一重建网格进行网格参数化处理，得到第二重建网格。
可以理解的是，本申请实施例中编码端与解码端进行网格参数化处理的方式应保持一致，这样有利于更为准确的重建所需应用的网格。因此，本实施例采用正交投影算法对所述第一重建网格进行网格参数化处理，实现的复杂度较低，便于应用于解码端。
可选地，所述编码端采用正交投影算法对所述第一重建网格进行网格参数化处理，得 到第二重建网格，包括：
所述编码端将所述第一重建网格划分为至少两个连通区域，其中，每个所述连通区域均包括所述第一重建网格的至少一个面，投影面相同且在三维空间中连通的面属于同一连通区域；
所述编码端对所述至少两个连通区域进行合并处理，得到至少一个第一连通区域；
所述编码端分别对所述至少一个第一连通区域执行第一处理，其中，所述第一处理包括对每个所述第一连通区域的各个面进行投影以进行重叠区域检测，以及将存在重叠区域的第一连通区域拆分为不重叠的至少两个连通区域；
所述编码端对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域；
所述编码端将所述至少一个第二连通区域的投影区域进行二维排列，以获得第二重建网格。
以下结合图5对本实施例进行举例说明。
如图5所示，编码端采用正交投影算法对所述第一重建网格进行网格参数化处理依次包括如下过程：初始分区，合并分区，检测重叠区域，简单区域合并，二维排列。其中，上述初始分区也即上述编码端将所述第一重建网格划分为至少两个连通区域。上述合并分区也即编码端对所述至少两个连通区域进行合并处理，得到至少一个第一连通区域。上述检测重叠区域也即编码端分别对所述至少一个第一连通区域执行第一处理。上述简单区域合并也即所述编码端对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域。上述二维排列也即所述编码端将所述至少一个第二连通区域的投影区域进行二维排列，以获得第二重建网格。以下以第一重建网格的面为三角面为例对上述各个过程进行说明。
示例性的，上述初始分区可以包括如下过程：
定义第一重建网格的任一三角面T，其三个顶点为A,B,C，(xi,yi,zi)为第i个顶点的几何坐标，则当前面T的法向量为AB×AC＝((yB-yA)*(zC-zA)-(yC-yA)*(zB-zA),(zB-zA)*(xC-xA)-(zC-zA)*(xB-xA),(xB-xA)*(yC-yA)-(xC-xA)*(yB-yA))，其中，AB、AC代表三角面T的两条边对应的向量，×表示向量叉乘。通过上式计算得到法向量后再对法向量进行归一化得到当前三角面的单位法向量。同时，当前三角面的面积S可以通过计算法向量的模长得到，即
在得到第一重建网格的每个三角面的法向量与面积后，根据预先设定的投影面，确定每个三角面的投影面，该过程是通过计算当前三角面法向量与预先设定的每个投影面的法向量的点积，取最大的积点对应的投影面作为当前三角面的投影面。然后，将第一重建网格中属于同一个投影面且在三维空间中连通的三角面划分为同一个连通区域。通过该过程可以将第一重建网格拆分为多个连通区域。其中，上述预先设定的投影面可以根据实际需求进行合理设定，例如，上述预先设定的投影面可以为正方体的6个面。
经过初始分区之后，可以尝试合并邻接的连通区域，合并时需要根据一个代价函数来判断两个连通区域是否可以合并，该代价函数可以由连通区域的周长和拉伸程度定义。示例性的，上述合并分区可以包括如下过程：
首先，对第一重建网格划分得到的连通区域进行排序，具体的，可以根据各个连通区域所包含的三角面的数量，以及各个连通区域的平均法向量与对应投影方向的接近程度(即两向量点积的大小)中的至少一项对连通区域进行排序。例如，先根据各个连通区域所包含的三角面的数量按照从小到大的顺序进行排序；对于包含的三角面的数量相同的多个连通区域，可以根据各个连通区域的平均法向量与对应投影方向的接近程度进行排序，接近程度越高的连通区域排序越靠前。其中，上述连通区域的平均法向量可以是指上述连通区域所包含的所有三角面的法向量的平均值。
然后，每次取连通区域排序的第一个连通区域，获取与该连通区域邻接的所有连通区域构成候选列表，对于候选列表的每个候选区域，计算该连通区域与该候选区域合并后的代价函数：COST(Mi+Mj)＝L2(Mi+Mj)+λ*PERIMETER(Mi+Mj)，获取合并后的代价函数小于合并前两个区域的代价函数之和(即COST(Mi+Mj)＜COST(Mi)+COST(Mj))的所有候选区域，选择其中使代价函数最小的候选区域，将当前的连通区域与该候选区域合并，更新合并后的连通区域的投影面，并更新连通区域的排序，进行迭代，直到遍历完所有连通区域。
其中，两个连通区域邻接可以理解为两个连通区域之间共享至少一条边。上述Mi和Mj一个表示连通区域排序的第一个连通区域，另一个表示第一个连通区域的候选区域。COST(Mi+Mj)表示上述Mi和Mj合并后的连通区域的代价函数。L2(Mi+Mj)表示上述Mi和Mj合并后的连通区域投影后的拉伸程度。PERIMETER(Mi+Mj)表示上述Mi和Mj合并后的连通区域的周长。λ表示权重系数。COST(Mi)表示上述Mi的代价函数。COST(Mj)上述Mj的代价函数。
示例性的，上述连通区域投影后的拉伸程度可以通过如下方式确定：
假设一个带有二维纹理坐标p1,p2,p3的三角面T，其中pi＝(si,ti)，i依次取值为1、2和3，相应的三维坐标表示为q1,q2,q3,仿射映射S(p)＝S(s,t)＝q的计算过程如公式(1)所示：
其中，<p,p2,p3>表示三角面pp2p3的面积，<p,p3,p1>表示三角面pp3p1的面积，<p,p1,p2>表示三角面pp1p2的面积，p表示三角面p1p2p3上的任一点的坐标。由于该映射是仿射的，所以它的偏导数在(s,t)上是常数，其计算过程如公式(2)和公式(3)所示：
其中，A＝<p1,p2,p3>＝((s2-s1)(t3-t1)-(s3-s1)(t2-t1))/2。
然后计算得雅可比(Jacobian)矩阵[SS,St]的较大和较小奇异值，计算过程如公式(4)和公式(5)所示：

其中，a＝Ss·Ss,b＝Ss·St,c＝St·St。奇异值γmax，γmin分别表示单位长度向量从二维纹理域映射到三维表面时获得的最大长度和最小长度，即最大和最小的局部“拉伸”。在三角面T上的拉伸测度定义如公式(6)所示：
在整个三维网格M＝{Ti}上拉伸测度的定义如式(7)所示：
其中，A′(Ti)是三角面Ti在三维空间中的表面面积。
由上可知，本示例的代价函数通过L2(Mi)和PERIMETER(Mi)定义，通过λ参数对两个变量进行权衡。经过合并分区步骤后，连通区域的分区数量相比于初始分区后会减少。
在经过合并分区过程后，可以将各个连通区域进行投影，在投影过程中，需要防止当前三角面的投影区域遮挡已投影区域。示例性的，上述检测重叠区域可以包括如下过程：
首先遍历当前连通区域所有三角面，将每个三角面按照当前连通区域对应的投影面进行投影。即计算当前三角面上三个顶点投影后的二维坐标。然后判断当前三角面的投影区域是否与当前连通区域已投影三角面的投影区域重叠。示例性的，投影区域重叠可以分为三种情况，分别是：当前三角面的投影区域上的一个或多个顶点在当前连通区域已投影三角面的投影区域内；当前连通区域已投影三角面的投影区域的顶点在当前三角面的投影区域内；当前三角面的投影区域的一个或多个边与当前连通区域已投影三角面的投影区域的边相交。
通过判断以上三种情况，若当前三角面的投影区域与当前连通区域已投影三角面的投影区域重叠发生重叠，则先跳过当前三角面的投影，继续遍历当前连通区域的下一个三角面。然后在一轮遍历结束后，判断当前连通区域是否存在未投影的三角面，并将这些未投影的三角面作为新的连通区域进行投影并判断是否存在投影区域重叠，如此迭代直到最终的投影区域不产生重叠。通过该过程，可以将存在重叠区域的连通区域拆分为互不重叠的多个连通区域。例如，如图6a至图6c所示，将一个存在重叠区域的连通区域(即图6a所示的连通区域)拆分为不重叠的两个连通区域(即图6b和图6c所示的两个连通区域)。
可以理解的是，对连通区域的三角面的投影过程也即获取该三角面的顶点的初始的纹理坐标的过程。
在通过上述检测重叠区域过程之后，对于一些面数较少的连通区域，可以尝试将这些连通区域和与其邻接的连通区域进行合并，以减少连通区域的数量。示例性的，上述简单区域合并可以是将一些面数较少的连通区域和与其连接的连通区域进行合并，例如，将面 数小于预设值的连通区域和与其连接的连通区域进行合并，其中，上述预设值可以根据实际需求进行合理设置，例如，4或3或2等。可以理解的是，在进行简单区域合并的过程中，需要保证合并后的连通区域的各个面的投影区域之间不存在重叠。
在简单分区合并完成后，即可得到第一重建网格的参数化最终分区的结果，将这些分区按照一定的规则排列到二维图像中得到所有顶点最终的纹理坐标，也即第一重建网格最终的参数化结果。示例性的，上述二维排列可以包括如下过程：
首先计算整个第一重建网格顶点的缩放比例以使其能排列到指定大小的二维图像中，然后分别计算每一个连通区域投影后占用的宽度和高度，接着对该连通区域所占用的图像进行光栅化，即将三角面占用的区域的像素值设为1，其余区域设为0。
接着，对于每一个连通区域，尝试将其排列到最终的图像上，即对于当前待排列的连通区域，遍历二维图像上的每一个像素块，作为该连通区域的左上角，然后尝试不同的放置方向，若发现该连通区域可以放置到以当前像素为左上角的区域(即当前连通区域在二维图像上的占用区域不占用其他已放置的区域)，则将该区域所有像素设为已占用并调整当前连通区域所有点的纹理坐标(包括旋转、缩放以及平移)，然后记录当前连通区域左上角在二维图像上的位置以及当前连通区域的宽和高；若遍历所有像素后仍未成功放置，则对当前连通区域进行缩放，然后重新尝试将缩放后的连通区域排列到二维图像上。若缩放后当前连通区域仍不能成功放置，则对整个网格进行缩放然后重新排列。
经过以上步骤，即可得到第一重建网格最终的参数化结果，也即第一重建网格的纹理坐标，相应也就得到了包括纹理坐标的第一重建网格。
在一些可选的实施例中，在检测重叠区域的步骤与简单区域合并的步骤之间，还可以包括提取连通区域的步骤，如图7a所示。具体的，在检测并拆分重叠区域后，一个连通区域可能会被拆分为多个连通区域，在这些拆分出来的连通区域中，可能会存在一些互相不连通的面或区域，如图6c所示，上述提取连通区域也即将在一个连通区域中互不连通的区域提取出来，各自作为独立的连通区域进行投影，如图7b所示。相应的，上述简单区域合并可以是对经过提取连通区域后所得到的连通区域进行简单区域合并。
本实施例中编码端在分别对所述至少一个第一连通区域执行第一处理之后，再次对所述第一处理后所得到的连通区域进行合并处理，这样可以进一步减少连通区域的数量，进而可以进一步提高编码效率。
可选地，所述编码端对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域，包括：
所述编码端获取第三连通区域的候选区域集合，其中，所述候选区域集合包括至少一个候选区域，所述候选区域为所述第一处理后所得到的连通区域中与所述第三连通区域邻接的连通区域，所述第三连通区域为所述第一处理后所得到的连通区域中的任意连通区域；
所述编码端获取所述合并候选区域集合的至少一个目标候选区域，其中，在所述第三连通区域合并至所述目标候选区域的情况下所述第三连通区域的投影区域与所述目标候选 区域的投影区域不重叠；
所述编码端将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域。
本实施例中，上述目标候选区域为与第三连通区域合并后，其投影区域与第三连通区域的投影区域不重叠的候选区域。其中，在上述目标候选区域与第三连通区域合并的情况下，上述目标候选区域和第三连通区域的投影面可以为上述目标候选区域对应的投影面，或者可以为合并连通区域对应的投影面等。
上述将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域，例如，可以将第三连通区域合并至任一目标候选区域，或者，可以将第三连通区域合并中面数最大的目标候选区域等。
在一些可选的实施例中，上述第三连通区域可以为所述第一处理后所得到的连通区域中包含的面数小于预设值的连通区域，或者，由存在重叠区域的连通区域拆分得到的连通区域。
示例性的，以上述合并候选区域集合为合并候选区域列表为例，编码端在确定合并候选区域列表后，遍历合并候选区域列表中的各个候选区域，尝试将当前连通区域合并到候选区域中，然后检测其投影后是否会产生重叠，投影面可以为当前合并的候选区域对应的投影面，此处检测重叠过程与上述重叠区域检测过程相同。若投影后的投影区域产生重叠，则将当前候选区域从合并候选区域列表中删除。这样合并候选区域列表中剩余的候选区域为可与当前区域合并且投影后不会重叠的区域，即上述目标候选区域。
本实施例通过从与第三连通区域邻接的连通区域中选择合并后投影区域与第三连通区域的投影区域不重叠的候选区域进行合并，这样可以保证合并后各个连通区域的投影区域不重叠，进而可以保证所得到的网格的纹理坐标的准确性。
可选地，所述编码端将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域，包括：
所述编码端将所述第三连通区域合并至第一目标候选区域；
其中，所述第一目标候选区域为所述至少一个目标候选区域中投影面与所述第三连通区域的法向量之间的夹角最小的目标候选区域，或者，所述第一目标候选区域为所述至少一个目标候选区域中包含的面数最多的目标候选区域。
本实施例中，上述投影面与第三连通区域的法向量之间的夹角最小的目标候选区域，也即投影面与第三连通区域的法向量最相近的目标候选区域，这样可以保证投影后的失真最小。
示例性的，可以遍历各个目标候选区域，选择投影面与第三连通区域的法向量最相近的目标候选区域作为第一目标候选区域，以保证投影后失真最小；在投影面与第三连通区域的法向量最相近的目标候选区域的数量为多个的情况下，可以从该多个目标候选区域中选择面数最多的目标候选区域作为第一目标候选区域，这样有利于减少连通区域的数量。
可以理解的是，将第三连通区域合并至第一目标候选区域即将第三连通区域中的所有面添加到第一目标候选区域中，并删除原有的第三连通区域，并进一步得到合并后的连通区域中新添加顶点的纹理坐标。
可选地，所述候选区域与所述第三连通区域之间存在至少两个重合顶点。
可选地，所述第一网格根据简化网格确定，所述简化网格为对所述待编码的三维网格进行网格简化处理所得到的网格；
其中，所述简化网格的顶点数小于所述待编码的三维网格的顶点数，或者，所述简化网格的面数小于所述待编码的三维网格的面数。
本实施例中，上述第一网格根据简化网格确定，例如，第一网格可以为上述简化网格，或者，第一网格可以为对简化网格的各个顶点进行位移后的所得到的网格。
上述网格简化处理可以理解为将待编码的三维网格简化到一个点数和面数相对较少的网格，并尽可能地保持原始网格的形状。
示例性的，如图8所示，将待编码的三维网格的边两端的顶点合并为一个顶点并删除这两个顶点间的连接，整个网格按照一定规则重复这一过程以将网格的面数和顶点数减少到目标值，其中，上述目标值可以根据有损编码所需要的数量确定。具体的，在网格简化处理的过程中，可以选择某一误差度量来最优化简化的结果，例如，可以选取顶点所有相邻面的方程系数之和作为该顶点的误差度量，也即通过顶点所有相邻面的方程系数之和来度量该顶点的误差，相应的边的误差即为该边上的两个顶点的误差之和，也即一条边合并产生的误差为合并后的顶点到边的原始两个顶点所有相邻面的距离之和。确定好误差度量之后，就开始迭代进行网格简化，例如，计算待编码的三维网格的顶点的误差，从而得到待编码的三维网格的每条边的误差。然后将每条边按误差从小到大排列，每次选取误差最小的边进行合并。同时，计算合并后的顶点位置，并更新所有与合并后的顶点相关的边的误差。即更新边排列的顺序，保证每次迭代都是基于全局的误差度量。通过迭代将待编码的三维网格的面简化到满足有损编码所需要的数量。
本实施例通过待编码的三维网格进行网格简化处理得到简化网格，并对由简化网格所确定的第一网格进行压缩编码，由于网格简化处理后网格的面数或者顶点数减少，这样不仅可以提高网格编码的效率，还可以进一步减少网格码流所需占用的传输资源或存储资源等。
可选地，所述方法还包括：
所述编码端对所述简化网格进行细分变形，得到第二网格；
所述编码端根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，包括：
所述编码端根据所述第二重建网格和所述第二网格计算所述第二网格的各个顶点的位移；
所述编码端对所述第二网格的各个顶点的位移进行编码，得到位移码流；
所述编码端根据所述第一重建网格和所述位移码流进行三维网格重建，得到第三重建网格；
所述编码端根据所述待编码的三维网格和所述第三重建网格对第一纹理图进行转换，得到第二纹理图。
示例性的，细分变形的基本思路可以如图9所示，对输入的2D曲线(由2D折线表示)，称为原始曲线，首先进行下采样以生成基本曲线/折线，称为简化曲线。然后可以基于简化曲线的多段线生成细分曲线。随后对细分曲线的多段线进行变形，以获得与原始曲线更为接近的变形曲线，即为细分曲线的每个顶点计算位移矢量，如图9中的箭头所示，以使细分变形曲线的形状尽可能接近原始曲线的形状；进而可以按照细分曲线的每个顶点的位移矢量对应移动各个顶点，得到接近原始曲线的形状的细分变形曲线。
具体的，可以先对简化网格进行细分，得到细分网格，其中，细分的具体实现方式可以任意选取，一种可选的实现方式为中点细分方式，以简化网格包括的面为三角面为例，在每次细分迭代中可以将每个三角面细分为四个子三角面，如图10所示。在简化网格的每条边的中间引入新的顶点，几何信息与属性信息的细分是独立进行的，因为几何信息与属性信息的连接关系通常是不同的。
针对上述细分方式，可以采用公式(8)计算新引入的边(v1,v2)的中点v12的位置Pos(v12)：
其中，Pos(v1)与Pos(v2)分别为顶点v1和v2的几何坐标。对于中点v12的纹理坐标，可以采用与几何坐标相同的计算方式，在此不做赘述。
对于细分后的网格，寻找其每个点在简化网格上的最近邻点(包括简化网格面上的点)，可选的，可以通过k维树(kdTree)等数据结构加速查找。通过计算细分网格上的每个顶点和其在简化网格上的最近邻点的几何坐标之间的距离得到细分网格的每个顶点的几何坐标的位移，进而可以基于细分网格的每个顶点的几何坐标的位移移动对应顶点，以获得细分变形网格，即上述第二网格。
编码端在获得细分变形网格之后，可以基于细分变形网格和第二重建网格计算细分变形网格的各个顶点的位移，例如，可以对第二重建网格进行细分，确定细分后的第二重建网格的各个顶点和细分变形网格的各个顶点的对应关系，并分别根据细分后的第二重建网格的各个顶点与对应的细分变形网格的各个顶点计算各个顶点的位移。进而编码端可以对计算得到的位移进行编码，其中，本实施例对上述对计算得到的位移进行编码的方式不做限定，例如，可以采用视频编码方法对计算得到的位移进行编码，具体可以包括如下步骤：转换位移所在的坐标系，即将每个顶点的位移的坐标系转换到以该位移对应的顶点的法向量及与法向量相切的两个分量构建的坐标系中，然后对位移进行变换，如小波变换等，将变换后的系数进行量化，并按照扫描的顺序排列到图像中，对图像应用视频编码；或者，可以对计算得到的位移直接使用熵编码进行编码。
其中，上述视频编码方法可以包括如下步骤：
(1)、坐标系转换：
生成的位移处于与输入网格相同的全局坐标系下。一种可选的优化方法是将其转换到局部坐标系下，每个顶点的局部坐标系由细分变形网格上的顶点的法向量定义。该方式的优点在于位移的法向分量比两个切向分量对重建网格的质量影响更显著，因此，可以对切向分量设置更大的量化参数。
(2)、小波变换：
对于量化后的位移，可以对其应用小波变换，一种可选的小波变换如线性小波变换如下所示：
其预测过程定义如公式(12)所示：
其中，v为边(v1,v2)上新插入的中点，Signal(v),Signal(v1)以及Signal(v2)分别为顶点v,v1及v2对应的位移，“←”表示通过箭头右侧内容更新箭头左侧的内容。
更新过程定义如公式(13)所示：
其中v*为顶点v的相邻的所有顶点的集合。
(3)、小波系数量化：
小波系数量化的方式有多种，一种方法如式(14)和(15)所示：
disp[v].d[k]＝floor(disp[v].d[k]*scale[k])        -(14)
其中，disp[v]表示第v个顶点位移的小波系数，d[k]表示其位移第k个系数，floor表示向下取整。bitDepthPosition表示当前网格位置的比特位深，qp[k]表示第k个系数的量化参数。如前所述，将位移转换坐标系后，其法向分量相比于切向分量对质量的影响效果更显著，因此可以对切向分量使用更大的量化参数。
同时，根据小波变换的特性，还可以对细分新产生的顶点和原有的顶点使用不同的量化参数。即对于细分后的顶点，量化参数更新如式(16)所示：
scale[k]＝scale[k]*lodScale[k]     -(16)
其中，lodScale[k]表示当前细分层级量化参数的系数。
(4)、小波系数排列：
对于小波变换后生成的小波系数，按如下方式将其排列到二维图像上：
按从低频到高频的顺序遍历小波系数；
对于每个系数，确定其所在NxM像素块的索引(例如N＝M＝16)，它应该按照块的光栅扫描顺序存储在其中；
根据Morton顺序计算对应NxM像素块在图像上的位置。
此处不对排列的方式进行限定，也即可以使用其它的排列方案，如zigzag顺序、光栅 顺序等。编码器可以在码流中显示地指定相应的排列方案。
(5)、视频编码：
将小波系数排列到二维图像上后，可以直接使用视频编码器对其进行编码。本申请的方案与使用哪种视频编码技术无关，可以在码流中显示地指定视频编码器的类型。同时，若量化应用于视频编码前，则此处视频编码可以使用无损的方法，此时无须在编码端重建位移。
上述对计算得到的位移直接使用已有的熵编码进行压缩编码，例如，可以采用自适应二进制算术编码(Context Adaptive Binary Arithmetic Coding，CABAC)等对计算得到的位移进行压缩编码。需要说明的是，本实施例也可以对小波变换后的系数直接采用熵编码。熵编码前可以进行相应的量化。
在得到位移码流之后，编码端可以基于位移码流重建位移，并根据重建的位移和第一重建网格进行三维网格重建，得到第三重建网格，例如，可以对第一重建网格进行网格细分，得到细分的第一重建网格，进而可以分别基于重建的各个顶点的位移对应的移动细分的第一重建网格的各个顶点，得到第三重建网格，该第三重建网格也即重建的细分变形网格，进而可以根据待编码的三维网格和第三重建网格对第一纹理图进行转换，得到第二纹理图。其中，根据待编码的三维网格和第三重建网格对第一纹理图进行转换的过程可以参见前述纹理图转换方式的相关说明，在此不做赘述。
本实施例中在对待编码的三维网格进行网格简化处理的情况下，对所述简化网格进行细分变形得到第二网格，根据所述第二重建网格和所述第二网格计算所述第二网格的各个顶点的位移，对所述第二网格的各个顶点的位移进行编码，得到位移码流，并根据所述第一重建网格和所述位移码流进行三维网格重建得到第三重建网格，这样可以保证重建的网格的质量。
在一些可选的实施例中，在对所述简化网格进行细分变形的情况下，所述方法还包括：
所述编码端根据细分网格的各个顶点的几何坐标的位移确定所述简化网格的各个顶点的几何坐标的位移；所述编码端分别根据所述简化网格的各个顶点的几何坐标的位移移动所述简化网格的各个顶点，得到第一网格。其中，上述细分网格的每个顶点的几何坐标的位移可以参见前述实施例的相关说明，在此不做赘述。
可选地，所述方法还包括：
所述编码端对第一编码信息执行第一操作，其中，所述第一编码信息包括所述网格码流和所述纹理图码流，所述第一操作包括如下至少一项：存储，传输。
可选地，所述第一编码信息还包括如下至少一项：
位移码流；
第一辅助信息，所述第一辅助信息包括如下至少一项：与网格编码相关的信息，与纹理图编码相关的信息，与网格细分变形相关的信息，与网格位移相关的信息，与网格参数化处理相关的信息。
可以理解的是，编码端在编码得到上述位移码流的情况下，上述第一编码信息还包括位移码流，在未编码得到上述位移码流的情况下，上述第一编码信息不包括位移码流。
上述与网格编码相关的信息，例如，帧内编码标识，用于指示当前帧是否需要根据参考帧网格进行构建；参考帧列表，用于指示当前帧需要用到的参考帧的索引，该参考帧列表可以应用于第一网格的重建；静态网格编码器类型等。上述与纹理图编码相关的信息，例如，视频编码器类型。上述与网格细分变形相关的信息，例如，细分变形标识，用于指示是否需要对重建的网格进行细分变形处理；细分方式，即重建细分变形网格时网格细分的方式。上述与网格位移相关的信息，例如，位移的编码方式；位移变换方式；小波系数排列方式等。上述与网格参数化处理相关的信息，例如，网格参数化处理所采用的算法(例如，Isochart算法、正交投影算法等)。
可以理解的是，上述第一辅助信息可以用于辅助解码端进行解码，例如，解码端根据与网格编码相关的信息进行网格码流解码，根据与纹理图编码相关的信息进行纹理图码流进行解码，根据与网格细分变形相关的信息对解码得到的网格进行细分变形，根据与网格参数化处理相关的信息对解码得到的网格进行网格参数化处理，根据与网格位移相关的信息对位移码流进行解码，这样有利于解码端更为准确的对编码的网格进行解码。
以下结合图11对本申请实施例提供的三维网格编码方法进行举例说明。
参见图11，编码端对输入网格进行网格简化处理，得到简化网格，对该简化网格进行细分变形，分别得到基础网格(即第一网格)和细分变形网格(即第二网格)。对基础网格进行压缩编码，得到网格码流。基于网格码流进行基础网格重建，得到重建的基础网格(即第一重建网格)。对第一重建网格进行网格参数化处理，得到包含纹理坐标的第一重建网格(即第二重建网格)。根据细分变形网格和第二重建网格计算细分变形网格各个顶点的位移，该位移可以包括几何坐标的位移和纹理坐标的位移，对计算得到的各个顶点的位移进行变换、量化等处理，并对处理后的位移进行编码，得到位移码流。进一步的，编码端基于位移码流进行位移重建，得到重建的位于，并基于重建的位移和第一重建网格重建细分变形网格(即第三重建网格)，并基于重建的细分变形网格、输入网格对输入纹理图(即第一纹理图)进行转换，得到转换后的纹理图(即第二纹理图)，并利用视频编码器对转换后的纹理图进行编码，得到纹理图码流，进而编码端可以存储或传输所得到的网格码流、纹理图码流以及位移码流。
请参见图12，图12是本申请实施例提供的一种三维网格解码方法的流程图，该方法可以由解码端执行，如图12所示，包括以下步骤：
步骤1201、解码端获取网格码流和纹理图码流。
本实施例中，上述网格码流和纹理图码流为编码端根据上述三维网格编码方法生成的网格码流和纹理图码流。示例性的，解码端可以从编码端接收上述网格码流和纹理图码流。
步骤1202、所述解码端对所述网格码流进行解码得到第三网格，并对所述纹理图码流进行解码得到第三纹理图。
本实施例中，上述第三网格为网格码流经解码后所得到的网格。其中，上述第三网格也可以称为重建的基础网格。上述第三纹理图为纹理图码流经解码后所得到的纹理图。
具体的，解码端应采用与编码端的网格编码方式对应的解码方式对网格码流进行解码。例如，若编码端采用帧内模式编码，则解码端也采用帧内模式解码；若编码端采用帧间模式编码，则解码端也采用帧间模式解码；若编码端采用skip模式编码，则解码端也采用skip模式解码。以下结合图13对上述三种解码方式进行举例说明：
如图13所示，若为帧内模式，则使用静态网格解码器进行解码；若为帧间模式，则解码相应的运动向量，并根据网格缓冲区中的参考帧重建相应的基础网格；若为skip模式，则直接将网格缓冲区中对应的参考帧作为当前帧的基础网格。
同样的，解码端应采用与编码端的纹理图编码方式对应的解码方式对纹理图码流进行解码，例如，编码端采用视频编码器，则解码端采用对应的视频编码器对纹理图码流进行解码。在一些可选的实施例中，解码端还可以对解码所得到的纹理图进行颜色空间转换以得到与编码端输入纹理图一致的图像格式。
步骤1203、所述解码端对所述第三网格进行网格参数化处理，得到第四网格，其中，所述第四网格为包括纹理坐标的网格。
本实施例中，解码端可以采用与编码端相同的网格参数化处理方式，对第三网格进行网格参数化处理，得到包含纹理坐标的第三网格，也即上述第四网格。进而解码端可以应用上述第三纹理图和上述第四网格。
本申请实施例通过解码端对网格码流解码得到的网格进行网格参数化处理以获取网格的纹理坐标，这样可以在保证解码可以获得的网格的纹理坐标的同时，减少三维网格的码流对存储资源或传输资源的占用。
可选地，所述解码端对所述第三网格进行网格参数化处理，得到第四网格，包括：
所述解码端采用正交投影算法对所述第三网格进行网格参数化处理，得到第四网格。
可选地，所述解码端采用正交投影算法对所述第三网格进行网格参数化处理，得到第四网格，包括：
所述解码端将所述第三网格划分为至少两个连通区域，其中，每个所述连通区域均包括所述第三网格的至少一个面，投影面相同且在三维空间中连通的面属于同一连通区域；
所述解码端对所述至少两个连通区域进行合并处理，得到至少一个第一连通区域；
所述解码端分别对所述至少一个第一连通区域执行第一处理，其中，所述第一处理包括对每个所述第一连通区域的各个面进行投影以进行重叠区域检测，以及将存在重叠区域的第一连通区域拆分为不重叠的至少两个连通区域；
所述解码端对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域；
所述解码端将所述至少一个第二连通区域的投影区域进行二维排列，以获得第四网格。
可选地，所述解码端对所述第一处理后所得到的连通区域进行合并处理，得到至少一 个第二连通区域，包括：
所述解码端获取第三连通区域的候选区域集合，其中，所述候选区域集合包括至少一个候选区域，所述候选区域为所述第一处理后所得到的连通区域中与所述第三连通区域邻接的连通区域，所述第三连通区域为所述第一处理后所得到的连通区域中的任意连通区域；
所述解码端获取所述合并候选区域集合的至少一个目标候选区域，其中，在所述第三连通区域合并至所述目标候选区域的情况下所述第三连通区域的投影区域与所述目标候选区域的投影区域不重叠；
所述解码端将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域。
可选地，所述解码端将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域，包括：
所述解码端将所述第三连通区域合并至第一目标候选区域；
其中，所述第一目标候选区域为所述至少一个目标候选区域中投影面与所述第三连通区域的法向量之间的夹角最小的目标候选区域，或者，所述第一目标候选区域为所述至少一个目标候选区域中包含的面数最多的目标候选区域。
可选地，所述候选区域与所述第三连通区域之间存在至少两个重合顶点。
可选地，所述方法还包括：
所述解码端获取位移码流；
所述解码端对所述第四网格进行细分变形，得到第五网格；
所述解码端根据所述第五网格和所述位移码流经解码得到的位移进行三维网格重建，得到第四重建网格。
本实施例中，解码端在获取到位移码流的情况下，对位移码流进行解码，例如，若编码端通过视频编码的方式压缩位移，则解码端通过相应的视频解码器进行解码，并根据排列方案将其从二维图像中按照对应的顺序恢复出来。再对其执行反变换、反量化等操作使其恢复出与编码端一致的位移。如采用熵编码的方式，则直接对其进行熵解码即可。
在基础网格以及位移解码重建完成后，根据该两部分重建细分变形的网格。示例性的，若位移编码过程使用了参考帧作为参考，即解码得到的位移中与参考帧匹配的区域中的顶点的位移为当前帧顶点相对于参考帧顶点位移的残差，则首先通过参考帧重建的细分变形网格的顶点以及残差恢复当前帧匹配区域顶点的位移，然后进行后续重建步骤。即对重建的基础网格进行细分，与编码端细分方式保持一致。对细分后的网格的每个顶点加上对应的位移，如公式(17)所示：
deformedmesh[i].v[k]＝subdivmesh[i].v[k]+displacement[k]    -(17)
其中，subdivmesh[i].v[k]即当前帧(索引为i)基础网格细分后的第k个顶点的几何坐标，displacement[k]即第k个顶点对应的位移，deformedmesh[i].v[k]即当前帧细分变形后的第k个顶点的几何坐标。
可选地，所述方法还包括：
所述解码端获取第一辅助信息，其中，所述第一辅助信息包括如下至少一项：与网格编码相关的信息，与纹理图编码相关的信息，与网格细分变形相关的信息，与网格位移相关的信息，与网格参数化处理相关的信息。
本实施例中，解码端在获取到第一辅助信息的情况下，可以基于第一辅助信息进行解码，例如，解码端根据与网格编码相关的信息进行网格码流解码，根据与纹理图编码相关的信息进行纹理图码流进行解码，根据与网格细分变形相关的信息对解码得到的网格进行细分变形，根据与网格参数化处理相关的信息对解码得到的网格进行网格参数化处理，根据与网格位移相关的信息对位移码流进行解码，这样有利于解码端更为准确的对编码的网格进行解码。
需要说明的是，上述第一辅助信息可以编码成独立的码流，也可以是携带于上述网格码流、纹理图码流以及位移码流，例如，与网格编码相关的信息携带于上述网格码流，与纹理图编码相关的信息携带于上述纹理图码流，与网格位移相关的信息携带于位移码流等。
需要说明的是，上述各个实施方式的实现方式可以参见图3所示的实施例的相关说明，此处不作赘述。
以下结合图14对本申请实施例提供的三维网格解码方法进行举例说明。
参见图14，解码端获取到各部分码流解码并进行重建，基础网格重建后进行网格参数化处理得到相应的包含纹理坐标的基础网格，将该基础网格进行细分并应用解码得到的位移得到解码端重建的细分变形网格，即解码端最终输出的重建网格。
需要说明的是，本申请实施例提供的三维网格编码方法，执行主体可以为三维网格编码装置，或者，该三维网格编码装置中的用于执行三维网格编码方法的控制模块。本申请实施例中以三维网格编码装置执行三维网格编码方法为例，说明本申请实施例提供的三维网格编码装置。
请参见图15，图15是本申请实施例提供的一种三维网格编码装置的结构图，如图15所示，三维网格编码装置1500包括：
第一编码模块1501，用于对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；
第一重建模块1502，用于基于所述网格码流进行三维网格重建，得到第一重建网格；
第一处理模块1503，用于对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；
转换模块1504，用于根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；
第二编码模块1505，用于对所述第二纹理图进行编码，得到纹理图码流。
可选地，所述第一处理模块具体用于：
采用正交投影算法对所述第一重建网格进行网格参数化处理，得到第二重建网格。
可选地，所述第一处理模块具体用于：
将所述第一重建网格划分为至少两个连通区域，其中，每个所述连通区域均包括所述第一重建网格的至少一个面，投影面相同且在三维空间中连通的面属于同一连通区域；
对所述至少两个连通区域进行合并处理，得到至少一个第一连通区域；
分别对所述至少一个第一连通区域执行第一处理，其中，所述第一处理包括对每个所述第一连通区域的各个面进行投影以进行重叠区域检测，以及将存在重叠区域的第一连通区域拆分为不重叠的至少两个连通区域；
对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域；
将所述至少一个第二连通区域的投影区域进行二维排列，以获得第二重建网格。
可选地，所述第一处理模块具体用于：
获取第三连通区域的候选区域集合，其中，所述候选区域集合包括至少一个候选区域，所述候选区域为所述第一处理后所得到的连通区域中与所述第三连通区域邻接的连通区域，所述第三连通区域为所述第一处理后所得到的连通区域中的任意连通区域；
获取所述合并候选区域集合的至少一个目标候选区域，其中，在所述第三连通区域合并至所述目标候选区域的情况下所述第三连通区域的投影区域与所述目标候选区域的投影区域不重叠；
将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域。
可选地，所述第一处理模块具体用于：
将所述第三连通区域合并至第一目标候选区域；
其中，所述第一目标候选区域为所述至少一个目标候选区域中投影面与所述第三连通区域的法向量之间的夹角最小的目标候选区域，或者，所述第一目标候选区域为所述至少一个目标候选区域中包含的面数最多的目标候选区域。
可选地，所述候选区域与所述第三连通区域之间存在至少两个重合顶点。
可选地，所述第一网格根据简化网格确定，所述简化网格为对所述待编码的三维网格进行网格简化处理所得到的网格；
其中，所述简化网格的顶点数小于所述待编码的三维网格的顶点数，或者，所述简化网格的面数小于所述待编码的三维网格的面数。
可选地，所述装置还包括：
第一细分变形模块，用于对所述简化网格进行细分变形，得到第二网格；
所述转换模块具体用于：
根据所述第二重建网格和所述第二网格计算所述第二网格的各个顶点的位移；
对所述第二网格的各个顶点的位移进行编码，得到位移码流；
根据所述第一重建网格和所述位移码流进行三维网格重建，得到第三重建网格；
根据所述待编码的三维网格和所述第三重建网格对第一纹理图进行转换，得到第二纹理图。
可选地，所述装置还包括：
第一执行模块，用于对第一编码信息执行第一操作，其中，所述第一编码信息包括所述网格码流和所述纹理图码流，所述第一操作包括如下至少一项：存储，传输。
可选地，所述第一编码信息还包括如下至少一项：
位移码流；
第一辅助信息，所述第一辅助信息包括如下至少一项：与网格编码相关的信息，与纹理图编码相关的信息，与网格细分变形相关的信息，与网格位移相关的信息，与网格参数化处理相关的信息。
本申请实施例中的三维网格编码装置可以是电子设备，例如具有操作系统的电子设备，也可以是电子设备中的部件，例如集成电路或芯片。该电子设备可以是终端或者网络侧设备，也可以为除终端或网络侧设备之外的其他设备。示例性的，终端可以包括但不限于上述所列举的终端11的类型，网络侧设备可以包括但不限于上述所列举的网络侧设备12的类型其他设备可以为服务器、网络附属存储器(Network Attached Storage，NAS)等，本申请实施例不作具体限定。
本申请实施例提供的三维网格编码装置能够实现图3的方法实施例实现的各个过程，并达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，本申请实施例提供的三维网格解码方法，执行主体可以为三维网格解码装置，或者，该三维网格解码装置中的用于执行三维网格解码方法的控制模块。本申请实施例中以三维网格解码装置执行三维网格解码方法为例，说明本申请实施例提供的三维网格解码装置。
请参见图16，图16是本申请实施例提供的一种三维网格解码装置的结构图，如图16所示，三维网格解码装置1600包括：
第一获取模块1601，用于获取网格码流和纹理图码流；
解码模块1602，用于对所述网格码流进行解码得到第三网格，并对所述纹理图码流进行解码得到第三纹理图；
第二处理模块1603，用于对所述第三网格进行网格参数化处理，得到第四网格，其中，所述第四网格为包括纹理坐标的网格。
可选地，所述第二处理模块具体用于：
采用正交投影算法对所述第三网格进行网格参数化处理，得到第四网格。
可选地，所述第二处理模块具体用于：
将所述第三网格划分为至少两个连通区域，其中，每个所述连通区域均包括所述第三网格的至少一个面，投影面相同且在三维空间中连通的面属于同一连通区域；
对所述至少两个连通区域进行合并处理，得到至少一个第一连通区域；
分别对所述至少一个第一连通区域执行第一处理，其中，所述第一处理包括对每个所述第一连通区域的各个面进行投影以进行重叠区域检测，以及将存在重叠区域的第一连通 区域拆分为不重叠的至少两个连通区域；
对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域；
将所述至少一个第二连通区域的投影区域进行二维排列，以获得第四网格。
可选地，所述第二处理模块具体用于：
获取第三连通区域的候选区域集合，其中，所述候选区域集合包括至少一个候选区域，所述候选区域为所述第一处理后所得到的连通区域中与所述第三连通区域邻接的连通区域，所述第三连通区域为所述第一处理后所得到的连通区域中的任意连通区域；
获取所述合并候选区域集合的至少一个目标候选区域，其中，在所述第三连通区域合并至所述目标候选区域的情况下所述第三连通区域的投影区域与所述目标候选区域的投影区域不重叠；
将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域。
可选地，所述第二处理模块具体用于：
将所述第三连通区域合并至第一目标候选区域；
其中，所述第一目标候选区域为所述至少一个目标候选区域中投影面与所述第三连通区域的法向量之间的夹角最小的目标候选区域，或者，所述第一目标候选区域为所述至少一个目标候选区域中包含的面数最多的目标候选区域。
可选地，所述候选区域与所述第三连通区域之间存在至少两个重合顶点。
可选地，所述装置还包括：
第二获取模块，用于获取位移码流；
第二细分变形模块，用于对所述第四网格进行细分变形，得到第五网格；
第二重建模块，用于根据所述第五网格和所述位移码流经解码得到的位移进行三维网格重建，得到第四重建网格。
可选地，所述装置还包括：
第三获取模块，用于获取第一辅助信息，其中，所述第一辅助信息包括如下至少一项：与网格编码相关的信息，与纹理图编码相关的信息，与网格细分变形相关的信息，与网格位移相关的信息，与网格参数化处理相关的信息。
本申请实施例中的三维网格解码装置可以是电子设备，例如具有操作系统的电子设备，也可以是电子设备中的部件，例如集成电路或芯片。该电子设备可以是网络侧设备或者终端，也可以为除网络侧设备或者终端之外的其他设备。示例性的，网络侧设备可以包括但不限于上述所列举的网络侧设备12的类型，终端可以包括但不限于上述所列举的终端11的类型，其他设备可以为服务器、网络附属存储器(Network Attached Storage，NAS)等，本申请实施例不作具体限定。
本申请实施例提供的三维网格解码装置能够实现图12的方法实施例实现的各个过程，并达到相同的技术效果，为避免重复，这里不再赘述。
可选的，如图17所示，本申请实施例还提供一种通信设备1700，包括处理器1701和 存储器1702，存储器1702上存储有可在所述处理器1701上运行的程序或指令，例如，该通信设备1700为编码端时，该程序或指令被处理器1701执行时实现上述三维网格编码方法实施例的各个步骤，且能达到相同的技术效果。该通信设备1700为解码端时，该程序或指令被处理器1701执行时实现上述三维网格解码方法实施例的各个步骤，且能达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例还提供一种编码端，包括处理器和通信接口，所述处理器用于对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；基于所述网格码流进行三维网格重建，得到第一重建网格；对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；对所述第二纹理图进行编码，得到纹理图码流。该编码端实施例与上述编码端侧方法实施例对应，上述方法实施例的各个实施过程和实现方式均可适用于该编码端实施例中，且能达到相同的技术效果。具体地，图18为实现本申请实施例的一种编码端的硬件结构示意图。
该编码端1800包括但不限于：射频单元1801、网络模块1802、音频输出单元1803、输入单元1804、传感器1805、显示单元1806、用户输入单元1807、接口单元1808、存储器1809以及处理器1810等中的至少部分部件。
本领域技术人员可以理解，编码端1800还可以包括给各个部件供电的电源(比如电池)，电源可以通过电源管理系统与处理器1810逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。图18中示出的编码端结构并不构成对编码端的限定，编码端可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置，在此不再赘述。
应理解的是，本申请实施例中，输入单元1804可以包括图形处理器(Graphics Processing Unit，GPU)18041和麦克风18042，图形处理器18041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。显示单元1806可包括显示面板18061，可以采用液晶显示器、有机发光二极管等形式来配置显示面板18061。用户输入单元1807包括触控面板18071以及其他输入设备18072中的至少一种。触控面板18071，也称为触摸屏。触控面板18071可包括触摸检测装置和触摸控制器两个部分。其他输入设备18072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆，在此不再赘述。
本申请实施例中，射频单元1801接收来自网络侧设备的下行数据后，可以传输给处理器1810进行处理；另外，射频单元1801可以向网络侧设备发送上行数据。通常，射频单元1801包括但不限于天线、放大器、收发信机、耦合器、低噪声放大器、双工器等。
存储器1809可用于存储软件程序或指令以及各种数据。存储器1809可主要包括存储程序或指令的第一存储区和存储数据的第二存储区，其中，第一存储区可存储操作系统、 至少一个功能所需的应用程序或指令(比如声音播放功能、图像播放功能等)等。此外，存储器1809可以包括易失性存储器或非易失性存储器。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synch link DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请实施例中的存储器1809包括但不限于这些和任意其它适合类型的存储器。
处理器1810可包括一个或多个处理单元；可选的，处理器1810集成应用处理器和调制解调处理器，其中，应用处理器主要处理涉及操作系统、用户界面和应用程序等的操作，调制解调处理器主要处理无线通信信号，如基带处理器。可以理解的是，上述调制解调处理器也可以不集成到处理器1810中。
其中，处理器1810，用于对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；基于所述网格码流进行三维网格重建，得到第一重建网格；对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；对所述第二纹理图进行编码，得到纹理图码流。
可以理解，本实施例中提及的各实现方式的实现过程可以参照前述方法实施例的相关描述，并达到相同或相应的技术效果，为避免重复，在此不再赘述。
本申请实施例还提供一种解码端，包括处理器和通信接口，所述处理器用于获取网格码流和纹理图码流；对所述网格码流进行解码得到第三网格，并对所述纹理图码流进行解码得到第三纹理图；对所述第三网格进行网格参数化处理，得到第四网格，其中，所述第四网格为包括纹理坐标的网格。该解码端实施例与上述解码端方法实施例对应，上述方法实施例的各个实施过程和实现方式均可适用于该解码端实施例中，且能达到相同的技术效果。
具体地，本申请实施例还提供了一种解码端。如图19所示，该解码端1900包括：天线1901、射频装置1902、基带装置1903、处理器1904和存储器1905。天线1901与射频装置1902连接。在上行方向上，射频装置1902通过天线1901接收信息，将接收的信息发送给基带装置1903进行处理。在下行方向上，基带装置1903对要发送的信息进行处理，并发送给射频装置1902，射频装置1902对收到的信息进行处理后经过天线1901发送出去。
以上实施例中解码端执行的方法可以在基带装置1903中实现，该基带装置1903包括 基带处理器。
基带装置1903例如可以包括至少一个基带板，该基带板上设置有多个芯片，如图19所示，其中一个芯片例如为基带处理器，通过总线接口与存储器1905连接，以调用存储器1905中的程序，执行以上方法实施例中所示的网络设备操作。
该解码端还可以包括网络接口1906，该接口例如为通用公共无线接口(Common Public Radio Interface，CPRI)。
具体地，本申请实施例的解码端1900还包括：存储在存储器1905上并可在处理器1904上运行的指令或程序，处理器1904调用存储器1905中的指令或程序执行图16所示各模块执行的方法，并达到相同的技术效果，为避免重复，故不在此赘述。
本申请实施例还提供一种可读存储介质，所述可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述三维网格编码方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述处理器为上述实施例中所述的终端中的处理器。所述可读存储介质，包括计算机可读存储介质，如计算机只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等。在一些示例中，可读存储介质可以是非瞬态的可读存储介质。
本申请实施例另提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现上述三维网格编码方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
应理解，本申请实施例提到的芯片还可以称为系统级芯片，系统芯片，芯片系统或片上系统芯片等。
本申请实施例另提供了一种计算机程序/程序产品，所述计算机程序/程序产品被存储在存储介质中，所述计算机程序/程序产品被至少一个处理器执行以实现上述三维网格编码方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例还提供了一种三维网格编解码系统，包括：编码端及解码端，所述编码端用于执行如图3及上述各个方法实施例的各个过程，所述解码端用于执行如图12及上述各个方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。此外，需要指出的是，本申请实施方式中的方法和装置的范围不限按示出或讨论的顺序来执行功能，还可包括根据所涉及的功能按基本同时的方式或按相反的顺序来执行功能，例如，可以按不同于所描述的次序来执行所描述的方法，并且还可以添加、省去或组合各种步骤。另外，参照 某些示例所描述的特征可在其他示例中被组合。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助计算机软件产品加必需的通用硬件平台的方式来实现，当然也可以通过硬件。该计算机软件产品存储在存储介质(如ROM、RAM、磁碟、光盘等)中，包括若干指令，用以使得终端或者网络侧设备执行本申请各个实施例所述的方法。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式的实施方式，这些实施方式均属于本申请的保护之内。

Claims (23)

  1. 一种三维网格编码方法，包括：
    编码端对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；
    所述编码端基于所述网格码流进行三维网格重建，得到第一重建网格；
    所述编码端对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；
    所述编码端根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；
    所述编码端对所述第二纹理图进行编码，得到纹理图码流。
  2. 根据权利要求1所述的方法，其中，所述编码端对所述第一重建网格进行网格参数化处理，得到第二重建网格，包括：
    所述编码端采用正交投影算法对所述第一重建网格进行网格参数化处理，得到第二重建网格。
  3. 根据权利要求2所述的方法，其中，所述编码端采用正交投影算法对所述第一重建网格进行网格参数化处理，得到第二重建网格，包括：
    所述编码端将所述第一重建网格划分为至少两个连通区域，其中，每个所述连通区域均包括所述第一重建网格的至少一个面，投影面相同且在三维空间中连通的面属于同一连通区域；
    所述编码端对所述至少两个连通区域进行合并处理，得到至少一个第一连通区域；
    所述编码端分别对所述至少一个第一连通区域执行第一处理，其中，所述第一处理包括对每个所述第一连通区域的各个面进行投影以进行重叠区域检测，以及将存在重叠区域的第一连通区域拆分为不重叠的至少两个连通区域；
    所述编码端对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域；
    所述编码端将所述至少一个第二连通区域的投影区域进行二维排列，以获得第二重建网格。
  4. 根据权利要求3所述的方法，其中，所述编码端对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域，包括：
    所述编码端获取第三连通区域的候选区域集合，其中，所述候选区域集合包括至少一个候选区域，所述候选区域为所述第一处理后所得到的连通区域中与所述第三连通区域邻接的连通区域，所述第三连通区域为所述第一处理后所得到的连通区域中的任意连通区域；
    所述编码端获取所述合并候选区域集合的至少一个目标候选区域，其中，在所述第三连通区域合并至所述目标候选区域的情况下所述第三连通区域的投影区域与所述目标候选区域的投影区域不重叠；
    所述编码端将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域。
  5. 根据权利要求4所述的方法，其中，所述编码端将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域，包括：
    所述编码端将所述第三连通区域合并至第一目标候选区域；
    其中，所述第一目标候选区域为所述至少一个目标候选区域中投影面与所述第三连通区域的法向量之间的夹角最小的目标候选区域，或者，所述第一目标候选区域为所述至少一个目标候选区域中包含的面数最多的目标候选区域。
  6. 根据权利要求4或5所述的方法，其中，所述候选区域与所述第三连通区域之间存在至少两个重合顶点。
  7. 根据权利要求1至6中任一项所述的方法，其中，所述第一网格根据简化网格确定，所述简化网格为对所述待编码的三维网格进行网格简化处理所得到的网格；
    其中，所述简化网格的顶点数小于所述待编码的三维网格的顶点数，或者，所述简化网格的面数小于所述待编码的三维网格的面数。
  8. 根据权利要求7所述的方法，还包括：
    所述编码端对所述简化网格进行细分变形，得到第二网格；
    所述编码端根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，包括：
    所述编码端根据所述第二重建网格和所述第二网格计算所述第二网格的各个顶点的位移；
    所述编码端对所述第二网格的各个顶点的位移进行编码，得到位移码流；
    所述编码端根据所述第一重建网格和所述位移码流进行三维网格重建，得到第三重建网格；
    所述编码端根据所述待编码的三维网格和所述第三重建网格对第一纹理图进行转换，得到第二纹理图。
  9. 根据权利要求1至8中任一项所述的方法，还包括：
    所述编码端对第一编码信息执行第一操作，其中，所述第一编码信息包括所述网格码流和所述纹理图码流，所述第一操作包括如下至少一项：存储，传输。
  10. 根据权利要求9所述的方法，其中，所述第一编码信息还包括如下至少一项：
    位移码流；
    第一辅助信息，所述第一辅助信息包括如下至少一项：与网格编码相关的信息，与纹理图编码相关的信息，与网格细分变形相关的信息，与网格位移相关的信息，与网格参数化处理相关的信息。
  11. 一种三维网格解码方法，包括：
    解码端获取网格码流和纹理图码流；
    所述解码端对所述网格码流进行解码得到第三网格，并对所述纹理图码流进行解码得到第三纹理图；
    所述解码端对所述第三网格进行网格参数化处理，得到第四网格，其中，所述第四网格为包括纹理坐标的网格。
  12. 根据权利要求11所述的方法，其中，所述解码端对所述第三网格进行网格参数化处理，得到第四网格，包括：
    所述解码端采用正交投影算法对所述第三网格进行网格参数化处理，得到第四网格。
  13. 根据权利要求12所述的方法，其中，所述解码端采用正交投影算法对所述第三网格进行网格参数化处理，得到第四网格，包括：
    所述解码端将所述第三网格划分为至少两个连通区域，其中，每个所述连通区域均包括所述第三网格的至少一个面，投影面相同且在三维空间中连通的面属于同一连通区域；
    所述解码端对所述至少两个连通区域进行合并处理，得到至少一个第一连通区域；
    所述解码端分别对所述至少一个第一连通区域执行第一处理，其中，所述第一处理包括对每个所述第一连通区域的各个面进行投影以进行重叠区域检测，以及将存在重叠区域的第一连通区域拆分为不重叠的至少两个连通区域；
    所述解码端对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域；
    所述解码端将所述至少一个第二连通区域的投影区域进行二维排列，以获得第四网格。
  14. 根据权利要求13所述的方法，其中，所述解码端对所述第一处理后所得到的连通区域进行合并处理，得到至少一个第二连通区域，包括：
    所述解码端获取第三连通区域的候选区域集合，其中，所述候选区域集合包括至少一个候选区域，所述候选区域为所述第一处理后所得到的连通区域中与所述第三连通区域邻接的连通区域，所述第三连通区域为所述第一处理后所得到的连通区域中的任意连通区域；
    所述解码端获取所述合并候选区域集合的至少一个目标候选区域，其中，在所述第三连通区域合并至所述目标候选区域的情况下所述第三连通区域的投影区域与所述目标候选区域的投影区域不重叠；
    所述解码端将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域。
  15. 根据权利要求14所述的方法，其中，所述解码端将所述第三连通区域合并至所述至少一个目标候选区域中的一个目标候选区域，包括：
    所述解码端将所述第三连通区域合并至第一目标候选区域；
    其中，所述第一目标候选区域为所述至少一个目标候选区域中投影面与所述第三连通区域的法向量之间的夹角最小的目标候选区域，或者，所述第一目标候选区域为所述至少一个目标候选区域中包含的面数最多的目标候选区域。
  16. 根据权利要求14或15所述的方法，其中，所述候选区域与所述第三连通区域之间 存在至少两个重合顶点。
  17. 根据权利要求11至16中任一项所述的方法，还包括：
    所述解码端获取位移码流；
    所述解码端对所述第四网格进行细分变形，得到第五网格；
    所述解码端根据所述第五网格和所述位移码流经解码得到的位移进行三维网格重建，得到第四重建网格。
  18. 根据权利要求11至17中任一项所述的方法，还包括：
    所述解码端获取第一辅助信息，其中，所述第一辅助信息包括如下至少一项：与网格编码相关的信息，与纹理图编码相关的信息，与网格细分变形相关的信息，与网格位移相关的信息，与网格参数化处理相关的信息。
  19. 一种三维网格编码装置，包括：
    第一编码模块，用于对第一网格进行压缩编码，得到网格码流，其中，所述第一网格为根据待编码的三维网格确定的网格；
    第一重建模块，用于基于所述网格码流进行三维网格重建，得到第一重建网格；
    第一处理模块，用于对所述第一重建网格进行网格参数化处理，得到第二重建网格，其中，所述第二重建网格为包括纹理坐标的第一重建网格；
    转换模块，用于根据所述待编码的三维网格和所述第二重建网格对第一纹理图进行转换，得到第二纹理图，其中，所述第一纹理图为所述待编码的三维网格对应的纹理图；
    第二编码模块，用于对所述第二纹理图进行编码，得到纹理图码流。
  20. 一种三维网格解码装置，包括：
    第一获取模块，用于获取网格码流和纹理图码流；
    解码模块，用于对所述网格码流进行解码得到第三网格，并对所述纹理图码流进行解码得到第三纹理图；
    第二处理模块，用于对所述第三网格进行网格参数化处理，得到第四网格，其中，所述第四网格为包括纹理坐标的网格。
  21. 一种电子设备，包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求1至10任一项所述的三维网格编码方法的步骤，或实现如权利要求11至18任一项所述的维网格解码方法的步骤。
  22. 一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如权利要求1至10任一项所述的三维网格编码方法的步骤，或实现如权利要求11至18任一项所述的维网格解码方法的步骤。
  23. 一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1至10任一项所述的三维网格编码方法的步骤，或者实现如权利要求11至18任一项所述的维网格解码方法的步骤。
PCT/CN2024/100835 2023-06-30 2024-06-24 三维网格编码方法、解码方法、装置、编码端及解码端 Ceased WO2025002019A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310800556.8 2023-06-30
CN202310800556.8A CN119232904B (zh) 2023-06-30 2023-06-30 三维网格编码方法、解码方法、装置、编码端及解码端

Publications (1)

Publication Number Publication Date
WO2025002019A1 true WO2025002019A1 (zh) 2025-01-02

Family

ID=93937630

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/100835 Ceased WO2025002019A1 (zh) 2023-06-30 2024-06-24 三维网格编码方法、解码方法、装置、编码端及解码端

Country Status (2)

Country Link
CN (1) CN119232904B (zh)
WO (1) WO2025002019A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210090301A1 (en) * 2019-09-24 2021-03-25 Apple Inc. Three-Dimensional Mesh Compression Using a Video Encoder
CN115690361A (zh) * 2022-11-09 2023-02-03 浙江大学 快速鲁棒的自由曲面三角剖分方法及装置
US20230050860A1 (en) * 2020-01-02 2023-02-16 Nokia Technologies Oy An apparatus, a method and a computer program for volumetric video
US20230068178A1 (en) * 2020-01-02 2023-03-02 Nokia Technologies Oy A method, an apparatus and a computer program product for volumetric video encoding and decoding

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101626509B (zh) * 2009-08-10 2011-02-02 北京工业大学 三维网格编码、解码方法及编码、解码装置
US20140092439A1 (en) * 2012-09-28 2014-04-03 Scott A. Krig Encoding images using a 3d mesh of polygons and corresponding textures
CN112017292B (zh) * 2019-05-31 2025-05-06 华为技术有限公司 网格译码方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210090301A1 (en) * 2019-09-24 2021-03-25 Apple Inc. Three-Dimensional Mesh Compression Using a Video Encoder
US20230050860A1 (en) * 2020-01-02 2023-02-16 Nokia Technologies Oy An apparatus, a method and a computer program for volumetric video
US20230068178A1 (en) * 2020-01-02 2023-03-02 Nokia Technologies Oy A method, an apparatus and a computer program product for volumetric video encoding and decoding
CN115690361A (zh) * 2022-11-09 2023-02-03 浙江大学 快速鲁棒的自由曲面三角剖分方法及装置

Also Published As

Publication number Publication date
CN119232904A (zh) 2024-12-31
CN119232904B (zh) 2025-09-09

Similar Documents

Publication Publication Date Title
TW202249488A (zh) 點雲屬性的預測方法、裝置及編解碼器
WO2025002021A1 (zh) 三维网格帧间预测编码方法、解码方法、装置及电子设备
WO2024193487A1 (zh) 三维网格位移信息编码方法、解码方法、装置及终端
WO2024217304A1 (zh) 三维网格位移信息编码方法、解码方法、装置及终端
US12524919B2 (en) Point cloud coding/decoding method and apparatus, device and storage medium
WO2025002019A1 (zh) 三维网格编码方法、解码方法、装置、编码端及解码端
CN116458158B (zh) 帧内预测方法及装置、编解码器、设备、存储介质
CN119182907B (zh) 网格编码方法、网格解码方法及相关设备
CN120958824A (zh) 点云编解码方法、装置、设备及存储介质
CN119182908B (zh) 网格编码方法、网格解码方法及相关设备
CN118678093B (zh) 编码处理方法、解码处理方法及相关设备
CN120321415A (zh) 位移信息视频生成方法、解码方法、装置及电子设备
US20260129213A1 (en) Three-dimensional mesh inter-frame prediction encoding method, decoding method, and apparatus, and electronic device
CN119815052B (zh) 编码方法、解码方法及相关设备
CN120321417A (zh) 数据优化方法及相关设备
CN121151562A (zh) 数据提取方法、数据排列方法、装置以及设备
CN121284235A (zh) 网格解码方法、网格编码方法、装置以及设备
CN120835141A (zh) 基于片的网格解码方法、编码方法及相关设备
KR20260021745A (ko) 속성 인코딩 방법, 속성 디코딩 방법 및 전자 디바이스
CN119232952A (zh) 三维网格编码方法、解码方法、装置及电子设备
WO2026085664A1 (zh) 编解码方法、编码器、解码器以及存储介质
CN121284234A (zh) 网格解码方法、网格编码方法、装置以及设备
WO2024207463A1 (zh) 点云编解码方法、装置、设备及存储介质
TW202425650A (zh) 點雲編解碼方法、裝置、設備及儲存媒介
WO2022183611A1 (zh) 帧内预测方法及装置、编解码器、设备、存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24830664

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE