WO2023174336A1 - 编码、解码方法、装置及设备 - Google Patents

编码、解码方法、装置及设备 Download PDF

Info

Publication number
WO2023174336A1
WO2023174336A1 PCT/CN2023/081640 CN2023081640W WO2023174336A1 WO 2023174336 A1 WO2023174336 A1 WO 2023174336A1 CN 2023081640 W CN2023081640 W CN 2023081640W WO 2023174336 A1 WO2023174336 A1 WO 2023174336A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
geometric
placeholder
map
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/081640
Other languages
English (en)
French (fr)
Inventor
邹文杰
张伟
杨付正
吕卓逸
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Publication of WO2023174336A1 publication Critical patent/WO2023174336A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder

Definitions

  • This application belongs to the field of coding and decoding technology, and specifically relates to a coding and decoding method, device and equipment.
  • Three-dimensional mesh can be considered the most popular representation method of three-dimensional models in the past many years, and it plays an important role in many applications. Its expression is simple, so it is widely integrated into the graphics processing unit (GPU) of computers, tablets and smartphones with hardware algorithms, specifically used to render three-dimensional meshes.
  • GPU graphics processing unit
  • the vertices and point clouds of Mesh are both a set of randomly distributed discrete points in space, they have similar characteristics. Therefore, 3D mesh geometry information can be compressed using point cloud compression algorithms. However, compared with point clouds, the vertices of three-dimensional grids are spatially distributed more sparsely and unevenly. Using point cloud compression algorithm to compress the geometric information of the three-dimensional mesh model, the coding efficiency is not high.
  • Embodiments of the present application provide an encoding and decoding method, device and equipment, which can solve the problem of low encoding efficiency of three-dimensional grid geometric information in the prior art.
  • the first aspect provides an encoding method, including:
  • the encoding end obtains the placeholder map and geometric map based on the geometric information of the target three-dimensional grid
  • the encoding end performs translation processing on the pixels of the projection point in the first image to obtain a second image, where the first image includes at least one of a placeholder map and a geometric map;
  • the encoding end encodes a third image and the second image, where the third image is a picture of the placeholder image and the geometric image that does not belong to the first image.
  • an encoding device including:
  • the first acquisition module is used to obtain placeholder images and geometric images based on the geometric information of the target three-dimensional grid
  • the second acquisition module is used to perform translation processing on the pixels of the projection point in the first image to obtain a second image, where the first image includes at least one of a placeholder map and a geometric map;
  • An encoding module configured to encode a third image and the second image, where the third image is the placeholder image and the geometric image that does not belong to the first image.
  • the third aspect provides a decoding method, including:
  • the decoding end decomposes the obtained code stream of the target three-dimensional grid, and obtains a second image and a third image, where the second image includes at least one of a closely arranged placeholder image and a closely arranged geometric image,
  • the third image is a picture among the placeholder picture and the geometric picture that does not belong to the second image;
  • the decoding end performs translation processing on the pixels of the projection point in the second image to obtain the first image
  • the decoding end obtains geometric information of the target three-dimensional grid based on the first image and the third image.
  • a decoding device including:
  • the third acquisition module is used to decompose the acquired code stream of the target three-dimensional grid and acquire the second image and the third image.
  • the second image includes closely arranged placeholder images and closely arranged geometric images. At least one of the items, the third image is a picture among the placeholder picture and the geometric picture that does not belong to the second image;
  • the fourth acquisition module is used to perform translation processing on the pixels of the projection point in the second image to obtain the first image
  • the fifth acquisition module is used to acquire the geometric information of the target three-dimensional grid according to the first image and the third image.
  • a coding device including a processor and a memory.
  • the memory stores programs or instructions that can be run on the processor. When the program or instructions are executed by the processor, the first The steps of the method described in this aspect.
  • an encoding device including a processor and a communication interface, wherein the processor is configured to obtain a placeholder map and a geometric map according to the geometric information of the target three-dimensional grid; Perform translation processing to obtain a second image, where the first image includes a placeholder image and At least one of the geometric figures; encoding a third image and the second image, where the third image is a figure among the placeholder image and the geometric figure that does not belong to the first image.
  • a decoding device including a processor and a memory.
  • the memory stores programs or instructions that can be run on the processor.
  • the program or instructions are executed by the processor, the third process is implemented. The steps of the method described in this aspect.
  • a decoding device including a processor and a communication interface, wherein the processor is used to decompose the obtained code stream of the target three-dimensional grid and obtain the second image and the third image, and the third
  • the two images include at least one of closely arranged placeholder images and closely arranged geometric figures, and the third image is a figure among the placeholder images and the geometric figures that does not belong to the second image; for the The pixels of the projection points in the second image are translated to obtain a first image; and the geometric information of the target three-dimensional grid is obtained according to the first image and the third image.
  • a communication system including: an encoding device and a decoding device.
  • the encoding device can be used to perform the steps of the method described in the first aspect
  • the decoding device can be used to perform the steps of the method described in the third aspect. steps of the method.
  • a readable storage medium is provided. Programs or instructions are stored on the readable storage medium. When the programs or instructions are executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method are implemented as described in the first aspect. The steps of the method described in the third aspect.
  • a chip in an eleventh aspect, includes a processor and a communication interface.
  • the communication interface is coupled to the processor.
  • the processor is used to run programs or instructions to implement the method described in the first aspect. The steps of a method, or steps of implementing a method as described in the third aspect.
  • a computer program product is provided, the computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps of the method as described in the first aspect, or Implementing the steps of the method as described in the third aspect.
  • a thirteenth aspect provides a communication device configured to perform the steps of the method described in the first aspect, or to perform the steps of the method described in the third aspect.
  • At least one of the placeholder map and the geometric map is translated to achieve a tight arrangement, thereby reducing the number of encoding bits and thereby improving compression efficiency.
  • Figure 1 is a schematic flow chart of the encoding method according to the embodiment of the present application.
  • Figure 2 is a schematic diagram of the fine division process based on grid
  • Figure 3 is a schematic diagram of the eight directions of patch arrangement
  • Figure 4 is a schematic diagram of the geometric diagram before and after pixel translation
  • Figure 5 is a schematic diagram of the video-based three-dimensional grid geometric information encoding framework
  • Figure 6 is a schematic module diagram of an encoding device according to an embodiment of the present application.
  • Figure 7 is a schematic structural diagram of an encoding device according to an embodiment of the present application.
  • Figure 8 is a schematic flow chart of the decoding method according to the embodiment of the present application.
  • Figure 9 is a schematic diagram of the video-based three-dimensional grid geometric information decoding framework
  • Figure 10 is a schematic module diagram of a decoding device according to an embodiment of the present application.
  • Figure 11 is a schematic structural diagram of a communication device according to an embodiment of the present application.
  • first, second, etc. in the description and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in sequences other than those illustrated or described herein, and that "first" and “second” are distinguished objects It is usually one type, and the number of objects is not limited.
  • the first object can be one or multiple.
  • “and/or” in the description and claims indicates at least one of the connected objects, and the character “/" generally indicates that the related objects are in an "or” relationship.
  • LTE Long Term Evolution
  • LTE-Advanced, LTE-A Long Term Evolution
  • CDMA Code Division Multiple Access
  • TDMA Time Division Multiple Access
  • FDMA Frequency Division Multiple Access
  • OFDMA Orthogonal Frequency Division Multiple Access
  • SC-FDMA single-carrier frequency division multiple access
  • NR New Radio
  • Three-dimensional models have become a new generation of digital media after audio, images, and videos.
  • Three-dimensional mesh and point cloud are two commonly used three-dimensional model representation methods.
  • 3D mesh models are more interactive and realistic, making them useful in various fields such as commerce, manufacturing, construction, education, medicine, entertainment, art, and military. has been increasingly widely used.
  • 3D mesh models obtained through 3D scanning equipment or 3D modeling software
  • the size and complexity of data are also growing rapidly. Therefore, how to efficiently compress 3D mesh data is the key to realizing convenient transmission, storage and processing of 3D mesh data.
  • a three-dimensional mesh often contains three main types of information: topological information, geometric information and attribute information.
  • Topological information is used to describe the connection relationship between elements such as vertices and patches in the mesh; geometric information is the three-dimensional coordinates of all vertices in the mesh; attribute information records other information attached to the mesh, such as normal vectors, Texture coordinates and colors, etc.
  • Some traditional general data compression methods can reduce a certain amount of 3D grid data, due to the particularity of 3D grid data, directly using these compression methods to compress 3D grid data often cannot achieve ideal results. Therefore, the compression of three-dimensional mesh data faces new challenges.
  • geometric data often takes up more storage space than topological data. Efficient compression of geometric data will be of extremely important significance in reducing the storage space of three-dimensional grid data. Therefore, the compression of three-dimensional mesh geometric information has become a research focus.
  • the 3D mesh geometric information compression algorithm can use the 3D geometric information compression algorithm of point cloud.
  • V-PCC Video-based Point Cloud Compression
  • G-PCC Geometry-based Point Cloud Compression
  • V-PCC The main idea of V-PCC is to project the geometric and attribute information of the point cloud into a two-dimensional video, and use existing video coding technology to compress the two-dimensional video to achieve the purpose of compressing the point cloud.
  • the geometric coding of V-PCC is achieved by projecting geometric information into placeholder video and geometric video, and using a video encoder to encode the two videos separately.
  • the process of V-PCC geometric information encoding mainly includes: first generating a three-dimensional patch (3D patch), which refers to a set of vertices with the same and connected projection plane in the point cloud.
  • the current method of generating 3D patches is to use nearby points to estimate the normal vector of each vertex, calculate the projection plane of each vertex based on the normal vector of each point and the normal vector of the preset plane, and connect the connected vertices with the same projection plane Make up a patch.
  • the 3D patch is projected onto a two-dimensional (2D) plane to form a 2D patch, and the 2D patches are arranged on a two-dimensional image. This process is called patch packing.
  • a placeholder map is an image that represents the placeholder information of vertices in a two-dimensional image.
  • the position value of the vertex projection in the placeholder map is 1, and the other position values are 0.
  • a placeholder image is generated. What is stored in the geometric graph is the distance from each vertex to the projection plane. The depth information of each vertex can be directly calculated using the three-dimensional coordinates of the vertex, the projection plane of the vertex and the placeholder map, thereby generating a geometric map.
  • Image filling methods include "push-pull" background filling algorithm, filling method based on sparse linear model (Sparse Linear Model), harmonic background filling (Harmonic Background Filling) and other methods.
  • the final geometric map is obtained.
  • the existing video encoder is used to compress the placeholder map and geometric map to obtain the video code stream.
  • the placeholder video code stream, the geometric video code stream, and the sub-code stream containing patch information are synthesized into the final total code stream.
  • this embodiment of the present application provides an encoding method, including:
  • Step 101 The encoding end obtains the placeholder map and geometric map based on the geometric information of the target three-dimensional grid;
  • the target three-dimensional grid mentioned in this application can be understood as the three-dimensional grid corresponding to any video frame.
  • the geometric information of the target three-dimensional grid can be understood as the coordinates of the vertices in the three-dimensional grid. These coordinates are usually Refers to three-dimensional coordinates.
  • Step 102 The encoding end performs translation processing on the pixels of the projection point in the first image to obtain the second image;
  • the first image includes at least one of a placeholder image and a geometric image.
  • the projection point refers to the point corresponding to the vertex in the two-dimensional image after projecting the vertex of the three-dimensional grid into a two-dimensional slice.
  • a pixel in a two-dimensional image corresponds to the projection point of a vertex.
  • the pixel value corresponding to the projected point in the placeholder map is usually 1, and the depth value corresponding to the projected point in the geometric map is usually greater than 0.
  • Step 103 The encoding end encodes the third image and the second image
  • the third image is a diagram among the placeholder diagram and the geometric diagram that does not belong to the first image.
  • the third image includes a geometric diagram
  • the third image includes a placeholder diagram
  • the third image includes a placeholder diagram and a geometric diagram.
  • the third image is an empty set, which can be understood as, in this case, there is no third image.
  • this application translates the pixels of the projection points of at least one of the occupied images and geometric images. , to achieve a tight arrangement of pixels, for example, keeping the placeholder map unchanged and translating the pixels of the projection points in the geometric map horizontally or vertically to improve the coding efficiency of the subsequent video encoder when encoding the geometric map.
  • step 101 is:
  • Step 1011 The encoding end divides the geometric information of the target three-dimensional grid into three-dimensional slices;
  • the main task is to divide the geometric information of the target three-dimensional grid into patches to obtain multiple three-dimensional patches;
  • the specific implementation method of this step is: the encoding end determines the geometry of the target three-dimensional grid. The projection plane of each vertex contained in the information; the encoding end performs slice division on the vertices contained in the geometric information of the target three-dimensional mesh according to the projection plane; the encoding end The vertices contained in the geometric information of the target three-dimensional mesh are clustered to obtain each divided piece.
  • the process of patch division mainly includes: first estimating the normal vector of each vertex, selecting the candidate projection plane with the smallest angle between the plane normal vector and the vertex normal vector as the projection plane of the vertex; then, according to the projection
  • the plane initially divides the vertices, and vertices with the same and connected projection planes are composed into patches; finally, a fine division algorithm is used to optimize the clustering results and obtain the final three-dimensional patch (3D patch).
  • the projection plane of each vertex is initially selected.
  • the normal vector of the candidate projection plane is Select the plane whose normal vector direction is closest to the vertex normal vector direction as the projection plane of the vertex.
  • the calculation process of plane selection is as shown in Formula 3:
  • the fine division process can use a grid-based algorithm to reduce the time complexity of the algorithm.
  • the grid-based fine division algorithm flow is shown in Figure 2, which specifically includes:
  • Step S201 Divide the (x, y, z) geometric coordinate space into voxels.
  • the geometric coordinate space here refers to the geometric coordinate space composed of the geometric information of the target three-dimensional grid.
  • Step S202 Find filled voxels.
  • Filled voxels refer to volumes containing at least one point in the grid. white.
  • Step S203 calculate the smoothing score of each filled voxel on each projection plane, recorded as voxScoreSmooth.
  • the voxel smoothing score of the voxel on a certain projection plane is the number of points gathered to the projection plane through the initial segmentation process.
  • Step S204 use KD-Tree partitioning to find neighboring filled voxels, recorded as nnFilledVoxels, that is, the nearest filled voxels of each filled voxel (within the search radius and/or limited to the maximum number of adjacent voxels).
  • Step S205 use the voxel smoothing score of the nearest neighbor filled voxel in each projection plane to calculate the smoothing score (scoreSmooth) of each filled voxel.
  • the calculation process is as shown in Formula 4:
  • Step S206 Calculate the normal score using the normal vector of the vertex and the normal vector of the candidate projection plane, recorded as scoreNormal.
  • the calculation process is as shown in Formula 5:
  • p is the index of the projection plane and i is the index of the vertex.
  • Step S207 use scoreSmooth and scoreNormal to calculate the final score of each voxel on each projection plane.
  • the calculation process is as shown in Formula 6:
  • i is the vertex index
  • p is the index of the projection plane
  • v is the voxel index where vertex i is located.
  • Step S208 Use the scores in step 207 to cluster the vertices to obtain finely divided patches.
  • Step 1012 The encoding end performs two-dimensional projection on the divided three-dimensional slice to obtain the two-dimensional slice;
  • this process is to project the 3D patch onto a two-dimensional plane to obtain a two-dimensional patch (2D patch).
  • patch partitioning converts 3D samples into 2D samples by using a strategy that provides the best compression performance on a given projection plane.
  • the goal of patch division is to decompose the vertices of a frame of 3D model into patches with the smallest number and smooth boundaries, while minimizing the reconstruction error.
  • Step 1013 The encoding end packages the two-dimensional slices to obtain two-dimensional image information
  • this step implements patch packing.
  • the purpose of patch packing is to arrange 2D patches on a two-dimensional image.
  • the basic principle of patch packing is to arrange patches on a two-dimensional image without overlapping or The pixel-free parts of the patch are partially overlapped and arranged on the two-dimensional image.
  • the patches are arranged more closely and have time domain consistency to improve coding performance.
  • the resolution of the 2D image is WxH
  • the minimum block size that defines the patch arrangement is T, which specifies the minimum distance between different patches placed on this 2D grid.
  • patches are inserted and placed on the 2D grid according to the non-overlapping principle.
  • Each patch occupies an area consisting of an integer number of TxT blocks.
  • the patches can choose a variety of different arrangement directions. For example, eight different arrangement directions can be adopted, as shown in Figure 3, including 0 degrees, 180 degrees, 90 degrees, 270 degrees and mirror images of the first four directions.
  • a patch arrangement method with temporal consistency is adopted.
  • a group of frames all patches of the first frame are arranged in order from largest to smallest.
  • the temporal consistency algorithm is used to adjust the order of patches.
  • the patch information can be obtained based on the information in the process of obtaining the two-dimensional image information. After that, the patch information can be encoded and the patch information sub-stream can be obtained;
  • the patch information records the information of each step operation in the process of obtaining two-dimensional image.
  • the patch information includes: patch division information, patch projection plane information, and patch packing position information.
  • Step 1014 The encoding end obtains a placeholder map and a geometric map based on the two-dimensional image information
  • the process of obtaining the placeholder image is mainly: using the patch arrangement information obtained by patch packing, setting the position of the vertex in the two-dimensional image to 1, and setting the remaining positions to 0. Get the placeholder image.
  • the main process is: in the process of obtaining the 2D patch through projection, the distance from each vertex to the projection plane is saved. This distance is called depth.
  • the depth value of each vertex in the 2D patch is arranged to At the position of the vertex in the placeholder map, the geometric map is obtained.
  • the specific implementation of the above step 102 is:
  • the encoding end scans the pixels of the projection points in the first image row by row or column by column along the first direction, and re-numbers the position index of the pixels in each row or column within the row or column; so The encoding end arranges the pixel values of each vertex in the pixel position specified by the position index corresponding to the projection point corresponding to the vertex according to the scanning order, and obtains the closely arranged second image after translation, and the first direction Either horizontally or vertically.
  • the pixels corresponding to the projection points in the placeholder map or geometric map are not closely adjacent to each other, but have a certain interval.
  • the scan renumbering mentioned in the embodiment of this application That is, pixels with gaps in the same row or column are renumbered in a pixel-adjacent manner. For example, in a certain row, there are 5 pixels, and their numbers from left to right are 1, 3, 5, 7, 8. After scanning from left to right, renumber the five pixels. After renumbering, the new numbers of the five pixels from left to right are 1, 2, 3, 4, and 5.
  • the pixel value at each pixel position is represented by the pixel value of the vertex corresponding to the pixel, thereby obtaining a tightly arranged geometric map or placeholder map.
  • the pixel value of the vertex is the corresponding value of the vertex in the placeholder information.
  • the pixel value of the vertex is 1; when the first image is a geometric image, the pixel value of the vertex is Refers to the depth value of the vertex.
  • this implementation method is to first scan the pixels of the projection points corresponding to the vertices in the geometric diagram (or placeholder diagram) and re-number them.
  • the pixels are scanned in the horizontal direction. Scan row by row, and renumber the index positions of the pixels in each row within the row.
  • scanning in the vertical direction scan column by column in the vertical direction, and renumber the index positions of the pixels in each column.
  • Figure 4 is a schematic diagram of the geometric map before and after pixel translation. In Figure 4, the depth values of the pixels are binarized to improve the visibility of the schematic map. In practice, the pixels are closely arranged Pixels have different depth values.
  • the specific implementation of the above step 102 is as follows:
  • the encoding end translates the pixels of the projection points in the placeholder map and the geometric map to the second direction, and the pixel value corresponding to the projection point in the placeholder map after the pixel translation is calculated using the distance of the translation of the pixel of the projection point. Representation, obtain closely arranged placeholder images and geometric images respectively;
  • the second direction is a horizontal direction or a vertical direction
  • the translation distance is the relative distance of the pixel translation of the projection point or the absolute distance of the pixel translation of the projection point.
  • this implementation method is to use the placeholder map and the geometric map in the same way (the so-called same way refers to the corresponding movement direction and distance of the pixels of the projection point in the placeholder map and the geometric map).
  • the movement direction and distance of the pixels of the projection point are the same) are gathered in a certain direction, and the pixel value corresponding to the projection point in the placeholder map is recorded as the absolute distance of the translation of the pixel of the projection point (or the translation of the pixel of the projection point Relative distance (for example, the difference between the pixel movement distance of the current projection point and the pixel movement of the previous projection point is used as the relative distance). That is to say, at this time, the pixel value in the placeholder map is no longer 1, but is recorded The value of the pixel translation distance of the projection point.
  • projection is used to perform patch division, patch projection, and patch arrangement to generate patch sequence compression information (including patch division information, projection information, and arrangement information), placeholder map, and geometric map; for pixel point distribution
  • the sparse geometric map translates the pixels of the projection points in the geometric map along the horizontal or vertical direction, so that the pixels are closely arranged to form a geometric map with closely arranged pixels; finally, the patch sequence compression information, placeholder map, and geometric map are encoded respectively.
  • the corresponding sub-code streams are obtained, and the multiple sub-code streams are mixed to obtain the final output code stream.
  • this application provides an implementation method of encoding the geometric information of a three-dimensional grid.
  • the distance between vertices is reduced. spacing, which can improve the compression efficiency and coding efficiency of three-dimensional mesh geometric information.
  • the execution subject may be an encoding device.
  • the encoding device performing the encoding method is taken as an example to illustrate the encoding device provided by the embodiment of the present application.
  • this embodiment of the present application provides an encoding device 600, which includes:
  • the first acquisition module 601 is used to acquire the placeholder map and geometric map according to the geometric information of the target three-dimensional grid
  • the second acquisition module 602 is used to perform translation processing on the pixels of the projection points in the first image to obtain a second image, where the first image includes at least one of a placeholder map and a geometric map;
  • the encoding module 603 is used to encode a third image and the second image, where the third image is a picture of the placeholder image and the geometric image that does not belong to the first image.
  • the first image is a placeholder image or a geometric image
  • the second acquisition module 602 includes:
  • a first scanning unit configured to scan the pixels of the projection points in the first image row by row or column by column along the first direction, and re-scan the position index of the pixels in each row or column within the row or column.
  • the first acquisition unit is configured to arrange the pixel depth value of each vertex at the pixel position specified by the position index corresponding to the projection point corresponding to the vertex according to the scanning order, and obtain the closely arranged second image after translation;
  • the first direction is a horizontal direction or a vertical direction.
  • the second acquisition module 602 includes:
  • the second acquisition unit is used to translate the pixels of the projection points in the placeholder map and the geometric map to the second direction, and use the pixel values corresponding to the projection points in the placeholder map after the pixels are translated to translate the pixels of the projection points. represent the distance, and obtain closely arranged placeholder images and geometric images respectively;
  • the second direction is a horizontal direction or a vertical direction
  • the translation distance is the relative distance of the pixel translation of the projection point or the absolute distance of the pixel translation of the projection point.
  • the first acquisition module 601 includes:
  • a dividing unit used to divide the geometric information of the target three-dimensional grid into three-dimensional slices
  • the third acquisition unit is used to perform two-dimensional projection on the divided three-dimensional slices to obtain the two-dimensional slices;
  • the fourth acquisition unit is used to package the two-dimensional slices and acquire two-dimensional image information
  • the fifth acquisition unit is used to acquire a placeholder map and a geometric map according to the two-dimensional image information.
  • the first acquisition module 601 further includes:
  • the sixth acquisition unit is used to acquire slice information based on the information in the process of acquiring two-dimensional image information
  • the seventh acquisition unit is used to encode the slice information and obtain the slice information sub-stream.
  • This device embodiment corresponds to the above-mentioned encoding method embodiment.
  • Each implementation process and implementation manner of the above-mentioned method embodiment can be applied to this device embodiment, and can achieve the same technical effect.
  • Embodiments of the present application also provide an encoding device, including a processor and a communication interface, wherein the processor is configured to obtain a placeholder map and a geometric map according to the geometric information of the target three-dimensional grid; Perform close arrangement processing to obtain a second image, where the first image includes at least one of a placeholder image and a geometric image; and encode a third image and the second image, where the third image is the placeholder image. Bitmaps and geometric figures that do not belong to the first image.
  • the encoding device 700 includes: a processor 701, a network interface 702, and a memory 703.
  • the network interface 702 is, for example, a common public radio interface (CPRI).
  • CPRI common public radio interface
  • the encoding device 700 in the embodiment of the present application also includes: instructions or programs stored in the memory 703 and executable on the processor 701.
  • the processor 701 calls the instructions or programs in the memory 703 to execute the modules shown in Figure 6
  • the implementation method and achieve the same technical effect will not be repeated here to avoid repetition.
  • this embodiment of the present application also provides a decoding method, including:
  • Step 801 The decoder decomposes the obtained code stream of the target three-dimensional grid and obtains the second image and the third image;
  • the second image includes at least one of closely arranged placeholder images and closely arranged geometric figures
  • the third image is one of the placeholder images and the geometric figures that does not belong to the Picture of the second image
  • Step 802 The decoder performs translation processing on the pixels of the projection point in the second image to obtain the first image
  • Step 803 The decoder acquires the geometric information of the target three-dimensional grid based on the first image and the third image.
  • the decoder after using the video decoder to decode and obtain at least one closely arranged placeholder map and geometric map, the placeholder map and geometric map arranged according to the projection position are restored, for example, using the The two-dimensional position information of the pixels in the bitmap translates the pixels in the geometric map to restore the geometric map; finally, the slice information, geometric map and placeholder map are used to restore the three-dimensional geometric information of the three-dimensional grid.
  • the translation process is performed on the pixels of the projection points in the second image to obtain the first Images, including:
  • the decoding end scans the pixels of the projection point of the third image along the first direction row by row or column by column, and obtains the position index within the row or column where each pixel in each row or column is located;
  • the decoder scans the pixels of the projection points in the second image row by row or column by column in the first direction, and scans each pixel in each row or column according to the same pattern as the projection point in the third image.
  • the position index corresponding to the pixel is re-numbered within the row or column;
  • the decoding end arranges the pixel values of each vertex at the pixel position specified by the position index corresponding to the projection point corresponding to the vertex in the third image according to the scanning order, and obtains the sparsely arranged third image after translation. 2 images;
  • the first direction is a horizontal direction or a vertical direction.
  • the pixels of the projection points in the second image are translated to obtain the first Images, including:
  • the decoding end translates the pixels of the projection points in the closely arranged placeholder map and the closely arranged geometric map along the second direction according to the pixel values of the projection points in the closely arranged placeholder map, and obtains the sparse array respectively.
  • the pixel value of the projection point in the closely arranged placeholder map is represented by the translation distance of the pixel of the projection point, the second direction is the horizontal direction or the vertical direction, and the translation distance is the translation distance of the projection point.
  • the pixel values in the closely arranged placeholder map are represented by the translation distance of the pixels of the projection point, when performing image restoration, the closely arranged pixels will be closely arranged according to the translation distance of the pixels of the projection point.
  • the placeholder map of the cloth and the closely arranged geometric map are moved in the opposite direction of the translation distance during the encoding process, so as to obtain the original placeholder map and geometric map, and realize the decoding of the geometric information of the three-dimensional grid.
  • decomposing the obtained code stream of the target three-dimensional grid and obtaining the second image and the third image includes:
  • the decoding end obtains a target sub-code stream according to the obtained code stream of the target three-dimensional grid.
  • the target sub-code stream includes: a slice information sub-stream, a placeholder map sub-stream and a geometric map sub-stream;
  • the decoding end obtains the second image and the third image according to the target sub-stream.
  • the geometric information reconstruction process is a process of reconstructing a three-dimensional geometric model using patch information, placeholder images, and geometric images.
  • the specific process is mainly divided into:
  • Step S1 obtain 2D patch
  • the closely arranged placeholder map and/or geometric map of the decoded pixels is restored according to the inverse process of the pixel translation rules in the encoder, and the placeholder map and geometric map whose pixel positions are two-dimensional projection positions are obtained.
  • Obtaining a 2D patch refers to using the patch information to segment the occupancy information and depth information of the 2D patch from the occupancy map and geometric map.
  • the patch information contains the position and size of the bounding box of each 2D patch in the placeholder map and geometric map.
  • the placeholder information and geometric information of the 2D patch can be directly obtained by using the patch information, placeholder map, and geometric map.
  • Step S2 reconstruct 3D patch
  • Reconstructing a 3D patch refers to using the occupancy information and geometric information in the 2D patch to reconstruct the vertices in the 2D patch into a 3D patch.
  • the placeholder information of a 2D patch contains the position of the vertex relative to the coordinate origin in the local coordinate system of the patch projection plane, and the depth information contains the depth value of the vertex in the normal direction of the projection plane. Therefore, the 2D patch can be reconstructed into a 3D patch in the local coordinate system using the occupancy information and depth information.
  • Step S3 Reconstruct the grid geometry information
  • the code stream is decomposed into patch information sub-streams, placeholder map sub-streams, and geometric map sub-streams, and decoded separately; the position information of the projected pixels in the decoded placeholder map is used to restore the decoded geometric map.
  • the original position of the pixel in the grid; the three-dimensional geometric information of the grid is reconstructed using the placeholder map, the restored geometric map and the patch information; finally, the grid is reconstructed using the reconstructed geometric information and the connection relationship information obtained by other encoding and decoding methods.
  • the embodiment of the present application is a method embodiment of the opposite end corresponding to the embodiment of the above encoding method.
  • the decoding process is the inverse process of encoding. All the above implementation methods on the encoding side are applicable to the embodiment of the decoding end. The same technical effect can also be achieved, which will not be described again here.
  • this embodiment of the present application also provides a decoding device 1000, which includes:
  • the third acquisition module 1001 is used to decompose the acquired code stream of the target three-dimensional grid, and acquire a second image and a third image.
  • the second image includes closely arranged placeholder images and closely arranged geometric images. At least one of the above, the third image is a picture among the placeholder picture and the geometric picture that does not belong to the second image;
  • the fourth acquisition module 1002 is used to perform translation processing on the pixels of the projection points in the second image to obtain the first image
  • the fifth acquisition module 1003 is used to acquire the geometric information of the target three-dimensional grid according to the first image and the third image.
  • the fourth acquisition module 1002 includes:
  • the eighth acquisition unit is used to scan the pixels of the projection point of the third image along the first direction row by row or column by column, and obtain the position index within the row or column where each pixel in each row or column is located. ;
  • the second scanning unit is used to scan the pixels of the projection points in the second image along the first direction row by row or column by column, and scan each pixel in each row or column according to the same pattern as in the third image.
  • the position index corresponding to the pixel of the projection point is renumbered within the row or column;
  • the ninth acquisition unit is used to arrange the pixel value of each vertex at the pixel position specified by the position index corresponding to the projection point corresponding to the vertex in the third image according to the scanning order, and obtain the sparse arrangement after translation.
  • the first direction is a horizontal direction or a vertical direction.
  • the fourth acquisition module 1002 includes:
  • the tenth acquisition unit is used to translate the pixels of the projection points in the closely arranged placeholder map and the closely arranged geometric map along the second direction according to the pixel values of the projection points in the closely arranged placeholder map, respectively. Obtain sparsely arranged placeholder images and geometric images;
  • the pixel value of the projection point in the closely arranged placeholder map is represented by the translation distance of the pixel of the projection point, the second direction is the horizontal direction or the vertical direction, and the translation distance is the translation distance of the projection point.
  • the third acquisition module 1001 includes:
  • the eleventh acquisition unit is used to acquire the target sub-code stream according to the acquired code stream of the target three-dimensional grid.
  • the target sub-code stream includes: slice information sub-stream, placeholder map sub-stream and geometric map sub-stream. flow;
  • the twelfth acquisition unit is used to acquire the second image and the third image according to the target sub-stream.
  • this device embodiment is a device corresponding to the above-mentioned method. All implementation methods in the above-mentioned method embodiment are applicable to this device embodiment and can achieve the same technical effect, which will not be described again here.
  • An embodiment of the present application also provides a decoding device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor.
  • a decoding device including a processor, a memory, and a program or instruction stored in the memory and executable on the processor.
  • the program or instruction is executed by the processor, the above decoding method is implemented.
  • Each process in the example can achieve the same technical effect. To avoid repetition, we will not repeat it here.
  • Embodiments of the present application also provide a readable storage medium.
  • Programs or instructions are stored on the computer-readable storage medium.
  • the program or instructions are executed by a processor, each process of the above-mentioned decoding method embodiment is implemented, and the same process can be achieved. To avoid repetition, the technical effects will not be repeated here.
  • the computer-readable storage medium is such as read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.
  • An embodiment of the present application also provides a decoding device, including a processor and a communication interface, wherein the processor is used to decompose the obtained code stream of the target three-dimensional grid and obtain the second image and the third image, and the
  • the second image includes at least one of closely arranged placeholder images and closely arranged geometric figures, and the third image is a figure among the placeholder images and the geometric figures that does not belong to the second image;
  • the pixels of the projection points in the second image are translated to obtain a first image; and the geometric information of the target three-dimensional grid is obtained according to the first image and the third image.
  • This decoding device embodiment corresponds to the above-mentioned decoding method embodiment.
  • Each implementation process and implementation manner of the above-mentioned method embodiment can be applied to this decoding device embodiment, and can achieve the same technical effect.
  • the embodiment of the present application also provides a decoding device.
  • the structure of the decoding device can be referred to the structure of the encoding device shown in Figure 7, which will not be described again here.
  • the decoding device in the embodiment of the present application also includes: instructions or programs stored in the memory and executable on the processor.
  • the processor calls the instructions or programs in the memory to execute the method executed by each module shown in Figure 10, and To achieve the same technical effect, to avoid repetition, we will not repeat them here.
  • Embodiments of the present application also provide a readable storage medium.
  • the readable storage medium may be non-volatile or volatile.
  • the readable storage medium stores programs or instructions, and the programs or instructions are stored on the readable storage medium.
  • the processor executes, each process of the above decoding method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, details will not be described here.
  • the processor is the processor in the decoding device described in the above embodiment.
  • the readable storage medium includes computer readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disk or optical disk, etc.
  • this embodiment of the present application also provides a communication device 1100, which includes a processor 1101 and a memory 1102.
  • the memory 1102 stores programs or instructions that can be run on the processor 1101.
  • the communication device 1100 is an encoding device, when the program or instruction is executed by the processor 1101, each step of the above encoding method embodiment is implemented, and the same technical effect can be achieved.
  • the communication device 1100 is a decoding device, when the program or instruction is executed by the processor 1101, each step of the above decoding method embodiment is implemented, and the same technical effect can be achieved. To avoid duplication, the details will not be described here.
  • An embodiment of the present application further provides a chip.
  • the chip includes a processor and a communication interface.
  • the communication interface is coupled to the processor.
  • the processor is used to run programs or instructions to implement the above encoding method or decoding method.
  • Each process in the example can achieve the same technical effect. To avoid repetition, we will not repeat it here.
  • chip mentioned in the embodiment of this application can also be called a system-level chip, system chip, System-on-a-chip or system-on-chip, etc.
  • An embodiment of the present application further provides a computer program product, the computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement each process of the above encoding method or decoding method embodiment, And can achieve the same technical effect. To avoid repetition, they will not be described again here.
  • Embodiments of the present application also provide a communication system, which at least includes: an encoding device and a decoding device.
  • the encoding device can be used to perform the steps of the encoding method as described above.
  • the decoding device can be used to perform the decoding method as described above. A step of. And can achieve the same technical effect. To avoid repetition, they will not be described again here.
  • the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation.
  • the technical solution of the present application can be embodied in the form of a computer software product that is essentially or contributes to the existing technology.
  • the computer software product is stored in a storage medium (such as ROM/RAM, disk , CD), including several instructions to cause a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of this application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了一种编码、解码方法、装置及设备，涉及编解码技术领域。该编码方法，包括：编码端根据目标三维网格的几何信息获取占位图和几何图；所述编码端对第一图像中投影点的像素进行平移处理，得到第二图像，所述第一图像包括占位图和几何图中的至少一项；所述编码端对第三图像以及所述第二图像进行编码，所述第三图像为所述占位图和所述几何图中不属于所述第一图像的图。

Description

编码、解码方法、装置及设备
相关申请的交叉引用
本申请主张在2022年03月18日在中国提交的中国专利申请No.202210273171.6的优先权，其全部内容通过引用包含于此。
技术领域
本申请属于编解码技术领域，具体涉及一种编码、解码方法、装置及设备。
背景技术
三维网格(Mesh)可以被认为是过去多年来最流行的三维模型的表示方法，其在许多应用程序中扮演着重要的角色。它的表示简便，因此被大量以硬件算法集成到电脑、平板电脑和智能手机的图形处理单元(Graphics Processing Unit，GPU)中，专门用于渲染三维网格。
由于Mesh的顶点与点云都是空间中一组无规则分布的离散点集，具有相似的特点。因此，三维网格几何信息可以用点云压缩算法进行压缩。但相比于点云，三维网格的顶点具有空间分布更加稀疏，更加不均匀的特点。使用点云压缩算法来压缩三维网格模型的几何信息，编码效率并不高。
发明内容
本申请实施例提供一种编码、解码方法、装置及设备，能够解决现有技术的对于三维网格几何信息的编码效率不高的问题。
第一方面，提供了一种编码方法，包括：
编码端根据目标三维网格的几何信息获取占位图和几何图；
所述编码端对第一图像中投影点的像素进行平移处理，得到第二图像，所述第一图像包括占位图和几何图中的至少一项；
所述编码端对第三图像以及所述第二图像进行编码，所述第三图像为所述占位图和所述几何图中不属于所述第一图像的图。
第二方面，提供了一种编码装置，包括：
第一获取模块，用于根据目标三维网格的几何信息获取占位图和几何图；
第二获取模块，用于对第一图像中投影点的像素进行平移处理，得到第二图像，所述第一图像包括占位图和几何图中的至少一项；
编码模块，用于对第三图像以及所述第二图像进行编码，所述第三图像为所述占位图和所述几何图中不属于所述第一图像的图。
第三方面，提供了一种解码方法，包括：
解码端对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像，所述第二图像包括紧密排布的占位图和紧密排布的几何图中的至少一项，所述第三图像为所述占位图和所述几何图中不属于第二图像的图；
所述解码端对所述第二图像中的投影点的像素进行平移处理，得到第一图像；
所述解码端根据所述第一图像和所述第三图像，获取所述目标三维网格的几何信息。
第四方面，提供了一种解码装置，包括：
第三获取模块，用于对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像，所述第二图像包括紧密排布的占位图和紧密排布的几何图中的至少一项，所述第三图像为所述占位图和所述几何图中不属于第二图像的图；
第四获取模块，用于对所述第二图像中的投影点的像素进行平移处理，得到第一图像；
第五获取模块，用于根据所述第一图像和所述第三图像，获取所述目标三维网格的几何信息。
第五方面，提供了一种编码设备，包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第一方面所述的方法的步骤。
第六方面，提供了一种编码设备，包括处理器及通信接口，其中，所述处理器用于根据目标三维网格的几何信息获取占位图和几何图；对第一图像中投影点的像素进行平移处理，得到第二图像，所述第一图像包括占位图和 几何图中的至少一项；对第三图像以及所述第二图像进行编码，所述第三图像为所述占位图和所述几何图中不属于所述第一图像的图。
第七方面，提供了一种解码设备，包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第三方面所述的方法的步骤。
第八方面，提供了一种解码设备，包括处理器及通信接口，其中，所述处理器用于对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像，所述第二图像包括紧密排布的占位图和紧密排布的几何图中的至少一项，所述第三图像为所述占位图和所述几何图中不属于第二图像的图；对所述第二图像中的投影点的像素进行平移处理，得到第一图像；根据所述第一图像和所述第三图像，获取所述目标三维网格的几何信息。
第九方面，提供了一种通信系统，包括：编码设备和解码设备，所述编码设备可用于执行如第一方面所述的方法的步骤，所述解码设备可用于执行如第三方面所述的方法的步骤。
第十方面，提供了一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如第一方面所述的方法的步骤，或者实现如第三方面所述的方法的步骤。
第十一方面，提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如第一方面所述的方法的步骤，或实现如第三方面所述的方法的步骤。
第十二方面，提供了一种计算机程序产品，所述计算机程序产品被存储在存储介质中，所述计算机程序产品被至少一个处理器执行以实现如第一方面所述的方法的步骤，或者实现如第三方面所述的方法的步骤。
第十三方面，提供一种通信设备，被配置为执行如第一方面所述的方法的步骤，或者，执行如第三方面所述的方法的步骤。
在本申请实施例中，通过将占位图和几何图中的至少一项进行平移处理以实现紧密排布，以此减少编码的比特数量，进而提高压缩效率。
附图说明
图1是本申请实施例的编码方法的流程示意图；
图2是基于网格的精细划分过程示意图；
图3是Patch排列的八种方向示意图；
图4是像素进行平移前及平移后的几何图的示意图；
图5是基于视频的三维网格几何信息编码框架示意图；
图6是本申请实施例的编码装置的模块示意图；
图7是本申请实施例的编码设备的结构示意图；
图8是本申请实施例的解码方法的流程示意图；
图9是基于视频的三维网格几何信息解码框架示意图；
图10是本申请实施例的解码装置的模块示意图；
图11是本申请实施例的通信设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施，且“第一”、“第二”所区别的对象通常为一类，并不限定对象的个数，例如第一对象可以是一个，也可以是多个。此外，说明书以及权利要求中“和/或”表示所连接对象的至少其中之一，字符“/”一般表示前后关联对象是一种“或”的关系。
值得指出的是，本申请实施例所描述的技术不限于长期演进型(Long Term Evolution，LTE)/LTE的演进(LTE-Advanced，LTE-A)系统，还可用于其他无线通信系统，诸如码分多址(Code Division Multiple Access，CDMA)、时分多址(Time Division Multiple Access，TDMA)、频分多址(Frequency Division Multiple Access，FDMA)、正交频分多址(Orthogonal Frequency  Division Multiple Access，OFDMA)、单载波频分多址(Single-carrier Frequency Division Multiple Access，SC-FDMA)和其他系统。本申请实施例中的术语“系统”和“网络”常被可互换地使用，所描述的技术既可用于以上提及的系统和无线电技术，也可用于其他系统和无线电技术。以下描述出于示例目的描述了新空口(New Radio，NR)系统，并且在以下大部分描述中使用NR术语，但是这些技术也可应用于NR系统应用以外的应用，如第6代(6th Generation，6G)通信系统。
下面对与本申请相关的现有技术简要说明如下。
近年来，随着多媒体技术的迅速发展，相关研究成果迅速产业化，并成为人们生活中不可或缺的重要组成部分。三维模型成为继音频、图像、视频之后的新一代数字化媒体。三维网格和点云是两种常用的三维模型表示方式。三维网格模型与传统的图像、视频等多媒体相比具有更强的交互性和逼真性的特点，使其在商业、制造业、建筑业、教育、医学、娱乐、艺术、军事等各个领域都得到了越来越广泛的应用。
而随着人们对三维网格模型在视觉效果上越来越高的需求，以及许多更加成熟的三维扫描技术和三维建模软件的涌现，通过三维扫描设备或三维建模软件获取的三维网格模型的数据规模和复杂度也在急剧地增长着。因此，如何高效压缩三维网格数据是实现三维网格数据方便地传输、存储和处理的关键。
一个三维网格往往同时包含了拓扑信息、几何信息及属性信息三种主要信息。拓扑信息用于描述网格中顶点和面片等元素之间的连接关系；几何信息是网格中所有顶点的三维坐标；属性信息则记录了附着在网格上的其它信息，比如法向量、纹理坐标以及颜色等。虽然一些传统的通用的数据压缩方法能够减少一定的三维网格数据量，但是由于三维网格数据的特殊性，直接将这些压缩方法用于压缩三维网格数据往往不能获得理想的效果。因此，对三维网格数据的压缩面临着新的挑战。在三维网格的数据中，几何数据往往比拓扑数据需要占用更多的存储空间，对几何数据进行高效压缩将对减少三维网格数据的存储空间具有极为重要的意义。因此，三维网格几何信息的压缩成为了人们的研究重点。
三维网格几何信息压缩算法可以使用点云的三维几何信息压缩算法。近年来点云压缩的国际标准主要有两种，分别是基于视频的点云压缩(Video-based Point Cloud Compression，V-PCC)和基于几何的点云压缩(Geometry-based Point Cloud Compression，G-PCC)。
V-PCC的主要思路是将点云的几何和属性信息投影成二维视频，利用已有的视频编码技术对二维视频进行压缩，从而达到压缩点云的目的。V-PCC的几何编码是通过将几何信息投影成占位视频和几何视频，利用视频编码器分别编码这两路视频来实现的。
V-PCC几何信息编码的过程主要包括：首先生成三维片(3D patch)，片(patch)是指点云中投影平面相同且连通的顶点集合。目前生成3D patch的方法是，利用近邻点估计每个顶点的法向量，根据每个点的法向量和预设平面的法向量，计算每个顶点的投影平面，将连通的具有相同投影平面顶点组成一个patch。然后，将3D patch投影到二维(2 Dimension，2D)平面上形成2D patch，并将2D patch排列在一张二维图像上，这个过程称为片打包(patch packing)。为了使patch排列的更加紧密从而提高压缩性能，目前的排列方法有：优先级排列、时域一致排列、全局patch分配等。然后，生成占位图和几何图。占位图是表示二维图像中顶点占位信息的图像，占位图中有顶点投影的位置值为1，其余位置值为0。按照一定规则将patch排列在二维图像中，就生成了占位图。几何图中存储的是每个顶点到投影平面的距离。利用顶点的三维坐标、顶点的投影平面以及占位图可以直接计算出每个顶点的深度信息，从而生成几何图。对于投影位置重复的顶点，将除第一个投影顶点外的顶点几何坐标排列进原始的(raw)patch里放入几何图或者单独进行编码。为了提高压缩效率，对几何图像执行图像填充过程。图像填充方法“推拉”背景填充算法、基于稀疏线性模型(Sparse Linear Model)的填充方法、谐波背景填充(Harmonic Background Filling)等方法。图像填充后，得到最终的几何图，使用现有的视频编码器压缩占位图和几何图，得到视频码流。最后，将占位视频码流、几何视频码流以及包含patch信息的子码流合成为最终的总码流。
下面结合附图，通过一些实施例及其应用场景对本申请实施例提供的编 码、解码方法、装置及设备进行详细地说明。
如图1所示，本申请实施例提供一种编码方法，包括：
步骤101，编码端根据目标三维网格的几何信息获取占位图和几何图；
需要说明的是，本申请中所说的目标三维网格可以理解为任意视频帧对应的三维网格，该目标三维网格的几何信息可以理解为是三维网格中顶点的坐标，该坐标通常指的是三维坐标。
步骤102，所述编码端对第一图像中投影点的像素进行平移处理，得到第二图像；
需要说明的是，所述第一图像包括占位图和几何图中的至少一项。
需要说明的是，投影点指的是将三维网格的顶点进行二维片投影后，顶点在二维图像中对应的点。二维图像中的一个像素对应一个顶点的投影点。在占位图中投影点所对应的像素值通常为1，在几何图中投影点所对应的深度值通常大于0。
步骤103，所述编码端对第三图像以及所述第二图像进行编码；
需要说明的是，所述第三图像为所述占位图和所述几何图中不属于所述第一图像的图。例如，当第一图像包括占位图时，第三图像包括几何图；当第一图像包括几何图时，第三图像包括占位图；当第一图像包括占位图和几何图时，第三图像为空集，可以理解为，在此种情况下，不存在第三图像。
需要说明的是，对于编码端而言，由于视频编码器对投影点的像素稀疏分布的图压缩率极低，本申请对占用图和几何图中的至少一种图的投影点的像素进行平移，以实现像素的紧密排布，例如保持占位图不变，将几何图中投影点的像素进行水平或者垂直方向的平移，以提高后续视频编码器编码几何图时的编码效率。
可选地，上述步骤101的具体实现方式为：
步骤1011，所述编码端对所述目标三维网格的几何信息进行三维片划分；
需要说明的是，此种情况下，主要是将目标三维网格的几何信息进行片(Patch)划分，得到多个三维片；此步骤的具体实现方式为：编码端确定目标三维网格的几何信息中包含的每个顶点的投影平面；编码端根据所述投影平面对所述目标三维网格的几何信息中所包含的顶点进行片划分；编码端对 所述目标三维网格的几何信息中所包含的顶点进行聚类，得到划分后的每一片。也就是说，对于Patch划分的过程主要包括：首先估计每个顶点的法向量，选择平面法向量与顶点法向量之间的夹角最小的候选投影平面作为该顶点的投影平面；然后，根据投影平面对顶点进行初始划分，将投影平面相同且连通的顶点组成patch；最后，使用精细划分算法优化聚类结果，得到最终的三维片(3D patch)。
下面对由目标三维网格的几何信息得到三维片的过程的具体实现进行详细说明如下。
首先估计每个点的法向量。切线平面和它对应的法线是根据每个点的最近的邻居顶点在一个预定义的搜索距离定义的。K-D树用于分离数据，并在点pi附近找到相邻点，该集合的重心用于定义法线。重心c的计算方法如下：
公式一：
使用特征分解法估计顶点法向量，计算过程公式二所示：
公式二：
在初始划分阶段，初步选择每个顶点的投影平面。设顶点法向量的估计值为候选投影平面的法向量为选择法向量方向与顶点法向量方向最接近的平面作为该顶点的投影平面，平面选择的计算过程如公式三所示：
公式三：
精细划分过程可以采用基于网格的算法来降低算法的时间复杂度，基于网格的精细划分算法流程如图2所示，具体包括：
先设置循环次数(numlter)为0，判断循环次数是否小于最大循环次数(需要说明的是，该最大循环次数可以根据使用需求设置)，若小于则执行下述过程：
步骤S201，将(x,y,z)几何坐标空间划分为体素。
需要说明的是，此处的几何坐标空间指的是由目标三维网格的几何信息所构成的几何坐标空间。例如，对于使用体素大小为8的10位Mesh，每个坐标上的体素数量将是1024/8＝128，此坐标空间中的体素总数将是128×128×128。
步骤S202，查找填充体素，填充体素是指网格中包含至少有一个点的体 素。
步骤S203，计算每个填充体素在每个投影平面上的平滑分数，记为voxScoreSmooth，体素在某投影平面的体素平滑分数是通过初始分割过程聚集到该投影平面的点的数量。
步骤S204，使用KD-Tree分区查找近邻填充体素，记为nnFilledVoxels，即每个填充体素(在搜索半径内和/或限制到最大数量的相邻体素)的最近的填充体素。
步骤S205，使用近邻填充体素在每个投影平面的体素平滑分数，计算每个填充体素的平滑分数(scoreSmooth)，计算过程如公式四所示：
公式四：
其中，p是投影平面的索引，v是近邻填充体素的索引。一个体素中所有点的scoreSmooth是相同的。
步骤S206，使用顶点的法向量与候选投影平面的法向量计算法向分数，记为scoreNormal，计算过程如公式五所示：
公式五：scoreNormal[i][p]＝normal[i]·orientation[p]；
其中，p是投影平面的索引，i是顶点的索引。
步骤S207，使用scoreSmooth和scoreNormal计算每个体素在各个投影平面上的最终分数，计算过程如公式六所示：
公式六：
其中，i为顶点索引，p为投影平面的索引，v是顶点i所在的体素索引。
步骤S208，使用步骤207中的分数对顶点进行聚类，得到精细划分的patch。
多次迭代上述过程，直到得到较为准确的patch。
步骤1012，所述编码端将划分的三维片进行二维投影，获取二维片；
需要说的是，此过程是将3D patch投影到二维平面得到二维片(2D patch)。
需要说明的是，Patch划分通过在给定投影平面上使用提供最佳压缩性能的策略将3D样本转换为2D样本。Patch划分的目标是将一帧三维模型的顶点分解成数量最小且边界光滑的Patch，同时最小化重构误差。
步骤1013，所述编码端将所述二维片进行打包，获取二维图像信息；
需要说明的是，此步骤实现的是片打包(Patch packing)，Patch packing的目的是将2D patch排列在一张二维图像上，Patch packing的基本原则是将patch不重叠的排列在二维图像上或者将patch的无像素部分进行部分重叠的排列在二维图像上，通过优先级排列、时域一致排列等算法，使patch排列的更加紧密，且具有时域一致性，提高编码性能。
假设，二维图像的分辨率为WxH，定义patch排列的最小块大小为T，它指定了放置在这个2D网格上的不同补丁之间的最小距离。
首先，patch按照不重叠的原则插入放置在2D网格上。每个patch占用由整数个TxT块组成的区域。此外，相邻patch之间要求至少有一个TxT块的距离。当没有足够的空间放置下一个patch时，图像的高度将变成原来的2倍，然后继续放置patch。
为了使patch排列的更加紧密，patch可以选择多种不同的排列方向。例如，可以采用八种不同的排列方向，如图3所示，包括0度、180度、90度、270度以及前四种方向的镜像。
为了获得更好的适应视频编码器帧间预测的特性，采用一种具有时域一致性的Patch排列方法。在一个一组帧(Group of frame，GOF)中，第一帧的所有patch按照从大到小的顺序依次排列。对于GOF中的其他帧，使用时域一致性算法调整patch的排列顺序。
这里还需要说明的是，在得到二维图像信息后便能根据获取二维图像信息过程中的信息得到patch信息，之后便可以进行片信息的编码，获取片信息子码流；
这里需要说明的是，在进行二维图像信息过程中需要记录patch划分的信息、patch投影平面的信息以及patch packing位置的信息，所以patch信息记录的是获取二维图像过程中各步骤操作的信息，即patch信息包括：patch划分的信息、patch投影平面的信息以及patch packing位置的信息。
步骤1014，所述编码端根据所述二维图像信息，获取占位图和几何图；
需要说的是，对于获取占位图的过程，主要为：利用patch packing得到的patch排列信息，将二维图像中存在顶点的位置设为1，其余位置设为0， 得到占位图。对于获取几何图的过程，主要为：在通过投影得到2D patch的过程中，保存了每个顶点到投影平面的距离，这个距离称为深度，将2D patch中每个顶点的深度值，排列到该顶点在占位图中的位置上，得到几何图。
可选地，在所述第一图像为占位图或几何图，上述的步骤102的具体实现方式为：
所述编码端对所述第一图像中的投影点的像素沿第一方向进行逐行或逐列扫描，对每行或每列中的像素的位置索引重新进行行内或列内的编号；所述编码端按照扫描顺序，将每一顶点的像素值排列在所述顶点对应的投影点所对应的位置索引指定的像素位置，获取平移后的紧密排布的第二图像，所述第一方向为水平方向或垂直方向。
需要说明的是，通常情况下，占位图或几何图中的投影点所对应的像素之间并不是紧密相邻的，而是有一定的间隔，本申请实施例中所说的扫描重新编号即将同一行或同一列中存在间距的像素以像素相邻的方式重新进行编号，例如，在某一行中，对应存在5个像素，由左向右其编号分别为1、3、5、7、8，在经过由左向右的扫描后，重新对这5个像素进行编号，则重新编号后，这5个像素由左向右的新编号为1、2、3、4、5。
在对像素进行扫描重新编号之后，将每个像素位置的像素值用与该像素对应顶点的像素值进行表示，以此得到紧密排布的几何图或占位图，需要说明的是，当第一图像为占位图时，顶点的像素值为该顶点在占位信息中对应的取值，通常此时，该顶点的像素值为1；当第一图像为几何图时，顶点的像素值指的是该顶点的深度值。
需要说明的是，此种实现方式是先将几何图(或占位图)中的顶点对应的投影点的像素进行扫描，重新进行编号，当进行的是水平方向的扫描时，是按照水平方向逐行进行扫描，对每行中的像素的索引位置重新进行行内的编号，当进行的是垂直方向的扫描时，是按照垂直方向逐列进行扫描，对每列中的像素的索引位置重新进行列内的编号；通过将几何图(或占位图)中的像素按照水平或者垂直方向进行平移，并将顶点的像素值排列到该顶点对应的投影点的像素位置，得到像素排列紧密的几何图(或占位图)，再利用视频编码器对几何图(或占位图)进行压缩，得到几何图子码流(或占位图子 码流)；例如，图4为像素进行平移前及平移后的几何图的示意图，其中，图4中像素点的深度值被二值化来提高示意图的可视性，实际中排布紧密的像素具有不同的深度值。
可选地，在所述第一图像包括几何图和占位图的情况下，上述的步骤102的具体实现方式为：
所述编码端将占位图和几何图中的投影点的像素向第二方向平移，将像素平移后的占位图中投影点所对应的像素值用对投影点的像素进行平移的距离进行表示，分别获取紧密排布的占位图和几何图；
其中，所述第二方向为水平方向或垂直方向，所述平移的距离为投影点的像素平移的相对距离或投影点的像素平移的绝对距离。
需要说明的是，此种实现方式为：将占位图和几何图以相同的方式(所谓相同的方式指的是占位图中的投影点的像素的移动方向和距离与几何图中的对应的投影点的像素的移动方向和距离相同)向某一方向聚集，将占位图中投影点对应的像素值记录为对投影点的像素进行平移的绝对距离(或投影点的像素进行平移的相对距离(例如，以当前投影点的像素相对于前一投影点的像素移动距离的差值作为相对距离))，也就是说，此时占位图中像素值不再是1，而是记录投影点的像素平移距离的值。
例如，以对几何图进行紧密排布为例，本申请实施例的基于视频的三维网格几何信息编码框架如图5所示，总体编码流程为：
首先，对于几何信息，采用投影的方式进行patch划分、patch投影、patch排列来生成patch序列压缩信息(包括patch的划分信息、投影信息、排列信息)、占位图和几何图；对像素点分布稀疏的几何图沿水平或者垂直方向对几何图中投影点的像素进行平移，使像素紧密排布，构成像素排布紧密的几何图；最后，编码patch序列压缩信息、占位图、几何图分别得到对应的子码流，并将多路子码流混流，得到最终输出码流。
需要说明的是，本申请给出了如何进行三维网格的几何信息进行编码的实现方式，通过将三维网格的几何信息对应的几何图或占位图进行紧密排布处理，减少顶点间的间距，以此能提高三维网格几何信息的压缩效率和编码效率。
本申请实施例提供的编码方法，执行主体可以为编码装置。本申请实施例中以编码装置执行编码方法为例，说明本申请实施例提供的编码装置。
如图6所示，本申请实施例提供一种编码装置600，包括：
第一获取模块601，用于根据目标三维网格的几何信息获取占位图和几何图；
第二获取模块602，用于对第一图像中投影点的像素进行平移处理，得到第二图像，所述第一图像包括占位图和几何图中的至少一项；
编码模块603，用于对第三图像以及所述第二图像进行编码，所述第三图像为所述占位图和所述几何图中不属于所述第一图像的图。
可选地，所述第一图像为占位图或几何图，所述第二获取模块602，包括：
第一扫描单元，用于对所述第一图像中的投影点的像素沿第一方向进行逐行或逐列扫描，对每行或每列中的像素的位置索引重新进行行内或列内的编号；
第一获取单元，用于按照扫描顺序，将每一顶点的像素深度值排列在所述顶点对应的投影点所对应的位置索引指定的像素位置，获取平移后的紧密排布的第二图像；
其中，所述第一方向为水平方向或垂直方向。
可选地，在所述第一图像包括几何图和占位图的情况下，所述第二获取模块602，包括：
第二获取单元，用于将占位图和几何图中的投影点的像素向第二方向平移，将像素平移后的占位图中投影点所对应的像素值用对投影点的像素进行平移的距离进行表示，分别获取紧密排布的占位图和几何图；
其中，所述第二方向为水平方向或垂直方向，所述平移的距离为投影点的像素平移的相对距离或投影点的像素平移的绝对距离
可选地，所述第一获取模块601，包括：
划分单元，用于对所述目标三维网格的几何信息进行三维片划分；
第三获取单元，用于将划分的三维片进行二维投影，获取二维片；
第四获取单元，用于将所述二维片进行打包，获取二维图像信息；
第五获取单元，用于根据所述二维图像信息，获取占位图和几何图。
可选地，在所述第四获取单元将所述二维片进行打包，获取二维图像信息之后，所述第一获取模块601，还包括：
第六获取单元，用于根据获取二维图像信息过程中的信息，获取片信息；
第七获取单元，用于对所述片信息进行编码，获取片信息子码流。
该装置实施例与上述编码方法实施例对应，上述方法实施例的各个实施过程和实现方式均可适用于该装置实施例中，且能达到相同的技术效果。
本申请实施例还提供了一种编码设备，包括处理器及通信接口，其中，所述处理器用于根据目标三维网格的几何信息获取占位图和几何图；对第一图像中的像素进行紧密排布处理，得到第二图像，所述第一图像包括占位图和几何图中的至少一项；对第三图像以及所述第二图像进行编码，所述第三图像为所述占位图和所述几何图中不属于所述第一图像的图。
具体地，本申请实施例还提供了一种编码设备，如图7所示，该编码设备700包括：处理器701、网络接口702和存储器703。其中，网络接口702例如为通用公共无线接口(common public radio interface，CPRI)。
具体地，本申请实施例的编码设备700还包括：存储在存储器703上并可在处理器701上运行的指令或程序，处理器701调用存储器703中的指令或程序执行图6所示各模块执行的方法，并达到相同的技术效果，为避免重复，故不在此赘述。
如图8所示，本申请实施例还提供一种解码方法，包括：
步骤801，解码端对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像；
需要说明的是，所述第二图像包括紧密排布的占位图和紧密排布的几何图中的至少一项，所述第三图像为所述占位图和所述几何图中不属于第二图像的图；
步骤802，所述解码端对所述第二图像中的投影点的像素进行平移处理，得到第一图像；
步骤803，所述解码端根据所述第一图像和所述第三图像，获取所述目标三维网格的几何信息。
需要说明的是，对于解码端而言，使用视频解码器解码得到至少有一个紧密排布的占位图和几何图后，恢复按照投影位置排布的占位图和几何图，例如，使用占位图中像素的二维位置信息平移几何图中的像素还原几何图；最后使用片信息、几何图和占位图还原三维网格的三维几何信息。
可选地，在所述第二图像包括紧密排布的占位图或紧密排布的几何图的情况下，所述对所述第二图像中的投影点的像素进行平移处理，得到第一图像，包括：
所述解码端对所述第三图像的投影点的像素沿第一方向进行逐行或逐列扫描，获取每行或每列中的每个像素所在的行内或列内的位置索引；
所述解码端对所述第二图像中的投影点的像素沿第一方向进行逐行或逐列扫描，对每行或每列中的每个像素按照与所述第三图像中的投影点的像素对应的位置索引重新进行行内或列内的编号；
所述解码端按照扫描顺序，将每一顶点的像素值排列在所述顶点对应的投影点在所述第三图像中所对应的位置索引指定的像素位置，获取平移后的稀疏排布的第二图像；
其中，所述第一方向为水平方向或垂直方向。
需要说明的是，因占位图和几何图中的一者有一个在编码过程中没有进行像素平移，因此在解码时，需要按照未进行像素平移的这个图恢复另一个像素平移的图中的顶点对应的投影点的像素的位置，以此保证图像的准确恢复。
可选地，在所述第二图像包括紧密排布的几何图和紧密排布的占位图的情况下，所述对所述第二图像中的投影点的像素进行平移处理，得到第一图像，包括：
所述解码端按照紧密排布的占位图中投影点的像素值，将紧密排布的占位图和紧密排布的几何图中的投影点的像素沿第二方向平移，分别获取稀疏排布的占位图和几何图；
其中，所述紧密排布的占位图中投影点的像素值用对投影点的像素进行平移的距离表示，所述第二方向为水平方向或垂直方向，所述平移的距离为投影点的像素平移的相对距离或投影点的像素平移的绝对距离。
需要说明的是，因为紧密排布的占位图中的像素值用对投影点的像素进行平移的距离表示的，因此在进行图像恢复时，按照投影点的像素进行平移的距离，将紧密排布的占位图和紧密排布的几何图按照编码过程中平移的距离的反方向移动，以此得到原始的占位图和几何图，实现对三维网格的几何信息的解码。
可选地，所述对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像，包括：
所述解码端根据获取的目标三维网格的码流，获取目标子码流，所述目标子码流包括：片信息子码流、占位图子码流和几何图子码流；
所述解码端根据所述目标子码流，获取第二图像以及第三图像。
需要说明的是，几何信息重建过程是利用patch信息、占位图、几何图来重建三维几何模型的过程。具体过程主要分为：
步骤S1、获取2D patch；
对于解码后的像素经过紧密排布的占位图和/或几何图按照编码器中像素平移规则的逆过程进行恢复，得到像素点位置为二维投影位置的占位图和几何图。获取2D patch是指利用patch信息从占位图和几何图中分割出2D patch的占位信息和深度信息。Patch信息中包含了每个2D patch的包围盒在占位图和几何图中的位置和大小，利用patch信息、占位图和几何图可以直接获取到2D patch的占位信息和几何信息。
步骤S2、重建3D patch；
重建3D patch是指利用2D patch中的占位信息和几何信息，将2D patch中的顶点重建为3D patch。2D patch的占位信息中包含了顶点在patch投影平面局部坐标系中相对于坐标原点的位置，深度信息包含了顶点在投影平面法线方向上的深度值。因此，利用占位信息和深度信息可以在局部坐标系中将2D patch重建为3D patch。
步骤S3、重建网格几何信息；
将重建的3D patch按照patch信息流中3D patch的划分信息恢复回完整的网格几何坐标信息。
例如，以对紧密排布的几何图进行恢复为例，本申请实施例的基于视频 的三维网格几何信息解码框架如图9所示，总体解码流程为：
首先，将码流分解成patch信息子码流、占位图子码流、几何图子码流，并分别进行解码；使用解码得到的占位图中投影像素的位置信息来还原解码得到几何图中像素的原始位置；使用占位图、复原的几何图和patch信息重建网格的三位几何信息；最终，使用重建的几何信息以及其他编解码方式得到的连接关系等信息重建网格。
需要说明的是，本申请实施例是与上述编码方法的实施例对应的对端的方法实施例，解码过程为编码的反过程，上述编码侧的所有实现方式均适用于该解码端的实施例中，也能达到与之相同的技术效果，在此不再赘述。
如图10所示，本申请实施例还提供一种解码装置1000，包括：
第三获取模块1001，用于对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像，所述第二图像包括紧密排布的占位图和紧密排布的几何图中的至少一项，所述第三图像为所述占位图和所述几何图中不属于第二图像的图；
第四获取模块1002，用于对所述第二图像中的投影点的像素进行平移处理，得到第一图像；
第五获取模块1003，用于根据所述第一图像和所述第三图像，获取所述目标三维网格的几何信息。
可选地，在所述第二图像包括紧密排布的占位图或紧密排布的几何图的情况下，所述第四获取模块1002，包括：
第八获取单元，用于对所述第三图像的投影点的像素沿第一方向进行逐行或逐列扫描，获取每行或每列中的每个像素所在的行内或列内的位置索引；
第二扫描单元，用于对所述第二图像中的投影点的像素沿第一方向进行逐行或逐列扫描，对每行或每列中的每个像素按照与所述第三图像中的投影点的像素对应的位置索引重新进行行内或列内的编号；
第九获取单元，用于按照扫描顺序，将每一顶点的像素值排列在所述顶点对应的投影点在所述第三图像中所对应的位置索引指定的像素位置，获取平移后的稀疏排布的第二图像；
其中，所述第一方向为水平方向或垂直方向。
可选地，在所述第二图像包括紧密排布的几何图和紧密排布的占位图的情况下，所述第四获取模块1002，包括：
第十获取单元，用于按照紧密排布的占位图中投影点的像素值，将紧密排布的占位图和紧密排布的几何图中的投影点的像素沿第二方向平移，分别获取稀疏排布的占位图和几何图；
其中，所述紧密排布的占位图中投影点的像素值用对投影点的像素进行平移的距离表示，所述第二方向为水平方向或垂直方向，所述平移的距离为投影点的像素平移的相对距离或投影点的像素平移的绝对距离。
可选地，所述第三获取模块1001，包括：
第十一获取单元，用于根据获取的目标三维网格的码流，获取目标子码流，所述目标子码流包括：片信息子码流、占位图子码流和几何图子码流；
第十二获取单元，用于根据所述目标子码流，获取第二图像以及第三图像。
需要说明的是，该装置实施例是与上述方法对应的装置，上述方法实施例中的所有实现方式均适用于该装置实施例中，也能达到相同的技术效果，在此不再赘述。
本申请实施例还提供一种解码设备，包括处理器，存储器，存储在存储器上并可在所述处理器上运行的程序或指令，该程序或指令被处理器执行时实现上述的解码方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例还提供一种可读存储介质，计算机可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述的解码方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述的计算机可读存储介质，如只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等。
本申请实施例还提供了一种解码设备，包括处理器及通信接口，其中，所述处理器用于对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像，所述第二图像包括紧密排布的占位图和紧密排布的几何图中的至少一项，所述第三图像为所述占位图和所述几何图中不属于第二图像的图； 对所述第二图像中的投影点的像素进行平移处理，得到第一图像；根据所述第一图像和所述第三图像，获取所述目标三维网格的几何信息。
该解码设备实施例是与上述解码方法实施例对应的，上述方法实施例的各个实施过程和实现方式均可适用于该解码设备实施例中，且能达到相同的技术效果。
具体地，本申请实施例还提供了一种解码设备。具体地，该解码设备的结构可参见如图7所示的编码设备的结构，在此不再赘述。具体地，本申请实施例的解码设备还包括：存储在存储器上并可在处理器上运行的指令或程序，处理器调用存储器中的指令或程序执行图10所示各模块执行的方法，并达到相同的技术效果，为避免重复，故不在此赘述。
本申请实施例还提供一种可读存储介质，所述可读存储介质可以是非易失的，也可以是易失的，所述可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述解码方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述处理器为上述实施例中所述的解码设备中的处理器。所述可读存储介质，包括计算机可读存储介质，如计算机只读存储器ROM、随机存取存储器RAM、磁碟或者光盘等。
如图11所示，本申请实施例还提供一种通信设备1100，包括处理器1101和存储器1102，存储器1102上存储有可在所述处理器1101上运行的程序或指令，例如，该通信设备1100为编码设备时，该程序或指令被处理器1101执行时实现上述编码方法实施例的各个步骤，且能达到相同的技术效果。该通信设备1100为解码设备时，该程序或指令被处理器1101执行时实现上述解码方法实施例的各个步骤，且能达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例另提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现上述编码方法或解码方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
应理解，本申请实施例提到的芯片还可以称为系统级芯片，系统芯片， 芯片系统或片上系统芯片等。
本申请实施例另提供了一种计算机程序产品，所述计算机程序产品被存储在存储介质中，所述计算机程序产品被至少一个处理器执行以实现上述编码方法或解码方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例还提供了一种通信系统，至少包括：编码设备和解码设备，所述编码设备可用于执行如上所述的编码方法的步骤，所述解码设备可用于执行如上所述的解码方法的步骤。且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。此外，需要指出的是，本申请实施方式中的方法和装置的范围不限按示出或讨论的顺序来执行功能，还可包括根据所涉及的功能按基本同时的方式或按相反的顺序来执行功能，例如，可以按不同于所描述的次序来执行所描述的方法，并且还可以添加、省去、或组合各种步骤。另外，参照某些示例所描述的特征可在其他示例中被组合。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以计算机软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的， 本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (22)

  1. 一种编码方法，包括：
    编码端根据目标三维网格的几何信息获取占位图和几何图；
    所述编码端对第一图像中投影点的像素进行平移处理，得到第二图像，所述第一图像包括占位图和几何图中的至少一项；
    所述编码端对第三图像以及所述第二图像进行编码，所述第三图像为所述占位图和所述几何图中不属于所述第一图像的图。
  2. 根据权利要求1所述的方法，其中，所述第一图像为占位图或几何图，所述对第一图像中投影点的像素进行平移处理，得到第二图像，包括：
    所述编码端对所述第一图像中的投影点的像素沿第一方向进行逐行或逐列扫描，对每行或每列中的像素的位置索引重新进行行内或列内的编号；
    所述编码端按照扫描顺序，将每一顶点的像素值排列在所述顶点对应的投影点所对应的位置索引指定的像素位置，获取平移后的紧密排布的第二图像；
    其中，所述第一方向为水平方向或垂直方向。
  3. 根据权利要求1所述的方法，其中，在所述第一图像包括几何图和占位图的情况下，所述对第一图像中投影点的像素进行平移处理，得到第二图像，包括：
    所述编码端将占位图和几何图中的投影点的像素向第二方向平移，将像素平移后的占位图中投影点所对应的像素值用对投影点的像素进行平移的距离进行表示，分别获取紧密排布的占位图和几何图；
    其中，所述第二方向为水平方向或垂直方向，所述平移的距离为投影点的像素平移的相对距离或投影点的像素平移的绝对距离。
  4. 根据权利要求1所述的方法，其中，所述根据目标三维网格的几何信息获取占位图和几何图，包括：
    所述编码端对所述目标三维网格的几何信息进行三维片划分；
    所述编码端将划分的三维片进行二维投影，获取二维片；
    所述编码端将所述二维片进行打包，获取二维图像信息；
    所述编码端根据所述二维图像信息，获取占位图和几何图。
  5. 一种解码方法，包括：
    解码端对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像，所述第二图像包括紧密排布的占位图和紧密排布的几何图中的至少一项，所述第三图像为所述占位图和所述几何图中不属于第二图像的图；
    所述解码端对所述第二图像中的投影点的像素进行平移处理，得到第一图像；
    所述解码端根据所述第一图像和所述第三图像，获取所述目标三维网格的几何信息。
  6. 根据权利要求5所述的方法，其中，在所述第二图像包括紧密排布的占位图或紧密排布的几何图的情况下，所述对所述第二图像中的投影点的像素进行平移处理，得到第一图像，包括：
    所述解码端对所述第三图像的投影点的像素沿第一方向进行逐行或逐列扫描，获取每行或每列中的每个像素所在的行内或列内的位置索引；
    所述解码端对所述第二图像中的投影点的像素沿第一方向进行逐行或逐列扫描，对每行或每列中的每个像素按照与所述第三图像中的投影点的像素对应的位置索引重新进行行内或列内的编号；
    所述解码端按照扫描顺序，将每一顶点的像素值排列在所述顶点对应的投影点在所述第三图像中所对应的位置索引指定的像素位置，获取平移后的稀疏排布的第二图像；
    其中，所述第一方向为水平方向或垂直方向。
  7. 根据权利要求5所述的方法，其中，在所述第二图像包括紧密排布的几何图和紧密排布的占位图的情况下，所述对所述第二图像中的投影点的像素进行平移处理，得到第一图像，包括：
    所述解码端按照紧密排布的占位图中投影点的像素值，将紧密排布的占位图和紧密排布的几何图中的投影点的像素沿第二方向平移，分别获取稀疏排布的占位图和几何图；
    其中，所述紧密排布的占位图中投影点的像素值用对投影点的像素进行平移的距离表示，所述第二方向为水平方向或垂直方向，所述平移的距离为 投影点的像素平移的相对距离或投影点的像素平移的绝对距离。
  8. 根据权利要求5所述的方法，其中，所述对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像，包括：
    所述解码端根据获取的目标三维网格的码流，获取目标子码流，所述目标子码流包括：片信息子码流、占位图子码流和几何图子码流；
    所述解码端根据所述目标子码流，获取第二图像以及第三图像。
  9. 一种编码装置，包括：
    第一获取模块，用于根据目标三维网格的几何信息获取占位图和几何图；
    第二获取模块，用于对第一图像中投影点的像素进行平移处理，得到第二图像，所述第一图像包括占位图和几何图中的至少一项；
    编码模块，用于对第三图像以及所述第二图像进行编码，所述第三图像为所述占位图和所述几何图中不属于所述第一图像的图。
  10. 根据权利要求9所述的装置，其中，所述第一图像为占位图或几何图，所述第二获取模块，包括：
    第一扫描单元，用于对所述第一图像中的投影点的像素沿第一方向进行逐行或逐列扫描，对每行或每列中的像素的位置索引重新进行行内或列内的编号；
    第一获取单元，用于按照扫描顺序，将每一顶点的像素值排列在所述顶点对应的投影点所对应的位置索引指定的像素位置，获取平移后的紧密排布的第二图像；
    其中，所述第一方向为水平方向或垂直方向。
  11. 根据权利要求9所述的装置，其中，在所述第一图像包括几何图和占位图的情况下，所述第二获取模块，包括：
    第二获取单元，用于将占位图和几何图中的投影点的像素向第二方向平移，将像素平移后的占位图中投影点所对应的像素值用对投影点的像素进行平移的距离进行表示，分别获取紧密排布的占位图和几何图；
    其中，所述第二方向为水平方向或垂直方向，所述平移的距离为投影点的像素平移的相对距离或投影点的像素平移的绝对距离。
  12. 根据权利要求9所述的装置，其中，所述第一获取模块，包括：
    划分单元，用于对所述目标三维网格的几何信息进行三维片划分；
    第三获取单元，用于将划分的三维片进行二维投影，获取二维片；
    第四获取单元，用于将所述二维片进行打包，获取二维图像信息；
    第五获取单元，用于根据所述二维图像信息，获取占位图和几何图。
  13. 一种编码设备，其中，包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求1至4任一项所述的编码方法的步骤。
  14. 一种解码装置，包括：
    第三获取模块，用于对获取的目标三维网格的码流进行分解，获取第二图像以及第三图像，所述第二图像包括紧密排布的占位图和紧密排布的几何图中的至少一项，所述第三图像为所述占位图和所述几何图中不属于第二图像的图；
    第四获取模块，用于对所述第二图像中的投影点的像素进行平移处理，得到第一图像；
    第五获取模块，用于根据所述第一图像和所述第三图像，获取所述目标三维网格的几何信息。
  15. 根据权利要求14所述的装置，其中，在所述第二图像包括紧密排布的占位图或紧密排布的几何图的情况下，所述第四获取模块，包括：
    第八获取单元，用于对所述第三图像的投影点的像素沿第一方向进行逐行或逐列扫描，获取每行或每列中的每个像素所在的行内或列内的位置索引；
    第二扫描单元，用于对所述第二图像中的投影点的像素沿第一方向进行逐行或逐列扫描，对每行或每列中的每个像素按照与所述第三图像中的投影点的像素对应的位置索引重新进行行内或列内的编号；
    第九获取单元，用于按照扫描顺序，将每一顶点的像素值排列在所述顶点对应的投影点在所述第三图像中所对应的位置索引指定的像素位置，获取平移后的稀疏排布的第二图像；
    其中，所述第一方向为水平方向或垂直方向。
  16. 根据权利要求14所述的装置，其中，在所述第二图像包括紧密排布的几何图和紧密排布的占位图的情况下，所述第四获取模块，包括：
    第十获取单元，用于按照紧密排布的占位图中投影点的像素值，将紧密排布的占位图和紧密排布的几何图中的投影点的像素沿第二方向平移，分别获取稀疏排布的占位图和几何图；
    其中，所述紧密排布的占位图中投影点的像素值用对投影点的像素进行平移的距离表示，所述第二方向为水平方向或垂直方向，所述平移的距离为投影点的像素平移的相对距离或投影点的像素平移的绝对距离。
  17. 根据权利要求14所述的装置，其中，所述第三获取模块，包括：
    第十一获取单元，用于根据获取的目标三维网格的码流，获取目标子码流，所述目标子码流包括：片信息子码流、占位图子码流和几何图子码流；
    第十二获取单元，用于根据所述目标子码流，获取第二图像以及第三图像。
  18. 一种解码设备，包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求5至8任一项所述的解码方法的步骤。
  19. 一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1至4任一项所述的编码方法的步骤或如权利要求5至8任一项所述的解码方法的步骤。
  20. 一种芯片，包括处理器和通信接口，其中，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如权利要求1至4任一项所述的编码方法的步骤，或者，实现如权利要求5至8任一项所述的解码方法的步骤。
  21. 一种计算机程序产品，其中，所述计算机程序产品被存储在非瞬态的可读存储介质中，所述计算机程序产品被至少一个处理器执行以实现如权利要求1至4任一项所述的编码方法的步骤，或者，所述计算机程序产品被至少一个处理器执行以实现如权利要求5至8任一项所述的解码方法的步骤。
  22. 一种通信设备，被配置为执行如权利要求1至4任一项所述的编码方法的步骤，或者，被配置为执行如权利要求5至8任一项所述的解码方法的步骤。
PCT/CN2023/081640 2022-03-18 2023-03-15 编码、解码方法、装置及设备 Ceased WO2023174336A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210273171.6 2022-03-18
CN202210273171.6A CN116800972A (zh) 2022-03-18 2022-03-18 编码、解码方法、装置及设备

Publications (1)

Publication Number Publication Date
WO2023174336A1 true WO2023174336A1 (zh) 2023-09-21

Family

ID=88022405

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/081640 Ceased WO2023174336A1 (zh) 2022-03-18 2023-03-15 编码、解码方法、装置及设备

Country Status (2)

Country Link
CN (1) CN116800972A (zh)
WO (1) WO2023174336A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101276483A (zh) * 2008-04-14 2008-10-01 中山大学 一种实现平移敏感的Laplacian网格编辑方法
CN103329169A (zh) * 2010-09-30 2013-09-25 汤姆逊许可公司 几何图案编码方法和设备，以及几何图案解码方法和设备
US20210090301A1 (en) * 2019-09-24 2021-03-25 Apple Inc. Three-Dimensional Mesh Compression Using a Video Encoder
US20210211703A1 (en) * 2020-01-07 2021-07-08 Apple Inc. Geometry information signaling for occluded points in an occupancy map video

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101276483A (zh) * 2008-04-14 2008-10-01 中山大学 一种实现平移敏感的Laplacian网格编辑方法
CN103329169A (zh) * 2010-09-30 2013-09-25 汤姆逊许可公司 几何图案编码方法和设备，以及几何图案解码方法和设备
US20210090301A1 (en) * 2019-09-24 2021-03-25 Apple Inc. Three-Dimensional Mesh Compression Using a Video Encoder
US20210211703A1 (en) * 2020-01-07 2021-07-08 Apple Inc. Geometry information signaling for occluded points in an occupancy map video

Also Published As

Publication number Publication date
CN116800972A (zh) 2023-09-22

Similar Documents

Publication Publication Date Title
US10798389B2 (en) Method and apparatus for content-aware point cloud compression using HEVC tiles
JP7604670B2 (ja) デコーダによるメッシュ圧縮のための適応サンプリング方法および装置
US11908169B2 (en) Dense mesh compression
KR20240032738A (ko) 정보 처리 장치 및 방법
CN116843771A (zh) 编码方法、解码方法及终端
WO2023155778A1 (zh) 编码方法、装置及设备
WO2023193709A1 (zh) 编码、解码方法、装置及设备
KR20230127324A (ko) 메시 압축을 위한 패치 시간 정렬의 디코딩
CN118266223A (zh) 信息处理设备和方法
CN116843855A (zh) 编码方法及终端
JP2025530443A (ja) メッシュ圧縮のためのｖ３ｃ構文拡張
US12183045B2 (en) Mesh patch simplification
KR20240163635A (ko) 점유 맵을 사용하지 않는 v-pcc 기반 동적 텍스처드 메시 코딩
WO2023174337A1 (zh) 编码、解码方法、装置及设备
WO2023174334A1 (zh) 编码、解码方法、装置及设备
KR20200007733A (ko) Pcc 데이터의 효과적인 압축을 위한 부호화/복호화 방법 및 장치
CN117242480A (zh) 使用几何信息和语义信息的曼哈顿布局估计
WO2023179705A1 (zh) 编码、解码方法、装置及设备
US20240404119A1 (en) Coding Method, Decoding Method, and Device
WO2023174336A1 (zh) 编码、解码方法、装置及设备
KR20260011162A (ko) 부호화 방법, 복호화 방법, 장치 및 기기
JP2025532407A (ja) オルソアトラス（ｏｒｔｈｏａｔｌａｓ）：正投影を使用した動的メッシュのテクスチャマップ生成
WO2023197990A1 (zh) 编码方法、解码方法及终端
KR20240152867A (ko) 동적 메시 코딩을 위한 패치 생성
US20240357147A1 (en) DISPLACEMENT PACKING USING SINGLE LoD PER BLOCK

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23769838

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23769838

Country of ref document: EP

Kind code of ref document: A1