WO2022166966A1 - 基于二维规则化平面投影的点云编解码方法及装置 - Google Patents

基于二维规则化平面投影的点云编解码方法及装置 Download PDF

Info

Publication number
WO2022166966A1
WO2022166966A1 PCT/CN2022/075407 CN2022075407W WO2022166966A1 WO 2022166966 A1 WO2022166966 A1 WO 2022166966A1 CN 2022075407 W CN2022075407 W CN 2022075407W WO 2022166966 A1 WO2022166966 A1 WO 2022166966A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional
information
point cloud
map
projection
Prior art date
Application number
PCT/CN2022/075407
Other languages
English (en)
French (fr)
Inventor
杨付正
张伟
杜雨欣
孙泽星
于有光
陈天
张可
Original Assignee
荣耀终端有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 荣耀终端有限公司 filed Critical 荣耀终端有限公司
Priority to US18/268,466 priority Critical patent/US20240080497A1/en
Priority to EP22749245.1A priority patent/EP4246974A4/en
Publication of WO2022166966A1 publication Critical patent/WO2022166966A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43—Hardware specially adapted for motion estimation or compensation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/004—Predictors, e.g. intraframe, interframe coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor

Definitions

  • the invention belongs to the technical field of encoding and decoding, and in particular relates to a point cloud encoding and decoding method and device based on two-dimensional regularized plane projection.
  • 3D point cloud With the improvement of hardware processing capability and the rapid development of computer vision, 3D point cloud has become a new generation of immersive multimedia after audio, image and video, and is widely used in virtual reality, augmented reality, autonomous driving and environment modeling, etc. .
  • 3D point cloud usually has a large amount of data, which is not conducive to the transmission and storage of point cloud data. Therefore, it is of great significance to study efficient point cloud encoding and decoding technology.
  • G-PCC Geometry-based Point Cloud Compression
  • the geometric information and attribute information of the point cloud are encoded separately.
  • the geometric encoding and decoding of G-PCC can be divided into geometric encoding and decoding based on octree and geometric encoding and decoding based on prediction tree.
  • Octree-based geometric encoding and decoding At the encoding end, first, the geometric information of the point cloud is preprocessed, which includes the coordinate transformation and voxelization process of the point cloud. Then, according to the order of breadth-first traversal, the bounding box where the point cloud is located is continuously divided into trees (octree/quadtree/binary tree). Finally, encode the placeholder code of each node, and encode the number of points contained in each leaf node to generate a binary code stream. At the decoding end, the placeholder code of each node is continuously parsed according to the order of breadth-first traversal. Then, the tree division is continuously performed in turn, until the division is stopped when a unit cube of 1x1x1 is obtained. Finally, the number of points contained in each leaf node is obtained through analysis, and finally the reconstructed point cloud geometric information is obtained.
  • the placeholder code of each node is continuously parsed according to the order of breadth-first traversal. Then, the tree division is continuously performed in
  • Prediction tree-based geometric encoding and decoding On the encoding side, the original point cloud is first sorted. Then, a prediction tree structure is established, by classifying each point to the corresponding laser scanner, and establishing the prediction tree structure according to different laser scanners. Next, traverse each node in the prediction tree, predict the geometric information of the node by selecting different prediction modes, and obtain the prediction residual, and use the quantization parameter to quantize the prediction residual. Finally, the prediction tree structure, quantization parameters and prediction residuals of node geometry information are encoded to generate a binary code stream.
  • the code stream is first parsed, then the prediction tree structure is reconstructed, and then the prediction residual and quantization parameters are obtained by analyzing the geometric information of each node, the prediction residual is inversely quantized, and finally the weight of each node is recovered.
  • the geometric information of the point cloud is reconstructed.
  • the present invention provides a point cloud encoding and decoding method and device based on two-dimensional regularized plane projection.
  • the technical problem to be solved by the present invention is realized by the following technical solutions:
  • a point cloud encoding method based on two-dimensional regularized plane projection comprising:
  • the several two-dimensional image information is encoded to obtain code stream information.
  • the pieces of two-dimensional map information include a placeholder information map.
  • code stream information including:
  • the occupancy information map is encoded to obtain the occupancy information code stream.
  • the occupancy information graph is encoded to obtain a occupancy information code stream, including:
  • the occupancy information of the pixels in the non-empty rows and non-empty columns in the occupancy information map is encoded to obtain the occupancy information code stream.
  • encoding the occupancy information of the pixels in the non-empty rows and non-empty columns in the occupancy information map includes:
  • the occupancy information of pixels in non-empty rows and non-empty columns is predicted using the reconstructed occupancy information of the encoded pixels and correspondingly encoded.
  • the plurality of two-dimensional map information further includes at least one of a depth information map, a projection residual information map, and a coordinate conversion error information map.
  • Another embodiment of the present invention also provides a point cloud encoding device based on two-dimensional regularized plane projection, including:
  • a first data acquisition module used for acquiring original point cloud data
  • a projection module for performing a two-dimensional regularized plane projection on the original point cloud data to obtain a two-dimensional projected plane structure
  • a data processing module configured to obtain several two-dimensional map information according to the two-dimensional projection plane structure
  • the encoding module is used for encoding the several two-dimensional image information to obtain the code stream information.
  • Another embodiment of the present invention also provides a point cloud decoding method based on two-dimensional regularized plane projection, including:
  • the point cloud is reconstructed using the two-dimensional projected plane structure.
  • reconstructing several two-dimensional map information according to the analytical data including:
  • Pixels located in empty rows or columns in the occupancy information map are directly reconstructed according to the identifiers of empty rows and columns in the occupancy information map in the parsed data, and residuals are predicted according to the occupancy information in the parsed data.
  • the difference reconstructs the pixels located in the non-empty rows and non-empty columns in the occupancy information map to obtain the reconstructed occupancy information map.
  • Yet another embodiment of the present invention also provides a point cloud decoding device based on two-dimensional regularized plane projection, including:
  • the second data acquisition module is used to acquire and decode the code stream information to obtain analytical data
  • a first reconstruction module configured to reconstruct several two-dimensional map information according to the analysis data
  • a second reconstruction module configured to obtain a two-dimensional projection plane structure according to the several two-dimensional map information
  • the point cloud reconstruction module is used for reconstructing the point cloud by using the two-dimensional projection plane structure.
  • the present invention performs regular correction on the vertical and horizontal directions of the point cloud by projecting the point cloud in the three-dimensional space into the corresponding two-dimensional regularized projection plane structure, and obtains the point cloud in the two-dimensional projection plane structure. Therefore, the sparseness existing in the three-dimensional representation structure is avoided, and the spatial correlation of the point cloud is better reflected; it enables several two-dimensional images obtained by projecting the two-dimensional regularized plane structure in the future.
  • the spatial correlation of the point cloud can be greatly utilized, the spatial redundancy can be reduced, and the encoding efficiency of the point cloud can be further improved;
  • the present invention encodes the occupancy information map obtained by the projection of the two-dimensional regularized plane, the empty row and empty column elements are not encoded, which greatly improves the encoding efficiency;
  • the present invention can use the occupancy information graph to assist other two-dimensional graphs in coding, so as to improve coding efficiency.
  • FIG. 1 is a schematic diagram of a point cloud encoding method based on two-dimensional regularized plane projection provided by an embodiment of the present invention
  • FIG. 2 is a schematic diagram of the correspondence between the cylindrical coordinates of a point and a pixel in a two-dimensional projection plane provided by an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a two-dimensional projection plane structure of a point cloud provided by an embodiment of the present invention.
  • FIG. 4 is a coding block diagram of a placeholder information graph provided by an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of prediction of current pixel occupancy information by using neighbor pixels according to an embodiment of the present invention.
  • FIG. 6 is a coding block diagram of another occupancy information graph provided by an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a point cloud encoding device based on two-dimensional regularized plane projection provided by an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of a point cloud decoding method based on two-dimensional regularized plane projection provided by an embodiment of the present invention.
  • FIG. 9 is a decoding block diagram of an occupancy information graph provided by an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of a point cloud decoding device based on two-dimensional regularized plane projection according to an embodiment of the present invention.
  • FIG. 1 is a schematic diagram of a point cloud encoding method based on two-dimensional regularized plane projection provided by an embodiment of the present invention, which specifically includes:
  • the original point cloud data usually consists of a set of three-dimensional space points, and each space point records its own geometric position information, as well as additional attribute information such as color, reflectance, and normal.
  • the geometric position information of the point cloud is generally represented based on the Cartesian coordinate system, that is, represented by the x, y, and z coordinates of the point.
  • Raw point cloud data can be obtained through 3D scanning equipment such as lidar, etc., or through public datasets provided by various platforms.
  • the acquired geometric position information of the original point cloud data is represented based on a Cartesian coordinate system. It should be noted that the representation method of the geometric position information of the original point cloud data is not limited to Cartesian coordinates.
  • the original point cloud data may also be preprocessed, such as voxelization, to facilitate subsequent encoding.
  • the regularization parameters are usually finely determined by the manufacturer and provided to consumers as one of the necessary data, such as the acquisition range of the lidar, the sampling angle resolution of the horizontal azimuth or the number of sampling points, and the distance correction factor of each laser scanner, the offset information V o and H o of the laser scanner along the vertical and horizontal directions, and the offset information ⁇ 0 of the laser scanner along the elevation and horizontal azimuth angles and a.
  • the regularization parameters are not limited to the parameters given above, which can use the given calibration parameters of the lidar, or can optimize the estimation and data simulation when the calibration parameters of the lidar are not given. obtained in an equal manner.
  • the two-dimensional regularized projection plane structure of the point cloud is a data structure containing M rows and N columns of pixels, and the points in the three-dimensional point cloud correspond to the pixels in the data structure after being projected.
  • the pixel (i, j) in the data structure can be associated with the cylindrical coordinate component ( ⁇ , ⁇ ), for example, the following formula can be used to find the pixel (i, j) corresponding to the cylindrical coordinate (r, ⁇ , ⁇ ) .
  • FIG. 2 is a schematic diagram of the correspondence between the cylindrical coordinates of a point and a pixel in a two-dimensional projection plane provided by an embodiment of the present invention.
  • the resolution of the two-dimensional regularized projection plane can be obtained from the regularization parameters. For example, if the resolution of the two-dimensional regularized projection plane is assumed to be M ⁇ N, the number of laser scanners in the regularization parameter can be used to initialize M. , and utilize the sampling angular resolution of the horizontal azimuth (or the number of sampling points of the laser scanner) to initialize N, for example, the following formula can be used, and finally the initialization of the two-dimensional projection plane structure can be completed, and a plane structure containing M ⁇ N pixels can be obtained.
  • mapping relationship between the original point cloud data and the two-dimensional projected plane structure is determined, so as to project the original point cloud data onto the two-dimensional projected plane structure.
  • This part judges the position of the original point cloud in the two-dimensional projection plane structure point by point, and maps the point cloud originally distributed randomly in the Cartesian coordinate system to the uniformly distributed two-dimensional regular projection plane structure. Specifically, for each point in the original point cloud, the corresponding pixel is determined in the two-dimensional projection plane structure, for example, the pixel with the smallest spatial distance from the projection position of the point in the two-dimensional plane can be selected as the corresponding pixel of the point.
  • the search area of the current point in the two-dimensional projected plane structure Specifically, the entire two-dimensional projection plane structure can be directly used as the search area. Further, in order to reduce the amount of calculation, the pitch angle ⁇ and azimuth angle ⁇ of the cylindrical coordinate components of the current point can also be used to determine the corresponding pixel in the two-dimensional Project the search area in a planar structure to reduce the search area.
  • the error Err is less than the current minimum error minErr, use it to update the minimum error minErr, and use the i and j corresponding to the current pixel to update the i and j of the pixel corresponding to the current point; if the error Err is greater than the minimum error minErr, do not perform the above update process.
  • the corresponding pixel (i, j) of the current point in the two-dimensional projection plane structure can be determined.
  • FIG. 3 is a schematic diagram of a two-dimensional projection plane structure of a point cloud provided by an embodiment of the present invention, wherein each point in the original point cloud data is mapped to a corresponding pixel in the structure.
  • the pieces of two-dimensional map information include a placeholder information map.
  • the occupancy information map is used to identify whether each pixel in the two-dimensional regularized projection plane structure is occupied, that is, whether each pixel corresponds to a point in the point cloud. If it is occupied, the pixel is said to be non-empty. Otherwise, the pixel is said to be empty.
  • 0 and 1 can be used to represent, 1: represents that the current pixel is occupied; 0: represents that the current pixel is not occupied, so the occupancy information map of the point cloud can be obtained according to the two-dimensional projection plane structure of the point cloud.
  • S4 Encode several two-dimensional image information to obtain code stream information.
  • encoding several two-dimensional image information to obtain the code stream information includes: encoding the occupancy information graph to obtain the occupancy information code stream.
  • FIG. 4 is a coding block diagram of an occupancy information graph provided by an embodiment of the present invention, which specifically includes:
  • existing coding techniques such as direct coding or differential predictive coding can be used to encode the identifiers of empty rows and empty columns.
  • the occupancy information of the pixels in the non-empty rows and non-empty columns can be predicted by using the reconstructed occupancy information of the encoded pixels and correspondingly encoded.
  • each pixel in the occupancy information graph is scanned in a certain order, such as zigzag scanning, etc., and it is determined whether the current pixel is located in an empty row or an empty column of the occupancy information graph. Then, for the current pixel in the non-empty row and non-empty column, the occupancy information of the current pixel is predicted using the reconstructed occupancy information of the encoded and decoded pixels.
  • FIG. 5 is a schematic diagram of predicting the occupancy information of a current pixel by using neighbor pixels according to an embodiment of the present invention; wherein, ⁇ represents the current pixel to be encoded, ⁇ represents the occupied pixel that has been encoded and decoded, Represents the encoded and decoded unoccupied pixels, and the pixels included in the dotted box in FIG. 5 are the encoded and decoded neighbor pixels adjacent to the current pixel.
  • the obtained prediction residual information can be encoded by using the existing context-based entropy encoding technology to obtain the occupancy information code stream.
  • the occupancy information of the current pixel is directly coded.
  • each pixel in the occupancy information graph may be judged and encoded in turn by means of traversal.
  • FIG. 6 is a coding block diagram of another occupancy information graph provided by an embodiment of the present invention, which specifically includes:
  • the pixels in the occupancy information map can be traversed in a certain scanning order, such as Z-scanning, etc., to determine whether the current pixel is in an empty row or column.
  • the specific judgment method is: traversing the current pixel respectively. The row and column where the pixel is located. If there is a non-empty pixel, the pixel is not in an empty row or column, otherwise, the pixel is in an empty row or column. If the current pixel is located in an empty row or column and it is the first element of the row or column, the empty row or column can be identified in the above-mentioned manner, and the identification can be encoded.
  • the occupancy information of the pixels in the non-empty rows and non-empty columns can also be predicted and encoded according to the above-mentioned prediction and encoding method.
  • the empty rows and empty columns in the occupancy information graph are marked, and only the pixels in the marks and non-empty rows and non-empty columns are encoded, which reduces the code stream and greatly improves the encoding. efficiency.
  • the present embodiment can also adopt a traditional encoding method, that is, instead of processing empty rows and empty columns, directly using the reconstructed occupancy information of the encoded pixels to predict the occupancy information of the current pixel and perform corresponding coding.
  • the placeholder information graph can also be encoded by means of image ⁇ video compression, and the encoding schemes that can be used here include but are not limited to: JPEG, JPEG2000, HEIF, H.264 ⁇ AVC, H.265 ⁇ HEVC, etc.
  • the invention By projecting the point cloud in the three-dimensional space into the corresponding two-dimensional regularized projection plane structure, the invention performs regular correction on the vertical direction and the horizontal direction of the point cloud, and obtains the point cloud on the two-dimensional projection plane structure.
  • Strong correlation representation which avoids the sparsity in the 3D representation structure, and better reflects the spatial correlation of the point cloud; it enables the subsequent coding of the occupancy information map and other 2D map information.
  • the spatial correlation of point clouds is greatly utilized to reduce spatial redundancy, thereby further improving the coding efficiency of point clouds.
  • several two-dimensional map information obtained according to the two-dimensional projection plane structure further includes at least one of a depth information map, a projection residual information map, and a coordinate conversion error information map.
  • the depth information map is used to represent the distance between the corresponding point of each occupied pixel and the coordinate origin in the 2D regularized projection plane structure.
  • the cylindrical coordinate r component of the corresponding point of the pixel can be used as the depth of the pixel. Based on this, each occupied pixel in the two-dimensional regularized projection plane structure will have a depth value, thereby obtaining the corresponding depth information map.
  • the projection residual information map is used to represent the residual between the corresponding position of each occupied pixel in the 2D regular projection plane structure and the actual projection position. Based on this, each occupied pixel in the two-dimensional regularized projection plane will have a projection residual, so as to obtain the projection residual information map corresponding to the point cloud.
  • the coordinate transformation error information map is used to represent the residual between the spatial position of each occupied pixel in the two-dimensional regularized projection plane structure and the spatial position of the corresponding original point of the pixel. Based on this, each occupied pixel in the two-dimensional regularized projection plane structure will have a coordinate transformation error, so as to obtain the coordinate transformation error information map corresponding to the point cloud.
  • the depth information map is traversed in a certain scanning order.
  • the reconstructed occupancy information map and the reconstructed depth information of the encoded and decoded pixels can be used for predictive coding.
  • the existing neighbor prediction can be performed.
  • the technology is combined with the occupancy information of the neighbor pixels to predict, that is, only the neighbor pixels with non-empty occupancy information are used to predict the depth information of the current pixel.
  • the existing entropy coding technology can be used for coding to obtain the depth information code stream.
  • FIG. 7 A schematic structural diagram of a point cloud encoding device for regularized plane projection, which includes:
  • the first data acquisition module 11 is used to acquire original point cloud data
  • the projection module 12 is used to perform a two-dimensional regularized plane projection on the original point cloud data to obtain a two-dimensional projected plane structure
  • the data processing module 13 is used to obtain several two-dimensional map information according to the two-dimensional projection plane structure
  • the encoding module 14 is used for encoding several two-dimensional image information to obtain code stream information.
  • the encoding apparatus provided in this embodiment can implement the encoding methods described in the first and second embodiments above, and the detailed process is not repeated here.
  • FIG. 8 is a schematic diagram of a point cloud decoding method based on two-dimensional regularized plane projection provided by an embodiment of the present invention.
  • the method includes:
  • Step 1 Obtain the code stream information and decode it to obtain parsed data.
  • the decoding end obtains the compressed code stream information, and uses the corresponding existing entropy decoding technology to decode the code stream information correspondingly to obtain parsed data.
  • Step 2 Reconstruct several two-dimensional map information according to the analytical data.
  • step 2 may include:
  • the pixels located in the empty rows or columns in the occupancy information graph are directly reconstructed according to the identifiers of the empty rows and columns in the occupancy information in the parsed data, and the residuals are predicted according to the occupancy information in the parsed data. Pixels located in non-empty rows and non-empty columns in the information map are reconstructed to obtain a reconstructed occupancy information map.
  • the code stream information at the decoding end also includes the placeholder information code stream.
  • the parsed data obtained by decoding the code stream information includes identifiers of empty rows and columns in the occupancy information graph and prediction residuals of the occupancy information.
  • FIG. 9 is a decoding block diagram of the occupancy information map provided by the embodiment of the present invention. The following process is performed according to the identifiers of empty rows and columns in the occupancy information map obtained by parsing and the prediction residual of the occupancy information.
  • the encoder uses a certain scanning order to traverse the pixels in the occupancy information map and encodes the occupancy information of the pixels in the non-empty rows and non-empty columns, the prediction residuals of the pixel occupancy information obtained by the decoding end It is also in this order, and the decoding end can obtain the resolution of the occupancy information map through the regularization parameter.
  • the decoding end can know the current position of the pixel to be reconstructed in the two-dimensional image according to the parsed empty row and empty column identifiers and the resolution of the occupancy information map.
  • the reconstructed occupancy information of the encoded and decoded pixels is used for prediction.
  • the reconstructed occupancy information of the neighbor pixels is predicted, and then the occupancy information of the current pixel is reconstructed according to the obtained predicted value and the parsed prediction residual.
  • the occupancy information of the pixels in the empty rows and empty columns in the occupancy information graph is reconstructed according to the parsed empty row and empty column identifiers. After the occupancy information of all pixels is reconstructed, the reconstructed occupancy information map is obtained.
  • Step 3 obtain a two-dimensional projection plane structure according to the two-dimensional map information
  • the occupancy information of each pixel in the 2D projection plane structure can be known, and the reconstructed 2D projection can be obtained. flat structure.
  • Step 4 Reconstruct the point cloud using the 2D projected plane structure.
  • the occupancy information of each pixel can be known. If the occupancy information of the current pixel (i, j) is non-empty, use other information, such as depth information, coordinate conversion error information, etc., to reconstruct the spatial point (x, y, z) corresponding to the pixel.
  • the corresponding position of the current pixel (i, j) can be expressed as ( ⁇ j , i), then the regularization parameters and other information, such as depth information r and coordinate conversion error information ( ⁇ x, ⁇ y, ⁇ z) can be used to recreate the The spatial point (x, y, z) corresponding to the current pixel is constructed, and the specific calculation is as follows:
  • each non-empty pixel in the two-dimensional projection structure can be reconstructed to its corresponding spatial point, thereby obtaining the reconstructed point cloud.
  • FIG. 10 A schematic structural diagram of a point cloud decoding device for plane projection, which includes:
  • the second data acquisition module 21 is used to acquire and decode the code stream information to obtain analytical data
  • the first reconstruction module 22 is used for reconstructing several two-dimensional map information according to the analysis data
  • the second reconstruction module 23 is used to obtain a two-dimensional projection plane structure according to several two-dimensional map information
  • the point cloud reconstruction module 24 is used for reconstructing the point cloud by using the two-dimensional projection plane structure.
  • the decoding apparatus provided in this embodiment can implement the decoding method described in the fifth embodiment, and the detailed process is not repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)

Abstract

本发明公开了一种基于二维规则化平面投影的点云编解码方法及装置，编码方法包括：获取原始点云数据；对原始点云数据进行二维规则化平面投影，得到二维投影平面结构；根据二维投影平面结构得到若干二维图信息；对若干二维图信息进行编码，得到码流信息。本发明通过二维规则化平面投影技术得到点云在二维投影平面结构上的强相关性表示，从而更好的体现了点云的空间相关性，使得后续在对二维规则化投影平面结构所得到的占位信息图进行编码时，能够极大地利用点云的空间相关性，提升点云了的编码效率；同时在对占位信息图进行编码时，对其中的空行和空列元素不进行编码，进一步提升了编码效率。

Description

基于二维规则化平面投影的点云编解码方法及装置
本申请要求于2021年02月08日提交中国专利局、申请号为202110172789.9、申请名称为“基于二维规则化平面投影的点云编解码方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明属于编解码技术领域，具体涉及一种基于二维规则化平面投影的点云编解码方法及装置。
背景技术
随着硬件处理能力的提升和计算机视觉的飞速发展，三维点云成为继音频、图像、视频之后的新一代沉浸式多媒体，被广泛的应用于虚拟现实、增强现实、自动驾驶和环境建模等。然而三维点云通常具有较大的数据量，十分不利于点云数据的传输及存储，因此研究高效的点云编解码技术具有重要意义。
在现有的基于几何的点云压缩编码(G-PCC,Geometry-based Point Cloud Compression)框架中，点云的几何信息和属性信息是分开进行编码的。目前G-PCC的几何编解码可分为基于八叉树的几何编解码和基于预测树的几何编解码。
基于八叉树的几何编解码：在编码端，首先，对点云的几何信息进行预处理，这包括点云的坐标转换和体素化过程。然后，按照广度优先遍历的顺序不断对点云所在的包围盒进行树划分(八叉树/四叉树/二叉树)。最后，对每个节点的占位码进行编码，并编码每个叶子节点中包含的点数，生成二进制码流。在解码端，首先按照广度优先遍历的顺序，不断解析得到每个节点的占位码。然后，依次不断进行树划分，直至划分得到1x1x1的单位立方体时停止划分。最后，解析得到每个叶子节点中包含的点数，最终得到重构的点 云几何信息。
基于预测树的几何编解码：在编码端，首先对原始点云进行排序。然后，建立预测树结构，通过将每个点归类到所属的激光扫描器上，并按照不同的激光扫描器建立预测树结构。接下来，遍历预测树中的每个节点，通过选取不同的预测模式对节点的几何信息进行预测得到预测残差，并利用量化参数对预测残差进行量化。最后，对预测树结构、量化参数以及节点几何信息的预测残差等进行编码，生成二进制码流。在解码端，首先解析码流，其次重构预测树结构，然后通过解析得到的每个节点的几何信息预测残差以及量化参数，对预测残差进行反量化，最终恢复得到每个节点的重构几何信息，即完成了点云几何信息的重建。
然而，由于点云具有较强的空间稀疏性，对于使用八叉树结构的点云编码技术而言，该结构会导致划分得到的空节点占比较高，且无法充分体现点云的空间相关性，从而不利于点云的预测及熵编码。基于预测树的点云编解码技术利用激光雷达设备的部分参数来建立树结构，在此基础上利用树结构进行预测编码，然而该树结构并未充分体现点云的空间相关性，从而不利于点云的预测及熵编码。因而，上述两种点云编解码技术均存在编码效率不够高的问题。
发明内容
为了解决现有技术中存在的上述问题，本发明提供了一种基于二维规则化平面投影的点云编解码方法及装置。本发明要解决的技术问题通过以下技术方案实现：
一种基于二维规则化平面投影的点云编码方法，包括：
获取原始点云数据；
对所述原始点云数据进行二维规则化平面投影，得到二维投影平面结构；
根据所述二维投影平面结构得到若干二维图信息；
对所述若干二维图信息进行编码，得到码流信息。
在本发明的一个实施例中，所述若干二维图信息包括占位信息图。
在本发明的一个实施例中，对所述若干二维图信息进行编码，得到码流信息，包括：
对所述占位信息图进行编码，得到占位信息码流。
在本发明的一个实施例中，对所述占位信息图进行编码，得到占位信息码流，包括：
对所述占位信息图中的空行和空列进行标识；
对所述占位信息图中空行和空列的标识进行编码；
对所述占位信息图中非空行和非空列中像素的占位信息进行编码，得到占位信息码流。
在本发明的一个实施例中，对所述占位信息图中非空行和非空列中像素的占位信息进行编码，包括：
利用已编码像素的重建占位信息预测非空行和非空列中像素的占位信息并进行相应的编码。
在本发明的一个实施例中，所述若干二维图信息还包括深度信息图、投影残差信息图以及坐标转换误差信息图中的至少一种。
本发明的另一个实施例还提供了一种基于二维规则化平面投影的点云编码装置，包括：
第一数据获取模块，用于获取原始点云数据；
投影模块，用于对所述原始点云数据进行二维规则化平面投影，得到二维投影平面结构；
数据处理模块，用于根据所述二维投影平面结构得到若干二维图信息；
编码模块，用于对所述若干二维图信息进行编码，得到码流信息。
本发明的又一个实施例还提供了一种基于二维规则化平面投影的点云解码方法，包括：
获取码流信息并进行解码，得到解析数据；
根据所述解析数据重构若干二维图信息；
根据所述若干二维图信息得到二维投影平面结构；
利用所述二维投影平面结构重建点云。
在本发明的一个实施例中，根据所述解析数据重构若干二维图信息，包括：
根据所述解析数据中的占位信息图中空行和空列的标识对占位信息图中位于空行或空列的像素直接进行重构，并根据所述解析数据中的占位信息预测残差对所述占位信息图中位于非空行和非空列的像素进行重构，得到重构的占位信息图。
本发明的再一个实施例还提供了一种基于二维规则化平面投影的点云解码装置，包括：
第二数据获取模块，用于获取码流信息并进行解码，得到解析数据；
第一重构模块，用于根据所述解析数据重构若干二维图信息；
第二重构模块，用于根据所述若干二维图信息得到二维投影平面结构；
点云重建模块，用于利用所述二维投影平面结构重建点云。
本发明的有益效果：
1、本发明通过将三维空间中的点云投影到对应的二维规则化投影平面结构当中，对点云在垂直方向和水平方向上进行了规则化校正，得到点云在二维投影平面结构上的强相关性表示，从而避免了三维表示结构中存在的稀疏性，又更好的体现了点云的空间相关性；使得后续在对二维规则化投影平面结构所得到的若干二维图信息进行编码时，能够极大地利用点云的空间相关性，减小空间冗余，从而进一步提升点云的编码效率；
2、本发明在对二维规则化平面投影得到的占位信息图进行编码时，对其中的空行和空列元素不进行编码，极大的提升了编码效率；
3、本发明可利用占位信息图辅助其他二维图进行编码，以提升编码效率。
以下将结合附图及实施例对本发明做进一步详细说明。
附图说明
图1是本发明实施例提供的一种基于二维规则化平面投影的点云编码方法示意图；
图2是本发明实施例提供的点的柱面坐标与二维投影平面中像素的对应关系示意图；
图3是本发明实施例提供的点云的二维投影平面结构示意图；
图4是本发明实施例提供的一种占位信息图的编码框图；
图5是本发明实施例提供的利用邻居像素对当前像素占位信息的预测示意图；
图6是本发明实施例提供的另一种占位信息图的编码框图；
图7是本发明实施例提供的一种基于二维规则化平面投影的点云编码装置结构示意图；
图8是本发明实施例提供的一种基于二维规则化平面投影的点云解码方法示意图；
图9是本发明实施例提供的占位信息图的解码框图；
图10是本发明实施例提供的一种基于二维规则化平面投影的点云解码装置结构示意图。
具体实施方式
下面结合具体实施例对本发明做进一步详细的描述，但本发明的实施方式不限于此。
实施例一
请参见图1，图1是本发明实施例提供的基于二维规则化平面投影的点云编码方法示意图，具体包括：
S1：获取原始点云数据。
具体地，原始点云数据通常由一组三维空间点组成，每个空间点都记录了自身的几何位置信息，以及颜色、反射率、法线等额外的属性信息。其中， 点云的几何位置信息一般是基于笛卡尔坐标系进行表示的，即利用点的x，y，z坐标进行表示。原始点云数据可通过3D扫描设备例如激光雷达等获取，也可通过各种平台提供的公共数据集获得。在本实施例中，设获取到的原始点云数据的几何位置信息基于笛卡尔坐标系进行表示。需要说明的是，原始点云数据的几何位置信息的表示方法不限于笛卡尔坐标。
S2：对原始点云数据进行二维规则化平面投影，得到二维投影平面结构。
具体的，在本实施例中，在对原始点云进行二维规则化平面投影之前，还可以对原始点云数据进行预处理，如体素化处理等，以方便后续编码。
首先，初始化二维投影平面结构。
初始化点云的二维规则化投影平面结构需要利用规则化参数。规则化参数通常由制造厂商进行精细测定并作为必备的数据之一提供给消费者，例如激光雷达的采集范围，水平方位角的采样角分辨率
Figure PCTCN2022075407-appb-000001
或采样点数，以及每个激光扫描器的距离校正因子、激光扫描器沿垂直方向和水平方向的偏移信息V o和H o、激光扫描器沿俯仰角和水平方位角的偏移信息θ 0和α。
需要说明的是，规则化参数不限于以上给出的这些参数，其可以利用给定的激光雷达的标定参数，也可以在激光雷达的标定参数没有给定的情况下，通过优化估计、数据拟合等方式得到。
点云的二维规则化投影平面结构为一个包含M行、N列像素的数据结构，三维点云中的点经过投影后与该数据结构中的像素对应。并且该数据结构中的像素(i,j)可与柱面坐标分量(θ,φ)相关联，如可利用以下公式找到柱面坐标(r,θ,φ)对应的像素(i,j)。
Figure PCTCN2022075407-appb-000002
Figure PCTCN2022075407-appb-000003
具体地，请参见图2，图2是本发明实施例提供的点的柱面坐标与二维投影平面中像素的对应关系示意图。
需要说明的是，此处像素的对应并不限于柱面坐标。
进一步地，二维规则化投影平面的分辨率可由规则化参数获得，如假设二维规则化投影平面的分辨率为M×N，则可利用规则化参数中激光扫描器的个数来初始化M，并利用水平方位角的采样角分辨率
Figure PCTCN2022075407-appb-000004
(或者激光扫描器的采样点数)来初始化N，例如可采用如下公式，最终即可完成二维投影平面结构的初始化，得到一个包含M×N个像素的平面结构。
M＝laserNum；
Figure PCTCN2022075407-appb-000005
或N＝pointNumPerLaser。
其次，确定原始点云数据与二维投影平面结构的映射关系，以将原始点云数据投影到二维投影平面结构上。
该部分通过逐点判断原始点云在二维投影平面结构中的位置，将原本在笛卡尔坐标系下杂乱分布的点云映射至均匀分布的二维规则化投影平面结构中。具体的，针对原始点云中的每一个点，在二维投影平面结构中确定对应的像素，例如可选择与点在二维平面中投影位置空间距离最小的像素作为该点的对应像素。
若利用柱面坐标系进行二维投影，则确定原始点云对应像素的具体流程如下：
a.确定原始点云数据中当前点的柱面坐标分量r，具体的，利用以下公式进行计算：
Figure PCTCN2022075407-appb-000006
b.确定当前点在二维投影平面结构中的搜索区域。具体的，可选择直接将整个二维投影平面结构作为搜索区域，进一步的，为了减小计算量，还可通过当前点的柱面坐标分量俯仰角θ和方位角φ来确定对应像素在二维投影平面结构中的搜索区域，以减小搜索区域。
c.确定搜索区域后，对其中的每个像素(i,j)，利用规则化参数即激光雷 达第i个激光扫描器的标定参数θ 0、V o、H o和α，计算当前像素在笛卡尔坐标系中的位置(xl,yl,zl)，具体计算公式如下：
θ i＝θ 0
Figure PCTCN2022075407-appb-000007
xl＝r·sin(φ j-α)-H o·cos(φ j-α)
yl＝r·cos(φ j-α)+H o·sin(φ j-α)
zl＝r·tanθ i+V o
d.得到当前像素在笛卡尔坐标系中的位置(xl,yl,zl)后，计算其与当前点(x,y,z)之间的空间距离并将其作为误差Err，即：
Err＝dist{(x,y,z),(xl,yl,zl)}
若该误差Err小于当前最小误差minErr，则用其更新最小误差minErr，并用当前像素对应的i和j更新当前点所对应像素的i和j；若该误差Err大于最小误差minErr，则不进行以上更新过程。
e.当搜索区域内的所有像素均被遍历完成后，即可确定当前点在二维投影平面结构中的对应像素(i,j)。
当原始点云中的所有点均完成上述操作后，即完成了点云的二维规则化平面投影。具体地，请参见图3，图3是本发明实施例提供的点云的二维投影平面结构示意图，其中，原始点云数据中的每个点均被映射至该结构中的对应像素。
需要说明的是，在点云的二维规则化平面投影过程中，可能会出现点云中的多个点对应到二维投影平面结构中的同一像素。若要避免这种情况发生，可选择在投影时将这些空间点投影到不同的像素中，例如，对某一点进行投影时，若其对应的像素中已有对应点，则将该点投影至该像素的邻近空像素中。此外，若点云中的多个点已投影到二维投影平面结构中的同一像素，则在基于二维投影平面结构进行编码时，应额外编码每个像素中的对应点数，并根据该点数对像素中的每个对应点信息进行编码。
S3：根据二维投影平面结构得到若干二维图信息。
在本实施例中，若干二维图信息包括占位信息图。
具体地，占位信息图用来标识二维规则化投影平面结构中每个像素是否被占据，即每个像素是否与点云中的点相对应，若被占据，则称该像素非空，否则，称该像素为空。如可采用0和1进行表示，1：代表当前像素被占用；0：代表当前像素没有被占用，由此可根据点云的二维投影平面结构得到其占位信息图。
S4：对若干二维图信息进行编码，得到码流信息。
相应的，对若干二维图信息进行编码，得到码流信息包括：对占位信息图进行编码，得到占位信息码流。
请参见图4，图4是本发明实施例提供的一种占位信息图的编码框图，具体包括：
41)对占位信息图中的空行和空列进行标识。
由于输入的占位信息图中可能会存在一整行或一整列中的像素均不被占据即存在空行或空列的情况，因此需要对占位信息图中的空行或空列进行标识，以更高效的进行编码。
具体的，按照某种顺序扫描占位信息图中的每一行像素，若当前像素非空，则该行非空，然后记下该非空像素所在的列号，那么该列号对应的列也非空，接着直接扫描下一行；若当前像素为空且当前像素为该行的最后一个像素，则该行为空，然后对该空行进行标识。接下来，按照某种顺序扫描占位信息图中的每一列，跳过前面已记下的非空列，若当前像素非空，则该列非空，接着直接扫描下一列；若当前像素为空且当前像素为该列的最后一个像素，则该列为空，然后对该空列进行标识。具体可采用行号和列号来标识空行和空列。
42)对占位信息图中空行和空列的标识进行编码。
具体的，可采用现有的编码技术如直接编码或差分预测编码方式对空行 和空列的标识进行编码。
43)对占位信息图中非空行和非空列中像素的占位信息进行编码。
在本实施例中，可利用已编码像素的重建占位信息预测非空行和非空列中像素的占位信息并进行相应的编码。
具体地，按照某种顺序扫描占位信息图中的每一个像素，如Z字扫描等，并确定当前像素是否位于占位信息图的空行或空列。然后对于非空行和非空列中的当前像素，利用已编解码像素的重建占位信息来预测当前像素的占位信息。
更具体地，例如可利用与当前像素相邻的已编解码的邻居像素的重建占位信息进行预测。请参见图5，图5是本发明实施例提供的利用邻居像素对当前像素占位信息的预测示意图；其中，☆代表当前待编码像素，○代表已编解码的占据像素，
Figure PCTCN2022075407-appb-000008
代表已编解码的未被占据的像素，图5中虚线框包含的像素即为当前像素相邻的已编解码的邻居像素。然后对得到的预测残差信息可采用现有的基于上下文的熵编码技术进行编码，得到占位信息码流。此外，如果当前像素没有可用的已编码像素，那么直接对当前像素的占位信息进行编码。
此外，在本发明的另一个实施例中，还可以通过遍历的方式依次对占位信息图中的每个像素进行判断并编码。请参见图6，图6是本发明实施例提供的另一种占位信息图的编码框图，具体包括：
4a)按照预设扫描顺序遍历占位信息图中的像素，若判断当前像素处于空行或者空列中，且当前像素为该行或者该列的首元素时，标识该空行或者空列，并对该标识进行编码。
具体的，在本实施例中，可按照一定的扫描顺序遍历占位信息图中的像素，如Z字扫描方式等，判断当前像素是否处于空行或者空列，具体判断方法是：分别遍历当前像素所在的行和列，若其中存在非空像素，则该像素不处于空行或空列中，否则，该像素处于空行或空列中。若当前像素位于空行 或空列中且其为该行或者该列的首元素时，可按照上述方式标识该空行或者空列，并对该标识进行编码。
4b)若当前像素不处于空行或者空列中，则对当前像素的占位信息进行预测和编码。
具体的，在本实施例中，同样可按照上述的预测和编码方法对非空行及非空列中像素的占位信息进行预测和编码。
至此，完成占位信息图的编码，得到占位信息码流。
本实施例通过对占位信息图中的空行和空列进行标识，并仅对标识和非空行和非空列中的像素进行了编码，减小了码流，极大的提高了编码效率。
进一步地，本实施例还可采用传统的编码方法，即不对空行和空列进行处理，而是直接利用已编码像素的重建占位信息预测当前像素的占位信息并进行相应的编码。
在本发明的另一个实施例中，还可以借助图像\视频压缩方式对占位信息图进行编码，此处可使用的编码方案包含但不限于：JPEG、JPEG2000、HEIF、H.264\AVC、H.265\HEVC等。
本发明通过将三维空间中的点云投影到对应的二维规则化投影平面结构当中，对点云在垂直方向和水平方向上进行了规则化校正，得到点云在二维投影平面结构上的强相关性表示，从而避免了三维表示结构中存在的稀疏性，又更好的体现了点云的空间相关性；使得后续在对占位信息图及其他二维图信息进行编码时，能够极大地利用点云的空间相关性，减小空间冗余，从而进一步提升点云的编码效率。
实施例二
在本实施例中，根据二维投影平面结构得到若干二维图信息还包括深度信息图、投影残差信息图以及坐标转换误差信息图中的至少一种。其中，
深度信息图用来表示二维规则化投影平面结构中每个被占据像素的对应点与坐标原点之间的距离。如可采用该像素对应点的柱面坐标r分量作为该像 素的深度。基于此，二维规则化投影平面结构中每个被占据的像素都会有一个深度值，从而得到对应的深度信息图。
投影残差信息图用来表示二维规则化投影平面结构中每个被占据像素的对应位置与实际投影位置之间的残差。基于此，二维规则化投影平面中每个被占据的像素都会有一个投影残差，从而得到点云对应的投影残差信息图。
坐标转换误差信息图用来表示二维规则化投影平面结构中每个被占据像素逆投影所得的空间位置与该像素对应原始点的空间位置之间的残差。基于此，二维规则化投影平面结构中每个被占据的像素都会有一个坐标转换误差，从而得到点云对应的坐标转换误差信息图。
则相应的，还需要根据实际情况对深度信息图、投影残差信息图以及坐标转换误差信息图进行编码。
具体地，对于上述四幅二维图，可采用分开编码的方式分别进行编码，还可以根据已编码的占位信息图辅助其他信息图依次进行编码，例如：
采用某种扫描顺序遍历深度信息图，对于深度信息图中的当前像素，可利用已重建的占位信息图和已编解码像素的重建深度信息来进行预测编码，具体可在现有的邻居预测技术之上结合邻居像素的占位信息来预测，即仅使用占位信息非空的邻居像素来预测当前像素的深度信息，预测值的计算可采用加权平均等方式，得到相应的预测残差后可采用现有的熵编码技术进行编码，得到深度信息码流。
对于投影残差信息图和坐标转换误差信息图，可采用与上述深度信息图相似的编码方法进行编码，具体不再赘述。
实施例三
在上述实施例一和二的基础上，本实施例提供了一种基于二维规则化平面投影的点云编码装置，请参见图7，图7是本发明实施例提供的一种基于二维规则化平面投影的点云编码装置结构示意图，其包括：
第一数据获取模块11，用于获取原始点云数据；
投影模块12，用于对原始点云数据进行二维规则化平面投影，得到二维投影平面结构；
数据处理模块13，用于根据二维投影平面结构得到若干二维图信息；
编码模块14，用于对若干二维图信息进行编码，得到码流信息。
本实施例提供的编码装置可以实现上述实施例一和二所述的编码方法，详细过程在此不再赘述。
实施例四
请参见图8，图8是本发明实施例提供的一种基于二维规则化平面投影的点云解码方法示意图，该方法包括：
步骤1：获取码流信息并进行解码，得到解析数据。
解码端获取压缩的码流信息，并采用相应的现有熵解码技术对码流信息进行相应的解码，得到解析后的数据。
步骤2：根据解析数据重构若干二维图信息。
在本实施例中，步骤2可以包括：
根据解析数据中的占位信息图中空行和空列的标识对占位信息图中位于空行或空列的像素直接进行重构，并根据解析数据中的占位信息预测残差对占位信息图中位于非空行和非空列的像素进行重构，得到重构的占位信息图。
具体地，由于在编码端，若干二维图信息可以包括占位信息图，也即对占位信息图进行了编码，相应的，解码端的码流信息也包括占位信息码流。
更具体地，码流信息通过解码得到的解析数据包括占位信息图中空行和空列的标识以及占位信息的预测残差。
请参见图9，图9是本发明实施例提供的占位信息图的解码框图，根据解析得到的占位信息图中空行和空列的标识以及占位信息的预测残差进行以下过程。
对于非空行和非空列中像素的占位信息进行预测和重构；同时，对于已标识的空行或空列中的像素，直接重构其占位信息。
由于编码端采用了某种扫描顺序遍历占位信息图中的像素并对其中非空行和非空列中像素的占位信息进行编码，那么解码端所得到的像素占位信息的预测残差同样是按照此种顺序，且解码端可通过规则化参数获得占位信息图的分辨率，具体参见实施例一中S2初始化二维投影平面结构部分。因此，解码端根据解析出的空行和空列标识以及占位信息图的分辨率可获知当前待重构像素在二维图中的位置。
一方面，对于占位信息图中的当前待重构像素，利用已编解码像素的重建占位信息来进行预测，预测方法与编码端保持一致，即采用与当前像素相邻的已编解码的邻居像素的重建占位信息进行预测，然后根据得到的预测值和解析出来的预测残差重建当前像素的占位信息。另一方面，根据解析出的空行和空列标识重构占位信息图中空行和空列中像素的占位信息。当重构完所有像素的占位信息后，即得到重构的占位信息图。
步骤3：根据二维图信息得到二维投影平面结构；
由于二维投影平面结构的分辨率与占位信息图一致，且占位信息图已被重构，因此可知二维投影平面结构中每个像素的占位信息，从而得到重构的二维投影平面结构。
步骤4：利用二维投影平面结构重建点云。
具体地，按照某一扫描顺序遍历重构的二维投影平面结构中的像素，可知每个像素的占位信息。若当前像素(i,j)的占位信息为非空，则利用其它信息，如深度信息、坐标转换误差信息等，重构该像素对应的空间点(x,y,z)。具体的，当前像素(i,j)的对应位置可表示为(φ j,i)，那么可利用规则化参数和其他信息，如深度信息r和坐标转换误差信息(Δx,Δy,Δz)重构当前像素对应的空间点(x,y,z)，具体计算如下：
Figure PCTCN2022075407-appb-000009
θ i＝θ 0
xl＝r·sin(φ j-α)-H o·cos(φ j-α)
yl＝r·cos(φ j-α)+H o·sin(φ j-α)
zl＝r·tanθ i+V o
(x,y,z)＝(xl,yl,zl)+(Δx,Δy,Δz)
最后根据以上计算即可对二维投影结构中的每个非空像素重构其对应的空间点，从而得到重建点云。
实施例五
在上述实施例四的基础上，本实施例提供了一种基于二维规则化平面投影的点云解码装置，请参见图10，图10是本发明实施例提供的一种基于二维规则化平面投影的点云解码装置结构示意图，其包括：
第二数据获取模块21，用于获取码流信息并进行解码，得到解析数据；
第一重构模块22，用于根据解析数据重构若干二维图信息；
第二重构模块23，用于根据若干二维图信息得到二维投影平面结构；
点云重建模块24，用于利用二维投影平面结构重建点云。
本实施例提供的解码装置可以实现上述实施例五所述的解码方法，详细过程在此不再赘述。
以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干简单推演或替换，都应当视为属于本发明的保护范围。

Claims (10)

  1. 一种基于二维规则化平面投影的点云编码方法，其特征在于，包括：
    获取原始点云数据；
    对所述原始点云数据进行二维规则化平面投影，得到二维投影平面结构；
    根据所述二维投影平面结构得到若干二维图信息；
    对所述若干二维图信息进行编码，得到码流信息。
  2. 根据权利要求1所述的基于二维规则化平面投影的点云编码方法，其特征在于，所述若干二维图信息包括占位信息图。
  3. 根据权利要求2所述的基于二维规则化平面投影的点云编码方法，其特征在于，对所述若干二维图信息进行编码，得到码流信息，包括：
    对所述占位信息图进行编码，得到占位信息码流。
  4. 根据权利要求3所述的基于二维规则化平面投影的点云编码方法，其特征在于，对所述占位信息图进行编码，得到占位信息码流，包括：
    对所述占位信息图中的空行和空列进行标识；
    对所述占位信息图中空行和空列的标识进行编码；
    对所述占位信息图中非空行和非空列中像素的占位信息进行编码，得到占位信息码流。
  5. 根据权利要求4所述的基于二维规则化平面投影的点云编码方法，其特征在于，对所述占位信息图中非空行和非空列中像素的占位信息进行编码，包括：
    利用已编码像素的重建占位信息预测非空行和非空列中像素的占位信息并进行相应的编码。
  6. 根据权利要求2所述的基于二维规则化平面投影的点云编码方法，其特征在于，所述若干二维图信息还包括深度信息图、投影残差信息图以及坐标转换误差信息图中的至少一种。
  7. 一种基于二维规则化平面投影的点云编码装置，其特征在于，包括：
    第一数据获取模块(11)，用于获取原始点云数据；
    投影模块(12)，用于对所述原始点云数据进行二维规则化平面投影，得到二维投影平面结构；
    数据处理模块(13)，用于根据所述二维投影平面结构得到若干二维图信息；
    编码模块(14)，用于对所述若干二维图信息进行编码，得到码流信息。
  8. 一种基于二维规则化平面投影的点云解码方法，其特征在于，包括：
    获取码流信息并进行解码，得到解析数据；
    根据所述解析数据重构若干二维图信息；
    根据所述若干二维图信息得到二维投影平面结构；
    利用所述二维投影平面结构重建点云。
  9. 根据权利按要求8所述的基于二维规则化平面投影的点云解码方法，其特征在于，根据所述解析数据重构若干二维图信息，包括：
    根据所述解析数据中的占位信息图中空行和空列的标识对占位信息图中位于空行或空列的像素直接进行重构，并根据所述解析数据中的占位信息预测残差对所述占位信息图中位于非空行和非空列的像素进行重构，得到重构的占位信息图。
  10. 一种基于二维规则化平面投影的点云解码装置，其特征在于，包括：
    第二数据获取模块(21)，用于获取码流信息并进行解码，得到解析数据；
    第一重构模块(22)，用于根据所述解析数据重构若干二维图信息；
    第二重构模块(23)，用于根据所述若干二维图信息得到二维投影平面结构；
    点云重建模块(24)，用于利用所述二维投影平面结构重建点云。
PCT/CN2022/075407 2021-02-08 2022-02-07 基于二维规则化平面投影的点云编解码方法及装置 WO2022166966A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US18/268,466 US20240080497A1 (en) 2021-02-08 2022-02-07 Point cloud encoding and decoding method and device based on two-dimensional regularization plane projection
EP22749245.1A EP4246974A4 (en) 2021-02-08 2022-02-07 METHOD AND APPARATUS FOR POINT CLOUD ENCODING/DECODING BASED ON TWO-DIMENSIONAL REGULARIZATION PLANE PROJECTION

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110172789.9 2021-02-08
CN202110172789.9A CN114915794B (zh) 2021-02-08 2021-02-08 基于二维规则化平面投影的点云编解码方法及装置

Publications (1)

Publication Number Publication Date
WO2022166966A1 true WO2022166966A1 (zh) 2022-08-11

Family

ID=82742019

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/075407 WO2022166966A1 (zh) 2021-02-08 2022-02-07 基于二维规则化平面投影的点云编解码方法及装置

Country Status (4)

Country Link
US (1) US20240080497A1 (zh)
EP (1) EP4246974A4 (zh)
CN (1) CN114915794B (zh)
WO (1) WO2022166966A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190156520A1 (en) * 2017-11-22 2019-05-23 Apple Inc. Point cloud occupancy map compression
WO2020026846A1 (ja) * 2018-08-02 2020-02-06 ソニー株式会社 画像処理装置および方法
US20200107033A1 (en) * 2018-10-02 2020-04-02 Samsung Electronics Co., Ltd. Point cloud compression using continuous surface codes
CN111739111A (zh) * 2019-03-20 2020-10-02 上海交通大学 一种点云投影编码的块内偏移优化方法及系统
CN112153391A (zh) * 2019-06-28 2020-12-29 腾讯美国有限责任公司 视频编码的方法、装置、电子设备及存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110708529B (zh) * 2018-07-09 2020-08-14 上海交通大学 动态点云投影压缩方法、系统、装置及介质
EP3595180B1 (en) * 2018-07-10 2021-12-08 BlackBerry Limited Methods and devices for neighbourhood-based occupancy prediction in point cloud compression
US20200296401A1 (en) * 2019-03-15 2020-09-17 Mediatek Inc. Method and Apparatus of Patch Segmentation for Video-based Point Cloud Coding
EP3926959A4 (en) * 2019-03-21 2022-03-23 LG Electronics Inc. POINT CLOUD DATA TRANSMITTER DEVICE, POINT CLOUD DATA TRANSMITTER METHOD, POINT CLOUD DATA RECEIVE DEVICE, AND POINT CLOUD DATA RECEIVE METHOD

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190156520A1 (en) * 2017-11-22 2019-05-23 Apple Inc. Point cloud occupancy map compression
WO2020026846A1 (ja) * 2018-08-02 2020-02-06 ソニー株式会社 画像処理装置および方法
US20200107033A1 (en) * 2018-10-02 2020-04-02 Samsung Electronics Co., Ltd. Point cloud compression using continuous surface codes
CN111739111A (zh) * 2019-03-20 2020-10-02 上海交通大学 一种点云投影编码的块内偏移优化方法及系统
CN112153391A (zh) * 2019-06-28 2020-12-29 腾讯美国有限责任公司 视频编码的方法、装置、电子设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4246974A4

Also Published As

Publication number Publication date
EP4246974A1 (en) 2023-09-20
US20240080497A1 (en) 2024-03-07
EP4246974A4 (en) 2024-05-01
CN114915794A (zh) 2022-08-16
CN114915794B (zh) 2023-11-14

Similar Documents

Publication Publication Date Title
JP7303992B2 (ja) 点群表現を介したメッシュ圧縮
US20230162402A1 (en) Method and apparatus for processing a point cloud
JP7425899B2 (ja) 点群の符号化及び復号方法
WO2022166963A1 (zh) 基于二维规则化平面投影的点云编解码方法及装置
KR20230021675A (ko) 인코딩 및 디코딩 방법, 인코더, 디코더 및 소프트웨어
JP2001186516A (ja) 画像データの符号化復号化方法及び装置
WO2022166961A1 (zh) 面向大规模点云的二维规则化平面投影及编解码方法
WO2022166968A1 (zh) 基于二维规则化平面投影的点云编解码方法及装置
WO2022166967A1 (zh) 基于二维规则化平面投影的点云编解码方法及装置
WO2022166969A1 (zh) 基于二维规则化平面投影的点云序列编解码方法及装置
WO2022166966A1 (zh) 基于二维规则化平面投影的点云编解码方法及装置
WO2022166958A1 (zh) 基于二维规则化平面投影的点云编解码方法及装置
CN117581537A (zh) 用于对点云进行编码和解码的方法
WO2023123284A1 (zh) 一种解码方法、编码方法、解码器、编码器及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22749245

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18268466

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2022749245

Country of ref document: EP

Effective date: 20230615

NENP Non-entry into the national phase

Ref country code: DE