CN113453009B - Point cloud space scalable coding geometric reconstruction method based on fitting plane geometric error minimum - Google Patents

Abstract

The invention relates to a point cloud space scalable coding geometric reconstruction method based on minimum fitting plane geometric error, belonging to the technical field of video coding and decoding. And fitting a local plane of the current node by using neighbor node information through a local space formed by each node and 26 neighbor nodes around, constructing a geometric error function according to a point-to-point geometric error measurement principle, and solving a corresponding coordinate value when the geometric error minimum value is solved to serve as a reconstructed geometric coordinate. The problem of the geometric error is great in the space scalable back geometry reconstruction process is solved.

Description

Scalable Coding Geometry Reconstruction of Point Cloud Space Based on Minimal Fitting Plane Geometry Error method

technical field

The present invention relates to the technical field of video encoding and decoding, in particular to a point cloud space scalable encoding geometric reconstruction method based on the minimum fitting plane geometric error.

Background technique

In the point cloud G-PCC encoder framework, after dividing the input point cloud into slices, the slices are encoded independently. In the slice, the geometric information of the point cloud and the attribute information corresponding to the points in the point cloud are encoded separately. The G-PCC encoder first encodes geometric information. The encoder performs coordinate conversion on the geometric information, so that all point clouds are contained in a bounding box (bounding box); and then quantized. This step of quantization mainly plays a role in scaling. Due to the rounding of quantization, the geometric information of some points Similarly, according to the parameters to determine whether to remove duplicate points, not removing duplicate points is called geometric lossless compression, removing duplicate points is called geometric lossy compression, and the process of quantizing and removing duplicate points is also called voxelization process. Next, divide the bounding box based on octree (octree). Both the geometric lossless compression and the geometric lossy compression process have a geometric reconstruction process when the octree division is completed.

In the point cloud G-PCC decoder framework, the geometric bitstream information of the point cloud and the attribute bitstream corresponding to the points in the point cloud are decoded separately. The G-PCC decoder first decodes the geometry bitstream. The decoder performs arithmetic decoding on the geometric bit stream, and decodes the bounding box (bounding box) of the point cloud and the occupancy bits based on the octree (octree) (1 is not empty, 0 is empty); according to the octree when encoding Depending on the depth of the division level, the decoding of geometric information is divided into two frameworks based on octree and trisoup (triangle soup, triangular patch set).

Spatial scalability (spatial scalability) is an important function of G-PCC. It generates point cloud thumbnails by decoding part of the point cloud bitstream information. Currently, it only works in the octree-based geometric information decoding framework. skip Layer (skip layer) is the octree level that decodes less at the decoder. As shown in Figure 1, the octree geometric encoding reaches the Kth layer. When the spatial scalability process is not used for decoding, the K layer is completely decoded; when the spatial scalability process is performed, it is partially decoded, and the decoding ends at the M layer (M=K–skip Layer).

Its function is controlled by the parameter scalable_lifting_enabled_flag. When scalable_lifting_enabled_flag=0, the spatial scalability function is not performed; when scalable_lifting_enabled_flag=1, the spatial scalability function is performed.

In the octree-based geometric information decoding framework, the largest cubic box of the current point cloud in space is calculated according to the bounding box, and then the non-empty sub-cube is continued to be divided into eight equal parts according to the occupancy bits, and the obtained leaves are usually divided When the node is a unit cube of 1×1×1, the division is stopped, but if the decoding process performs Spatial scalability (spatial scalability), it is divided into the specified skip Layer to generate a cube of 2 ^skipLayer × 2 ^skipLayer × 2 ^skipLayer . After that, the geometric coordinates are generated through the cube. For the unit cube of 1×1×1 for the leaf node, the geometric coordinate is the coordinate of the left front and lower corner of the cube, and for the unit cube of 2 ^skipLayer × 2 ^skipLayer × 2 ^skipLayer for the leaf node, the geometry The coordinates adopt different reconstruction strategies according to the skip Layer.

The geometric reconstruction method of scalable coding in the current standard was proposed by Hyejung Hur and Sejin Oh of LG Electronics Inc in the proposal m52315 in January 2020, and was adopted by the MPEG G-PCC standard (ie, MPEG-I (ISO/IEC23090) Part 9) Receive. The technical solution is to adopt different reconstruction strategies according to the different levels of the skip Layer.

The specific implementation of the decoder is described as follows:

When skip Layer=1, the geometric coordinates are the coordinates of the lower left corner of the cube, as shown in Figure 3.Q point position;

When skip Layer>1, the geometric coordinates are the coordinates of the center position of the cube, as shown in Figure 3.P point position;

Currently the technology is in Appendix C of the standard.

C.3 Position shifting process for decoding

When MinGeomNodeSizeLog2 is greater than 1, the process operates on the octree-based geometric information decoding framework for each slice of the current point cloud image as follows:

MinGeomNodeSizeLog2 is the minimum node side length of the current octree, which is numerically equal to skip Layer;

PointCount is the total number of nodes when the current slice is decoded to the MinGeomNodeSizeLog2 level;

PointPos[i][0] is the x-axis coordinate of the reconstructed geometric point of the current node;

PointPos[i][1] is the y-axis coordinate of the reconstructed geometric point of the current node;

PointPos[i][2] is the z-axis coordinate of the reconstructed geometric point of the current node;

mask is the middle mask;

The initial value of PointPos[i][0]PointPos[i][1]PointPos[i][2] is the coordinates of the left/front/bottom corner of the cube whose side length is MinGeomNodeSizeLog2, as shown in Figure 3.Q. After the shift operation of the above code, the value of PointPos[i][0]PointPos[i][1]PointPos[i][2] will be equal to the coordinates of the center position of the current node cube, as shown in Figure 3.P.

At present, G-PCC uses the following two methods to measure the geometric error of the process.

B为伸缩编码后的稀疏点云，A为原始点云，根据所有点i∈B的点对点距离

以N_B为点云B中的点数，定义整个点云的点对点误差D1为：(1) Using the point-to-point distance representation, the calculation process of the point-to-point geometric error measurement is shown in Figure 5. The black point (b _i ) in the figure is the point generated after point cloud scaling encoding and decoding, and the red point (a _j ) is The point closest to it in the original point cloud. The difference between the coordinates of the black point and the red point (E(i,j)=bi _- a _j ) is a point-to-point error vector. The length of the error vector is the point-to-point geometric error, that is:

B is the sparse point cloud after scaling encoding, A is the original point cloud, according to the point-to-point distance of all points i∈B

Taking N _B as the number of points in point cloud B, define the point-to-point error D1 of the entire point cloud as:

这样，点对平面误差计算为:(2) Using the point-to-plane distance representation, project the error vector E(i,j) along the normal direction N _j to obtain a new error vector

Thus, the point-to-plane error is calculated as:

In the above technology, only part of the geometric bitstream information is decoded, and in the node space with a side length of skip Layer, a geometric point represents all points within the range of the node, so the geometric reconstruction process after spatial scalability is a lossy process. the process of.

Contents of the invention

technical problem to be solved

In order to solve the geometric reconstruction process after the existing space scaling, the difference in the distribution of internal points of different nodes is not considered, and there is a problem that the geometric error in the geometric reconstruction process after the space scaling is large. The invention proposes a point cloud space scalable coding geometric reconstruction method based on the minimum fitting plane geometric error.

Technical solutions

A point cloud space scalable coding geometric reconstruction method based on the minimum fitting plane geometric error, characterized in that: through the local space formed by each node and the surrounding 26 neighbor nodes, the information of the neighbor nodes is used to fit the current node The local plane of the point, and then according to the point-to-point geometric error measurement principle, the geometric error function is constructed, and the coordinate value corresponding to the minimum value of the geometric error is obtained as the reconstructed geometric coordinate.

The further technical solution of the present invention is: the surrounding 26 neighbor nodes include 6 co-planar neighbor nodes, 12 co-edge neighbor nodes, and 8 co-point neighbor nodes.

The further technical solution of the present invention is: use the neighbor node information to fit the local plane of the current node, specifically:

1) Plane fitting judgment of K neighbor nodes

Retrieve the 26 neighbors of the current neighbor node, and set the number of neighbors in the 26 neighbors as: neighNum;

When neighNum≥K, use the neighNum nodes existing in the 26 neighbor nodes around the node to calculate the K neighbor nodes closest to the coordinates of the center of the node, and use the coordinates of point P and the octree node The side length octreeSize and the location of the neighbors are used to calculate the coordinates of the center of the neighbor node;

2) Least squares plane fitting

Using the least square method plane fitting principle, the final equation for plane fitting through the center position coordinates of N1～Nk is:

a ₀ *x+a _l *y+a ₂ -z=O

Among them, x _i , y _i , and z _i are the coordinates of the center position of the neighbor node.

The further technical scheme of the present invention is: it is characterized in that described K value and point cloud image plane normal vector search neighbor number are equal, can take 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16.

The further technical scheme of the present invention is: when solving the minimum value of the geometric error, the corresponding coordinate values are used as the reconstructed geometric coordinates as:

Beneficial effect

A point cloud space scalable encoding geometric reconstruction method based on the minimum fitting plane geometric error proposed by the present invention makes full use of the spatial correlation of the point cloud and reduces the geometric reconstruction error after spatial scalability. The geometric information PSNR represents: compared with the prior art, under the same code rate situation, the geometric error brought by the method of the present invention is reduced (PSNR is a positive value) or increased (PSNR is a negative value) compared with the geometric error brought by the prior art. value).

Description of drawings

The drawings are for the purpose of illustrating specific embodiments only and are not to be considered as limitations of the invention, and like reference numerals refer to like parts throughout the drawings.

Figure 1 Schematic diagram of space scalability;

Figure 2G-PCC decoder frame diagram;

Fig. 3 Geometry reconstruction method after scalable coding;

Fig. 4 is a schematic diagram of the position of the present invention in the point cloud G-PCC decoder framework;

Figure 5 schematic diagram of point-to-point error;

Figure 6 is a schematic diagram of the center positions of nodes and neighbor nodes.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

Nouns and terms involved in the present invention are applicable to the following explanations:

1) Point Cloud Compression (PCC)

2) Geometry-based Point Cloud Compression (G-PCC)

3) slice (slice)

4) Bounding box (bounding box)

5) Octree (octree)

6) Intra prediction (intra prediction)

7) Triangle face set (triangle soup, trisoup)

8) Context-based Adaptive Binary Arithmetic Coding (CABAC)

9) block (block)

10) Intersection (vertex)

11) Level of Detail (LOD)

12) Region Adaptive Hierarchal Transform (RAHT)

13) skip layer (skip Layer)

14) Spatial Scalability

15) Moving Picture Experts Group (MPEG)

16) International Standardization Organization (ISO)

17) International Electrotechnical Commission (IEC)

18) The log2 logarithm of the minimum geometry node side length (Minimum Geometry Node Size Log2, MinGeomNodeSizeLog2)

19) Direct encoding points (Direct Point Count, DirectPointCount)

The present invention proposes a brand-new point cloud space scalable coding geometric reconstruction method based on the minimum geometric error of the fitting plane, through the local space formed by each node and the surrounding 26 neighbor nodes, using neighbor node information to fit The local plane of the current node, and then according to the point-to-point geometric error measurement principle, construct the geometric error function, and solve the corresponding coordinate value when the geometric error minimum value is used as the reconstructed geometric coordinate. The specific process is as follows:

N _j is the plane normal vector of point a _j , and it is the normal vector of fitting the local plane by using point a _j and its neighbor points (a total of K points). After performing scalable, it is impossible to obtain the specific details of the point cloud at the decoding end. The position information of the current node cannot calculate the accurate local plane and plane normal vector, so the neighbor occupancy information of the current node is used for approximate fitting of the local plane, and the local plane equation is calculated.

1. K neighbor node plane fitting judgment

Retrieve the 26 neighbors of the current neighbor node, and set the number of neighbors in the 26 neighbors as: neighNum;

When neighNum≥K, use the neighNum nodes existing in the 26 neighbor nodes around the node to calculate the nearest K neighbors from the coordinates of the center position of the node (point P in Figure 6 is the center position of the node) For the node, use the coordinates of point P, the side length of the octree node octreeSize and the location of the neighbors to calculate the coordinates of the center of the neighbor node (N1-Nk in Figure 6 are the coordinates of the center of the neighbor node).

2. Least square method plane fitting

The plane fitting principle of the least square method is adopted, and the plane fitting is carried out through the center position coordinates of N1～Nk. According to the plane fitting principle of the least square method and the general plane formula:

z＝a ₀ *x+a ₁ *y+a ₂

According to the method of least squares:

S＝minΣ[(a ₀ *x _i +a ₁ *y _i +a ₂ )-2 _i ] ²

For the above formula, take the partial derivatives of a ₀ , a ₁ , and a ₂ respectively:

Then convert the above formula into a matrix form after shifting:

Then find out the determinant expression of a ₀ , a ₁ , a ₂ through Cramer's rule for the above formula;

Namely: using the least square method plane fitting principle, the final equation for plane fitting through the center position coordinates of N1～Nk is:

a ₀ *x+a _l *y+a ₂ -z=O

3. Calculation of geometric reconstruction coordinates after plane fitting

According to the prior condition, the coordinate point after geometric reconstruction cannot exceed the node space, that is, the selected point coordinate point P ₁ (x, y, z) satisfies the following constraints:

(The point P ₁ (x, y, z) is a geometric coordinate satisfying the above conditions, and the point P ₁ (x, y, z) is on the fitting plane.)

It can be seen from the point-to-point error measure that the point P ₁ (x, y, z) is in the local space (as shown in the largest cube space in Figure 3), and the point-to-point geometric error equation is:

And because the point P ₁ (x, y, z) satisfies the above fitting plane equation, namely:

a ₀ *x+a _l *y+a ₂ ＝z

So, the above error function is equivalent to:

The partial derivative of f(x,y) in the x direction:

时,when

hour,

The partial derivative of f(x,y) in the y direction:

时，when

hour,

in,

The minimum value of the above error equation is:

also because

z＝a ₀ *x+a ₁ *y+a ₂

So the final geometric coordinates of the selected point are:

This value is the minimum solution of the above geometric error equation.

The following compares the situation when skip Layer=3 different skip layers.

D1-PSNR is calculated as follows:

D2-PSNR is calculated as follows:

(In the formula, p is the peak constant value of each reference point cloud defined in the table, which is determined by the point cloud sequence, as shown in the bold part of Table 2.)

Table 1. Geometric Information PSNR

Table 2. Point cloud sequence peak constant value

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of various equivalents within the technical scope disclosed by the present invention. Modifications or replacements shall all fall within the protection scope of the present invention.

Claims (1)

1. A point cloud space scalable encoding geometric reconstruction method based on fitting plane geometric error minimum, is characterized in that: Step 1: Through the local space formed by each node and the surrounding 26 neighbor nodes, the surrounding 26 neighbor nodes include 6 co-planar neighbor nodes, 12 co-edge neighbor nodes, and 8 co-point neighbor nodes point; Step 2: Use the neighbor node information to fit the local plane of the current node: 1) Plane fitting judgment of K neighbor nodes Retrieve the 26 neighbors of the current node, and set the number of neighbors in the 26 neighbors as: neighNum;

When neighNum≥K, use the neighNum nodes that exist in the 26 neighbor nodes around the node to calculate the K neighbor nodes closest to the center position coordinates of the node, the K value and the point cloud image plane The normal vector searches for the same number of neighbors, and the value of K is any one of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16; use the coordinates of point P, The octree node side length octreeSize and the position of the neighbor calculate the coordinates of the center position of the neighbor node; the coordinates of the P point are the coordinates of the center position of the current node; 2) Least squares plane fitting Using the least square method plane fitting principle, the final equation for plane fitting through the center position coordinates of N1～Nk is: a ₀ *x+a ₁ *y+a ₂ -z=0

Among them, x _i , y _i , z _i are the coordinates of the center position of the neighbor node; Step 3: According to the point-to-point geometric error measurement principle, construct the geometric error function, and solve the coordinate value corresponding to the minimum value of the geometric error as the reconstructed geometric coordinate:

Images