WO2022116118A1 - Procédé de prédiction, encodeur, décodeur et support de stockage - Google Patents
Procédé de prédiction, encodeur, décodeur et support de stockage Download PDFInfo
- Publication number
- WO2022116118A1 WO2022116118A1 PCT/CN2020/133707 CN2020133707W WO2022116118A1 WO 2022116118 A1 WO2022116118 A1 WO 2022116118A1 CN 2020133707 W CN2020133707 W CN 2020133707W WO 2022116118 A1 WO2022116118 A1 WO 2022116118A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- point
- points
- repetition
- repeating
- point cloud
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 162
- 238000004590 computer program Methods 0.000 claims description 15
- 230000001174 ascending effect Effects 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 16
- 230000008569 process Effects 0.000 description 15
- 238000002310 reflectometry Methods 0.000 description 9
- 230000006835 compression Effects 0.000 description 8
- 238000007906 compression Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 6
- 238000013139 quantization Methods 0.000 description 5
- 230000003252 repetitive effect Effects 0.000 description 5
- 230000007423 decrease Effects 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 101000638069 Homo sapiens Transmembrane channel-like protein 2 Proteins 0.000 description 1
- 241000023320 Luma <angiosperm> Species 0.000 description 1
- 102100032054 Transmembrane channel-like protein 2 Human genes 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000009365 direct transmission Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
Definitions
- the embodiments of the present application relate to the technical field of encoding and decoding, and in particular, to a prediction method, an encoder, a decoder, and a storage medium.
- G-PCC Geometry-based Point Cloud Compression
- the repeated points are predicted according to the following:
- the original point cloud order is used to select the prediction reference point, but the selected prediction reference point is not the best prediction reference point, thus affecting the prediction performance of repeated points.
- Embodiments of the present application provide a prediction method, an encoder, a decoder, and a storage medium, which utilize the sequence of sorted repetition points to determine a prediction reference point, which is beneficial to reduce output code streams and improve encoding and decoding performance.
- an embodiment of the present application provides a prediction method, which is applied to an encoder, and the method includes:
- Attribute prediction is performed on the repeating point based on the repeating point sequence, and an attribute prediction result of the repeating point is determined.
- an embodiment of the present application provides a prediction method, which is applied to a decoder, and the method includes:
- Attribute prediction is performed on the repeating point based on the repeating point sequence, and an attribute prediction result of the repeating point is determined.
- an encoder comprising:
- the first determination unit is configured to determine the geometric information of the point cloud to be encoded; based on the geometric information of the point cloud to be encoded, determine a set of repeated points; wherein, the set of repeated points includes at least two of the point clouds to be encoded. point;
- a first sorting unit configured to sort the attribute information of the repetition points in the repetition point set, and determine the sequence of the repetition points
- the first predicting unit is configured to perform attribute prediction on the repeated points based on the repeating point sequence, and determine the attribute prediction result of the repeated points.
- an encoder comprising:
- the second determination unit is configured to parse the code stream and determine the reconstructed geometric information of the point cloud to be decoded; based on the geometric information of the point cloud to be decoded, determine a set of repeated points; wherein the set of repeated points includes the to-be-decoded point at least two points in the cloud;
- a second sorting unit configured to sort the attribute information of the repetition points in the repetition point set, and determine the sequence of the repetition points
- the second prediction unit is configured to perform attribute prediction on the repeated points based on the repeating point sequence, and determine an attribute prediction result of the repeated points.
- an embodiment of the present application further provides an encoder, including: a first memory and a first processor; the first memory stores a computer program that can run on the first processor, and the first The prediction method of the encoder is implemented when the processor executes the program.
- an embodiment of the present application further provides a decoder, including: a second memory and a second processor; the second memory stores a computer program that can be run on the second processor, the second memory The prediction method of the decoder is implemented when the processor executes the program.
- an embodiment of the present application provides a storage medium, comprising: a computer program is stored thereon, and when the computer program is executed by the first processor, the prediction method of the encoder is implemented; or, the computer program is When executed by the second processor, the prediction method of the decoder is implemented.
- Embodiments of the present application provide a prediction method, an encoder, a decoder, and a storage medium.
- the method includes: determining geometric information of a point cloud to be encoded/decoded; based on the geometric information of the point cloud to be encoded/decoded, determining a repeating point set; wherein the repeating point set includes at least two points in the point cloud to be encoded/decoded; sorting the attribute information of the repeating points in the repeating point set to determine the repeating point sequence; based on The repeating point sequence performs attribute prediction on the repeating point, and determines the attribute prediction result of the repeating point.
- the repeated points with continuous attribute information are obtained, and the sequence of the sorted repeated points is used to determine the prediction reference point, which is beneficial to reduce the output code stream and improve the encoding and decoding performance.
- FIG. 1 is a block diagram of an exemplary encoding process provided by an embodiment of the present application.
- FIG. 2 is a block diagram of an exemplary decoding process provided by an embodiment of the present application.
- FIG. 3 is a first schematic flowchart of a prediction method provided by an embodiment of the present application.
- FIG. 4 is a second schematic flowchart of the prediction method provided by the embodiment of the present application.
- FIG. 5 is a schematic diagram of a spatial relationship between a current node and a neighbor node provided by the related art
- FIG. 6 is a schematic diagram of a Morton code relationship between neighbor nodes that are coplanar with the current node and have been encoded and decoded within a preset neighbor range provided by the related art;
- FIG. 7 is a schematic diagram of the Morton code relationship between neighbor nodes that are collinear with the current node and have been encoded and decoded within a preset neighbor range provided by the related art;
- FIG. 8 is a third schematic flowchart of the prediction method provided by the embodiment of the present application.
- FIG. 9 is a schematic diagram of a first composition structure of an encoder in an embodiment of the present application.
- FIG. 10 is a schematic diagram of a second composition structure of an encoder in an embodiment of the present application.
- FIG. 11 is a schematic diagram of a first composition structure of a decoder in an embodiment of the present application.
- FIG. 12 is a schematic diagram of a second composition structure of a decoder in an embodiment of the present application.
- first ⁇ second ⁇ third involved in the embodiments of the present application is only to distinguish similar objects, and does not represent a specific ordering of objects. It is understandable that “first ⁇ second ⁇ third” "Where permitted, the specific order or sequence may be interchanged to enable the embodiments of the application described herein to be practiced in sequences other than those illustrated or described herein.
- nouns and terms involved in the embodiments of the present application are described.
- the nouns and terms involved in the embodiments of the present application are applicable to the following explanations: point cloud compression (Point Cloud Compression, PCC), Point Cloud Reference Model (PCRM), Morton code (Morton), bounding box, octree, Red-Green-Blue (RGB), Luminance-Chrominance (YUV), Look Up Table (LUT), Peak Signal to Noise Ratio (PSNR), Audio Video coding Standard (AVS).
- a point cloud refers to a collection of massive three-dimensional points, and the points in the point cloud can include the location information of the point and the attribute information of the point.
- the position information of the point may be three-dimensional coordinate information of the point.
- the position information of the point may also be referred to as the geometric information of the point.
- the attribute information of the points may include color information and/or reflectivity, among others.
- the color information may be information in any color space.
- the color information may be RGB information. Among them, R represents red (Red, R), G represents green (Green, G), and B represents blue (Blue, B).
- the color information may be luminance chrominance (YcbCr, YUV) information. Among them, Y represents brightness (Luma), Cb (U) represents blue color difference, and Cr (V) represents red color difference.
- the points in the point cloud may include the three-dimensional coordinate information of the point and the laser reflection intensity (reflectance) of the point.
- the points in the point cloud may include three-dimensional coordinate information of the point and color information of the point.
- a point cloud is obtained by combining the principles of laser measurement and photogrammetry, and the points in the point cloud may include three-dimensional coordinate information of the point, laser reflection intensity (reflectance) of the point, and color information of the point.
- Point clouds can be divided into:
- the first type of static point cloud that is, the object is static, and the device that obtains the point cloud is also static;
- the second type of dynamic point cloud the object is moving, but the device that obtains the point cloud is stationary;
- the third type of dynamic point cloud acquisition the device that acquires the point cloud is moving.
- point clouds are divided into two categories according to their use:
- Category 1 Machine perception point cloud, which can be used in scenarios such as autonomous navigation systems, real-time inspection systems, geographic information systems, visual sorting robots, and rescue and relief robots;
- Category 2 Human eye perception point cloud, which can be used in point cloud application scenarios such as digital cultural heritage, free viewpoint broadcasting, 3D immersive communication, and 3D immersive interaction.
- the point cloud is a collection of massive points, storing the point cloud will not only consume a lot of memory, but also is not conducive to transmission, and there is no such a large bandwidth to support the direct transmission of the point cloud at the network layer without compression. cloud for compression.
- the point cloud coding framework that can compress the point cloud can be the Geometry Point Cloud Compression (G-PCC) codec framework provided by the Moving Picture Experts Group (MPEG). Or a video-based point cloud compression (Video Point Cloud Compression, V-PCC) codec framework, or the AVS-PCC codec framework provided by the Audio Video Standard (AVS).
- G-PCC codec framework can be used to compress the first type of static point cloud and the third type of dynamically acquired point cloud, and the V-PCC codec framework can be used to compress the second type of dynamic point cloud.
- the G-PCC codec framework is also called point cloud codec TMC13, and the V-PCC codec framework is also called point cloud codec TMC2.
- the input point cloud is divided into slices, and then the slices are independently encoded.
- FIG. 1 is an exemplary coding flow block diagram provided by an embodiment of the present application.
- the geometric information and attribute information corresponding to each point cloud are coded separately in the coding process of the point cloud.
- the coordinates of the geometric position are transformed so that all point clouds are contained in a bounding box, and then quantization is performed.
- This step of quantization mainly plays the role of scaling. , so that the geometric information of a part of the point cloud is the same, so the process of quantizing and removing duplicate points is also called the voxelization process based on parameters to decide whether to remove duplicate points. Then perform octree division on the bounding box.
- the bounding box is divided into eight sub-cubes, and the non-empty sub-cubes (including points in the point cloud) are further divided into eight equal parts until the obtained leaf nodes are divided into eight equal parts.
- the point is a 1x1x1 unit cube
- the division is stopped, and the points in the leaf nodes are entropy encoded to generate a geometric code stream.
- attribute encoding is mainly carried out for color and reflectivity information. First, determine whether to perform color space conversion. If color space conversion is performed, convert the color information from RGB color space to YUV color space. Then, the reconstructed point cloud is recolored with the original point cloud so that the unencoded attribute information corresponds to the reconstructed geometric information. In the color information coding, after the point cloud is sorted by Morton code, the subsequent differential prediction is directly performed, and finally the prediction residual is quantized and encoded to generate an attribute code stream.
- the high-level syntax elements, geometric code stream and attribute code stream form the binary code stream of the 3D image, and the encoder sends the binary code stream to the decoder.
- FIG. 2 is an exemplary decoding process block diagram provided by an embodiment of the present application; as shown in FIG. 2 , the decoder obtains a binary code stream, and independently decodes the geometric code stream and the attribute code stream in the binary code stream.
- the geometric information of the point cloud is obtained through entropy decoding-octree reconstruction-inverse coordinate quantization-inverse coordinate translation; when decoding the attribute code stream, through entropy decoding-inverse quantization-attribute Reconstruction-inverse space transformation, get the attribute information of the point cloud, and restore the 3D image model of the point cloud data based on the geometric information and attribute information.
- a point cloud encoding and decoding method provided in the embodiment of the present application mainly acts on the attribute prediction part and the attribute reconstruction part enclosed by the dotted frame in FIG. 1 and FIG. 2 .
- FIG. 3 is a first schematic flowchart of the prediction method provided by the embodiment of the present application. As shown in FIG. 3 , the method may include:
- Step 101 determine the geometric information of the point cloud to be encoded
- the geometric information specifically refers to the three-dimensional geometric coordinates of the point.
- the geometric information is reconstructed geometric information obtained during the encoding process. After the geometric encoding is completed, the geometric information is reconstructed to guide the attribute encoding.
- Step 102 Determine a set of repeated points based on the geometric information of the point cloud to be encoded; wherein, the set of repeated points includes at least two points in the point cloud to be encoded;
- duplicate points refer to the same or similar points in the point cloud.
- the distance between the two points can be used to judge whether the two points coincide.
- the determining the repeated point set based on the geometric information of the point cloud to be encoded includes: determining the distance from the target point in the point cloud to be encoded based on the geometric information of the point cloud to be encoded Points smaller than the distance threshold constitute the repeated point set.
- the distance threshold is greater than or equal to 0.
- the repeated points include points with the same or similar geometric coordinates; when the distance threshold is equal to 0, the repeated points only include points with the same geometric coordinates.
- the determining a set of repeated points based on the geometric information of the point cloud to be encoded includes: based on the geometric information of the point cloud to be encoded, determining that the points with the same geometric information form the repeated points gather. That is, when the distance threshold is equal to 0, it is determined that the points whose geometric information coordinates are exactly equal form a repeated point set.
- the determining the repeated point set based on the geometric information of the point cloud to be encoded includes: generating a Morton code (Morton code) or Hill of the point cloud to be encoded based on the geometric information Bert code (Hilbert code); points that determine the equality of Morton codes or the equality of Hilbert codes constitute the set of repeating points.
- the geometric coordinates can be converted into Morton codes or Hilbert codes, and the points with equal Morton codes or equal Hilbert codes can be determined. Set of repeating points.
- the point cloud to be encoded includes at least one repeated point set, each repeated point set corresponds to the same geometric information, and each repeated point set has the same prediction method.
- Step 103 Sort the attribute information of the repetition points in the repetition point set, and determine the repetition point sequence
- the attribute information and the attribute value are two different expressions of the same encoded data.
- sorting the attribute information of the repetition points in the repetition point set and determining the repetition point sequence includes: sorting the repetition point set in ascending order of attribute information (ie, attribute values) Sort the repeating points in the set to determine the repeating point order; or, sort the repeating points in the repeating point set according to the attribute information in descending order to determine the repeating point order.
- attribute information ie, attribute values
- the existing duplicate points contain multiple attribute values, and the distribution of these attribute values is discontinuous according to the original point cloud order. impact on the prediction performance.
- the repeating point sequence is: 0, 0, 25, 50, 0, 0, 50, 25, 50, 50. If the first N coded repetition points are selected as prediction reference points for the current repetition point in this order, it is not easy to select the best prediction reference point.
- the present application sorts the attribute values of the repeated points according to the attribute values, and the sequence of the repeated points is: 0, 0, 0, 0, 25, 25, 50, 50, 50, 50, and according to this repeated point sequence is the current repeated point When the first N coded repetition points are selected as prediction reference points, the best prediction reference point will be selected, thereby improving coding performance.
- Step 104 Perform attribute prediction on the repeating points based on the repeating point sequence, and determine the attribute prediction result of the repeating points.
- the prediction reference point of the current repetition point to be encoded is determined based on the repetition point sequence, the prediction reference point is the coded repetition point, and the prediction attribute information of the current repetition point is determined according to the reconstruction attribute information of the prediction reference point, and the prediction attribute information is The predicted result for the current repeat point.
- the reconstruction attribute information of the prediction reference point is directly used as the prediction attribute information; when there are at least two prediction reference points, the reconstruction attribute information can be averaged or weighted to obtain Reconstruction attribute information for the current repeat point.
- the method further includes: making a residual between the real attribute information and the predicted attribute information, quantizing the residual, entropy encoding, and writing the code stream.
- the repeated points with continuous attribute information are obtained, and the sequence of the sorted repeated points is used to determine the prediction reference point, which is beneficial to reduce the output code stream and improve the coding performance.
- FIG. 4 is a second schematic flowchart of the prediction method provided by the embodiment of the present application. As shown in FIG. 4 , the method includes:
- Step 201 when the current repetition point is the first repetition point in the repetition point sequence, determine the encoded neighbor point of the current repetition point based on the point cloud sequence of the point cloud to be encoded; take the neighbor point as the current repetition point. Predicted reference points for repeat points;
- the neighbor points include: K neighbor points with the smallest distance from the current duplicate point; wherein, K is a positive integer.
- K may be 3, may also be 6, and may even be 8 or other values.
- the value of K is specifically set according to the actual situation, and no limitation is made here.
- the distance may be Euclidean distance, Manhattan distance, Earth Mover's Distance (EMD), or the like.
- EMD Earth Mover's Distance
- the Manhattan distance is preferred.
- the nearest K neighbor points are selected as the prediction reference points.
- Step 202 when the current repetition point is not the first repetition point in the repetition point sequence, determine the prediction reference point of the current repetition point based on the repetition point sequence;
- the determining the prediction reference point of the current repetition point based on the repetition point sequence includes: determining the top N repetition points of the current repetition point based on the repetition point sequence; wherein N is a positive integer ; Use the first N repetition points as the prediction reference point of the current repetition point.
- N can be an integer such as 1, 2, 3, 4, 5, etc.
- N 1, the prediction is performed by using the reconstruction attribute information of the previous repetition point of the current repetition point.
- the reconstructed attribute information of the previous repetition point is used as the predicted attribute information of the current repetition point.
- the prediction is performed using the reconstructed attribute information of the previous repetition points of the current repetition point.
- the reconstructed attribute information of the previous multiple repetition points is used as the predicted attribute information of the current repetition point. That is to say, when N is greater than 1, starting from the N+1th repetition point, the first N repetition points are taken as the prediction reference points, and for the 2nd to Nth repetition points, the nearest neighbors can be taken at the same time. point and one or more preceding repeat points as prediction reference points.
- the number of prediction reference points may be fixed or not.
- the number of prediction reference points will increase as the number of repeated points decreases, and the number of neighbor points will decrease as the number of repeated points increases.
- the number of prediction reference points is not fixed, the previous When predicting 3 duplicate points, only one nearest neighbor point is selected as the neighbor point, and all the previous coded points are selected for the duplicate point, 0, 1 or 2.
- the determining the prediction reference point of the current repetition point based on the repetition point order includes: determining, based on the point cloud order of the point cloud to be encoded, an encoded neighbor point of the current repetition point; According to the repeating point sequence, the first M repeating points of the current repeating point are determined; wherein, M is a positive integer; the neighbor point and the first M repeating points are used as the prediction reference points of the current repeating point.
- the performing point cloud reordering based on the geometric information of the point cloud to be encoded, and determining the point cloud order of the point cloud to be encoded includes: when the point cloud to be encoded is a dense point cloud, Perform Morton reordering based on the geometric information of the point cloud to be encoded to obtain a first Morton order; add the offset to the geometric information of the point cloud to be encoded and perform Morton reordering to obtain a second Morton order. Morton order;
- determining the encoded neighbor points of the current duplicate point based on the point cloud sequence of the point cloud to be encoded includes: from the first Morton sequence, determining the top P neighbor points of the current duplicate point; From the second Morton sequence, determine the first Q neighbors of the current repeat point; from the first P neighbors and the first Q neighbors, determine the K neighbors with the smallest distance from the current repeat point point; among them, both P and Q are positive integers, and K is less than the sum of P+Q.
- performing point cloud reordering based on the reconstructed geometric information of the point cloud to be encoded, and determining the point cloud order of the point cloud to be encoded includes: when the point cloud to be encoded is a sparse point cloud , perform Hilbert reordering based on the geometric information of the point cloud to be encoded to obtain the Hilbert order;
- determining the encoded neighbor points of the current repeat point based on the point cloud order of the point cloud to be encoded includes: from the Hilbert order, determining the first L neighbor points of the current repeat point; Among the first L neighbor points, determine the K neighbor points with the smallest distance from the current duplicate point; among them, L is a positive integer, and K is less than L.
- the K points closest to the current repetition point are searched among the first L (L can be the first maximum number of neighbors (maxNumOfNeighbours)) points of the current repetition point to be encoded.
- maxNumOfNeighbours is 128, K is 3, and the distance calculation method is the Manhattan distance calculation method.
- Step 203 Based on the reconstructed attribute information of the prediction reference point, perform attribute prediction on the current repetition point.
- the performing attribute prediction on the current repetition point based on the reconstructed attribute information of the prediction reference point includes: when the number of the prediction reference points is greater than 1, reconstructing the attribute of the prediction reference point The information is weighted to obtain the predicted attribute information of the current repetition point.
- the method further includes: determining a weight value for each prediction reference point based on the distance between each prediction reference point and the current repetition point.
- the accuracy of prediction is higher by using a point with a closer distance from the current repeated point. Therefore, determining the weight value of the reference point according to the distance can improve the prediction performance. For example, the smaller the distance, the larger the weight value, and the larger the distance, the smaller the weight value.
- the method further includes: when the prediction reference point includes at least two repeating points, determining the difference between the at least two repeating points based on the positions of the at least two repeating points in the repeating point sequence. Weights.
- the distances of all the repeating points in the repeating point set are 0. Therefore, the distance cannot be used to set the weight value, so the position of the selected repeating point and the current repeating point can be determined according to the repeating point sequence. Set the weight value near and far. For example, the closer to the current repetition point, the larger the weight value, and the farther away from the current repetition point, the smaller the weight value.
- each repetition point uses the first N points for weighted prediction, and the weight is related to the sorting position of the prediction reference point to the current repetition point, that is, the weight of the first Nth repetition point of the current repetition point is 1/N, N The larger the weight, the smaller the weight, and the smaller the N, the greater the weight.
- the attribute prediction method may specifically include the following:
- the Morton code is first used to find the spatial neighbor points of the current point, and then the attributes of the current point are predicted according to the found neighbor points.
- Morton codes can also be called z-order codes because their coding order is in the spatial z order.
- the specific method of calculating the Morton code is described as follows. For the three-dimensional coordinates (x, y, z) represented by d-bit binary numbers for each component, the three components are represented by the following implementation:
- the Morton code M starts from the highest bit for x, y, and z, and arranges x l , y l , z l to the lowest bit in turn.
- the calculation formula of M is as follows:
- the points in the point cloud are arranged in order of Morton code from small to large, and the weight w of each point is set to 1. Represented in computer language, similar to the combination of z
- the Morton code of the current node is first calculated, and then the Morton code of the reference point is located according to the Morton code of the current node.
- the Morton code of the reference point is the smallest Morton code among the neighbor nodes of the current node.
- the three-dimensional coordinates of the current node are (x, y, z)
- the three-dimensional coordinates of the reference point are (x-1, y-1, z-1)
- the calculated Morton code is the smallest. of.
- the coordinate difference (offset) between the reference point and the N neighbor nodes to be searched can be calculated, and the N coordinate difference values can be converted into N Morton codes, and then the N Morton codes Stored in a preset lookup table.
- the spatial range of the neighbor point is shown in Figure 5, the current point to be encoded is the middle gray mark block, and its corresponding preset neighborhood range is 3 ⁇ 3 ⁇ 3 size; wherein, X represents the horizontal coordinate axis, and Y represents the vertical coordinate axis , Z represents the z-coordinate axis.
- the total number of neighbor nodes of the current node is 26; among them, the number of neighbor nodes that are coplanar with the current node is 6, the number of neighbor nodes that are collinear with the current node is 12, and the number of neighbor nodes that are co-located with the current node. There are 8.
- the preset neighborhood range is not limited to 3 ⁇ 3 ⁇ 3, but can also be 4 ⁇ 4 ⁇ 4, 5 ⁇ 5 ⁇ 5, 6 ⁇ 6 ⁇ 6, or even larger than 6 ⁇
- the size of 6 ⁇ 6, etc., is not specifically limited in the embodiments of the present application.
- the search range of the neighbor node is a preset neighborhood range with a size of 3 ⁇ 3 ⁇ 3 of the current node to be encoded.
- the Morton code of the current point node is used to obtain the block with the smallest Morton code value in the neighborhood, and this block is used as the reference block, and then the reference block is used to find the coded code that is coplanar and collinear with the current point node to be coded.
- Decode neighbor nodes The Morton code relationship between neighboring nodes that are coplanar with the current node to be encoded and have been encoded/decoded (may be referred to as “encoded and decoded”) within the preset neighborhood range is shown in FIG. 6 .
- FIG 7 The Morton code relationship between neighbor nodes that are collinear with the current node and have been encoded and decoded within the domain is shown in Figure 7.
- the block filled with 7 represents the current node
- the block filled with 3, 5, and 6 is the neighbor node coplanar with the current node.
- the block filled with 7 still represents the current node
- the block filled with 1, 2, and 4 is the neighbor node that is collinear with the current node.
- the 6 neighbor nodes filled with 1, 2, 3, 4, 5, and 6 must complete the attribute encoding and decoding before the current node, but in addition to these neighbor nodes, they can also include some Collinear neighbor nodes may also complete attribute encoding and decoding before the current node.
- Collinear neighbor nodes may also complete attribute encoding and decoding before the current node.
- FIG. 6 and FIG. 7 currently only three coplanar and three collinear neighbor nodes as shown in FIG. 6 and FIG. 7 are considered, so some coded and decoded collinear neighbor nodes will be missed. Then, after the six coplanar or collinear neighbor nodes cannot be found, it is spatially inaccurate to directly use the reconstructed attribute value of the previously encoded and decoded node whose Morton code is smaller than the current node as the predicted attribute value.
- the method further includes: performing Morton reordering based on the geometric information of the point cloud to be encoded to obtain a Morton order (ie point cloud order); determining a set of repeated points according to the Morton order; The attribute information of the repeating points in the repeating point set is sorted to determine the sequence of the repeating points; the attribute prediction is performed on the repeating points based on the repeating point sequence, and the attribute prediction result of the repeating points is determined.
- a Morton order ie point cloud order
- the attribute information of the repeating points in the repeating point set is sorted to determine the sequence of the repeating points; the attribute prediction is performed on the repeating points based on the repeating point sequence, and the attribute prediction result of the repeating points is determined.
- the current repeat point is the first repeat point in the repeat point sequence
- determine the encoded neighbor point of the current repeat point based on the point cloud sequence of the point cloud to be encoded; take the neighbor point as the current repeat point A prediction reference point; based on the reconstructed attribute information of the prediction reference point, perform attribute prediction on the current repetition point.
- the prediction reference point of the current repetition point is determined based on the repetition point sequence; Attribute prediction.
- the method for determining the prediction reference point includes: based on the repeating point sequence, determining the top N repeating points of the current repeating point; wherein, N is a positive integer; and using the first N repeating points as the prediction of the current repeating point reference point.
- the encoded neighbor points of the current repeat point are determined; based on the repeat point order, the first M repeat points of the current repeat point are determined; wherein M is a positive integer;
- the neighbor point and the first M repetition points are used as prediction reference points for the current repetition point.
- the attribute prediction method may specifically include the following:
- both the repeated points and the non-repeated points need to be reordered first to obtain the point cloud order (ie Morton order or Hilbert order).
- Morton reordering Generate the corresponding Morton code according to the geometric coordinates of the point cloud. After reordering according to the Morton code to obtain Morton order 1 (ie the first Morton order), add all point coordinates to a fixed value to obtain new coordinates , use the new coordinates to generate the Morton code corresponding to the point cloud, and obtain Morton order 2 (ie, the second Morton order) according to the Morton order.
- Morton order 1 ie the first Morton order
- add all point coordinates to a fixed value to obtain new coordinates use the new coordinates to generate the Morton code corresponding to the point cloud, and obtain Morton order 2 (ie, the second Morton order) according to the Morton order.
- Decode according to Morton order 1 find the nearest neighbor point of the point to be decoded, select the top P points of the current point in Morton order 1 as candidates, and select the top Q points of the current point in Morton order 2
- Hilbert reordering Generate the corresponding Hilbert code according to the geometric coordinates of the point cloud, get the Hilbert order after sorting, and encode according to the Hilbert order. In the Hilbert order, find the K points closest to the current point among the top maximum neighbors (maxNumOfNeighbours) points of the current point to be encoded, where maxNumOfNeighbours is 128 by default, K is 3, and the distance calculation method is Manhattan distance.
- the prediction is made according to the geometric position, which is called the "geometric position-based prediction method". If the residual error generated by the geometric position-based prediction method is large, the attribute-based prediction method Value prediction methods usually reduce prediction residuals and improve coding efficiency.
- the attribute value-based prediction method is divided into the following steps:
- PCRM In PCRM, first use the prediction method based on geometric position to perform attribute prediction, and count the obtained residual values, that is, accumulate and record the number of residual values greater than or equal to 3. After encoding 512 points, calculate the average value of residual values greater than or equal to 3 and use this value to compare the preset threshold. When the calculated residual mean value exceeds the threshold, switch to attribute value-based prediction method.
- the threshold for lossless encoding is 50 and the threshold for lossy encoding is 25.
- the prediction method provided by the embodiment of the present application is applied to the existing point cloud prediction.
- the repeated points with continuous attribute information are obtained, and the sequence of the sorted repeated points is used to determine the prediction reference point, which is beneficial to Reduce the output code stream and improve the codec performance at the same time.
- Methods for encoding attribute information of non-repeating points include the following:
- Method 1 Directly encode the attribute information of the first point, perform quantization, entropy encoding, and write it into the code stream; refer to the attribute information of the previous encoded point for the attribute information of the second and subsequent points, and make predictions , the residual is quantized, entropy encoded and written into the code stream;
- Method 2 Directly encode the attribute information of the first point, perform quantization, entropy encoding, and write it into the code stream; refer to the attribute information of the first encoded point for the attribute information of the second point, make predictions, The residual is quantized, entropy encoded, and written into the code stream; for the third point and later points, the attribute information of the specified number of points in the previously encoded points is selected to construct a predicted value, and the prediction is performed, and the residual is quantized and entropy. Encode and write to the codestream.
- FIG. 8 is a third schematic flowchart of the prediction method provided by the embodiment of the present application. As shown in FIG. 8 , the method includes:
- Step 301 Parse the code stream to determine the reconstructed geometric information of the point cloud to be decoded
- the geometric code stream is parsed and the geometric information is predicted to obtain the reconstructed geometric information, the reconstructed geometric information is used to guide attribute decoding, and the geometric information specifically points to the three-dimensional geometric coordinates.
- the parsing process may perform entropy decoding and inverse quantization on the code stream.
- Step 302 Determine a set of repeated points based on the geometric information of the point cloud to be decoded; wherein, the set of repeated points includes at least two points in the point cloud to be decoded;
- duplicate points refer to the same or similar points in the point cloud.
- the distance between the two points can be used to judge whether the two points coincide.
- the determining the repeated point set based on the geometric information of the point cloud to be decoded includes: determining the distance from the target point in the point cloud to be decoded based on the geometric information of the point cloud to be decoded Points smaller than the distance threshold constitute the repeated point set.
- the distance threshold is greater than or equal to 0.
- the repeated points include points with the same or similar geometric coordinates; when the distance threshold is equal to 0, the repeated points only include points with the same geometric coordinates.
- the determining a set of repeated points based on the geometric information of the point cloud to be decoded includes: based on the geometric information of the point cloud to be decoded, determining that the points with the same geometric information form the repeated points gather. That is, when the distance threshold is equal to 0, it is determined that the points whose geometric information coordinates are exactly equal form a repeated point set.
- the determining the repeated point set based on the geometric information of the point cloud to be decoded includes: generating a Morton code (Morton code) or a Hill code of the point cloud to be decoded based on the geometric information Bert code (Hilbert code); points that determine the equality of Morton codes or the equality of Hilbert codes constitute the set of repeating points.
- a Morton code Morton code
- a Hill code of the point cloud to be decoded based on the geometric information Bert code
- points that determine the equality of Morton codes or the equality of Hilbert codes constitute the set of repeating points.
- the geometric coordinates can be converted into Morton codes or Hilbert codes, and the points with equal Morton codes or equal Hilbert codes can be determined. Set of repeating points.
- the point cloud to be decoded includes at least one repeated point set, each repeated point set corresponds to the same geometric information, and each repeated point set has the same prediction method.
- Step 303 Sort the attribute information of the repetition points in the repetition point set, and determine the repetition point sequence
- the attribute information and the attribute value are two different expressions of the same decoded data.
- sorting the attribute information of the repetition points in the repetition point set and determining the repetition point sequence includes: sorting the repetition point set in ascending order of attribute information (ie, attribute values) Sort the repeating points in the set to determine the repeating point order; or, sort the repeating points in the repeating point set according to the attribute information in descending order to determine the repeating point order.
- attribute information ie, attribute values
- the existing duplicate points contain multiple attribute values, and the distribution of these attribute values is discontinuous according to the original point cloud order. impact on the prediction performance.
- the repeating point sequence is: 0, 0, 25, 50, 0, 0, 50, 25, 50, 50. If the first N decoded repetition points are selected as prediction reference points for the current repetition point in this order, it is not easy to select the best prediction reference point.
- the present application sorts the attribute values of the repeated points according to the attribute values, and the sequence of the repeated points is: 0, 0, 0, 0, 25, 25, 50, 50, 50, 50, and according to this repeated point sequence is the current repeated point When the first N decoded repetition points are selected as prediction reference points, the best prediction reference point will be selected, thereby improving decoding performance.
- Step 304 Predict the attributes of the repeated points based on the sequence of the repeated points, and determine the attribute prediction result of the repeated points.
- the prediction reference point of the current repetition point to be decoded is determined based on the repetition point sequence, the prediction reference point is the decoded repetition point, and the prediction attribute information of the current repetition point is determined according to the reconstruction attribute information of the prediction reference point, and the prediction attribute information is The predicted result for the current repeat point.
- the reconstruction attribute information of the prediction reference point is directly used as the prediction attribute information; when there are at least two prediction reference points, the reconstruction attribute information can be averaged or weighted to obtain Reconstruction attribute information for the current repeat point.
- Step 201 when the current repetition point is the first repetition point in the repetition point sequence, determine the decoded neighbor point of the current repetition point based on the point cloud sequence of the point cloud to be decoded; take the neighbor point as the current repetition point. Predicted reference points for repeat points;
- the neighbor points include: K neighbor points with the smallest distance from the current duplicate point; wherein, K is a positive integer.
- the distance can be Euclidean distance, Manhattan distance, and Earth Mover's Distance (EMD).
- EMD Earth Mover's Distance
- the nearest K neighbor points are selected as the prediction reference points.
- Step 202 when the current repetition point is not the first repetition point in the repetition point sequence, determine the prediction reference point of the current repetition point based on the repetition point sequence;
- the determining the prediction reference point of the current repetition point based on the repetition point sequence includes: determining the top N repetition points of the current repetition point based on the repetition point sequence; wherein N is a positive integer ; Use the first N repetition points as the prediction reference point of the current repetition point.
- N can be an integer such as 1, 2, 3, 4, 5, etc.
- N 1, the prediction is performed by using the reconstruction attribute information of the previous repetition point of the current repetition point.
- the reconstructed attribute information of the previous repetition point is used as the predicted attribute information of the current repetition point.
- the prediction is performed using the reconstructed attribute information of the previous repetition points of the current repetition point.
- the reconstructed attribute information of the previous multiple repetition points is used as the predicted attribute information of the current repetition point. That is to say, when N is greater than 1, starting from the N+1th repetition point, the first N repetition points are taken as the prediction reference points, and for the 2nd to Nth repetition points, the nearest neighbors can be taken at the same time. point and one or more preceding repeat points as prediction reference points.
- the number of prediction reference points may be fixed or not.
- the number of prediction reference points When the number of prediction reference points is fixed, the number of neighbor points will increase as the number of repeated points decreases, and the number of neighbor points will decrease as the number of repeated points increases.
- the number of prediction reference points is not fixed, the previous When predicting 3 repeated points, only one nearest neighbor point is selected for the neighbor point, and all the previous points to be decoded are selected for the repeated point, 0, 1 or 2.
- the determining the prediction reference point of the current repetition point based on the repetition point sequence includes: determining the decoded neighbor point of the current repetition point based on the point cloud sequence of the point cloud to be decoded; According to the repeating point sequence, the first M repeating points of the current repeating point are determined; wherein, M is a positive integer; the neighbor point and the first M repeating points are used as the prediction reference points of the current repeating point.
- the performing point cloud reordering based on the geometric information of the point cloud to be decoded, and determining the point cloud order of the point cloud to be decoded includes: when the point cloud to be decoded is a dense point cloud, Perform Morton reordering based on the geometric information of the point cloud to be decoded to obtain a first Morton order; add the offset to the geometric information of the point cloud to be decoded and perform Morton reordering to obtain a second Morton order. Morton order;
- determining the decoded neighbor points of the current duplicate point based on the point cloud sequence of the point cloud to be decoded includes: from the first Morton sequence, determining the top P neighbor points of the current duplicate point; From the second Morton sequence, determine the first Q neighbors of the current repeat point; from the first P neighbors and the first Q neighbors, determine the K neighbors with the smallest distance from the current repeat point point; among them, both P and Q are positive integers, and K is less than the sum of P+Q.
- the performing point cloud reordering based on the reconstructed geometric information of the point cloud to be decoded, and determining the point cloud order of the point cloud to be decoded includes: when the point cloud to be decoded is a sparse point cloud , perform Hilbert reordering based on the geometric information of the point cloud to be decoded to obtain the Hilbert order;
- determining the decoded neighbor points of the current duplicate point based on the point cloud order of the point cloud to be decoded includes: from the Hilbert order, determining the first L neighbor points of the current duplicate point; Among the first L neighbor points, determine the K neighbor points with the smallest distance from the current duplicate point; among them, L is a positive integer, and K is less than L.
- the K points closest to the current repeated point are searched among the first L (L can be the first maximum number of neighbors (maxNumOfNeighbours)) points of the current repeated point to be decoded.
- maxNumOfNeighbours is 128, K is 3, and the distance calculation method is the Manhattan distance calculation method.
- Step 203 Based on the reconstructed attribute information of the prediction reference point, perform attribute prediction on the current repetition point.
- the performing attribute prediction on the current repetition point based on the reconstructed attribute information of the prediction reference point includes: when the number of the prediction reference points is greater than 1, reconstructing the attribute of the prediction reference point The information is weighted to obtain the predicted attribute information of the current repetition point.
- the method further includes: determining a weight value for each prediction reference point based on the distance between each prediction reference point and the current repetition point.
- the accuracy of prediction is higher by using a point with a closer distance from the current repeated point. Therefore, determining the weight value of the reference point according to the distance can improve the prediction performance. For example, the smaller the distance, the larger the weight value, and the larger the distance, the smaller the weight value.
- the method further includes: when the prediction reference point includes at least two repeating points, determining the difference between the at least two repeating points based on the positions of the at least two repeating points in the repeating point sequence. Weights.
- the distances of all the repeating points in the repeating point set are 0. Therefore, the distance cannot be used to set the weight value, so the position of the selected repeating point and the current repeating point can be determined according to the repeating point sequence. Set the weight value near and far. For example, the closer to the current repetition point, the larger the weight value, and the farther away from the current repetition point, the smaller the weight value.
- each repetition point uses the first N points for weighted prediction, and the weight is related to the sorting position of the prediction reference point to the current repetition point, that is, the weight of the first Nth repetition point of the current repetition point is 1/N, N The larger the weight, the smaller the weight, and the smaller the N, the greater the weight.
- the attribute prediction method may specifically include the following:
- the Morton code is first used to find the spatial neighbor points of the current point, and then the attributes of the current point are predicted according to the neighbor points found.
- the spatial range of the neighbor point is shown in Figure 5
- the neighbor point that is coplanar with the current point is shown in Figure 6
- the neighbor point that is collinear with the current point is shown in Figure 7.
- the method further includes: performing Morton reordering based on the geometric information of the point cloud to be decoded to obtain a Morton order (ie, point cloud order); determining a set of repeated points according to the Morton order; The attribute information of the repeating points in the repeating point set is sorted to determine the sequence of the repeating points; the attribute prediction is performed on the repeating points based on the repeating point sequence, and the attribute prediction result of the repeating points is determined.
- a Morton order ie, point cloud order
- the attribute information of the repeating points in the repeating point set is sorted to determine the sequence of the repeating points; the attribute prediction is performed on the repeating points based on the repeating point sequence, and the attribute prediction result of the repeating points is determined.
- the current repetition point is the first repetition point in the repetition point sequence
- determine the decoded neighbor point of the current repetition point based on the point cloud sequence of the point cloud to be decoded; take the neighbor point as the current repetition point A prediction reference point; based on the reconstructed attribute information of the prediction reference point, perform attribute prediction on the current repetition point.
- the prediction reference point of the current repetition point is determined based on the repetition point sequence; Attribute prediction.
- the method for determining the prediction reference point includes: based on the repeating point sequence, determining the top N repeating points of the current repeating point; wherein, N is a positive integer; and using the first N repeating points as the prediction of the current repeating point reference point.
- the decoded neighbor points of the current repeat point are determined; based on the repeat point order, the first M repeat points of the current repeat point are determined; wherein, M is a positive integer;
- the neighbor point and the first M repetition points are used as prediction reference points for the current repetition point.
- the attribute prediction method may specifically include the following:
- Morton reordering is used for single-frame dense point clouds
- Hilbert reordering is used for multi-frame sparse point clouds.
- both the repeated points and the non-repeated points need to be reordered first to obtain the point cloud order (ie Morton order or Hilbert order).
- Morton reordering Generate the corresponding Morton code according to the geometric coordinates of the point cloud. After reordering according to the Morton code to obtain Morton order 1 (ie the first Morton order), add all point coordinates to a fixed value to obtain new coordinates , use the new coordinates to generate the Morton code corresponding to the point cloud, and obtain Morton order 2 (ie, the second Morton order) according to the Morton order.
- Morton order 1 ie the first Morton order
- add all point coordinates to a fixed value to obtain new coordinates use the new coordinates to generate the Morton code corresponding to the point cloud, and obtain Morton order 2 (ie, the second Morton order) according to the Morton order.
- Decode according to Morton order 1 find the nearest neighbor point of the point to be decoded, select the top P points of the current point in Morton order 1 as candidates, and select the top Q points of the current point in Morton order 2
- Hilbert reordering Generate the corresponding Hilbert code according to the geometric coordinates of the point cloud, get the Hilbert order after sorting, and decode according to the Hilbert order. In Hilbert order, find the K points closest to the current point among the points with the maximum number of neighbors (maxNumOfNeighbours) of the current point to be decoded, where maxNumOfNeighbours is 128 by default, K is 3, and the distance calculation method is Manhattan distance.
- the prediction is made according to the geometric position, which is called the "geometric position-based prediction method". If the residual error generated by the geometric position-based prediction method is large, the attribute-based prediction method Value prediction methods usually reduce prediction residuals and improve decoding efficiency.
- the attribute value-based prediction method is divided into the following steps:
- PCRM In PCRM, first use the prediction method based on geometric position to perform attribute prediction, and count the obtained residual values, that is, accumulate and record the number of residual values greater than or equal to 3. After 512 points are decoded, the average value of residual values greater than or equal to 3 is calculated and used to compare the preset threshold. When the calculated residual mean value exceeds the threshold, switch to attribute value-based prediction method.
- the lossless decoding threshold is 50 and the lossy decoding threshold is 25.
- the prediction method provided by the embodiment of the present application is applied to the above point cloud prediction, and by sorting the attribute information of the repeated points, the repeated points with continuous attribute information are obtained, and the sequence of the sorted repeated points is used to determine the prediction reference point, so as to improve the decoding performance.
- FIG. 9 is a schematic diagram of the first composition structure of the encoder in the embodiment of the present application. As shown in FIG. 9 , the encoder 90 includes:
- the first determining unit 901 is configured to determine the geometric information of the point cloud to be encoded; based on the geometric information of the point cloud to be encoded, determine a set of repeated points; wherein the set of repeated points includes at least one of the point clouds to be encoded. two o'clock;
- a first sorting unit 902 configured to sort the attribute information of the repetition points in the repetition point set, and determine the sequence of the repetition points
- the first predicting unit 903 is configured to perform attribute prediction on the repeated points based on the repeating point sequence, and determine an attribute prediction result of the repeated points.
- the first determining unit 901 is configured to, based on the geometric information of the point cloud to be encoded, determine a point in the point cloud to be encoded whose distance from the target point is less than a distance threshold to form the repeated point set .
- the first determining unit 901 is configured to, based on the geometric information of the point cloud to be encoded, determine that the points with the same geometric information form the repeated point set.
- the first determining unit 901 is configured to, based on the geometric information, generate a Morton code or a Hilbert code of the point cloud to be encoded; determine that the Morton code is equal or the Hilbert code is equal
- the points constitute the repeating point set.
- the first sorting unit 902 is configured to sort the repetition points in the repetition point set according to the attribute information in ascending order, and determine the repetition point order; or, according to the attribute information in ascending order
- the small order sorts the repeating points in the repeating point set to determine the repeating point order.
- the first prediction unit 903 is configured to, when the current repetition point is not the first repetition point in the repetition point sequence, determine the prediction reference point of the current repetition point based on the repetition point sequence; The reconstructed attribute information of the prediction reference point is used to predict the attribute of the current repeat point.
- the first prediction unit 903 is configured to determine, based on the sequence of the repetition points, the first N repetition points of the current repetition point; wherein, N is a positive integer; and the first N repetition points are regarded as the current repetition Point's forecast reference point.
- the first prediction unit 903 is configured to determine, based on the point cloud order of the point cloud to be encoded, the encoded neighbor points of the current repeat point; and based on the repeat point order, determine the top M of the current repeat point repeating points; wherein, M is a positive integer; the neighbor point and the first M repeating points are used as the prediction reference points of the current repeating point.
- the first predicting unit 903 is configured to, when the current repetition point is the first repetition point in the repetition point sequence, determine the current repetition point based on the point cloud sequence of the point cloud to be encoded. Encoding a neighbor point; using the neighbor point as a prediction reference point of the current repetition point; and performing attribute prediction on the current repetition point based on the reconstructed attribute information of the prediction reference point.
- the neighbor points include: K neighbor points with the smallest distance from the current duplicate point; wherein, K is a positive integer.
- the distance is the Manhattan distance.
- the first sorting unit 902 is configured to perform point cloud reordering based on the geometric information of the point cloud to be encoded, and determine the point cloud order of the point cloud to be encoded.
- the first sorting unit 902 is configured to perform Morton reordering based on the geometric information of the point cloud to be encoded when the point cloud to be encoded is a dense point cloud to obtain a first Morton order; After adding the offset to the geometric information of the point cloud to be encoded, Morton reordering is performed to obtain a second Morton order;
- the first prediction unit 903 is configured to determine the top P neighbor points of the current repetition point from the first Morton sequence; and determine the top Q neighbor points of the current repetition point from the second Morton sequence Neighbor point; from the first P neighbor points and the first Q neighbor points, determine the K neighbor points with the smallest distance from the current duplicate point; wherein, P and Q are both positive integers, and K is less than the value of P+Q and.
- the first sorting unit 902 is configured to perform Hilbert reordering based on the geometric information of the point cloud to be encoded when the point cloud to be encoded is a sparse point cloud to obtain a Hilbert order;
- the first prediction unit 903 is configured to determine, from the Hilbert sequence, the first L neighbors of the current repetition point; from the first L neighbors, determine the K neighbors with the smallest distance from the current repetition point. point; among them, L is a positive integer, and K is less than L.
- the first prediction unit 903 is configured to perform a weighted operation on the reconstruction attribute information of the prediction reference point when the number of the prediction reference points is greater than 1, to obtain the prediction attribute information of the current repetition point.
- the first prediction unit 903 is configured to determine the weight value of each prediction reference point based on the distance between each prediction reference point and the current repetition point.
- the first prediction unit 903 is configured to, when the prediction reference point includes at least two repetition points, determine at least two repetitions based on the positions of the at least two repetition points in the repetition point sequence Point weight value.
- the first determining unit 901 is configured to determine the reconstructed geometric information of the point cloud to be encoded.
- FIG. 10 is a schematic diagram of a second composition structure of the encoder in the embodiment of the present application.
- the encoder 100 includes: a first memory 1001 and a first processor 1002;
- the first memory 1001 stores a computer program that can be run on the first processor 1002, and the first processor 1002 executes the program, and a point cloud encoding method on the encoder side is used.
- repeating points with the same attribute information are continuously arranged by sorting the repeating points.
- the output code stream can be reduced and the coding performance can be improved.
- FIG. 11 is a schematic diagram of the first composition structure of the decoder in the embodiment of the present application.
- the decoder 110 includes:
- the second determining unit 1101 is configured to parse the code stream and determine the geometric information of the point cloud to be decoded; based on the geometric information of the point cloud to be decoded, determine a set of repeated points; wherein the set of repeated points includes the points to be decoded at least two points in the cloud;
- the second sorting unit 1102 is configured to sort the attribute information of the repetition points in the repetition point set, and determine the sequence of the repetition points;
- the second predicting unit 1103 is configured to perform attribute prediction on the repeated points based on the repeating point sequence, and determine an attribute prediction result of the repeated points.
- the second determining unit 1101 is configured to, based on the geometric information of the point cloud to be decoded, determine a point in the point cloud to be decoded whose distance from the target point is less than a distance threshold to form the repeated point set .
- the second determining unit 1101 is configured to, based on the geometric information of the point cloud to be decoded, determine that the points with the same geometric information form the repeated point set.
- the second determining unit 1101 is configured to, based on the geometric information, generate a Morton code or a Hilbert code of the point cloud to be decoded; determine that the Morton code or the Hilbert code is equal
- the points constitute the repeating point set.
- the second sorting unit 1102 is configured to sort the repetition points in the repetition point set according to the attribute information in ascending order, and determine the repetition point order; or, according to the attribute information in ascending order The small order sorts the repeating points in the repeating point set to determine the repeating point order.
- the second prediction unit 1103 is configured to, when the current repetition point is not the second repetition point in the repetition point sequence, determine the prediction reference point of the current repetition point based on the repetition point sequence; The reconstructed attribute information of the prediction reference point is used to predict the attribute of the current repeat point.
- the second prediction unit 1103 is configured to determine, based on the sequence of the repetition points, the first N repetition points of the current repetition point; wherein, N is a positive integer; and the first N repetition points are used as the current repetition Point's forecast reference point.
- the second prediction unit 1103 is configured to, based on the point cloud order of the point cloud to be decoded, determine the decoded neighbor points of the current duplicate point; based on the repeat point order, determine the top M of the current duplicate point repeating points; wherein, M is a positive integer; the neighbor point and the first M repeating points are used as the prediction reference points of the current repeating point.
- the second predicting unit 1103 is configured to, when the current repetition point is the second repetition point in the repetition point sequence, determine the current repetition point based on the point cloud sequence of the point cloud to be decoded. Decoding the neighbor point; using the neighbor point as the prediction reference point of the current repetition point; and performing attribute prediction on the current repetition point based on the reconstructed attribute information of the prediction reference point.
- the neighbor points include: K neighbor points with the smallest distance from the current duplicate point; wherein, K is a positive integer.
- the distance is the Manhattan distance.
- the second sorting unit 1102 is configured to perform point cloud reordering based on the geometric information of the point cloud to be decoded, and determine the point cloud order of the point cloud to be decoded.
- the second sorting unit 1102 is configured to perform Morton reordering based on the geometric information of the point cloud to be decoded when the point cloud to be decoded is a dense point cloud to obtain a second Morton order; After adding the offset to the geometric information of the point cloud to be decoded, Morton reordering is performed to obtain a second Morton order;
- the second prediction unit 1103 is configured to determine the top P neighbor points of the current repetition point from the second Morton sequence; determine the top Q neighbor points of the current repetition point from the second Morton sequence Neighbor point; from the first P neighbor points and the first Q neighbor points, determine the K neighbor points with the smallest distance from the current duplicate point; wherein, P and Q are both positive integers, and K is less than the value of P+Q and.
- the second sorting unit 1102 is configured to perform Hilbert reordering based on the geometric information of the to-be-decoded point cloud when the point cloud to be decoded is a sparse point cloud to obtain a Hilbert order;
- the second prediction unit 1103 is configured to determine, from the Hilbert sequence, the first L neighbors of the current repetition point; from the first L neighbors, determine the K neighbors with the smallest distance from the current repetition point. point; among them, L is a positive integer, and K is less than L.
- the second prediction unit 1103 is configured to perform a weighted operation on the reconstruction attribute information of the prediction reference point when the number of the prediction reference points is greater than 1 to obtain the prediction attribute information of the current repetition point.
- the second prediction unit 1103 is configured to determine the weight value of each prediction reference point based on the distance between each prediction reference point and the current repetition point.
- the second prediction unit 1103 is configured to, when the prediction reference point includes at least two repetition points, determine at least two repetitions based on the positions of the at least two repetition points in the repetition point sequence Point weight value.
- the second determining unit 1101 is configured to determine the reconstructed geometric information of the point cloud to be decoded.
- FIG. 12 is a schematic diagram of the second composition structure of the decoder in the embodiment of the present application.
- the decoder 120 includes: a second memory 1201 and a second processor 1202;
- the second memory 1201 stores a computer program that can be run on the second processor 1202, and the second processor 1202 executes the program on the decoder side of a point cloud decoding method.
- repeating points with the same attribute information are continuously arranged by sorting repeating points, and decoding performance is improved when repeating point attribute prediction is performed using the sorting result of repeating points.
- a "unit” may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, it may also be a module, and it may also be non-modular.
- each component in this embodiment may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
- the above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of software function modules.
- the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium.
- the technical solution of this embodiment is essentially or Said part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product
- the computer software product is stored in a storage medium and includes several instructions for making a computer device (which can be It is a personal computer, a server, or a network device, etc.) or a processor (processor) that executes all or part of the steps of the method described in this embodiment.
- the aforementioned storage medium includes: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.
- the processor may be an integrated circuit chip with signal processing capabilities.
- each step of the above-mentioned method may be completed by an integrated logic circuit of hardware in the first processor 1803 or an instruction in the form of software.
- the above-mentioned first processor 1803 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a ready-made programmable gate array (Field Programmable Gate Array, FPGA) Or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
- DSP Digital Signal Processor
- ASIC Application Specific Integrated Circuit
- FPGA Field Programmable Gate Array
- the methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed.
- a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
- the steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor.
- the software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art.
- an embodiment of the present application provides a computer storage medium on which a computer program is stored.
- the computer program is executed by the first processor, the point prediction method on the encoder side is implemented; or, the computer program is stored by the second processor.
- the prediction method on the decoder side is implemented.
- Embodiments of the present application provide a prediction method, an encoder, a decoder, and a storage medium.
- the method includes: determining geometric information of a point cloud to be encoded/decoded; based on the geometric information of the point cloud to be encoded/decoded, determining a repeating point set; wherein the repeating point set includes at least two points in the point cloud to be encoded/decoded; sorting the attribute information of the repeating points in the repeating point set to determine the repeating point sequence; based on The repeating point sequence performs attribute prediction on the repeating point, and determines the attribute prediction result of the repeating point.
- the repeated points with continuous attribute information are obtained, and the sequence of the sorted repeated points is used to determine the prediction reference point, which is beneficial to reduce the output code stream and improve the encoding and decoding performance.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
La présente invention concerne un procédé de prédiction, un encodeur, un décodeur et un support de stockage. Le procédé consiste : à déterminer des informations géométriques d'un nuage de points à encoder/décoder ; à déterminer un ensemble de points de répétition sur la base des informations géométriques dudit nuage de points, l'ensemble de points de répétition comprenant au moins deux points dans ledit nuage de points ; à trier des informations d'attribut de points de répétition dans l'ensemble de points de répétition, de façon à déterminer une séquence de points de répétition ; et à réaliser une prédiction d'attribut sur les points de répétition sur la base de la séquence de points de répétition, de façon à déterminer un résultat de prédiction d'attribut des points de répétition. De cette manière, des informations d'attribut de points de répétition sont triées de façon à obtenir des points de répétition ayant des informations d'attribut continues, et un point de référence de prédiction est déterminé en utilisant la séquence des points de répétition triés, de sorte que la réduction des flux de code de sortie est facilitée, et les performances d'encodage et de décodage sont également améliorées.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/133707 WO2022116118A1 (fr) | 2020-12-03 | 2020-12-03 | Procédé de prédiction, encodeur, décodeur et support de stockage |
CN202080107249.6A CN116472556B (zh) | 2020-12-03 | 2020-12-03 | 预测方法、编码器、解码器以及存储介质 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/133707 WO2022116118A1 (fr) | 2020-12-03 | 2020-12-03 | Procédé de prédiction, encodeur, décodeur et support de stockage |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022116118A1 true WO2022116118A1 (fr) | 2022-06-09 |
Family
ID=81852824
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/133707 WO2022116118A1 (fr) | 2020-12-03 | 2020-12-03 | Procédé de prédiction, encodeur, décodeur et support de stockage |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN116472556B (fr) |
WO (1) | WO2022116118A1 (fr) |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019055772A1 (fr) * | 2017-09-14 | 2019-03-21 | Apple Inc. | Compression de nuage de points |
CN109979008A (zh) * | 2019-04-09 | 2019-07-05 | 北京大学深圳研究生院 | 一种基于属性的点云条带划分方法 |
CN110572655A (zh) * | 2019-09-30 | 2019-12-13 | 北京大学深圳研究生院 | 一种基于邻居权重的参数选取和传递的点云属性编码和解码的方法及设备 |
US20200021844A1 (en) * | 2018-07-10 | 2020-01-16 | Tencent America LLC | Method and apparatus for video coding |
WO2020072319A1 (fr) * | 2018-10-02 | 2020-04-09 | Tencent America LLC | Procédé et appareil de codage vidéo |
CN111145090A (zh) * | 2019-11-29 | 2020-05-12 | 鹏城实验室 | 一种点云属性编码方法、解码方法、编码设备及解码设备 |
CN111340899A (zh) * | 2020-02-14 | 2020-06-26 | 福州大学 | 一种彩色点云的压缩采样及重构方法 |
CN111405281A (zh) * | 2020-03-30 | 2020-07-10 | 北京大学深圳研究生院 | 一种点云属性信息的编码方法、解码方法、存储介质及终端设备 |
CN111512342A (zh) * | 2017-12-22 | 2020-08-07 | 三星电子株式会社 | 在点云压缩中处理重复点的方法和装置 |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10911787B2 (en) * | 2018-07-10 | 2021-02-02 | Apple Inc. | Hierarchical point cloud compression |
WO2020123469A1 (fr) * | 2018-12-11 | 2020-06-18 | Futurewei Technologies, Inc. | Encodage d'attribut d'arbre hiérarchique par points médians dans un codage en nuage de points |
-
2020
- 2020-12-03 CN CN202080107249.6A patent/CN116472556B/zh active Active
- 2020-12-03 WO PCT/CN2020/133707 patent/WO2022116118A1/fr active Application Filing
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019055772A1 (fr) * | 2017-09-14 | 2019-03-21 | Apple Inc. | Compression de nuage de points |
CN111512342A (zh) * | 2017-12-22 | 2020-08-07 | 三星电子株式会社 | 在点云压缩中处理重复点的方法和装置 |
US20200021844A1 (en) * | 2018-07-10 | 2020-01-16 | Tencent America LLC | Method and apparatus for video coding |
WO2020072319A1 (fr) * | 2018-10-02 | 2020-04-09 | Tencent America LLC | Procédé et appareil de codage vidéo |
CN109979008A (zh) * | 2019-04-09 | 2019-07-05 | 北京大学深圳研究生院 | 一种基于属性的点云条带划分方法 |
CN110572655A (zh) * | 2019-09-30 | 2019-12-13 | 北京大学深圳研究生院 | 一种基于邻居权重的参数选取和传递的点云属性编码和解码的方法及设备 |
CN111145090A (zh) * | 2019-11-29 | 2020-05-12 | 鹏城实验室 | 一种点云属性编码方法、解码方法、编码设备及解码设备 |
CN111340899A (zh) * | 2020-02-14 | 2020-06-26 | 福州大学 | 一种彩色点云的压缩采样及重构方法 |
CN111405281A (zh) * | 2020-03-30 | 2020-07-10 | 北京大学深圳研究生院 | 一种点云属性信息的编码方法、解码方法、存储介质及终端设备 |
Also Published As
Publication number | Publication date |
---|---|
CN116472556B (zh) | 2024-10-01 |
CN116472556A (zh) | 2023-07-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11450031B2 (en) | Significant coefficient flag encoding for point cloud attribute compression | |
US20240205430A1 (en) | Block-Based Predictive Coding For Point Cloud Compression | |
US20210264640A1 (en) | Adaptive Distance Based Point Cloud Compression | |
US20210258617A1 (en) | Hierarchical Point Cloud Compression | |
US20210118190A1 (en) | Point Cloud Compression | |
US20210103780A1 (en) | Trimming Search Space For Nearest Neighbor Determinations in Point Cloud Compression | |
US10964102B2 (en) | Adaptive sub-band based coding of hierarchical transform coefficients of three-dimensional point cloud | |
US20190081638A1 (en) | Hierarchical point cloud compression | |
TW202249488A (zh) | 點雲屬性的預測方法、裝置及編解碼器 | |
KR20230173695A (ko) | 엔트로피 인코딩, 디코딩 방법 및 장치 | |
WO2022121650A1 (fr) | Procédé de prédiction d'attribut de nuage de points, codeur, décodeur et support d'enregistrement | |
WO2022100173A1 (fr) | Procédé et appareil de compression de trames vidéo et procédé et appareil de décompression de trames vidéo | |
WO2022116118A1 (fr) | Procédé de prédiction, encodeur, décodeur et support de stockage | |
WO2022116117A1 (fr) | Procédé de prédiction, codeur, décodeur et support de stockage informatique | |
KR20240006667A (ko) | 점군 속성 정보 부호화 방법, 복호화 방법, 장치 및 관련 기기 | |
WO2022257528A1 (fr) | Procédé et appareil de prédiction d'attribut de nuage de points, et dispositif associé | |
WO2023024842A1 (fr) | Procédé, appareil et dispositif de codage/décodage de nuage de points, et support de stockage | |
WO2022140937A1 (fr) | Procédé et système de codage de nuage de points, procédé et système de décodage de nuage de points, codeur de nuage de points et décodeur de nuage de points | |
WO2024065406A1 (fr) | Procédés de codage et de décodage, train de bits, codeur, décodeur et support de stockage | |
CN117061770B (zh) | 一种点云处理方法、装置、设备、存储介质及产品 | |
US20240089459A1 (en) | Point cloud encoding processing method and apparatus and point cloud decoding processing method and apparatus | |
WO2023098820A1 (fr) | Procédé et appareil de codage de nuage de points, procédé et appareil de décodage de nuage de points et dispositif de communication | |
WO2024065269A1 (fr) | Procédé et appareil de codage et de décodage de nuage de points, dispositif, et support de stockage | |
WO2023249999A1 (fr) | Système et procédé de codage en nuage de points géométriques | |
CN117221551A (zh) | 点云处理方法、装置、存储介质及电子设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20963964 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202080107249.6 Country of ref document: CN |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20963964 Country of ref document: EP Kind code of ref document: A1 |