TW202406343A - Encoding method, decoding method, decoder, encoder and computer readable storage medium - Google Patents

Abstract

Embodiments of the present application provide an encoding method, a decoding method, a decoder, an encoder and a computer readable storage medium, capable of improving the encoding efficiency, the decoding efficiency and the compression performance, thereby improving the encoding performance and the decoding performance. The decoding method comprises: parsing a bitstream, and determining occupation indication information corresponding to a first scale point cloud; determining a second scale point cloud, performing up-sampling and division processing on the second scale point cloud, and determining m groups of up-sampled voxels, wherein the second scale point cloud is previous decoded point cloud data corresponding to the first scale point cloud, and m is an integer greater than or equal to 1; sequentially performing occupation probability prediction on the m groups of up-sampled voxels, determining m groups of occupation probabilities, decoding the occupation indication information according to the m groups of occupation probabilities, and determining m groups of respective occupation symbols; and on the basis of the m groups of occupation symbols, determining reconstructed geometric data corresponding to the first scale point cloud.

Description

Codec methods, decoders, encoders, code streams and computer-readable storage media

This application relates to point cloud compression coding and decoding technology, and in particular, to a coding and decoding method, a decoder, an encoder and a computer-readable storage medium.

A point cloud is a collection of points that can store the geometric position and related attribute information of each point to accurately describe objects in space. The amount of point cloud data is huge, and one frame of point cloud can contain millions of points, which also brings great difficulties and challenges to the effective storage and transmission of point clouds. Therefore, compression technology is used to reduce redundant information in point cloud storage to facilitate subsequent processing.

At present, point cloud geometric compression technology based on neural networks is mainly divided into geometric distortion compression and geometric distortion-free compression. For geometric distortion-free compression, on the encoder side, it is often necessary to use the surrounding context such as parent nodes, neighbor nodes, etc. as input. After processing by the neural network (such as convolution, fully connected) layer, each volume in the geometric data of the point cloud is output. The occupancy probability of each voxel is then used, and an entropy encoder is used to convert the voxel occupancy symbol corresponding to the occupancy probability of each voxel into a code stream. Correspondingly, on the decoder side, the occupancy probability of each voxel is predicted according to the same process. Based on the predicted occupancy probability, the entropy decoder is used to decode the voxel occupancy symbols from the code stream and reconstruct the geometric data of the point cloud.

It can be seen that the method of predicting the occupancy probability of each voxel in the point cloud and then encoding and decoding based on the occupancy probability will lead to too long encoding and decoding time, thereby reducing the encoding and decoding efficiency, thereby reducing the encoding and decoding performance.

Embodiments of the present application provide a coding and decoding method, a decoder, an encoder and a computer-readable storage medium, which can improve coding and decoding efficiency and compression performance, thereby improving coding and decoding performance.

The technical solution of this application is implemented as follows:

The embodiment of this application provides a decoding method, including:

Analyze the code stream to determine the occupancy indication information corresponding to the first scale point cloud;

Determine the second scale point cloud, perform upsampling and division processing on the second scale point cloud, and determine m groups of upsampled voxels; wherein the second scale point cloud is the corresponding point cloud of the first scale point cloud. The previous decoded point cloud data; m is an integer greater than or equal to 1;

Predict the occupancy probabilities of the m groups of upsampled voxels in sequence, determine the m groups of occupancy probabilities, decode the occupancy indication information according to the m groups of occupancy probabilities, and determine the respective occupancy symbols of the m groups;

Based on the m sets of occupancy symbols, the reconstructed geometric data corresponding to the first scale point cloud is determined.

The embodiment of the present application provides an encoding method, including:

Perform voxel downsampling on the first-scale point cloud to determine the second-scale point cloud;

Perform upsampling and division processing on the second scale point cloud to determine m groups of upsampled voxels; m is an integer greater than or equal to 1;

Predict the occupancy probability of the m groups of upsampled voxels in sequence, determine the m group of occupancy probabilities, and determine the respective occupancy symbols of the m groups according to the m group of occupancy probabilities;

The m groups of respective occupancy symbols are encoded to determine the occupancy indication information corresponding to the first scale point cloud, and the occupancy indication information is written into the code stream.

This embodiment of the present application provides a decoder, including:

The parsing part is configured to parse the code stream and determine the occupancy indication information corresponding to the first scale point cloud;

The first grouping part is configured to determine the second scale point cloud, perform upsampling and dividing processing on the second scale point cloud, and determine m groups of upsampled voxels; wherein the second scale point cloud is the The previous decoded point cloud data corresponding to the first scale point cloud; m is an integer greater than or equal to 1;

The first prediction part is configured to predict the occupancy probabilities of the m groups of upsampled voxels in sequence, determine the m groups of occupancy probabilities, and decode the occupancy indication information according to the m groups of occupancy probabilities to determine the m groups of occupancy probabilities. respective occupancy symbols;

The decoding part is configured to determine the reconstructed geometric data corresponding to the first scale point cloud based on the m groups of occupied symbols.

An embodiment of the present application provides an encoder, including:

The downsampling part is configured to perform voxel downsampling on the first-scale point cloud and determine the second-scale point cloud;

The second grouping part is configured to perform upsampling and dividing processing on the second scale point cloud, and determine m groups of upsampled voxels; m is an integer greater than or equal to 1;

The second prediction part is configured to predict the occupancy probability of the m groups of upsampled voxels in sequence, determine the m group of occupancy probabilities, and determine the respective occupancy symbols of the m groups according to the m group of occupancy probabilities;

The encoding part is configured to encode the respective occupancy symbols of the m groups, determine the occupancy indication information corresponding to the first scale point cloud, and write the occupancy indication information into the code stream.

This embodiment of the present application provides a code stream, including:

The code stream is generated by bit encoding based on the information to be encoded; wherein the information to be encoded at least includes: occupancy indication information corresponding to the first scale point cloud.

This embodiment of the present application provides a decoder, including:

a first memory configured to store executable instructions;

The first processor is configured to implement the decoding method as described in any one of the above when executing executable instructions stored in the first memory.

An embodiment of the present application provides an encoder, including:

a second memory configured to store executable instructions;

The second processor is configured to implement the encoding method as described in any one of the above when executing the executable instructions stored in the second memory.

Embodiments of the present application provide a computer-readable storage medium that stores executable instructions, which are used to cause the first processor to implement the above decoding method when executed, or to cause the second processor to implement the above encoding when executed. method.

An embodiment of the present application provides a computer program product, including a computer program or an instruction. When the computer program or instruction is executed by a first processor, the decoding method provided by an embodiment of the application is implemented; or, the computer program or instruction is executed by a second processor. When executed by the second processor, the encoding method provided by the embodiment of the present application is implemented.

Embodiments of the present application provide a coding and decoding method, a decoder, an encoder and a computer-readable storage medium, which parse the occupancy indication information corresponding to the first-scale point cloud from the code stream, that is, represent the occupancy indication information in the first-scale point cloud. When encoding information about the occupancy of each voxel, you can upsample and divide the previously decoded second-scale point cloud corresponding to the first-scale point cloud, and upsample the second-scale point cloud to the first scale. The upsampled voxels are divided into m groups, and the occupancy probability predictions are performed on the groups in sequence to obtain m groups of occupancy probabilities. Through group prediction, the prediction speed of occupancy probability is accelerated and the processing efficiency of the prediction process is improved. In this way, m groups of occupancy probabilities are used to decode the occupancy indication information corresponding to the first-scale point cloud, and m groups of decoded occupancy symbols are obtained, and the geometric data of the first-scale point cloud is reconstructed to complete the decoding of the first-scale point cloud. Improved decoding efficiency and decoding performance.

In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be described in further detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. Those of ordinary skill in the art will not make any All other embodiments obtained under the premise of creative work belong to the scope of protection of this application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or a different subset of all possible embodiments, and Can be combined with each other without conflict.

In the following description, the terms "first\second\third" are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understandable that "first\second\third" Where permitted, the specific order or sequence may be interchanged so that the embodiments of the application described herein can be practiced in an order other than that illustrated or described herein.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are applicable to the following explanations.

Voxel: Voxel is the abbreviation of volume element and is the smallest unit of digital data in three-dimensional space division. Through voxels, the 3D space can be meshed and each mesh can be given characteristics. For example, a voxel may be a fixed-sized cube in three-dimensional space. Voxels can be widely used in fields such as three-dimensional imaging, scientific data, and medical imaging.

Point cloud compression algorithms include Geometry-based Point Cloud Compression (G-PCC). Geometric compression in G-PCC is mainly implemented through octree models and/or triangular surface models.

With the development of artificial intelligence technology, neural networks are applied to geometry-based point cloud compression technology. Point cloud geometric compression technology based on neural networks can be roughly divided into geometric distortion compression and distortion-free compression. Among them, the distortion-free compression algorithm mainly focuses on the design of the prediction model of voxel occupancy probability. The data representation of voxels usually uses octree models, volume models, sparse tensor representations, etc. In the design of prediction models, geometric distortion-free compression is often required. On the encoder side, it is often necessary to use parent nodes, neighbor nodes, etc. The surrounding context is the input, and after processing by the neural network (such as convolution, fully connected) layer, the occupancy probability of each voxel in the geometric data of the point cloud is output, and then an entropy encoder is used to convert the occupancy probability of each voxel corresponding to Voxels occupy symbols and are converted into code streams. Correspondingly, on the decoder side, the occupancy probability of each voxel is predicted according to the same process. Based on the predicted occupancy probability, the entropy decoder is used to decode the voxel occupancy symbols from the code stream and reconstruct the geometric data of the point cloud.

It can be seen that the method of predicting the occupancy probability of each voxel in the point cloud and then encoding and decoding based on the occupancy probability often has performance or complexity flaws: for example, the surrounding context of the location of the voxel to be predicted is insufficient. , leading to inaccurate prediction, which in turn leads to poor coding and compression performance; or the occupancy probability prediction for each voxel overuses context information, resulting in redundant code rates, resulting in slow encoding and decoding times and low encoding and decoding efficiency. In summary, the encoding and decoding performance is reduced.

Embodiments of the present application provide a coding and decoding method, a decoder, an encoder and a computer-readable storage medium, which can improve coding and decoding efficiency and coding and decoding performance. In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, a flow chart of G-PCC encoding and a flow chart of G-PCC decoding are first provided. It should be noted that the flow chart of G-PCC encoding and the flow chart of G-PCC decoding described in the embodiment of the present application are only for the purpose of explaining the technical solutions of the embodiment of the present application more clearly, and do not constitute the implementation of the present application. Limitation of the technical solution provided by the example. Those skilled in the art know that with the evolution of point cloud compression technology and the emergence of new business scenarios, the technical solutions provided by the embodiments of this application are also applicable to point cloud encoding and decoding architectures similar to G-PCC. The cloud can be a point cloud in the video, but is not limited to this.

In the point cloud G-PCC encoder framework, after dividing the point cloud of the input three-dimensional image model into slices, each slice is independently encoded.

As shown in the flow chart of G-PCC encoding shown in Figure 1, it is applied to the encoder. For the point cloud data to be encoded, the point cloud data is first divided into multiple slices through strip division. In each slice, the geometric information and attribute information of the point cloud are encoded separately. In the process of geometric encoding, the coordinates of the geometric information are converted so that all point clouds are contained in a bounding box, and then quantized. The quantization mainly plays the role of scaling. Due to the quantization rounding, part of the point cloud The geometric information is the same, and it can be decided whether to remove duplicate points based on parameters. The process of quantizing and removing duplicate points is also called the voxelization process. Then the bounding box is divided into octrees. In the octree-based geometric information encoding process, the bounding box is divided into eight equal parts into eight sub-cubes, and the non-empty sub-cubes (containing points in the point cloud) continue to be divided into eight equal parts until the leaf structure is obtained. The division stops when the point is a 1x1x1 unit cube, and the points in the leaf nodes are arithmetic encoded to generate a binary geometric bit stream, that is, a geometric code stream. In the process of geometric information encoding based on triangle patch set (triangle soup, trisoup), octree division is also required first, but unlike octree-based geometric information encoding, this trisoup does not need to divide the point cloud step by step. It is divided into a unit cube with a side length of 1x1x1, but is divided into sub-blocks (blocks). The division stops when the side length is W. Based on the surface formed by the distribution of point clouds in each block, twelve links between the surface and the block are obtained. At most twelve intersection points (vertex) generated by the edges, the vertex is arithmetic encoded (surface fitting based on the intersection points) to generate a binary geometric bit stream, that is, a geometric code stream. Vertex is also used in the implementation of the geometric reconstruction process, and the reconstructed geometric information is used when encoding the attributes of the point cloud.

During the attribute encoding process, color conversion is performed to convert color information (ie, attribute information) from RGB color space to YUV color space. Then, the point cloud is recolored using the reconstructed geometric information, so that the uncoded attribute information corresponds to the reconstructed geometric information. In the process of color information encoding, there are two main transformation methods. One is the distance-based lifting transformation that relies on Level of Detail (LOD) division, and the other is the direct Region Adaptive Hierarchal Transform (Region Adaptive Hierarchal Transform). , RAHT) transformation, both methods will convert the color information from the spatial domain to the frequency domain, obtain high-frequency coefficients and low-frequency coefficients through transformation, and finally quantize the coefficients (i.e., quantization coefficients). Finally, the octree will be passed After the geometric encoding data of division and surface fitting and the attribute encoding data of quantization coefficient processing are slice-synthesized, the vertex coordinates of each block are sequentially encoded (ie, arithmetic encoding), and a binary attribute bit stream is generated, which is the attribute code stream.

The flow chart of G-PCC decoding shown in Figure 2 is applied to the decoder. The decoder obtains the binary code stream and independently decodes the geometric bit stream (i.e., the geometric code stream) and the attribute bit stream in the binary code stream. When decoding the geometry bit stream, the geometric information of the point cloud is obtained through arithmetic decoding-octree synthesis-surface fitting-reconstructed geometry-inverse coordinate transformation; when decoding the attribute bit stream, through arithmetic decoding -Inverse quantization - LOD-based inverse lifting or RAHT-based inverse transformation - inverse color conversion to obtain the attribute information of the point cloud, and restore the three-dimensional image model of the point cloud data to be encoded based on the geometric information and attribute information.

The encoding method of the embodiment of the present application can be applied to the geometric information encoding process of G-PCC as shown in Figure 1. After the voxelization is completed, voxel down-sampling is performed on the voxelized first-scale point cloud. , determine the second scale point cloud; perform upsampling and division processing on the second scale point cloud, and determine m groups of upsampled voxels; predict the occupancy probability of the m groups of upsampled voxels in sequence, and determine the m group of occupancy probabilities. According to m The group occupancy probability determines the respective occupancy symbols of the m groups, as the data that needs to be entropy encoded, thereby replacing the octree allocation and surface fitting processes in Figure 1. The coding process in this embodiment of the present application may adopt the arithmetic coding method in Figure 1, such as entropy coding. The m groups of respective occupancy symbols are encoded through the arithmetic coding process, the occupancy indication information corresponding to the first scale point cloud is determined, and the geometry bit stream (code stream) is written.

The decoding method of the embodiment of the present application can be applied to the geometric information decoding process of G-PCC as shown in Figure 2. By analyzing the geometric bit stream, the occupancy indication information corresponding to the first scale point cloud is determined; the second scale is determined point cloud, and perform upsampling and division processing on the second-scale point cloud to determine m groups of upsampled voxels; among them, the second-scale point cloud is the previous decoded point cloud data corresponding to the first-scale point cloud; for M groups of upsampled voxels are sequentially predicted for occupancy probabilities to determine m groups of occupancy probabilities, and the occupancy indication information is decoded through the m groups of occupancy probabilities to determine the respective occupancy symbols of the m groups. The encoding process in this embodiment of the present application may adopt the arithmetic decoding method in Figure 2, such as entropy decoding. After arithmetic decoding, there is no need for the processing of octree synthesis and surface fitting in Figure 2. In the process of reconstructing the geometry, the reconstructed geometry data corresponding to the first-scale point cloud is determined directly based on m groups of occupied symbols.

It should be noted that the encoding method and decoding method in the embodiment of the present application can also be used in other point cloud encoding and decoding processes besides G-PCC.

The following describes the encoding method applied to the encoder provided by the embodiment of the present application.

Referring to Figure 3, Figure 3 is an optional flow diagram of the encoding method provided by the embodiment of the present application, which will be described in conjunction with the steps shown in Figure 3. Since the encoding method provided by the embodiments of the present application is applied to the process of generating a geometric bit stream in Figure 1, the first-scale point cloud, second-scale point cloud, etc. in the following embodiments specifically refer to the geometric information of the first-scale point cloud. , geometric information of second-scale point clouds, etc.

S101. Perform voxel downsampling on the first-scale point cloud to determine the second-scale point cloud.

In the embodiment of the present application, before the encoder performs voxel downsampling on the first-scale point cloud, it needs to complete voxelization of the first-scale point cloud to represent the first-scale point cloud in the form of a voxel grid.

In the embodiment of the present application, for the voxelization process, a point in the point cloud can correspond to an occupied voxel (i.e., a non-empty voxel), and an unoccupied voxel (i.e., an empty voxel) represents the volume. There are no points falling into the point cloud at the prime position. In some embodiments, occupied voxels may be marked as 1 and unoccupied voxels may be marked as 0. In this way, the voxelized point cloud can represent the geometric data of the point cloud through the occupancy symbols of voxels at various positions in the voxel grid.

In the embodiment of the present application, the encoder performs voxel downsampling on the voxelized first-scale point cloud to obtain a second-scale point cloud. In some embodiments, the encoder can implement voxel downsampling through pooling, such as using a stride of The maximum pooling layer merges 8 voxels of the first-scale point cloud into 1 voxel of the second-scale point cloud. Each downsampling reduces the size of the point cloud in three dimensions to half of the original size. In this way, the first-scale point cloud can be called a high-scale point cloud relative to the second-scale point cloud, and the second-scale point cloud can be called a low-scale point cloud relative to the first-scale point cloud.

Please refer to Figure 4, which as an example of a first-scale point cloud is a The voxel scale p point cloud, after one voxel downsampling, the second scale point cloud obtained, that is, the scale p-1 point cloud includes voxels. The occupied voxels in the point cloud are represented by solid cubes, which represent the locations of the points in the point cloud. Unoccupied voxels in the point cloud are represented by empty cubes, representing locations where there are no points in the point cloud. As shown in Figure 4, there are 6 voxels occupied on the side of the scale p point cloud facing the paper. The corresponding occupancy symbols are shown in Figure 5A. The occupancy symbol of the voxel is 1, which means that the voxel is occupied, and 0, which means that the volume is occupied. The element is not occupied. After voxel downsampling, 3 of the 4 voxels corresponding to the scale p-1 point cloud are occupied and 1 voxel is unoccupied. The corresponding occupancy symbols are shown in Figure 5B. However, the point cloud in Figure 4 is only exemplary, and the actual point cloud may include more voxels.

S102. Perform upsampling and division processing on the second scale point cloud, and determine m groups of upsampled voxels.

In the embodiment of the present application, the encoder performs upsampling processing on the second scale point cloud, and upsamples the second scale point cloud to the first scale. It can be understood that since the second scale is lower than the first scale, when the second scale point cloud is upsampled, multiple upsampled voxels corresponding to one second scale voxel will be obtained, and each of the multiple upsampled voxels will be obtained. Whether each upsampled voxel contains points in the point cloud, that is, whether it is occupied, needs to be predicted through the subsequent occupancy probability prediction process. Here, the encoder divides each upsampled voxel obtained through upsampling, determines m groups of upsampled voxels, and performs subsequent prediction processing in a group-based form. Among them, m is an integer greater than or equal to 1.

In the embodiment of the present application, after the encoder performs upsampling processing on the second scale point cloud, the upsampled voxels of the first scale obtained by the upsampling are numbered and grouped to implement division processing according to the numbering, and m groups of upsampling are obtained. voxels.

In some embodiments, based on Figure 3, as shown in Figure 6, S102 can be implemented by executing S1021-S1022, as follows:

S1021. Perform voxel upsampling on the second scale point cloud, and determine n upsampled voxels of the first scale; n is an integer greater than 1.

S1022. Number each of the n upsampled voxels, group them based on the number corresponding to each upsampled voxel, and determine m groups of upsampled voxels.

In some embodiments, based on Figure 6, as shown in Figure 7, S1022 can be implemented through S201-S202, as follows:

S201. Number each upsampled voxel among the n upsampled voxels, and determine the number corresponding to each upsampled voxel.

In the embodiment of this application, the encoder can determine the numbering range based on the number of voxels corresponding to the voxels in a second-scale point cloud after upsampling; using the numbering range, n number of voxels obtained by sampling on the second-scale point cloud In upsampling voxels, multiple upsampled voxels corresponding to each voxel in the second scale point cloud among the n upsampled voxels are numbered in the same manner to obtain the number corresponding to each upsampled voxel.

S202. Group n upsampled voxels according to the number of each upsampled voxel, and determine m groups of upsampled voxels.

In some embodiments, among n upsampled voxels, the encoder can divide upsampled voxels with the same number into a group according to the number of each upsampled voxel, and determine m groups of upsampled voxels.

In some embodiments, the encoder may also determine at least two consecutive numbers among the numbers used to mark the upsampled voxels; and based on the number of each upsampled voxel, assign at least two consecutive numbers to n The corresponding upsampled voxels in the sampled voxels are treated as a group, and m groups of upsampled voxels are determined.

In some embodiments, as shown in Figure 8, for the scale p-1 point cloud, that is, the second scale point cloud, After upsampling, n (n=24) upsampled voxels of scale p are obtained, that is, n upsampled voxels of the first scale. It can be seen that one voxel of the scale p-1 point cloud corresponds to 8 voxels of the scale p after upsampling, and the number range can be determined to be 1-8. For a voxel in the scale p-1 point cloud, use the numbers 1-8 to number the corresponding 8 upsampled voxels of the voxel in the scale p point cloud; for the scale p-1 point cloud Three voxels, use the same numbering method to number the 8 upsampled voxels corresponding to the voxels in each p-1 scale point cloud, and obtain 3 upsampled voxels numbered 1 and 3 upsampled voxels numbered 2. Sample voxels..., to 3 upsampled voxels numbered 8. The encoder groups each upsampled voxel according to its number. Figure 8 shows three optional grouping methods, as follows:

The first grouping method: divide upsampled voxels with the same number into one group; for example, group each upsampled voxel numbered 1 into one group, and divide each upsampled voxel numbered 2 into One group, and so on, get 8 groups of upsampled voxels as m groups of upsampled voxels.

The second grouping method: Determine number 1 and number 2 as consecutive numbers, number 3 and number 4 as consecutive numbers, and number 5 to number 8 as consecutive numbers; use the upsampled voxels corresponding to number 1 and number 2 as a Group, the upsampled voxels corresponding to numbers 3 and 4 are taken as a group, the upsampled voxels corresponding to numbers 5 to 8 are taken as a group, and 3 groups of upsampled voxels are obtained as m groups of upsampled voxels.

The third grouping method: treating all n upsampled voxels as a group is equivalent to determining the numbers 1-8 as consecutive numbers, and obtaining 1 group of upsampled voxels as m groups of upsampled voxels.

S103. Predict the occupancy probabilities of the m groups of upsampled voxels in sequence, determine the m groups of occupancy probabilities, and determine the respective occupancy symbols of the m groups according to the m groups of occupancy probabilities.

In the embodiment of the present application, based on the determined m groups of upsampled voxels, the encoder can sequentially predict the occupancy probability of the m groups of upsampled voxels in a progressive manner. When predicting the occupancy probability of the current group of upsampled voxels, the previously predicted prediction results of one or more groups of upsampled voxels (such as the occupancy symbols corresponding to the occupancy probability) are used to predict the occupancy probability of the current group of upsampled voxels. Prediction is made to determine the occupancy probability corresponding to the current group of upsampled voxels; until the m group of occupancy probabilities are determined, the respective occupancy symbols of the m groups are determined based on the m group of occupancy probabilities.

In some embodiments, based on any one of Figure 3, Figure 6, or Figure 7, as shown in Figure 9, S103 can be implemented by executing S1031-S1034, as follows:

S1031. In the m group of upsampled voxels, predict the occupancy probability of the first group of upsampled voxels, determine the first group of occupancy probabilities, use the preset occupancy symbols to represent the first group of occupancy probabilities, and determine the first group of occupancy symbols. .

In the embodiment of the present application, the first group of upsampled voxels does not have the prediction results of other groups of upsampled voxels that have been predicted before. Therefore, the encoder can directly perform the first group of upsampled voxels in the m groups of upsampled voxels. Carry out occupancy probability prediction and determine the first set of occupancy probabilities. Moreover, the preset occupancy symbols are used to represent the first group of occupancy probabilities, that is, the first group of occupancy probabilities are identified by the occupancy symbols, so as to distinguish whether the corresponding voxels represented by the first group of occupancy probabilities are occupied, and determine the first group of occupancy probabilities. A set of occupancy symbols.

S1032. When i is greater than 1 and less than or equal to m, use the completed probability prediction and at least k groups of occupancy symbols corresponding to at least k groups of upsampled voxels represented by the occupancy symbols to predict the occupancy probability of the i-th group of upsampled voxels. , determine the occupancy probability of the i-th group; where i is an integer, and k is an integer greater than or equal to 1 and less than i.

In the embodiment of the present application, when i is greater than 1 and less than or equal to m, that is, for the second to mth group of upsampled voxels in the m group of upsampled voxels, the encoder When predicting occupancy probability for the i-th group of up-sampled voxels in the group of up-sampled voxels, at least k groups of occupancy symbols corresponding to at least k groups of up-sampled voxels represented by the completed probability prediction and occupancy symbols are used to predict the i-th group of up-sampled voxels. Sampling voxels are used to predict the occupancy probability and determine the occupancy probability of the i-th group.

In some embodiments, the encoder can perform feature extraction on the i-th group of upsampled voxels to determine the i-th group of voxel features; determine the i-kth group of occupancy symbols to the i-1th group that have completed probability prediction and occupancy symbol representation The occupancy symbols are at least k groups of occupancy symbols; based on at least k groups of occupancy symbols and combined with the characteristics of the i-th group of voxels, the occupancy probability is predicted for each upsampled voxel in the i-th group of upsampled voxels, and the i-th group of upsampling is determined The occupancy probability corresponding to each upsampled voxel in the voxel is the i-th group occupancy probability.

In some embodiments, the encoder may use the occupancy symbols corresponding to all upsampled voxel groups represented by the occupancy symbols that have completed probability prediction before the i-th group of upsampled voxels as at least the k group of occupancy symbols; it may also be the i-th group of upsampled voxels. Before the group of upsampled voxels, there are at least k groups of occupancy symbols represented by the most recent completed probability prediction and occupancy symbols. The specific selection is based on the actual situation and is not limited in the embodiment of this application.

S1033. Use the preset occupancy symbols to represent the occupancy probability of the i-th group, and determine the occupancy symbols of the i-th group.

In the embodiment of the present application, the encoder uses the first occupancy symbol in the preset occupancy symbol to represent the occupancy probability in the i-th group of occupancy probabilities that is greater than or equal to the preset probability threshold based on the obtained i-th group occupancy probability; and uses the preset occupancy symbol. Assume that the second occupancy symbol in the occupancy symbol represents the occupancy probability of the i-th group of occupancy probabilities that is less than the preset probability threshold, thereby determining the i-th group of occupancy symbols.

For example, the preset probability threshold may be 90%, and the preset occupiers may include 0 and 1. If the occupancy probability of an upsampled voxel in the i-th group of occupancy probabilities is greater than or equal to the preset probability threshold, it means that the upsampled voxel has a greater probability of containing the point cloud midpoint, and 1 is used to represent the occupancy probability, and Let the preset occupancy symbol 1 correspond to the upsampled voxel corresponding to the occupancy probability.

It should be noted that the process of using preset occupancy characters to represent the first group of occupancy probabilities and determining the first group of occupied symbols in S1031 is the same as the method of using the preset occupancy characters to represent the i-th group of occupancy probability and determining the i-th group of occupied symbols in S1033. are consistent.

S1034. Using at least k groups of occupancy symbols including the i-th group of occupancy symbols, continue to perform occupancy probability prediction and occupancy symbol representation on the i+1 group of upsampled voxels until the respective occupancy symbols of the m groups are determined.

In the embodiment of the present application, in the process of occupancy probability prediction and occupancy symbol representation, the encoder uses at least k groups of occupancy symbols including the ith group of occupancy symbols. For example, the above-mentioned occupancy symbols from the 1st group to the ith group of occupancy symbols are obtained , continue to perform occupancy probability prediction and occupancy symbol representation on the i+1th group of upsampled voxels, and obtain the i+1th group of occupancy symbols corresponding to the i+1th group of upsampled voxels. The occupancy probability prediction and occupancy symbol representation are continued in this manner until m groups of respective occupancy symbols are determined.

In some embodiments, the above-mentioned processes of S1021-S1022 and S1031-S1034 can be implemented through a probabilistic prediction model. Here, the probabilistic prediction model is a trained deep learning network that performs feature extraction, upsampling, and probabilistic prediction processing. Illustratively, taking the example of the second scale point cloud (scale p-1 point cloud) shown in Figure 4, the sequence of occupied symbols shown in Figure 5B can be input into the probability prediction model to perform feature extraction, upsampling and Occupancy probability prediction. For unoccupied voxels in the low-scale point cloud (i.e., the second-scale point cloud) (voxels marked as 0 in Figure 5B), multiple upsampling of the high-scale (i.e., the first scale) The sampled voxels are also unoccupied and do not require probabilistic predictions. The high-scale multiple voxels obtained by upsampling the occupied voxels in the low-scale point cloud need to be predicted. by Taking the method of upsampling as an example, one voxel of scale p-1 in Figure 5B will be decomposed into 8 upsampled voxels of scale p. It is uncertain whether these 8 upsampled voxels are occupied, and they all need to be occupied. Probabilistic predictions. The process of occupancy probability prediction can be shown in Figure 10, as follows:

In Figure 10, the scale p-1 point cloud represents the second scale point cloud. The probability prediction model uses a convolutional neural network (Convolution Neural Network, CNN) to upsample the scale p-1 point cloud to obtain the first scale, also That is, n (n=24) upsampled voxels of scale p. Among them, each voxel at scale p-1 corresponds to 8 upsampled voxels at scale p. Among the 24 upsampled voxels, the encoder uses the numbers 1-8 to number the 8 upsampled voxels obtained by decomposing p-1 voxels at each scale one by one to obtain each of the 24 upsampled voxels. The number corresponding to the upsampled voxel. The probabilistic prediction model groups upsampled voxels with the same number as a group, resulting in 8 groups of upsampled voxels, and enters the occupancy probability prediction process.

In the first stage of the occupancy probability prediction process, the probability prediction model uses the CNN network to predict the first group of upsampled voxels, and obtains the occupancy probability corresponding to each upsampled voxel in the first group of upsampled voxels. The probabilistic prediction model performs occupancy symbol representation based on the occupancy probability corresponding to each upsampled voxel in the first group of upsampled voxels, and obtains the occupancy symbol corresponding to each upsampled voxel in the first group of upsampled voxels.

As shown in Figure 10, in the second stage of the occupancy probability prediction process, combined with the occupancy symbols corresponding to each upsampled voxel in the first group of upsampled voxels obtained in the first stage, the second group of upsampled voxels are Occupancy probability prediction and occupancy symbol representation are used to obtain the occupancy symbol corresponding to each upsampled voxel in the second group of upsampled voxels. In the third stage of the occupancy probability prediction process, the occupancy symbols corresponding to each upsampled voxel in the first group of upsampled voxels obtained in the first stage are combined with the occupancy symbols corresponding to each upsampled voxel in the second group of upsampled voxels obtained in the second stage. For the occupancy symbols corresponding to the upsampled voxels, the occupancy probability prediction and occupancy symbol representation are performed for the third group of upsampled voxels, and the occupancy symbols corresponding to each upsampled voxel in the third group of upsampled voxels are obtained. By analogy, the occupancy symbol corresponding to each upsampled voxel in the 8th group of upsampled voxels is obtained. In this way, the occupancy symbol corresponding to each upsampled voxel in the obtained eight groups of upsampled voxels can be used to obtain the geometric information of the first scale point cloud, that is, the scale p point cloud. In this way, by sampling from low scale to high scale, the voxels in the high-scale point cloud are recovered, and then through grouping and occupancy probability prediction, the occupancy probability of the points in the point cloud contained in each high-scale voxel is predicted, and based on the occupancy The probability is to mark each high-scale voxel with a preset occupancy symbol, thereby predicting the geometric data of the high-scale point cloud.

In some embodiments, the occupancy probability corresponding to each of the 24 upsampled voxels predicted through the above eight stages can be shown in Figure 11. Figure 11 shows 24 upsampled voxels of scale p. Among the voxels, the occupancy probability corresponding to each upsampled voxel facing the side of the paper. The predicted probabilities in Figure 11 are only for convenience of explanation and cannot be understood as the results of actual calculations. It can be seen that if the occupancy probability predicted for the actually occupied voxels is closer to 1, and the occupancy probability predicted for the actual unoccupied voxels is closer to 0, the prediction is more accurate. The more accurate the prediction, the less coding data will be obtained when encoding occupancy symbols determined based on the occupancy probability of voxels in high-scale point clouds. That is, the more accurate the probability prediction, the better the compression performance of point cloud geometric information. good.

S104. Encode the respective occupancy symbols of the m groups, determine the occupancy indication information corresponding to the first scale point cloud, and write the occupancy indication information into the code stream.

In the embodiment of the present application, the encoder can perform entropy coding on each group of m groups of occupancy symbols, and determine that the m groups of coded occupancy indication information are the occupancy indication information corresponding to the first scale point cloud.

In the embodiment of this application, whether a voxel in a high-scale point cloud is occupied can be represented by an occupancy symbol, and the occupancy symbol can be directly set to 0 for voxels that do not have occupancy probability prediction. The encoder encodes m groups of respective occupancy symbols, such as a sequence of m groups of occupancy symbols, and obtains the first encoded data corresponding to the first scale point cloud, that is, the occupancy indication information corresponding to the first scale point cloud, thereby achieving high-level Distortionless compression of geometric data from scaled point clouds.

In some embodiments, the entropy encoding may use a Context-based Adaptive Binary Arithmetic Coding (CABAC) algorithm, but is not limited thereto. According to the principle of entropy coding, the more accurate the prediction of occupancy probability is, the smaller the information entropy will be, and the actual code rate and bandwidth will be saved. In this way, the encoder writes the occupancy indication information corresponding to the first-scale point cloud into the code stream and sends it to the decoder. The decoder extracts the occupancy indication information corresponding to the first-scale point cloud and combines it with the previously decoded low-scale point cloud. The geometric data (such as the second-scale point cloud) is fed into the entropy decoder, and the geometric data of the lossless high-scale point cloud can be reconstructed, that is, the reconstructed geometric data corresponding to the first-scale point cloud can be reconstructed.

It can be understood that the group prediction and coding method in the embodiment of the present application, compared with the method of predicting and coding the occupancy probability voxel by voxel based on the parent node and neighbor node of each voxel, on the one hand, greatly reduces the coding complexity, improving the encoding speed and compression performance; on the other hand, as the encoding proceeds, the occupancy symbols of all upsampled voxel groups that have previously completed occupancy probability prediction can be used as context information and jointly input to the convolutional network , indicating that the occupancy probability of the current upsampled voxel group is predicted, thereby improving the accuracy of the occupancy prediction probability prediction, thereby improving the coding accuracy, that is, improving the coding performance.

In some embodiments, the encoder can encode m groups of respective occupied symbols in a parallel manner, thereby further speeding up the encoding speed and improving encoding efficiency and encoding performance.

It can be understood that in the embodiment of the present application, the second scale point cloud is obtained by downsampling the first scale point cloud, and the second scale point cloud is upsampled and divided, and the second scale point cloud is upsampled to In the first scale, the upsampled voxels are divided into m groups, and the occupancy probability predictions are performed on the groups in turn to obtain m groups of occupancy probabilities. In this way, through group prediction, the prediction speed of occupancy probability is accelerated, and the processing efficiency and coding efficiency of the prediction process are improved. In this way, m groups of occupancy symbols corresponding to m groups of occupancy probabilities are encoded, and the occupancy indication information corresponding to the first-scale point cloud is determined and written into the code stream, thereby completing the encoding of the first-scale point cloud and improving coding efficiency and coding performance.

In some embodiments, FIG. 4 and FIG. 10 are an example process of performing down-sampling, up-sampling and dividing processing on a first-scale point cloud, such as a scale p point cloud, and predicting occupancy probability. In actual compression, the first-scale point cloud can be down-sampled more times, such as 2 times, 3 times, 4 times or more, to obtain multiple low-scale point cloud geometric data, and based on multiple The point cloud geometry data at a low scale are subjected to upsampling and division processing, occupancy probability prediction, and occupancy symbol representation and coding processing to obtain occupancy indication information corresponding to multiple scales.

In some embodiments, the encoding method provided by the embodiment of the present application also includes: performing at least S times of voxel downsampling based on the second scale point cloud, and determining the third scale point cloud to the S+2th scale point cloud; S is greater than Or an integer equal to 1; perform voxel upsampling and division processing on the third scale point cloud to the S+2th scale point cloud respectively, and perform occupancy probability prediction and coding on the divided groups in sequence to determine the second scale point The occupancy indication information of the cloud to the occupancy indication information of the S+1th scale point cloud.

In the above embodiment, when the encoder performs two or more voxel downsamplings on the first scale point cloud, the two point clouds before and after each voxel downsampling can be used as the high-scale point cloud and the two point clouds respectively. The low-scale point cloud is divided, processed, occupancy probability predicted and entropy encoded in a similar manner to obtain the occupancy indication information of the second scale point cloud to the occupancy indication information of the S+1th scale point cloud; the encoder converts the first scale point cloud into The occupancy indication information of the point cloud, the occupancy indication information of the second scale point cloud and the occupancy indication information of the S+1th scale point cloud are written into the code stream and sent to the decoder.

It can be understood that the coding method provided by the embodiments of the present application can be repeatedly applied between multiple adjacent scales, and the coding between each group of adjacent scales is independent of each other. Therefore, scale-scalable coding can be flexibly implemented.

The following describes the decoding method applied to the decoder provided by the embodiment of the present application.

Referring to Figure 12, Figure 12 is an optional flow diagram of the decoding method provided by the embodiment of the present application, which will be described in conjunction with the steps shown in Figure 12. Since the decoding method provided by the embodiment of the present application is applied to the process of decoding the geometric bit stream in Figure 2, the first-scale point cloud, the second-scale point cloud, etc. in the following embodiments specifically refer to the first-scale point cloud. Geometric information, geometric information of second-scale point clouds, etc.

S401. Analyze the code stream and determine the occupancy indication information corresponding to the first scale point cloud.

In the embodiment of the present application, the decoder analyzes the received code stream to obtain the first encoded data corresponding to the first scale point cloud, that is, the occupancy indication information corresponding to the first scale point cloud.

S402. Determine the second scale point cloud, perform upsampling and division processing on the second scale point cloud, and determine m groups of upsampled voxels; wherein the second scale point cloud is the previous decoded corresponding to the first scale point cloud. point cloud data; m is an integer greater than or equal to 1.

In the embodiment of the present application, the code stream contains occupancy indication information of point clouds of multiple scales. When decoding, the decoder decodes in order from low scale to high scale. Here, when decoding the occupancy indication information corresponding to the first-scale point cloud, the decoder determines that the previously decoded point cloud data before the first-scale point cloud is the second-scale point cloud; thus, the second-scale point cloud is used As the decoded low-scale point cloud geometric data, the first-scale point cloud is decoded, predicted and reconstructed.

In some embodiments, S402 can be implemented by executing S4021-S4022, as follows:

S4021. Perform voxel upsampling on the second scale point cloud, and determine n upsampled voxels of the first scale; n is an integer greater than 1.

S4022. Number each of the n upsampled voxels, group them based on the number corresponding to each upsampled voxel, and determine m groups of upsampled voxels.

Here, the process of S4021 and S4022 is consistent with the description of the process of S1021 and S1022 in the encoding method, and will not be described again here.

In some embodiments, S4022 can be implemented by executing S501-S502, as follows:

S501. Number each upsampled voxel among the n upsampled voxels, and determine the number corresponding to each upsampled voxel.

S502. Group n upsampled voxels according to the number of each upsampled voxel, and determine m groups of upsampled voxels.

In some embodiments, the decoder may divide the upsampled voxels with the same number into a group according to the number of each upsampled voxel among the n upsampled voxels, and determine m groups of upsampled voxels. Alternatively, at least two consecutive numbers can also be determined; according to the number of each upsampling voxel, the upsampling voxels corresponding to the at least two consecutive numbers in the n upsampling voxels are taken as a group to determine m groups of upsampling voxels. The specific selection is made according to the actual situation, and is not limited by the embodiments of this application.

Here, the process of S501 and S502 is consistent with the description of the process of S201 and S202 in the encoding method, and will not be described again here.

It can be seen that in the embodiment of the present application, the process of upsampling and dividing in the decoder is the same as the process of upsampling and dividing in the encoder.

S403: Predict occupancy probabilities for m groups of upsampled voxels in sequence, determine m groups of occupancy probabilities, decode the occupancy indication information according to m groups of occupancy probabilities, and determine respective occupancy symbols of m groups.

In the embodiment of the present application, the decoder performs occupancy probability prediction on the obtained m groups of upsampled voxels in a progressive manner, and determines m groups of occupancy probabilities. When predicting the occupancy probability of the current group of upsampled voxels, the previously predicted prediction results of one or more groups of upsampled voxels (such as the occupancy symbols corresponding to the occupancy probability) are used to predict the occupancy probability of the current group of upsampled voxels. Prediction is made to determine the occupancy probability corresponding to the current group of upsampled voxels; until the m group of occupancy probabilities are determined, the respective occupancy symbols of the m groups are determined based on the m group of occupancy probabilities.

In some embodiments, based on Figure 12, as shown in Figure 13, S403 can be implemented by executing S4031-S4033, as follows:

S4031. In the m group of upsampled voxels, predict the occupancy probability of the first group of upsampled voxels, determine the occupancy probability of the first group, and decode the occupancy indication information according to the occupancy probability of the first group to determine the occupancy of the first group. symbol.

In S4031, the occupancy probability is predicted for the first group of upsampled voxels. The process of determining the occupancy probability of the first group is consistent with the corresponding process description in S1031 in the encoding method, and will not be described again here. The decoder decodes the occupancy indication information according to the first group of occupancy probabilities to determine the first group of occupancy symbols.

S4032. When i is greater than 1 and less than or equal to m, use at least k groups of decoded occupancy symbols to predict the occupancy probability of the i-th group of upsampled voxels, determine the i-th group of occupancy probabilities, and calculate the occupancy probability based on the i-th group of occupancy symbols. , decode the i-th group of occupancy indication information in the occupancy indication information, and determine the i-th group of occupancy symbols; where i is an integer, and k is an integer greater than or equal to 1 and less than i.

In S4032, when i is greater than 1 and less than or equal to m, that is, for the 2nd to mth group of upsampled voxels in the m group of upsampled voxels, the decoder is upsampling the 2nd to mth groups. When predicting the occupancy probability of the i-th group of upsampled voxels in the voxel, use at least k groups of occupancy symbols corresponding to the decoded at least k groups of upsampled voxels to predict the occupancy probability of the i-th group of upsampled voxels, and determine The occupancy probability of the i-th group.

Here, the decoder uses at least k groups of occupancy symbols corresponding to at least k groups of decoded upsampled voxels to predict the occupancy probability of the i-th group of upsampled voxels, and the process of determining the i-th group of occupancy probabilities is the same as the corresponding probability in S1032 The description of the prediction process is consistent and will not be repeated here.

In some embodiments, the encoder can perform feature extraction on the i-th group of upsampled voxels to determine the i-th group of voxel features; determine that the decoded i-k-th group of occupied symbols to the i-1-th group of occupied symbols are at least k groups Occupancy symbols; Based on at least k groups of occupancy symbols, combined with the characteristics of the i-th group of voxels, predict the occupancy probability for each upsampled voxel in the i-th group of upsampled voxels, and determine the predicted occupancy probability of the i-th group of upsampled voxels. The occupancy probability corresponding to each upsampled voxel is the i-th group occupancy probability.

In some embodiments, the decoder can perform entropy decoding on each occupancy indication information in the i-th group of occupancy indication information according to the i-th group of occupancy probability, and determine the occupancy symbol corresponding to each occupancy indication information in the i-th group of occupancy indication information. , thereby determining the i-th group of occupancy symbols; where the occupancy symbols include: a first occupancy symbol and a second occupancy symbol; the first occupancy symbol represents that the corresponding voxel is occupied; the second occupancy symbol represents that the corresponding voxel is not occupied. For example, the first occupancy character may be 1, and the second occupancy character may be 0. The specific selection is made according to the actual situation, and is not limited by the embodiments of this application.

S4033. Continue the next group of decoding based on the i-th group of occupied symbols until the respective occupied symbols of the m groups are determined.

In the embodiment of the present application, the decoder can continue decoding the next group of symbols based on the i-th group of occupied symbols in the same manner as above until the respective occupied symbols of the m groups are determined. The decoder can use at least k groups of occupancy symbols including the i-th group of occupancy symbols to predict the occupancy probability of the i+1th group of upsampled voxels, and predict the i+th group of occupancy indication information based on the i+1th group of occupancy probabilities. 1 group of occupancy indication information is decoded to determine the i+1th group of occupancy symbols, until the decoding of the occupancy indication information is completed and m groups of respective occupancy symbols are determined.

S404. Based on the m sets of occupancy symbols, determine the reconstructed geometric data corresponding to the first scale point cloud.

In the embodiment of the present application, when the decoder completes the occupancy probability prediction and decoding of m groups of upsampled voxels and obtains m groups of occupancy symbols, it can determine the reconstructed geometric data corresponding to the first scale point cloud based on the m groups of occupancy symbols, and reconstruct Geometric information of first-scale point clouds.

It can be understood that in the embodiment of the present application, the second scale point cloud is upsampled to the first scale by upsampling and dividing the previous decoded second scale point cloud corresponding to the first scale point cloud. The upsampled voxels are divided into m groups, and the occupancy probability predictions are performed on the groups in sequence to obtain m groups of occupancy probabilities. Through group prediction, the prediction speed of occupancy probability is accelerated and the processing efficiency of the prediction process is improved. In this way, m groups of occupancy probabilities are used to decode the occupancy indication information corresponding to the first-scale point cloud, and m groups of decoded occupancy symbols are obtained, and the geometric data of the first-scale point cloud is reconstructed to complete the decoding of the first-scale point cloud. Improved decoding efficiency and decoding performance.

Furthermore, the group prediction and coding method in the embodiment of this application, compared with the method of predicting and decoding the occupancy probability voxel by voxel based on the parent node and neighbor node of each voxel, on the one hand, greatly reduces the decoding complexity , improving the decoding speed; on the other hand, as decoding proceeds, the occupied symbols of all previously decoded upsampled voxel groups can be used as contextual information and jointly input into the convolutional network to predict the current upsampled volume. The occupancy probability of the element group is improved, thereby improving the accuracy of the occupancy prediction probability prediction, thereby improving the decoding accuracy, that is, improving the decoding performance.

In some embodiments, before determining the second scale point cloud, embodiments of the present application may also perform the process of S601-S605 for decoding to obtain the second scale point cloud, as follows:

S601. Analyze the code stream to determine the T-th scale point cloud and the T-1th occupancy indication information corresponding to the T-1th scale point cloud.

In S601, the decoder can obtain the occupancy indication information corresponding to the T-th scale point cloud by parsing the code stream, that is, the T-th occupancy indication information; and determine the previous decoded low-scale point cloud of the T-1th scale point cloud, That is, the T-th scale point cloud is used to decode the T-1th occupancy indication information. Among them, j=T,T-1,…,3; T is an integer greater than or equal to 3;

For example, the decoder can obtain the occupancy indication information corresponding to the second scale point cloud by parsing the code stream, that is, the second occupancy indication information; and determine the decoded third scale point cloud before the second scale point cloud. Thus, the decoded third-scale point cloud is used to decode the second occupancy indication information corresponding to the second-scale point cloud. Alternatively, the decoder can also start decoding from the third occupancy indication information corresponding to the third scale point cloud until the second scale point cloud is decoded: obtain the third occupancy indication information from the code stream and determine the decoded fourth scale point cloud. , use the fourth scale point cloud to decode the third occupancy indication information first to obtain the decoded third scale point cloud; then use the third scale point cloud to decode the second occupancy indication information analyzed in the code stream to obtain Second scale point cloud.

Here, for each decoding process in the above process, you can start from j=T and perform the following processing on each value of j in order from large to small:

S602. Perform voxel upsampling and division processing based on the j-th scale point cloud, perform occupancy probability prediction on the divided groups in sequence, and determine m group occupancy probabilities corresponding to m groups in the j-1th scale point cloud.

In S602, the decoder performs voxel upsampling and division processing based on the decoded j-th scale point cloud through a process similar to the above-mentioned S402-S403, and sequentially performs occupancy probability prediction on the divided groups to determine the j-th- The occupancy probability of m groups corresponding to m groups in the 1-scale point cloud.

S603. Perform entropy decoding on the j-1th occupancy indication information according to the m sets of occupancy probabilities corresponding to the m groups in the j-1th scale point cloud, and determine the m sets of occupancy symbols corresponding to the j-1th occupancy indication information.

S604. Reconstruct the j-1th scale point cloud based on the m sets of occupancy symbols corresponding to the j-1th occupancy indication information.

In S603-S604, the decoder performs entropy decoding on the j-1th occupancy indication information based on the predicted m group of occupancy probabilities corresponding to the m groups in the j-1th scale point cloud, and determines the correspondence of the j-1th occupancy indication information. m groups occupy symbols. Then, the geometric data is reconstructed based on the m sets of occupancy symbols corresponding to the j-1th occupancy indication information, and the j-1th scale point cloud is reconstructed.

S605. Continue to perform the next processing based on the next value of j until j=3, and reconstruct the second scale point cloud.

In S605, the decoder performs the next processing on the next value of j and reconstructs the next scale (more high-scale) point cloud, until j=3, the second-scale point cloud is reconstructed.

In this way, when the decoder decodes adjacent scales, it can use the known geometric data of the previously decoded low-scale point cloud to decode the occupancy indication information of the high-scale point cloud to be decoded, and reconstruct the high-scale point cloud until The second-scale point cloud is reconstructed.

It should be noted that the decoding method in the embodiment of the present application can be applied to a scalable encoding and decoding method, that is, for multiple occupancy indication information of multiple scale point clouds sent by the encoder side, the decoder can decode according to the actual To meet the needs of accuracy, decode and reconstruct point clouds of any scale in the order of decoding from low scale to high scale. For example, the encoder writes and sends the occupancy indication information of the first scale point cloud, the occupancy indication information of the second scale point cloud, and the occupancy indication information of the S+1th scale point cloud in the code stream, and the decoder According to the preset accuracy requirements, the decoding method provided by the embodiment of the present application can be used to decode from the S+1th scale point cloud to the third scale point cloud, and then end the decoding after reconstructing the geometric data of the third scale point cloud, and no longer perform the decoding of the third scale point cloud. The second-scale occupancy indication information is decoded with the occupancy indication information corresponding to the first-scale point cloud. The specific selection is made according to the actual situation, and is not limited by the embodiments of this application.

It can be understood that the decoding method provided by the embodiments of the present application can be repeatedly applied between multiple adjacent scales, and the decoding between each group of adjacent scales is independent of each other. Therefore, scale-scalable decoding can be flexibly implemented.

It should be noted that each decoding process of the above-mentioned decoder takes the decoded low-scale point cloud as known information and decodes the occupancy indication information of the high-scale point cloud. For the first decoding process of the decoder, its known information may be a preset number of unencoded point cloud information sent by the encoder side. The encoder can send a preset number of point cloud information, such as the coordinates of 100 points in the point cloud, as the first known information, directly to the decoder in an unencoded manner, so that the decoder does not need to process the first known information. Decode and directly use the position information of the preset number of points sent by the encoder to reconstruct the point cloud of the corresponding scale to continue the subsequent decoding process.

Next, with reference to Figure 14, the application of the encoding and decoding method provided by the embodiment of the present application in actual scenarios is explained.

As shown in Figure 14, the point cloud image A is first voxelized to obtain the scale p+1 point cloud. During the downsampling process of the encoder, the scale p+1 point cloud is voxelized to obtain the scale p Point cloud. In the upsampling process of the encoder, the point cloud of scale p is upsampled through the CNN network, and the n upsampled voxels of scale p+1 obtained by upsampling are numbered and grouped. For example, they are divided into 8 sets. From group 1 to group 8, the occupancy probability (occupancy probability) of each group of upsampled voxels is predicted sequentially through the CNN network. Here, the process of occupancy probability prediction can be referred to Figure 10.

For example, in the first stage prediction, the first group of upsampled voxels in the eight groups of upsampled voxels is processed by CNN, and the occupancy probability of the first group of upsampled voxels is output, and then the occupancy probability of the first group of upsampled voxels is output according to the first group of upsampled voxels. Voxel occupancy probability is determined to determine the first group of occupation symbols corresponding to the first group of upsampled voxels. As the encoding proceeds, in the prediction process from the second stage to the eighth stage, the input data of each prediction process includes not only the current group of upsampled voxels, but also all other upsampled voxel groups obtained in the previous prediction stage. The occupancy symbol is used as the contextual information of the current group of upsampled voxels, and is jointly input into the CNN to help predict the occupancy probability of the current group of upsampled voxels, and convert the occupancy probability of the current group of upsampled voxels into occupancy symbols. . In this way, the encoder obtains the occupancy symbols of each group of upsampled voxels based on the predicted occupancy probability of each group of upsampled voxels, and performs arithmetic coding based on the occupancy symbols of each group of upsampled voxels to obtain a scale p+1 point cloud. The standard p+1 encoded data is written into the code stream. Among them, the scale p+1 coding data contains 8 sets of codes corresponding to 8 sets of occupied symbols. The encoder performs the next downsampling based on the scale p point cloud to obtain the scale p-1 point cloud, and through the same process as above, predicts and codes based on the scale p-1 point cloud to obtain the scale p coded data of the scale p point cloud. Write code stream. By analogy, the encoder obtains a code stream containing scale p-1 coded data, scale p coded data and scale p+1 coded data of the scale p-1 point cloud, and sends the code stream to the decoder.

In the decoder, decoding is performed from low scale to high scale according to the decoding method provided by the embodiment of the present application. For example, the scale p-1 encoded data is decoded and a scale p-1 point cloud is reconstructed. In the upsampling process of the decoder, the scale p-1 point cloud is upsampled to obtain n upsampled voxels of scale p. The n upsampled voxels of scale p are numbered and grouped with the same number, and the occupancy probability prediction is performed in groups to obtain the occupancy probability corresponding to each group of upsampled voxels of scale p. Using each group of upsampled voxels of scale p Corresponding occupancy probability, perform arithmetic decoding on the scale p coded data analyzed in the code stream, and obtain the occupancy symbols corresponding to each upsampled voxel in the n upsampled voxels of scale p, thereby reconstructing the geometry of the scale p point cloud. data to complete the decoding of standard p-encoded data. Then, based on the decoded point cloud of scale p, the same process is used to decode the encoded data of scale p+1, and reconstruct the geometric data of the point cloud of scale p+1.

It can be understood that compared with the traditional G-PCC method, the embodiment of the present application has good compression performance and parallelism, while maintaining a faster encoding and decoding speed and improving encoding and decoding efficiency. The applicant conducted a comparative test between the encoding and decoding method of the embodiment of the present application and the traditional G-PCC method. The results are shown in Table 1, as follows: Table 1 G-PCC 1-stage 3-stage 8-stage Code rate (bpp) 1.029 1.086 0.703 0.633 Gain - +5.5% -31.7% -38.5% encoding time 5.8 0.7 1.1 1.9 decoding time 3.3 0.7 1.0 1.8

In Table 1, 1-stage indicates that 1 group is used in the grouping process, and occupancy probability prediction and encoding and decoding are performed on 1 group of upsampled voxels; 3-stage indicates that 3 groups are used in the grouping process. method, and sequentially perform occupancy probability prediction and encoding and decoding on 3 groups of upsampled voxels; 8-stage means using 8 groups of grouping methods in the group division processing, and sequentially perform occupancy probability prediction and encoding on 8 groups of upsampled voxels. Decode. It can be seen that compared with the traditional G-PCC method, the encoding time and decoding time of the embodiment of the present application under the three packet prediction methods are much smaller than the encoding and decoding time of the traditional G-PCC method. Moreover, the bit rates (bits per pixel, bpp) under 3-stage and 8-stage are also smaller than the bit rate of traditional G-PCC, resulting in gains of -31.7% under 3-stage and 8-stage respectively. vs. -38.5%. Here, the value of gain is inversely related to performance. The smaller the gain, such as the smaller the value expressed as a negative value, the better the compression performance. This information illustrates the improvement in encoding and decoding performance and saves transmission bandwidth.

This embodiment of the present application provides a decoder 1, as shown in Figure 15, including:

The parsing part 11 is configured to parse the code stream and determine the occupancy indication information corresponding to the first scale point cloud;

The first grouping part 12 is configured to determine the second scale point cloud, perform upsampling and dividing processing on the second scale point cloud, and determine m groups of upsampled voxels; wherein the second scale point cloud is The previous decoded point cloud data corresponding to the first scale point cloud;

The first prediction part 13 is configured to predict the occupancy probabilities of the m groups of upsampled voxels in sequence, determine the m groups of occupancy probabilities, and decode the occupancy indication information according to the m groups of occupancy probabilities to determine the m groups of occupancy probabilities. The occupied symbols of each group; m is an integer greater than or equal to 1;

The decoding part 14 is configured to determine the reconstructed geometric data corresponding to the first scale point cloud based on the m groups of occupied symbols.

In some embodiments, the first grouping part 12 is also configured to perform voxel upsampling on the second scale point cloud, and determine n upsampled voxels of the first scale; n is an integer greater than 1; Each of the n upsampled voxels is numbered, and grouped based on the number corresponding to each upsampled voxel, to determine the m groups of upsampled voxels.

In some embodiments, the first prediction part 13 is also configured to predict the occupancy probability of the first group of upsampled voxels in the m groups of upsampled voxels, determine the occupancy probability of the first group, and determine the occupancy probability according to The first group of occupancy probabilities decodes the occupancy indication information to determine the first group of occupancy symbols; when i is greater than 1 and less than or equal to m, use at least k groups of decoded occupancy symbols to upsample the i-th group The voxel performs occupancy probability prediction, determines the i-th group of occupancy probabilities, and decodes the i-th group of occupancy indication information in the occupancy indication information according to the i-th group of occupancy probabilities to determine the i-th group of occupancy symbols; wherein, i is an integer, k is an integer greater than or equal to 1 and less than i; based on the i-th group of occupied symbols, the next group of decoding is continued until the respective occupied symbols of the m groups are determined.

In some embodiments, the first grouping part 12 is further configured to number each of the n upsampled voxels, and determine the number corresponding to each of the upsampled voxels; According to the number of each upsampled voxel, the n upsampled voxels are grouped to determine the m groups of upsampled voxels.

In some embodiments, the first grouping part 12 is further configured to divide upsampled voxels with the same number among the n upsampled voxels according to the number of each upsampled voxel. A group is determined to determine the m groups of upsampled voxels.

In some embodiments, the first grouping part 12 is further configured to determine at least two consecutive numbers; according to the number of each upsampled voxel, the at least two consecutive numbers are assigned to the n The corresponding upsampled voxels in the sampled voxels are taken as a group to determine the m groups of upsampled voxels.

In some embodiments, the first prediction part 13 is also configured to perform feature extraction on the i-th group of upsampled voxels, determine the i-th group of voxel features, and determine the decoded i-k group of occupied symbols to the i-th group of occupied symbols. The i-1 group of occupancy symbols are the at least k groups of occupancy symbols; according to the at least k groups of occupancy symbols, combined with the i-th group of voxel features, each upsampled volume in the i-th group of upsampled voxels is The occupancy probability is predicted for each voxel, and the predicted occupancy probability corresponding to each upsampled voxel in the i-th group of up-sampled voxels is determined to be the i-th group of occupancy probabilities.

In some embodiments, the parsing part 11, the first grouping part 12, the first prediction part 13 and the decoding part 14 are also configured to parse the code stream and determine the T-th point cloud before determining the second scale point cloud. Scale point cloud, and the T-1th occupancy indication information corresponding to the T-1st scale point cloud; where, j=T, T-1,...,3; T is an integer greater than or equal to 3; starting from j=T , perform the following processing on each value of j in order from large to small: perform voxel upsampling and division processing based on the jth scale point cloud, and predict the occupancy probability of the divided groups in sequence to determine the jth- m groups of occupancy probabilities corresponding to m groups in the 1-scale point cloud; entropy decoding of the j-1th occupancy indication information based on m groups of occupancy probabilities corresponding to m groups in the j-1th scale point cloud, Determine the m group of occupancy symbols corresponding to the j-1th occupancy indication information; reconstruct the j-1th scale point cloud based on the m group of occupancy symbols corresponding to the j-1th occupancy indication information; continue to take the next step based on j The value is processed for the next time until j=3, and the second scale point cloud is reconstructed.

In some embodiments, the decoding part 14 is further configured to perform entropy decoding on each occupancy indication information in the i-th group of occupancy indication information according to the i-th group of occupancy probability, and determine the i-th group of occupancy information. The occupancy symbols corresponding to each occupancy indication information in the indication information are used to determine the i-th group of occupancy symbols; wherein the occupancy symbols include: a first occupancy symbol and a second occupancy symbol; the first occupancy symbol represents the corresponding entity The voxel is occupied; the second occupancy character indicates that the corresponding voxel is not occupied.

In some embodiments, the decoding part 14 is further configured to continue the next group of decoding based on the i-th group of occupied symbols until the respective m groups of occupied symbols are determined, including: using the i-th group of occupied symbols to determine At least k groups of occupancy symbols of the group of occupancy symbols, perform occupancy probability prediction on the i+1th group of upsampled voxels, and predict the i+1th group of occupancy indication information in the occupancy indication information based on the i+1th group of occupancy probabilities. Decoding is performed to determine the i+1th group of occupied symbols, until the decoding of the occupancy indication information is completed, and the respective occupied symbols of the m groups are determined.

In some embodiments, the embodiment of the present application provides an encoder 2, as shown in Figure 16, including:

The downsampling part 21 is configured to perform voxel downsampling on the first scale point cloud and determine the second scale point cloud;

The second grouping part 22 is configured to perform upsampling and dividing processing on the second scale point cloud, and determine m groups of upsampled voxels; m is an integer greater than or equal to 1;

The second prediction part 23 is configured to predict the occupancy probability of the m groups of upsampled voxels in sequence, determine the m group of occupancy probabilities, and determine the respective occupancy symbols of the m groups according to the m group of occupancy probabilities;

The encoding part 24 is configured to encode the respective occupancy symbols of the m groups, determine the occupancy indication information corresponding to the first scale point cloud, and write the occupancy indication information into the code stream.

In some embodiments, the second grouping part 22 is also configured to perform voxel upsampling on the second scale point cloud, and determine n upsampled voxels of the first scale; n is an integer greater than 1; Each of the n upsampled voxels is numbered, and grouped based on the number corresponding to each upsampled voxel, to determine m groups of upsampled voxels.

In some embodiments, the second prediction part 23 is also configured to predict the occupancy probability of the first group of upsampled voxels in the m groups of upsampled voxels, determine the occupancy probability of the first group, and use The preset occupancy symbol represents the first group of occupancy probabilities, and the first group of occupancy symbols is determined; when i is greater than 1 and less than or equal to m, the completed probability is used to predict the number corresponding to at least k groups of upsampled voxels represented by the occupancy symbol. At least k groups of occupancy symbols are used to predict the occupancy probability of the i-th group of upsampled voxels to determine the i-th group of occupancy probabilities; where i is an integer, k is an integer greater than or equal to 1 and less than i; use the preset occupancy symbol represents the i-th group of occupancy probabilities, and determines the i-th group of occupancy symbols; using at least k groups of occupancy symbols including the i-th group of occupancy symbols, continue to predict the occupancy probability and occupancy of the i+1 group of upsampled voxels symbol representation until the respective occupied symbols of the m groups are determined.

In some embodiments, the second grouping part 22 is further configured to number each of the n upsampled voxels, and determine the number corresponding to each of the upsampled voxels;

According to the number of each upsampled voxel, the n upsampled voxels are grouped to determine the m groups of upsampled voxels.

In some embodiments, the second grouping part 22 is further configured to divide upsampled voxels with the same number among the n upsampled voxels according to the number of each upsampled voxel. A group is determined to determine the m groups of upsampled voxels.

In some embodiments, the second grouping part 22 is further configured to determine at least two consecutive numbers; according to the number of each upsampled voxel, the at least two consecutive numbers are placed on the n The corresponding upsampled voxels in the sampled voxels are taken as a group to determine the m groups of upsampled voxels.

In some embodiments, the second prediction part 23 is also configured to perform feature extraction on the i-th group of upsampled voxels, determine the i-th group of voxel features, and determine the i-th group of voxels that have completed probability prediction and occupancy symbol representation. The i-k groups of occupancy symbols to the i-1th group of occupancy symbols are the at least k groups of occupancy symbols; according to the at least k groups of occupancy symbols, combined with the i-th group of voxel features, the i-th group of voxels are upsampled The occupancy probability is predicted for each upsampled voxel in the i-th group of up-sampled voxels, and the occupancy probability corresponding to each up-sampled voxel in the i-th group of up-sampled voxels is determined to be the i-th group of occupancy probabilities.

In some embodiments, the second prediction part 23 is also configured to use the first occupancy symbol in the preset occupancy symbols to represent the occupancy probability in the i-th group of occupancy probabilities that is greater than or equal to the preset probability threshold. ; And use the second occupancy symbol in the preset occupancy symbol to represent the occupancy probability of the i-th group of occupancy probabilities that is less than the preset probability threshold, and determine the i-th group of occupancy symbols.

In some embodiments, the encoding part 24 is also configured to perform entropy encoding on each group of occupancy symbols in the m groups of respective occupancy symbols, and determine that the encoded m groups of occupancy indication information are the first scale point cloud Corresponding occupancy indication information.

In some embodiments, the downsampling part 21, the second grouping part 22, the second prediction part 23 and the encoding part 24 are also configured to perform at least S times of voxel downsampling based on the second scale point cloud, determining The third scale point cloud to the S+2th scale point cloud; S is an integer greater than or equal to 1; perform voxel upsampling and division processing on the third scale point cloud to the S+2th scale point cloud respectively. , and perform occupancy probability prediction and coding on the divided groups in sequence, and determine the occupancy indication information of the second scale point cloud to the occupancy indication information of the S+1th scale point cloud; The occupancy indication information, the occupancy indication information of the second scale point cloud to the occupancy indication information of the S+1th scale point cloud are written into the code stream.

It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

In some embodiments, the embodiment of the present application also provides a decoder. Figure 17 is an optional structural schematic diagram of the decoder 3 provided by the embodiment of the present application. As shown in FIG. 17 , the decoder 3 includes: a first memory 32 and a first processor 33 . Among them, the first memory 32 and the first processor 33 are connected through the first communication bus 34; the first memory 32 is used to store executable instructions; the first processor 33 is used to execute the first memory 32. When storing executable instructions, the decoding method provided by the embodiment of the present application is implemented.

In some embodiments, the embodiment of the present application also provides an encoder. Figure 17 is an optional structural schematic diagram of the encoder 4 provided by the embodiment of the present application. As shown in FIG. 18 , the encoder 4 includes: a second memory 42 and a second processor 43 . Among them, the second memory 42 and the second processor 43 are connected through the second communication bus 44; the second memory 42 is used to store executable instructions; the second processor 43 is used to execute the second memory 42. When storing executable instructions, the coding method provided by the embodiment of the present application is implemented.

Embodiments of the present application provide a computer-readable storage medium storing executable instructions. The executable instructions are stored therein. When the executable instructions are executed by a first processor, they will cause the first processor to execute any of the above implementations of the present application. or, when the executable instruction is executed by the second processor, it will cause the second processor to execute any of the above encoding methods provided by the embodiments of the present application.

In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; it may also include the above memory One or any combination of various devices.

In some embodiments, executable instructions may take the form of programs, software, software modules, scripts, or code written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, And it can be deployed in any form, including deployed as a stand-alone program or deployed as a module, component, subroutine or other unit suitable for use in a computing environment.

As an example, executable instructions may, but do not necessarily correspond to, files in a file system and may be stored as part of a file that holds other programs or data, for example, in a Hyper Text Markup Language (HTML) document. of one or more scripts, stored in a single file dedicated to the program in question, or stored in multiple collaborative files (e.g., files storing one or more modules, subprograms, or code portions).

As examples, executable instructions may be deployed to execute on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed across multiple locations and interconnected by a communications network. execute on.

Those skilled in the art should understand that embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory and optical memory, etc.) containing computer-usable program code.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine that causes instructions to be executed by the processor of the computer or other programmable data processing device. Means are generated for implementing the functions specified in the process or processes of the flowchart diagrams and/or the block or blocks of the block diagrams.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. , the instruction device implements the functions specified in one process or multiple processes of the flow chart and/or one block or multiple blocks of the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby causing the computer or other programmable device to perform a computer-implemented process. The instructions executed on provide steps for implementing the functions specified in a process or processes of the flow diagrams and/or a block or blocks of the block diagrams.

The above descriptions are only preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of this application are included in the protection scope of this application.

1:Decoder 11: Analysis part 12: The first grouping part 13: First prediction part 14: Decoding part 2: Encoder 21: Downsampling part 22:Second grouping part 23: Second prediction part 24: Coding part 3:Decoder 32: First memory 33:First processor 34:First communication bus 4: Encoder 42: Second memory 43: Second processor 44: Second communication bus S101~S104: Steps S1021~S1022: Steps S1031~S1034: steps S201~S202: steps S401~S404: steps S4031~S4033: steps

Figure 1 is a flow chart of G-PCC encoding;

Figure 2 is a flow chart of G-PCC decoding;

Figure 3 is an optional flow diagram of the encoding method provided by the embodiment of the present application;

Figure 4 is a schematic diagram of an optional process of voxel downsampling provided by an embodiment of the present application;

Figure 5A is an optional schematic diagram of the occupancy symbols of voxels in the scale p point cloud provided by the embodiment of the present application;

Figure 5B is an optional schematic diagram of the occupancy symbols of voxels in the scale p-1 point cloud provided by the embodiment of the present application;

Figure 6 is an optional flow diagram of the encoding method provided by the embodiment of the present application;

Figure 7 is an optional flow diagram of the encoding method provided by the embodiment of the present application;

Figure 8 is a schematic diagram of an optional process of upsampling and dividing provided by the embodiment of the present application;

Figure 9 is an optional flow diagram of the encoding method provided by the embodiment of the present application;

Figure 10 is an optional schematic diagram of the group sequential occupancy probability prediction process provided by the embodiment of the present application;

Figure 11 is an optional effect diagram of the occupancy probability of voxels in the point cloud provided by the embodiment of the present application;

Figure 12 is an optional flow diagram of the decoding method provided by the embodiment of the present application;

Figure 13 is an optional flow diagram of the decoding method provided by the embodiment of the present application;

Figure 14 is an optional process schematic diagram of applying the encoding and decoding method provided by the embodiment of the present application to an actual scenario;

Figure 15 is an optional structural schematic diagram of a decoder provided by an embodiment of the present application;

Figure 16 is an optional structural schematic diagram of an encoder provided by an embodiment of the present application;

Figure 17 is an optional structural schematic diagram of a decoder provided by an embodiment of the present application;

Figure 18 is an optional structural schematic diagram of an encoder provided by an embodiment of the present application.

S101~S104: Steps

Claims (26)

A decoding method including: Analyze the code stream to determine the occupancy indication information corresponding to the first scale point cloud; Determine the second scale point cloud, perform upsampling and division processing on the second scale point cloud, and determine m groups of upsampled voxels; wherein the second scale point cloud is the corresponding point cloud of the first scale point cloud. The previous decoded point cloud data; m is an integer greater than or equal to 1; Predict the occupancy probabilities of the m groups of upsampled voxels in sequence, determine the m groups of occupancy probabilities, decode the occupancy indication information according to the m groups of occupancy probabilities, and determine the respective occupancy symbols of the m groups; Based on the m sets of occupancy symbols, the reconstructed geometric data corresponding to the first scale point cloud is determined. The method according to claim 1, wherein the upsampling and dividing process on the second scale point cloud to determine m groups of upsampled voxels includes: Perform voxel upsampling on the second scale point cloud to determine n upsampled voxels of the first scale; n is an integer greater than 1; Each of the n upsampled voxels is numbered, and grouped based on the number corresponding to each upsampled voxel, to determine the m groups of upsampled voxels. The method according to claim 1 or 2, wherein the occupancy probability prediction is performed on the m groups of upsampled voxels in sequence, the m groups of occupancy probabilities are determined, and the occupancy indication information is calculated according to the m groups of occupancy probabilities. Decode and determine the occupied symbols of each of the m groups, including: In the m groups of upsampled voxels, the occupancy probability is predicted for the first group of upsampled voxels, the first group of occupancy probabilities are determined, and the occupancy indication information is decoded according to the first group of occupancy probabilities to determine The first group of occupied symbols; When i is greater than 1 and less than or equal to m, use at least k groups of decoded occupancy symbols to predict the occupancy probability of the i-th group of upsampled voxels, determine the i-th group of occupancy probabilities, and use the i-th group of occupancy probabilities to , decode the i-th group of occupancy indication information in the occupancy indication information to determine the i-th group of occupancy symbols; where i is an integer, k is an integer greater than or equal to 1 and less than i; Decoding continues with the next set of occupancy symbols based on the i-th set of occupancy symbols until the respective occupancy symbols of the m sets are determined. The method according to claim 2, wherein each of the n upsampled voxels is numbered and grouped based on the number corresponding to each upsampled voxel to determine the m groups of upsampled voxels, including: Number each of the n upsampled voxels, and determine the number corresponding to each of the upsampled voxels; According to the number of each upsampled voxel, the n upsampled voxels are grouped to determine the m groups of upsampled voxels. The method according to claim 4, wherein said grouping the n upsampled voxels according to the number of each upsampled voxel and determining the m groups of upsampled voxels includes: Among the n upsampled voxels, the upsampled voxels with the same number are divided into one group according to the number of each upsampled voxel, and the m groups of upsampled voxels are determined. The method according to claim 4, wherein said grouping the n upsampled voxels according to the number of each upsampled voxel and determining the m groups of upsampled voxels includes: Identify at least two consecutive numbers; According to the number of each upsampled voxel, the upsampled voxels corresponding to the at least two consecutive numbers in the n upsampled voxels are taken as a group to determine the m group of upsampled voxels. The method according to claim 3, wherein the use of at least k groups of decoded occupancy symbols to predict the occupancy probability of the i-th group of upsampled voxels and determine the i-th group of occupancy probabilities includes: Perform feature extraction on the i-th group of upsampled voxels to determine the characteristics of the i-th group of voxels; Determine that the decoded i-kth group of occupied symbols to the i-1th group of occupied symbols are the at least k groups of occupied symbols; According to the at least k groups of occupancy symbols, combined with the i-th group of voxel features, the occupancy probability is predicted for each up-sampled voxel in the i-th group of up-sampled voxels, and the predicted i-th group of voxels are determined. The occupancy probability corresponding to each upsampled voxel in the upsampled voxels is the i-th group of occupancy probabilities. The method according to any one of claims 1, 2, 4 to 7, wherein before determining the second scale point cloud, the method further includes: Analyze the code stream to determine the T-th scale point cloud and the T-1th occupancy indication information corresponding to the T-1th scale point cloud; where, j=T, T-1,...,3; T is greater than or equal to an integer of 3; Starting from j=T, perform the following processing on each value of j in order from large to small: Perform voxel upsampling and division processing based on the j-th scale point cloud, and perform occupancy probability prediction on the divided groups in sequence to determine m group occupancy probabilities corresponding to m groups in the j-1th scale point cloud; According to m groups of occupancy probabilities corresponding to m groups in the j-1th scale point cloud, entropy decoding is performed on the j-1th occupancy indication information to determine m groups of occupancy corresponding to the j-1th occupancy indication information. symbol; Based on the m sets of occupancy symbols corresponding to the j-1th occupancy indication information, reconstruct the j-1th scale point cloud; Continue to perform the next processing according to the next value of j until j=3, and reconstruct the second scale point cloud. The method according to claim 3, wherein decoding the i-th group of occupancy indication information in the occupancy indication information according to the i-th group of occupancy probabilities and determining the i-th group of occupancy symbols includes: According to the i-th group of occupancy probabilities, entropy decoding is performed on each occupancy indication information in the i-th group of occupancy indication information, and the occupancy symbol corresponding to each occupancy indication information in the i-th group of occupancy indication information is determined, thereby determining The i-th group of occupied symbols; where, The occupancy symbols include: a first occupancy symbol and a second occupancy symbol; the first occupancy symbol represents that the corresponding voxel is occupied; and the second occupancy symbol represents that the corresponding voxel is not occupied. The method according to claim 3, wherein said continuing the next group of decoding based on the i-th group of occupied symbols until the respective m groups of occupied symbols are determined includes: Using at least k groups of occupancy symbols including the i-th group of occupancy symbols, perform occupancy probability prediction on the i+1 group of upsampled voxels, and predict the i-th group of occupancy indication information based on the i+1-th group of occupancy probabilities. The +1 group of occupancy indication information is decoded to determine the i+1 group of occupancy symbols, until the decoding of the occupancy indication information is completed, and the respective occupancy symbols of the m groups are determined. A coding method that includes: Perform voxel downsampling on the first-scale point cloud to determine the second-scale point cloud; Perform upsampling and division processing on the second scale point cloud to determine m groups of upsampled voxels; m is an integer greater than or equal to 1; Predict the occupancy probability of the m groups of upsampled voxels in sequence, determine the m group of occupancy probabilities, and determine the respective occupancy symbols of the m groups according to the m group of occupancy probabilities; The m groups of respective occupancy symbols are encoded to determine the occupancy indication information corresponding to the first scale point cloud, and the occupancy indication information is written into the code stream. The method according to claim 11, wherein the upsampling and dividing process on the second scale point cloud to determine m groups of upsampled voxels includes: Perform voxel upsampling on the second scale point cloud to determine n upsampled voxels of the first scale; n is an integer greater than 1; Each of the n upsampled voxels is numbered, and grouped based on the number corresponding to each upsampled voxel, to determine m groups of upsampled voxels. The method according to claim 11, wherein the occupancy probability prediction is performed on the m groups of upsampled voxels in sequence, the m groups of occupancy probabilities are determined, and the respective occupancy symbols of the m groups are determined according to the m groups of occupancy probabilities. ,include: In the m groups of upsampled voxels, the occupancy probability of the first group of upsampled voxels is predicted to determine the occupancy probability of the first group, and the preset occupancy symbol is used to represent the occupancy probability of the first group, and the occupancy probability of the first group is determined. occupancy symbol; When i is greater than 1 and less than or equal to m, use the completed probability prediction and at least k groups of occupancy symbols corresponding to at least k groups of upsampled voxels represented by the occupancy symbols to predict the occupancy probability of the i-th group of upsampled voxels, and determine Occupancy probability of the i-th group; where i is an integer, k is an integer greater than or equal to 1 and less than i; Use the preset occupancy symbols to represent the i-th group of occupancy probabilities and determine the i-th group of occupancy symbols; Using at least k groups of occupancy symbols including the i-th group of occupancy symbols, continue to perform occupancy probability prediction and occupancy symbol representation on the i+1-th group of upsampled voxels until the respective occupancy symbols of the m groups are determined. The method according to claim 12, wherein each of the n upsampled voxels is numbered and grouped based on the number corresponding to each upsampled voxel to determine m groups Upsampled voxels, including: Number each of the n upsampled voxels, and determine the number corresponding to each of the upsampled voxels; According to the number of each upsampled voxel, the n upsampled voxels are grouped to determine the m groups of upsampled voxels. The method according to claim 14, wherein said grouping the n upsampled voxels according to the number of each upsampled voxel and determining the m groups of upsampled voxels includes: Among the n upsampled voxels, the upsampled voxels with the same number are divided into one group according to the number of each upsampled voxel, and the m groups of upsampled voxels are determined. The method according to claim 14, wherein said grouping the n upsampled voxels according to the number of each upsampled voxel and determining the m groups of upsampled voxels includes: Identify at least two consecutive numbers; According to the number of each upsampled voxel, the upsampled voxels corresponding to the at least two consecutive numbers in the n upsampled voxels are taken as a group to determine the m group of upsampled voxels. The method according to claim 13, wherein the occupancy probability prediction is performed on the i-th group of upsampled voxels using at least k groups of occupancy symbols corresponding to at least k groups of upsampled voxels represented by completed probability predictions and occupancy symbols. , determine the occupancy probability of the i-th group, including: Perform feature extraction on the i-th group of upsampled voxels to determine the characteristics of the i-th group of voxels; Determine that the i-kth group of occupied symbols to the i-1th group of occupied symbols that have completed probability prediction and occupancy symbol representation are the at least k groups of occupied symbols; According to the at least k groups of occupancy symbols, combined with the i th group of voxel characteristics, the occupancy probability is predicted for each upsampled voxel in the i th group of upsampled voxels, and the i th group of upsampled voxels are determined The occupancy probability corresponding to each upsampled voxel in the voxel is the ith group of occupancy probabilities. The method according to claim 13, wherein said using the preset occupancy symbols to represent the i-th group of occupancy probabilities and determining the i-th group of occupancy symbols includes: Using the first occupancy symbol in the preset occupancy symbols represents the occupancy probability in the i-th group of occupancy probabilities that is greater than or equal to the preset probability threshold; and using the second occupancy symbol in the preset occupancy symbols represents The occupancy probability of the i-th group of occupancy probabilities that is less than the preset probability threshold determines the i-th group of occupied symbols. The method according to any one of claims 11 to 18, wherein encoding the m groups of respective occupancy symbols and determining the occupancy indication information corresponding to the first scale point cloud includes: Entropy encoding is performed on each group of occupancy symbols in the m groups of respective occupancy symbols, and the m groups of occupancy indication information obtained by encoding are determined to be occupancy indication information corresponding to the first scale point cloud. The method according to any one of claims 11 to 18, wherein the method further includes: Perform at least S times of voxel downsampling based on the second scale point cloud to determine the third scale point cloud to the S+2th scale point cloud; S is an integer greater than or equal to 1; Perform voxel upsampling and division processing on the third scale point cloud to the S+2th scale point cloud respectively, and perform occupancy probability prediction and coding on the divided groups in sequence to determine the second scale point. The occupancy indication information of the cloud to the occupancy indication information of the S+1th scale point cloud; The occupancy indication information of the first scale point cloud, the occupancy indication information of the second scale point cloud to the occupancy indication information of the S+1th scale point cloud are written into the code stream. A decoder including: The parsing part is configured to parse the code stream and determine the occupancy indication information corresponding to the first scale point cloud; The first grouping part is configured to determine the second scale point cloud, perform upsampling and dividing processing on the second scale point cloud, and determine m groups of upsampled voxels; wherein the second scale point cloud is the The previous decoded point cloud data corresponding to the first scale point cloud; m is an integer greater than or equal to 1; The first prediction part is configured to predict the occupancy probabilities of the m groups of upsampled voxels in sequence, determine the m groups of occupancy probabilities, and decode the occupancy indication information according to the m groups of occupancy probabilities to determine the m groups of occupancy probabilities. respective occupancy symbols; The decoding part is configured to determine the reconstructed geometric data corresponding to the first scale point cloud based on the m groups of occupied symbols. An encoder consisting of: The downsampling part is configured to perform voxel downsampling on the first-scale point cloud and determine the second-scale point cloud; The second grouping part is configured to perform upsampling and dividing processing on the second scale point cloud, and determine m groups of upsampled voxels; m is an integer greater than or equal to 1; The second prediction part is configured to predict the occupancy probability of the m groups of upsampled voxels in sequence, determine the m group of occupancy probabilities, and determine the respective occupancy symbols of the m groups according to the m group of occupancy probabilities; The encoding part is configured to encode the respective occupancy symbols of the m groups, determine the occupancy indication information corresponding to the first scale point cloud, and write the occupancy indication information into the code stream. A code stream, including: The code stream is generated by bit encoding based on the information to be encoded; wherein the information to be encoded at least includes: occupancy indication information corresponding to the first scale point cloud. A decoder including: a first memory configured to store executable instructions; The first processor is configured to implement the method described in any one of claims 1 to 10 when executing executable instructions stored in the first memory. An encoder consisting of: a second memory configured to store executable instructions; The second processor is configured to implement the method described in any one of claims 11 to 20 when executing the executable instructions stored in the second memory. A computer-readable storage medium storing executable instructions for causing the first processor to execute the method described in any one of claim items 1 to 10, or for causing the second processor to execute the request The method described in any one of items 11 to 20.