CN115086658A

CN115086658A - Point cloud data processing method and device, storage medium and encoding and decoding equipment

Info

Publication number: CN115086658A
Application number: CN202110269951.9A
Authority: CN
Inventors: 朱文婕
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-03-12
Filing date: 2021-03-12
Publication date: 2022-09-20
Anticipated expiration: 2041-03-12
Also published as: CN115086658B

Abstract

The embodiment of the application provides a point cloud data processing method, a point cloud data processing device, a storage medium and encoding and decoding equipment, wherein the method comprises the following steps: acquiring point cloud data, wherein the point cloud data comprises a current point to be processed and a plurality of processed points; selecting N processed points from the plurality of processed points as N alternative points; according to the distance between the N alternative points and the current point, k alternative points are selected from the N alternative points to serve as k neighbor points of the current point, wherein N and k are positive integers larger than 1; and determining the attribute predicted value of the current point according to the attribute values of the k neighbor points. According to the method and the device, the alternative points can be selected by utilizing the geometric relation and the attribute relation of the processed points, and the specific neighbor points are selected from the alternative points, so that richer neighbor point selection modes are provided, and the prediction efficiency of the point cloud attributes is improved.

Description

Point cloud data processing method and device, storage medium and encoding and decoding equipment

Technical Field

The embodiment of the application relates to the technical field of image processing, in particular to a point cloud data processing method and device, a storage medium and encoding and decoding equipment.

Background

In the existing point cloud attribute and reflectivity prediction method, for a certain point to be coded, a neighbor point needs to be determined, and the attribute value of the neighbor point is adopted to predict the attribute value of the current point to be coded. Considering the problem of computational complexity, the number of neighbor point selections is limited. Therefore, how to select neighboring points and how to improve the efficiency of point cloud attribute prediction in selecting a limited number of neighboring points have become important research subjects in the industry.

Disclosure of Invention

The embodiment of the application provides a point cloud data processing method and device, a storage medium and encoding and decoding equipment, which can select alternative points by using the geometric relationship and the attribute relationship of the processed points and select specific neighbor points from the alternative points, thereby providing richer neighbor point selection modes and improving the prediction efficiency of point cloud attributes.

In a first aspect, a method for processing point cloud data is provided, including:

acquiring point cloud data, wherein the point cloud data comprises a current point to be processed and a plurality of processed points;

selecting N processed points from the plurality of processed points as N alternative points;

selecting k candidate points from the N candidate points as k neighbor points of the current point according to the distance between the N candidate points and the current point, wherein N and k are positive integers greater than 1;

and determining the attribute predicted value of the current point according to the attribute values of the k neighbor points.

In a second aspect, an apparatus for processing point cloud data is provided, including:

an acquisition unit configured to acquire point cloud data including a current point to be processed and a plurality of processed points;

a first selection unit configured to select N processed points from the plurality of processed points as N alternative points;

a second selecting unit, configured to select k candidate points from the N candidate points as k neighbor points of the current point according to distances from the N candidate points to the current point, where N and k are positive integers greater than 1;

and the determining unit is used for determining the attribute predicted value of the current point according to the attribute values of the k neighbor points.

In a third aspect, a computer-readable storage medium stores a computer program, the computer program being adapted to be loaded by a processor to perform the method of the first aspect or its implementations.

In a fourth aspect, an encoding device is provided, which includes a processor and a memory, where the memory stores a computer program, and the processor executes the method in the first aspect or each implementation manner by calling the computer program stored in the memory.

In a fifth aspect, a decoding device is provided, which includes a processor and a memory, where the memory stores a computer program, and the processor executes the method in the first aspect or each implementation manner by calling the computer program stored in the memory.

The embodiment of the application provides a point cloud data processing method, a point cloud data processing device, a storage medium and a coding and decoding device, wherein the point cloud data is obtained and comprises a current point to be processed and a plurality of processed points, then N processed points are selected from the plurality of processed points to serve as N alternative points, then k alternative points are selected from the N alternative points to serve as k neighbor points of the current point according to the distance between the N alternative points and the current point, wherein N and k are positive integers larger than 1, and finally an attribute predicted value of the current point is determined according to the attribute values of the k neighbor points. According to the method and the device, the alternative points can be selected by utilizing the geometric relation and the attribute relation of the processed points, and the specific neighbor points are selected from the alternative points, so that richer neighbor point selection modes are provided, and the prediction efficiency of the point cloud attributes is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1a is a coding architecture diagram of G-PCC provided in an embodiment of the present application.

FIG. 1b is a schematic diagram of a Morton sequence according to an embodiment of the present application.

FIG. 1c is a schematic diagram of an offset Morton sequence according to an embodiment of the present application.

Fig. 1d is a schematic diagram of a spatial relationship between adjacent points according to an embodiment of the present application.

Fig. 1e is a schematic diagram of morton code relationships between neighboring points according to an embodiment of the present application.

Fig. 1f is a schematic block diagram of a coding/decoding system according to an embodiment of the present application.

Fig. 1g is a flowchart of a method for processing point cloud data according to an embodiment of the present disclosure.

Fig. 2 is a schematic block diagram of a processing apparatus for point cloud data provided in an embodiment of the present application.

Fig. 3 is a schematic block diagram of an electronic device provided in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.

Point cloud data (point cloud data) refers to a collection of vectors in a three-dimensional coordinate system. The point cloud data can be divided into dense point cloud and sparse point cloud according to the data acquisition mode; from the time-series type division of the data, the point cloud data can be divided into static point clouds and dynamic point clouds.

Point cloud data is used to represent three-Dimensional (3-Dimensional, 3D) scenes or objects in some emerging applications such as immersive Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), car navigation, robotic navigation, medical imaging, and the like. The point cloud comprises a collection of individual 3D points. Each point is associated with a set of 3D coordinates indicating the 3D location of the respective point and a number of other attributes such as color, surface normal, opacity, reflectivity, etc. A point cloud is a set of points in 3D space, each point having associated attributes, e.g., color, material attributes, etc. The point cloud may be used to reconstruct an object or scene as a composite of these points. The point cloud may be captured using multiple cameras and depth sensors in various settings, and may consist of thousands or even billions of points in order to truly represent the reconstructed scene.

Modern mainstream point cloud coding technologies can be classified into geometry-based point cloud coding and projection-based point cloud coding for different types of point cloud data. The description is given by taking Geometry-based Point Cloud Coding (G-PCC) in the international Video-Audio Coding and decoding Standard (MPEG) and Point Cloud Coding (PCC) Standard AVS-PCC in the chinese national digital Video Coding Standard (AVS) as an example.

G-PCC and AVS-PCC are both directed at static sparse point clouds, whose coding frames are substantially the same. The G-PCC encoding architecture diagram shown in fig. 1a is divided into a geometry encoding and an attribute information encoding.

For geometry coding, the main operations and processes are as follows:

(1) pre-processing (Pre-processing): including coordinate transformations (Transform coordinates) and voxelization (Voxelize). And converting the point cloud data in the 3D space into an integer form through operations of scaling and translation, and moving the minimum geometric position of the point cloud data to the coordinate origin. The voxel is a short name of a volume element, the voxel is a minimum unit for segmenting digital data on a three-dimensional space, and the concept of the voxel is similar to that of a pixel in a video image; the method comprises the steps of performing a voxelization process on points, wherein the voxelization process mainly comprises position quantization, repeated point deletion and attribute allocation; firstly, quantizing the geometric position information of points, rounding the geometric position information of the points to convert the geometric position information of the points into non-negative d-bit integers through a function Round (), and generating a large number of points with the same position after the step, wherein the points are called duplicate points (duplicate points); the duplicate points with the same quantized coordinates may be subjected to a deletion operation; different attribute values of the same position points are not deleted, but are fused into the attribute value of one point in a certain mode; the process of voxelization is to quantize all points in a voxel to the voxel center, combine the attributes of all points in the voxel and then assign the voxel. If a voxel contains any point in the point cloud, the voxel is called occupied.

(2) Geometric encoding (Geometry encoding): two modes are included in the geometric coding and can be used under different conditions.

(a) OctreeImage-based geometric coding (Octree): the octree is a tree-shaped data structure in which a predetermined bounding box is uniformly divided in 3D space division, and each node has eight child nodes. Whether each subnode of the octree is occupied or not is indicated by '1' and '0', and occupation code information (occupancy code) is obtained as a code stream of point cloud geometric position information.

(b) Geometric coding based on trigonometric representation (Trisoup): the point cloud is divided into blocks (blocks) with certain sizes, intersection points of the surface of the point cloud on the edges of the blocks are located, and a triangle is constructed. Compression of the geometric position information is achieved by encoding the intersection positions.

(3) Geometric quantification (Geometry quantification): the quantization fineness is usually determined by a Quantization Parameter (QP), which has a larger value, and coefficients representing a larger value range are quantized into the same output, thereby generally causing larger distortion and lower code rate; conversely, the QP value is smaller, and the coefficients representing a smaller value range will be quantized to the same output, thus usually causing less distortion and corresponding to a higher code rate. In point cloud coding, quantization is performed directly on the coordinate information of a point.

(4) Geometric entropy encoding (geometric entropy encoding): and (3) performing statistical compression coding on the occupied code information of the octree, and finally outputting a binary (0 or 1) compressed code stream. The statistical coding is a lossless coding mode, and can effectively reduce the code rate required by expressing the same signal. A commonly used statistical Coding method is context-based Binary Arithmetic Coding (CABAC).

For the attribute information encoding, the main operations and processes are as follows:

(1) in lossy encoding, after encoding the geometric position information, an encoding end needs to decode and reconstruct the geometric position information, namely recovering the coordinate information of each point of the 3D point cloud. And searching attribute information corresponding to one or more adjacent points in the original point cloud to serve as the attribute information of the reconstruction point.

(2) Attribute prediction coding (prediction) when in attribute prediction coding, one or more points are selected as predicted values through the adjacent relation of geometric position information or attribute information, and the final attribute predicted value is obtained by calculating weighted average, and the difference value between the actual value and the predicted value is coded.

(3) Attribute Transform coding (Transform): the attribute transform coding includes three modes, which can be used under different conditions.

(a) Predictive Transform coding (predictive Transform): and selecting a sub-point set according to the distance, dividing the point cloud into a plurality of different levels (LoD), and realizing the point cloud representation from coarse to fine. The prediction from bottom to top can be realized between adjacent layers, namely, the attribute information of the introduced points in the fine layer is predicted by the adjacent points in the coarse layer, and the corresponding residual signals are obtained. Wherein, the point of the bottom layer is used as reference information for coding.

(b) Lifting Transform coding (Lifting Transform): on the basis of LoD adjacent layer prediction, a weight updating strategy of neighborhood points is introduced, and finally, the prediction attribute values of all points are obtained, and corresponding residual signals are obtained.

(c) Hierarchical Adaptive Transform coding (RAHT): the attribute information is subjected to a RAHT transform to convert the signal into a transform domain, referred to as transform coefficients.

(4) Attribute information quantization (Attribute qualification): the degree of refinement of the quantization is typically determined by a Quantization Parameter (QP). In the predictive transform coding and the lifting transform coding, the residual error value is quantized and then entropy coded; in RAHT, transform coefficients are quantized and entropy encoded.

(5) Attribute entropy coding (Attribute entropy coding): the quantized attribute residual signal or transform coefficient is generally compressed finally using run length coding (run length coding) and arithmetic coding (arithmetric coding). And the corresponding information such as the coding mode, the quantization parameter and the like is also coded by adopting an entropy coder.

And at the encoding end, according to the encoding process, carrying out compression encoding on the geometric position information and the attribute information of the point cloud to obtain a compressed code stream.

At the decoding end, after the decoder obtains the compressed code stream output from the encoding end, entropy decoding is firstly carried out to obtain various mode information, and the quantized geometric position information and attribute information. Firstly, the geometric position information is subjected to inverse quantization to obtain the position information of the reconstructed 3D point. On the other hand, the attribute information is subjected to inverse quantization to obtain residual error information, and a reference signal is confirmed according to the adopted transformation mode to obtain reconstructed attribute information, and the reconstructed attribute information is in one-to-one correspondence with the geometric position information in sequence to generate output reconstructed point cloud data.

Taking the encoding end as an example, when performing attribute prediction encoding, attribute prediction may be performed based on a prediction value method based on a geometric position and/or a prediction value method based on an attribute value. The geometric position-based prediction value method comprises a reflectivity attribute prediction method and a color attribute prediction method.

The attribute prediction method of the reflectivity can be realized by adopting a selection mode of a Morton order neighbor point or a selection mode of a Hilbert order neighbor point.

The selection mode of the Morton sequence neighbor points finds k encoded points of the current point as neighbor points according to the method of the offset Morton code.

First, three-dimensional coordinates of all point clouds are obtained, and a morton sequence 1 is obtained according to the morton sorting, as shown in fig. 1B, the morton sequence 1 is the point cloud in the original morton sequence, for example, in fig. 1B, the morton code of a point a is 0, the morton code of a point B is 2, the morton code of a point C is 10, and the morton code of a point D is 16. Then, a fixed value (j1, j2, j3) is added to the coordinates (x, y, z) of all the point clouds, morton codes corresponding to the point clouds are generated by using the new coordinates (x + j1, y + j2, z + j3), and a morton sequence 2 is obtained according to the morton sorting, and as shown in fig. 1c, the morton sequence 2 is the point cloud in the offset morton sequence (for example, j1 ═ j2 ═ j3 ═ 1). When points a, B, C, D in fig. 1B move to different positions in fig. 1C, the corresponding morton codes change, but their relative positions remain the same. For example, in fig. 1c, the morton code for point D is 23, and the morton code for point B, which is a neighbor of point D, is 21, so that point B can be found by searching at most two points from point D. However, in FIG. 1B, at most 14 points are searched forward from point D (Morton code 16) to find point B (Morton code 2).

According to the Morton sequence decoding, searching the latest prediction point of the current point, selecting the first N points of the current point as alternative points in the Morton sequence 1, wherein the value range of N can be more than or equal to 1, selecting the first M points of the current point as alternative points in the Morton sequence 2, and the value range of M can be more than or equal to 1. In the N alternative points and the M alternative points, calculating the distance d from each alternative point to the current point, wherein the coordinates of the current point are (x, y, z), and the coordinates of the alternative points are (x ₁ ，y ₁ ，z ₁ ) The distance d is calculated by d ═ x-x ₁ |+|y-y ₁ |+|z-z ₁ And selecting k decoded points with the minimum distance from the N + M points as neighbor points of the current point. For example, in the PCEM software, j1 j2 j3, k 3, and N M4.

The selection mode of the Hilbert sequence neighbor point is that k points nearest to the current point are searched in the previous maxNumOfNeighbours points of the current coding point in the Hilbert sequence to serve as neighbor points, wherein the maxNumOfNeighbours represents the number of the candidate points, 128 is acquiescent to the maxNumOfNeighbours, 3 is acquiescent to the k, and the distance calculation method is Manhattan distance, namely d ═ x [ ] ₁ -x ₂ |+|y ₁ -y ₂ |+|z ₁ -z ₂ L. In the encoding, discrete hilbert curves are used for calculation when hilbert sorting is performed.

WhereinAnd when calculating the attribute predicted value of the current point, using the reciprocal of the Manhattan distance between the selected k neighbor points and the current point as a weight, and finally calculating the weighted average of the attribute reconstruction values of the k neighbor points to obtain the attribute predicted value of the current point. Let the serial number of the current point be i, and the geometric coordinate of the current point be (x) _i ，y _i ，z _i ) The geometric coordinate of each neighbor is (x) _ij ，y _ij ，z _ij ) Where j is 0, 1,2, … …, k, the weight w of each neighbor point _ij As shown in the following formula 1:

wherein, the attribute reconstruction value of each neighbor point is set as

Then the attribute prediction value of the current point

The following formula 2:

for the reflectivity attribute, if the components in x, y and z directions of the weight calculation in equation 1 adopt different weights, the weight w of each neighboring point _ij As shown in the following formula 3:

wherein a, b, c are different weights on the components in the x, y, z directions, respectively.

In the color attribute prediction method, firstly, a Morton code is used for searching a spatial neighbor point of a current point, and then the attribute prediction is carried out on the current point according to the searched spatial neighbor point. The spatial relationship of neighboring points is shown in fig. 1d, and the spatial relationship of neighboring points is shown in fig. 1 d. The current point to be coded is a thick-line marked block A, and the neighbor search range is a 3X3X3 neighborhood of the current point to be coded. Firstly, the block with the minimum Morton code value in the neighborhood is obtained by utilizing the Morton code of the current point, the block is used as a reference block, and the reference block is utilized to search the coded neighbor point which is coplanar and collinear with the current point 7 to be coded. The morton code relationship between the neighboring points coplanar and collinear with the current point 7 to be encoded within the neighborhood range, as shown in fig. 1e, the left diagram in fig. 1e shows the morton code relationship of the neighboring point 3, the neighboring point 5 and the neighboring point 6 coplanar with the current point 7 to be encoded, and the right diagram in fig. 1e shows the morton code relationship of the neighboring point 1, the neighboring point 2 and the neighboring point 4 collinear with the current point 7 to be encoded. The reference block is used to search k neighbor points (such as k is less than or equal to 6 when a neighborhood is selected) which are co-planar and collinear with the current point to be coded and are used to predict the attribute of the current point to be coded. For example, if k is 3, searching for a neighbor point coplanar with a current point to be encoded within a certain range [ j-maxNumOfNeighbours, j-1] of the encoded point, where an index of the current point is j, maxNumOfNeighbours represents the number of candidate points, and if a coplanar encoded and decoded neighbor point is found, assigning a weight of the coplanar neighbor point to be 2; continuously searching for a neighbor point which is collinear with the current point to be coded in the coded and decoded points, if the collinear neighbor point is found in the decoded point set, distributing the weight of the collinear neighbor point to be 1, and finally performing attribute prediction on the current point to be coded by using the searched neighbor point to perform weighted average; and if the coded and decoded neighbor point coplanar and collinear with the current point to be coded is not found, performing attribute prediction by using a point corresponding to a previous Morton code of the current point to be coded.

For example, when the attribute prediction is performed on the current point to be encoded by using the neighbor points to perform weighted average, if there is a coplanar point and a collinear point, the distance between the coplanar point and the current point is d1 (for example, d1 is 1), and the attribute value is r 1; the collinear point is separated from the current point by a distance d2 (e.g., d2 √ 2), and the attribute value is r2, so that the predicted value of the attribute of the current point to be encoded is r (2 × r1+ r2)/(2+ 1).

The attribute prediction method of the reflectivity and the attribute prediction method of the color are to determine the attribute prediction value according to the geometric position information, and the attribute prediction method of the reflectivity and the attribute prediction method of the color can be called as a prediction value method based on the geometric position. The geometric location-based predictor approach is generally suitable for denser, more predictable point clouds, such as human point clouds, or for cases where the prediction residual is small. In case of a large residual generated by the geometric position-based prediction method, the attribute value-based prediction method can generally reduce the prediction residual and improve the coding efficiency.

The attribute value-based prediction value method can be realized by the following steps:

storing 32 different attribute predicted values in the recently coded points in a candidate prediction value table;

selecting a point nearest to the attribute of the current point from the candidate prediction value table, and using the attribute value of the selected point as the attribute prediction value of the current point;

the sequence number of the selected point in the candidate prediction value table is binarized into 5 bits, and is encoded by context entropy coding.

In the common point cloud attribute and reflectivity prediction method, for a certain point to be coded, a neighbor point needs to be determined, and the attribute value of the current point to be coded is predicted by adopting the attribute value of the neighbor point. Considering the problem of computational complexity, the number of neighbor point selections is limited. In the selection process, the neighbor points need to be sorted, certain conditions are set, and the better neighbor points are selected as much as possible. Therefore, the embodiment of the application provides a method for processing point cloud data, which can select alternative points by using the geometric relationship and the attribute relationship of points to be processed, and select specific neighbor points from the alternative points, thereby providing richer neighbor point selection modes and improving the prediction efficiency of point cloud attributes.

The method and the device can be applied to the fields of image coding and decoding, video coding and decoding, hardware video coding and decoding, special circuit video coding and decoding, real-time video coding and decoding and the like. For example, the solution of the present application may be incorporated into the MPEG standard, AVS standard, or MPEG standard. Alternatively, the schemes of the present application may operate in conjunction with other proprietary or industry standards including ITU-T H.261, ISO/IECMPEG-1Visual, ITU-T H.262 or ISO/IECMPEG-2Visual, ITU-T H.263, ISO/IECMPEG-4Visual, ITU-T H.264 (also known as ISO/IECMPEG-4AVC), including Scalable Video Codec (SVC) and Multiview Video Codec (MVC) extensions. It should be understood that the techniques of this application are not limited to any particular codec standard or technique.

For ease of understanding, the codec system related to the embodiment of the present application will be described first with reference to fig. 1 f. Fig. 1f is a schematic block diagram of a coding/decoding system 100 according to an embodiment of the present application. It should be noted that fig. 1f is only an example, and the coding and decoding system according to the embodiment of the present application includes, but is not limited to, that shown in fig. 1 f. As shown in fig. 1f, the codec system 100 includes an encoding device 110 and a decoding device 120. The encoding device is used for encoding (can be understood as compressing) the point cloud data to generate a code stream, and transmitting the code stream to the decoding device. And decoding the code stream generated by the coding of the coding equipment by the decoding equipment to obtain decoded point cloud data.

The encoding apparatus 110 of the embodiment of the present application may be understood as an apparatus having audio encoding and video encoding functions, and the decoding apparatus 120 may be understood as an apparatus having audio decoding and video decoding functions, that is, the embodiment of the present application includes a broader range of devices for the encoding apparatus 110 and the decoding apparatus 120, including, for example, a smart phone, a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a television, a camera, a display device, a digital media player, a video game console, a vehicle mounted computer, and the like.

In some embodiments, the encoding device 110 may transmit the encoded point cloud data (e.g., code stream) to the decoding device 120 via the channel 130. The channel 130 may include one or more media and/or devices capable of transmitting the encoded point cloud data from the encoding device 110 to the decoding device 120.

In some embodiments, the channel 130 includes one or more communication media that enable the encoding device 110 to transmit the encoded point cloud data directly to the decoding device 120 in real-time. In this example, the encoding apparatus 110 may modulate the encoded point cloud data according to a communication standard and transmit the modulated point cloud data to the decoding apparatus 120. Wherein the communication medium comprises a wireless communication medium such as the radio frequency spectrum, and optionally a wired communication medium such as one or more physical transmission lines.

In some embodiments, the channel 130 includes a storage medium that can store point cloud data encoded by the encoding device 110. Storage media includes a variety of locally-accessed data storage media such as compact disks, DVDs, flash memory, and the like. In this example, the decoding apparatus 120 may acquire the encoded point cloud data from the storage medium.

In some embodiments, the channel 130 may include a storage server that may store the point cloud data encoded by the encoding device 110. In this example, the decoding device 120 may download the stored encoded point cloud data from the storage server. Alternatively, the storage server may store the encoded point cloud data and may transmit the encoded point cloud data to the decoding device 120, such as a web server (e.g., for a website), a File Transfer Protocol (FTP) server, and so forth.

In some embodiments, the encoding apparatus 110 includes an encoder 112 and an output interface 113. The output interface 113 may comprise, among other things, a modulator/demodulator (modem) and/or a transmitter.

In some embodiments, the encoding device 110 may include a video source 111 in addition to an encoder 112 and an input interface 113.

The video source 111 may include at least one of a video capture device (e.g., a video camera), a video archive, a video input interface for receiving point cloud data from a video content provider, a computer graphics system for generating point cloud data.

The encoder 112 encodes the point cloud data from the video source 111 to generate a code stream. The point cloud data may include one or more images (pictures) or sequences of images (pictures). The code stream contains the coding information of the picture or the sequence of pictures in the form of a bit stream. The encoded information may include encoded image data and associated data. The associated data may include Sequence Parameter Sets (SPS), Picture Parameter Sets (PPS), and other syntax structures. An SPS may contain parameters that apply to one or more sequences. A PPS may contain parameters that apply to one or more pictures. A syntax structure refers to a set of zero or more syntax elements in a codestream arranged in a specified order.

The encoder 112 transmits the encoded point cloud data directly to the decoding apparatus 120 via the output interface 113. The encoded point cloud data may also be stored on a storage medium or storage server for subsequent reading by the decoding apparatus 120.

In some embodiments, the decoding device 120 includes an input interface 121 and a decoder 122.

In some embodiments, the decoding apparatus 120 may further include a display device 123 in addition to the input interface 121 and the decoder 122.

The input interface 121 includes a receiver and/or a modem. The input interface 121 may receive the encoded point cloud data through the channel 130.

The decoder 122 is configured to decode the encoded point cloud data to obtain decoded point cloud data, and transmit the decoded point cloud data to the display device 123.

The display device 123 displays the decoded point cloud data. The display device 123 may be integrated with the decoding apparatus 120 or external to the decoding apparatus 120. The display device 123 may include a variety of display devices, such as a Liquid Crystal Display (LCD), a plasma display, an Organic Light Emitting Diode (OLED) display, or other types of display devices.

In addition, fig. 1f is only an example, and the technical solution of the embodiment of the present application is not limited to fig. 1 f.

The method and the device for predicting the candidate points design the priority of the candidate prediction points and select the specific neighbor points by utilizing the geometric relations and the attribute relations of all possible neighbor points of the points to be coded. Under different conditions, a single selection mode or a combination of multiple selection modes can be adopted to improve the prediction efficiency of the point cloud attributes. The selection mode can be determined by the encoding end and recorded in the code stream, and is analyzed by the decoding end; and can be calculated by the decoding end according to the situation.

The technical solutions provided in the embodiments of the present application are described in detail below with reference to specific embodiments.

It should be noted that the method of the present application is applicable to an encoding end and a decoding end, and the process of selecting the neighbor point by the encoding end and the decoding end is basically similar. Here, the method for processing point cloud data provided by the present application is described by taking a decoding end as an example, and the encoding end may refer to the decoding end.

Fig. 1g is a flowchart of a method for processing point cloud data according to an embodiment of the present disclosure. The specific process of the method can be as follows:

step 101, point cloud data is obtained, wherein the point cloud data comprises a current point to be processed and a plurality of processed points.

The execution subject of the embodiment of the present application includes, but is not limited to, the following devices: a decoder, an encoder, or an apparatus for processing point cloud data, such as a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a handset such as a smartphone, a television, a camera, a display device, a digital media player, a video game console, an on-board computer, or the like.

For example, at the encoding end, the current point to be processed is the current point to be encoded, and the processed point is the encoded point.

For example, at the decoding end, the current point to be processed is the current point to be decoded, and the processed point is the decoded point.

For example, taking a decoding end as an example, three-dimensional coordinates of point cloud data are acquired, and the point cloud data includes a current point to be encoded and a plurality of encoded points.

And 102, selecting N processed points from the plurality of processed points as N alternative points.

For example, taking the decoding end as an example, let the sequence number of the current point to be decoded be i, there are i-1 decoded points, and the decoded points are sorted in the decoding order or sorted according to the geometric position (e.g., calculating a morton code or a Hilbert code). N decoded points are selected as candidate points for neighbor points for attribute prediction.

In some embodiments, step 102 may be implemented by steps 1021 to 1023, specifically:

and step 1021, sequencing the point cloud data, wherein the sequence number of the current point in the sequenced point cloud data is i, and the number of the processed points is i-1.

And the serial numbers of the processed points in the sorted point cloud data are i-1 to 1.

In some embodiments, the ordering the point cloud data comprises:

sorting the point cloud data according to the processing sequence of the point cloud data; or

Sorting the point cloud data according to the geometric position information of the point cloud data; or alternatively

Sorting the point cloud data according to the attribute information of the point cloud data; or

And sorting the point cloud data according to the geometric position information and the attribute information of the point cloud data.

For example, at the encoding end, if the processed points are encoded points, the processing order is the encoding order. And sequencing according to the coding sequence of the point cloud data.

For example, at the encoding end, if the processed point is a decoded point, the processing order is a decoding order. And sequencing according to the decoding sequence of the point cloud data.

For example, the point cloud data are sorted according to the sequence of the attribute values in the attribute information of the point cloud data from large to small.

For example, the point cloud data are sorted according to the order of the attribute values in the attribute information of the point cloud data from small to large.

For example, when a plurality of points with the same geometric position information exist in the point cloud data, the plurality of points with the same geometric position information are sorted according to the sequence of the attribute values from large to small in the plurality of points with the same geometric position information. And performing geometric sorting on the points with different geometric position information.

In some embodiments, the ordering the point cloud data according to its geometric location information comprises:

performing Morton sorting on the point cloud data according to the geometric position information of the point cloud data; or alternatively

And performing Hilbert sorting on the point cloud data according to the geometric position information of the point cloud data.

For example, taking a decoding end as an example, morton sorting is performed on the point cloud data according to the geometric position information of the point cloud data, so as to obtain a morton code. The specific implementation manner of the morton sorting may refer to the content described in the selection manner of the morton sorting neighbor points.

For example, Hilbert ordering is performed on the point cloud data according to the geometric position information of the point cloud data, so that Hilbert codes are obtained.

A step 1022 of selecting all of the processed points in the plurality of processed points as the N alternative points when N ═ i-1;

for example, taking the decoding end as an example, when N ═ i-1, all decoded points are selected as alternative points.

Step 1023; when N < i-1, selecting the first N processing points of the plurality of processed points as the N alternative points.

For example, taking the decoding side as an example, when N < i-1, the first N decoded points are selected as the alternative points.

103, according to the distance between the N candidate points and the current point, selecting k candidate points from the N candidate points as k neighbor points of the current point, where N and k are positive integers greater than 1. Wherein k is less than or equal to N.

For example, taking the Hilbert sequence as an example, the current point to be decoded has sequence number i, and the decoded points of the preambles are i-1, i-2, …, 1, 0. The limited k points closest to the current point are selected as neighbor points. E.g. preambleHas been decoded, the distance value of the point is noted as d ₁ ，d ₂ ，…，d _k And d is ₁ <d ₂ <…<d _k 。

In some embodiments, when distances from all of the N candidate points to the current point are different, k points with the closest distance to the current point are selected from the N candidate points as the neighbor points.

For example, when all the candidate points are different from the current point in distance, the k closest candidate points are selected as the neighbor points.

In some embodiments, when there are k1 target points with the same distance to the current point in the N candidate points, k candidate points are selected from the N candidate points as k neighbor points of the current point according to a magnitude relation between the distances from the k1 target points to the current point and the distances from the N candidate points to the current point.

In some embodiments, when there are k1 target points with the same distance to the current point in the N candidate points, selecting k candidate points from the N candidate points as k neighbor points of the current point according to a size relationship between the distances from the k1 target points to the current point and the distances from the N candidate points to the current point includes:

selecting k candidate points with the nearest distance to the current point from the N candidate points as candidate neighbor points, wherein the distance from the 1 st candidate neighbor point to the current point is less than the distance from the kth candidate neighbor point to the current point;

and determining neighbor points from the k1 target points and the k candidate neighbor points according to the size relationship between the distances from the k1 target points to the current point and the distances from the k candidate neighbor points to the current point.

The 1 st candidate neighbor point is a point closest to the current point in the k candidate neighbor points, and the kth candidate neighbor point is a point farthest from the current point in the k candidate neighbor points.

For example, when there are k1 distance phasesAnd when the target points are the same, the distances between the k1 target points and the current point are set as d, and the neighbor points are determined according to different conditions. When there are k1 target points with the same distance to the current point in the N candidate points, in order to reduce the interference of the same distance to the neighbor point selection, it is necessary to further select neighbor points in the k1 target points and the N candidate points. Firstly, k candidate points with the nearest distance to the current point are selected from the N candidate points as candidate neighbor points, and the distance values of the k candidate neighbor points are recorded as d ₁ ，d ₂ ，…，d _k And d is ₁ <d ₂ <…<d _k . Then according to the distances d from the k1 target points to the current point and the distances (d) from the k candidate neighbor points to the current point ₁ ，d ₂ ，…，d _k ) The neighbor points are determined from the k1 target points and the k candidate neighbor points, including but not limited to the following methods:

(1) when d is>d _k Then, the k1 target points are skipped, and k candidate neighbor points are determined as the neighbor points.

(2) When d is _k-j <d…≤d _k And then j is more than or equal to 1 and less than or equal to k-1, j target points are selected from k1 target points as neighbor points, and the 1 st candidate neighbor point to the k-j candidate neighbor point are determined as the 1 st to the k-j neighbor points. The method for selecting j target points from the k1 target points as neighbor points includes, but is not limited to, the following methods:

selecting j target points which are the closest to the input sequence of the current point, namely j target points with the smallest difference with the input sequence number of the current point;

selecting j target points which are farthest from the input sequence of the current point, namely j target points which have the largest difference with the input sequence number of the current point;

selection and d ₁ The j target points with the smallest difference of the corresponding point attribute values, i.e. d ₁ The distance measure of the reflectivity or color of the corresponding point (single or multi-channel) is the smallest of the j target points;

calculating the attribute value mean value or weighted average value of other neighbor points (points with distances smaller than d, such as the 1 st to the k-j candidate neighbor points) as a first reference value, and selecting j target points with the minimum distance from the first reference value;

and calculating the mean value or weighted average of the attribute values of the neighbor points with different attribute values in the rest neighbor points (the distances are smaller than the points corresponding to d, such as the 1 st to the (k-j) th candidate neighbor points) as a second reference value, and selecting the (j) target points with the minimum distances from the second reference value.

(2.1) when j is 1, d ₁ <…<d _k-1 <d, selecting 1 target point from the k1 target points as a neighbor point (for example, as a k-th neighbor point), and determining the 1 st candidate neighbor point to the k-1 st candidate neighbor point as the 1 st to the k-1 st neighbor points. Among them, the method of selecting 1 target point as a neighbor point among the k1 target points includes, but is not limited to, the following methods:

selecting 1 target point which is the closest to the input sequence of the current point, namely 1 target point which has the smallest difference with the input sequence number of the current point;

selecting 1 target point which is farthest from the input sequence of the current point, namely 1 target point which has the largest difference with the input sequence number of the current point;

selection and d ₁ The 1 target point with the smallest difference of the corresponding point attribute values, i.e. d ₁ The 1 target point for which the distance measure of the reflectivity or color (single or multi-channel) of the corresponding point is minimal;

calculating the attribute value mean value or weighted average value of other neighbor points (points with distances smaller than d, such as the 1 st to the (k-1) th candidate neighbor points) as a first reference value, and selecting 1 target point with the minimum distance from the first reference value;

and calculating the mean value or weighted average of the attribute values of the neighbor points with different attribute values in the rest neighbor points (the distances are smaller than the points corresponding to d, such as the 1 st to the (k-1) th candidate neighbor points) as a second reference value, and selecting the 1 target point with the minimum distance from the second reference value.

(2.2) when j is k-1, d ₁ <d…≤d _k It is necessary to select k-1 target points among the k1 target points as neighbor points (2 nd to k th neighbor points), and determine the 1 st candidate neighbor point as the 1 st neighbor point. Wherein k-1 target points are selected from the k1 target points asMethods of neighbor points, including but not limited to the following:

selecting k-1 target points which are closest to the input sequence of the current point, namely k-1 target points which have the smallest difference with the input sequence number of the current point;

selecting k-1 target points which are farthest from the input sequence of the current point, namely k-1 target points which have the largest difference with the input sequence number of the current point;

selection and d ₁ K-1 target points with the smallest difference between the corresponding point attribute values, i.e. AND ₁ The k-1 target points for which the distance measure of reflectivity or color (single or multi-channel) of the corresponding point is minimal;

calculating the remaining neighbor points (points with distances less than d, e.g. distance d) ₁ When the candidate point corresponding to the distance d1 may be a single candidate point or a plurality of candidate points), selecting k-1 target points with the smallest distance from the first reference value as the first reference value;

calculating the remaining neighbor points (points with distances less than d, e.g. distance d) ₁ The corresponding candidate neighbor point, the candidate point corresponding to the distance d1 may be a single one, or may be a plurality of) the average or weighted average of the attribute values of the neighbor points having different attribute values is used as the second reference value, and k-1 target points having the smallest distance from the second reference value are selected.

(3) When d ═ d ₁ If so, selecting the k1 target points as neighbor points; or selecting points of the k1 target points whose attribute values are not the same as the neighbor points.

In some embodiments, the determining neighbor points from the k1 target points and the k candidate neighbor points according to the magnitude relationship between the distances from the k1 target points to the current point and the distances from the k candidate neighbor points to the current point includes:

if the distance from the k1 target points to the current point is greater than the distance from the kth candidate neighbor point to the current point, determining the k candidate neighbor points as neighbor points; or

If the distances from the k1 target points to the current point are greater than the distances from the k-j candidate neighbor points to the current point, and the distances from the k1 target points to the current point are not greater than the distance from the k candidate neighbor point to the current point, selecting j target points from the k1 target points as neighbor points according to a preset selection mode, and determining the 1 st candidate neighbor point to the k-j candidate neighbor points as the 1 st to the k-j neighbor points; or

If the distance from the k1 target points to the current point is equal to the distance from the 1 st candidate neighbor point to the current point, determining the k1 target points as neighbor points, or determining target points with different attribute values among the k1 target points as neighbor points.

For example, if j is 1, the distances from the k1 target points to the current point are greater than the distances from the k-1 st candidate neighbor point to the current point, and the distances from the k1 target points to the current point are not greater than the distance from the k candidate neighbor point to the current point, then one target point is selected from the k1 target points as the k-th neighbor point according to a preset selection manner, and the 1 st candidate neighbor point to the k-1 st candidate neighbor point are determined as the 1 st to the k-1 st neighbor points. For example, if j is 1, the preset selection mode includes any one of the following:

selecting a target point closest to the input order of the current point from the k1 target points as the kth neighbor point;

selecting a target point farthest from the input order of the current point from the k1 target points as the kth neighbor point;

selecting one target point with the smallest difference with the attribute value of the 1 st candidate neighbor point from the k1 target points as the k neighbor point;

taking the attribute value mean or weighted average of the 1 st candidate neighbor point to the (k-1) th candidate neighbor point as a first reference value, and selecting one target point with the minimum distance from the k1 target points to be the kth neighbor point;

and taking the mean value or weighted average of the attribute values of the candidate neighbor points with different attribute values from the 1 st candidate neighbor point to the (k-1) th candidate neighbor point as a second reference value, and selecting one target point with the minimum distance from the k1 target points to the second reference value as the kth neighbor point.

For example, if j is k-1, the distances from the k1 target points to the current point are greater than the distance from the 1 st candidate neighbor point to the current point, and the distances from the k1 target points to the current point are not greater than the distance from the 2 nd candidate neighbor point to the current point, then k-1 target points are selected from the k1 target points as the 2 nd to the kth neighbor points according to a preset selection manner, and the 1 st candidate neighbor point is determined as the 1 st neighbor point. For example, if j ═ k-1, the preset selection mode includes any one of the following:

selecting k-1 target points closest to the input order of the current point from the k1 target points as 2 nd to k th neighbor points;

selecting k-1 target points farthest from the input order of the current point from the k1 target points as 2 nd to k th neighbor points;

selecting k-1 target points with the minimum difference with the attribute value of the 1 st candidate neighbor point from the k1 target points as 2 nd to k th neighbor points;

taking the attribute value mean or weighted average of the candidate neighbor points of the k candidate neighbor points whose distances to the current point are less than the distances from the k1 target points to the current point as a first reference value, and selecting k-1 target points with the smallest distance from the first reference value from the k1 target points as 2 nd to k th neighbor points;

and taking the mean value or weighted average of the attribute values of the candidate neighbor points with different attribute values in the candidate neighbor points with the distances from the k candidate neighbor points to the current point smaller than the distances from the k1 target points to the current point as a second reference value, and selecting k-1 target points with the minimum distance from the second reference value from the k1 target points as 2 nd to k th neighbor points.

In some embodiments, the method further comprises:

if the number of the determined neighbor points is less than k, selecting the point with the closest distance to the current point from the alternative points excluding the determined neighbor points from the N alternative points according to the relationship that the distances from the N alternative points to the current point are from small to large so as to complement the k neighbor points.

For example, if the determined neighbor points are less than k points, the k points are complemented by the neighbor points according to the distance metric.

For example, if the determined neighbor points are less than k points, no operation may be performed.

In some embodiments, selecting k candidate points from the N candidate points as k neighbor points of the current point according to the distance between the N candidate points and the current point includes:

when the distances from all the N alternative points to the current point are the same, selecting k alternative points from the N alternative points as the k neighbor points; or alternatively

When the distances from all the N alternative points to the current point are the same, selecting k alternative points with different attribute values from the N alternative points as the k neighbor points;

when the distances from all the N alternative points to the current point are the same, selecting k alternative points which are the closest to the input sequence of the current point as the k neighbor points; or

And when the distances from all the N alternative points to the current point are the same, selecting k alternative points which are farthest from the input sequence of the current point as the k neighbor points.

And 104, determining the attribute predicted value of the current point according to the attribute values of the k neighbor points.

In some embodiments, the determining the predicted value of the attribute of the current point according to the attribute values of the k neighbor points includes:

calculating the weighted average value of the attribute values of the k neighbor points by taking the reciprocal of the distance between each neighbor point and the current point as the weight of each neighbor point so as to obtain the attribute predicted value of the current point; or

Setting the same or different weights for different neighbor points in the k neighbor points, and calculating the weighted average value of the attribute values of the k neighbor points, wherein the neighbor points with closer distances are set with larger weights; or

And taking the attribute value of the neighbor point closest to the current point in the k neighbor points as the attribute predicted value of the current point.

For example, the obtained attribute values of the neighbor points are used to calculate the attribute prediction value of the current point, and the specific calculation process includes, but is not limited to, the following methods:

(1) and calculating a weighted average value according to the distance between the neighbor point and the current point. For example, the euclidean distance metric is adopted, the reciprocal of the distance is used as the weight of each neighbor point, and the predicted value of the attribute of the current point is the weighted average value of the attribute values of each neighbor point.

(2) And calculating the weighted average value according to the initial setting, wherein the same or different weights can be set for different neighbor points by the encoding end, the corresponding weights are obtained by the decoding end through analyzing the code stream, and the attribute predicted value of the current point is the weighted average of the attribute values of all the neighbor points. For example, a larger weight value is set to a point closer to the center.

(3) And adopting the attribute value of the nearest neighbor point as the attribute predicted value of the current point.

For example, taking a coding end as an example, point cloud data is obtained, where the point cloud data includes a current point to be coded and a plurality of coded points, then N coded points are selected from the plurality of coded points as N candidate points, then k candidate points are selected from the N candidate points as k neighbor points of the current point to be coded according to the distance between the N candidate points and the current point, and N and k are positive integers greater than 1; and then determining an attribute predicted value of the current point to be coded according to the attribute values of the k neighbor points. And determining an attribute residual value of the current point according to the difference value between the attribute value of the current point and the attribute predicted value of the current point, and encoding the attribute residual value of the current point into a compressed code stream.

For example, taking a decoding end as an example, point cloud data is obtained, where the point cloud data includes a current point to be decoded and a plurality of decoded points, then N decoded points are selected from the plurality of decoded points as N candidate points, then k candidate points are selected from the N candidate points as k neighbor points of the current point to be decoded according to the distance between the N candidate points and the current point, and N and k are positive integers greater than 1; and then determining the attribute predicted value of the current point to be decoded according to the attribute values of the k neighbor points. And decoding the compressed code stream at a decoding end to obtain an attribute residual value of the current point to be decoded, finally calculating the sum of the attribute predicted value of the current point to be decoded and the attribute residual value of the current point to be decoded, determining an attribute reconstruction value of the current point to be decoded, and finishing attribute decoding.

All the above technical solutions can be combined arbitrarily to form the optional embodiments of the present application, and are not described herein again.

According to the method and the device, point cloud data are obtained, the point cloud data comprise a current point to be processed and a plurality of processed points, then N processed points are selected from the plurality of processed points to serve as N candidate points, then k candidate points are selected from the N candidate points to serve as k neighbor points of the current point according to the distance between the N candidate points and the current point, wherein N and k are positive integers larger than 1, and finally the attribute predicted value of the current point is determined according to the attribute values of the k neighbor points. According to the method and the device, the alternative points can be selected by utilizing the geometric relation and the attribute relation of the processed points, and the specific neighbor points are selected from the alternative points, so that richer neighbor point selection modes are provided, and the prediction efficiency of the point cloud attributes is improved.

In order to better implement the method for processing point cloud data according to the embodiments of the present application, an embodiment of the present application further provides a device for processing point cloud data. Referring to fig. 2, fig. 2 is a schematic block diagram of a device for processing point cloud data according to an embodiment of the present disclosure, which may belong to a decoding end, for example, a decoding apparatus. Alternatively, the apparatus may also belong to an encoding end, such as an encoding device.

As shown in fig. 2, the processing device 20 for point cloud data may include:

an acquisition unit 21 configured to acquire point cloud data including a current point to be processed and a plurality of processed points;

a first selecting unit 22 configured to select N processed points from the plurality of processed points as N alternative points;

a second selecting unit 23, configured to select k candidate points from the N candidate points as k neighbor points of the current point according to the distance between the N candidate points and the current point, where N and k are positive integers greater than 1;

and the determining unit 24 is configured to determine the predicted attribute value of the current point according to the attribute values of the k neighboring points.

In some embodiments, the first selecting unit 22 is specifically configured to:

sequencing the point cloud data, wherein the serial number of the current point in the sequenced point cloud data is i, and the number of the processed points is i-1;

when N-1, selecting all of the processed points as the N alternative points; or

When N < i-1, selecting the first N processing points of the plurality of processed points as the N alternative points.

In some embodiments, the first selecting unit 22 is configured to sort the point cloud data, and specifically includes:

Sorting the point cloud data according to the geometric position information of the point cloud data; or

In some embodiments, the first selecting unit 22 is configured to sort the point cloud data according to the geometric position information of the point cloud data, and specifically includes:

performing Morton sorting on the point cloud data according to the geometric position information of the point cloud data; or

In some embodiments, the second selecting unit 23 is configured to, when distances from all the N candidate points to the current point are different, select, as the neighbor point, k points with a closest distance to the current point from the N candidate points.

In some embodiments, the second selecting unit 23 is configured to, when there are k1 target points with the same distance to the current point in the N candidate points, select k candidate points from the N candidate points as k neighbor points of the current point according to a size relationship between the distances from the k1 target points to the current point and the distances from the N candidate points to the current point.

In some embodiments, the second selecting unit 23 is specifically configured to:

and according to the size relation between the distances from the k1 target points to the current point and the distances from the k candidate neighbor points to the current point, determining neighbor points from the k1 target points and the k candidate neighbor points.

In some embodiments, the second selecting unit 23 is configured to determine, according to a size relationship between distances from the k1 target points to the current point and distances from the k candidate neighbor points to the current point, neighbor points from the k1 target points and the k candidate neighbor points, and specifically includes:

In some embodiments, the preset selection mode comprises any one of:

selecting j target points which are closest to the input sequence of the current point from the k1 target points as neighbor points;

selecting j target points which are farthest from the input sequence of the current point from the k1 target points as neighbor points;

selecting j target points with the smallest attribute value difference with the 1 st candidate neighbor point from the k1 target points as neighbor points;

taking the attribute value mean value or weighted average value from the 1 st candidate neighbor point to the k-j candidate neighbor points as a first reference value, and selecting j target points with the minimum distance from the k1 target points to serve as neighbor points;

and taking the mean value or weighted average of the attribute values of the candidate neighbor points with different attribute values from the 1 st candidate neighbor point to the k-j candidate neighbor points as a second reference value, and selecting j target points with the minimum distance from the second reference value from the k1 target points as the neighbor points.

In some embodiments, the second selecting unit 23 is further configured to, if the number of the determined neighbor points is less than k, select, according to a relationship that distances from the N candidate points to the current point are from small to large, a point closest to the current point from candidate points excluding the determined neighbor points from the N candidate points, so as to complement the k neighbor points.

In some embodiments, the second selecting unit 23 is configured to select k candidate points of the N candidate points as the k neighbor points when distances from all the candidate points of the N candidate points to the current point are the same; or

In some embodiments, the determination unit 24 is configured to:

It is to be understood that apparatus embodiments and method embodiments may correspond to one another and that similar descriptions may refer to method embodiments. To avoid repetition, further description is omitted here. Specifically, the apparatus shown in fig. 2 may execute the above-mentioned embodiment of the method for processing point cloud data, and the foregoing and other operations and/or functions of each module in the apparatus are respectively for implementing the corresponding flow of the method embodiment corresponding to the decoding end, and are not described herein again for brevity.

The apparatus of the embodiments of the present application is described above in connection with the drawings from the perspective of functional modules. It should be understood that the functional modules may be implemented by hardware, by instructions in software, or by a combination of hardware and software modules. Specifically, the steps of the method embodiments in the present application may be implemented by integrated logic circuits of hardware in a processor and/or instructions in the form of software, and the steps of the method disclosed in conjunction with the embodiments in the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. Alternatively, the software modules may be located in random access memory, flash memory, read only memory, programmable read only memory, electrically erasable programmable memory, registers, and the like, as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps in the above method embodiments in combination with hardware thereof.

Fig. 3 is a schematic block diagram of an electronic device 30 provided in an embodiment of the present application, where the electronic device 30 may be a decoding device or an encoding device. The electronic device 30 may include:

a memory 31 and a processor 32, the memory 31 being arranged to store a computer program 34 and to transfer the program code 34 to the processor 32. In other words, the processor 32 may call and run the computer program 34 from the memory 31 to implement the method in the embodiment of the present application.

For example, the processor 32 may be configured to perform the following steps according to instructions in the computer program 34:

acquiring point cloud data, wherein the point cloud data comprises a current point to be processed and a plurality of processed points; selecting N processed points from the plurality of processed points as N alternative points; selecting k candidate points from the N candidate points as k neighbor points of the current point according to the distance between the N candidate points and the current point, wherein N and k are positive integers greater than 1; and determining the attribute predicted value of the current point according to the attribute values of the k neighbor points.

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

In some embodiments of the present application, the processor 32 may include, but is not limited to:

general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like.

In some embodiments of the present application, the memory 31 includes, but is not limited to:

volatile memory and/or non-volatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of example, but not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic random access memory (Synchronous DRAM, SDRAM), Double Data Rate Synchronous Dynamic random access memory (DDR SDRAM), Enhanced Synchronous SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

In some embodiments of the present application, the computer program 34 may be divided into one or more units, which are stored in the memory 31 and executed by the processor 32 to implement the methods provided by the embodiments of the present application. The one or more elements may be a series of computer program instruction segments capable of performing certain functions, which are used to describe the execution of the computer program 34 in the electronic device 30.

As shown in fig. 3, the electronic device 30 may further include:

a transceiver 33, the transceiver 33 being connectable to the processor 32 or the memory 31.

The processor 32 may control the transceiver 33 to communicate with other devices, and specifically, may transmit information or data to the other devices or receive information or data transmitted by the other devices. The transceiver 33 may include a transmitter and a receiver. The transceiver 33 may further include antennas, and the number of antennas may be one or more.

It should be understood that the various components in the electronic device 30 are connected by a bus system that includes a power bus, a control bus, and a status signal bus in addition to a data bus.

The present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a computer, enables the computer to execute the steps of any one of the methods for processing point cloud data provided in the embodiments of the present application. In other words, the present application also provides a computer program product containing instructions, which when executed by a computer, cause the computer to perform the steps of any one of the methods for processing point cloud data provided in the present application.

When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the present application occur, in whole or in part, when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Video Disk (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), among others.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the unit is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment. For example, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for processing point cloud data is characterized by comprising the following steps:

2. The method of processing point cloud data of claim 1, wherein said selecting N processed points from said plurality of processed points as N candidate points comprises:

when N-1, selecting all of the processed points as the N alternative points; or

3. The method of processing point cloud data of claim 2, wherein said sorting said point cloud data comprises:

4. The method of processing point cloud data of claim 3, wherein said sorting the point cloud data according to its geometric location information comprises:

5. The method for processing point cloud data according to claim 2, wherein said selecting k candidate points from the N candidate points as k neighbor points of the current point according to the distance between the N candidate points and the current point comprises:

and when the distances from all the N alternative points to the current point are different, selecting k points with the shortest distance from the N alternative points to the current point as the neighbor points.

6. The method for processing point cloud data according to claim 2, wherein said selecting k candidate points from the N candidate points as k neighbor points of the current point according to the distance between the N candidate points and the current point comprises:

when k1 target points with the same distance to the current point exist in the N alternative points, k alternative points are selected from the N alternative points as k neighbor points of the current point according to the size relationship between the distance from the k1 target points to the current point and the distance from the N alternative points to the current point.

7. The method for processing point cloud data according to claim 6, wherein when there are k1 target points with the same distance to the current point in the N candidate points, selecting k candidate points from the N candidate points as k neighbor points of the current point according to the magnitude relationship between the distances from the k1 target points to the current point and the distances from the N candidate points to the current point comprises:

8. The method for processing point cloud data of claim 7, wherein said determining neighbor points from said k1 target points and said k candidate neighbor points according to the magnitude relationship between the distances from said k1 target points to said current point and the distances from said k candidate neighbor points to said current point comprises:

9. The method for processing point cloud data according to claim 8, wherein the preset selection mode comprises any one of the following:

10. The method of processing point cloud data of claim 7, the method further comprising:

11. The method for processing point cloud data according to claim 2, wherein selecting k candidate points from the N candidate points as k neighbor points of the current point according to the distance between the N candidate points and the current point comprises:

when the distances from all the N alternative points to the current point are the same, selecting k alternative points from the N alternative points as the k neighbor points; or

12. The method for processing point cloud data according to any one of claims 2 to 11, wherein the determining the predicted value of the attribute of the current point according to the attribute values of the k neighbor points comprises:

calculating the weighted average value of the attribute values of the k neighbor points by taking the reciprocal of the distance between each neighbor point and the current point as the weight of each neighbor point so as to obtain the attribute predicted value of the current point; or alternatively

13. An apparatus for processing point cloud data, comprising:

14. A computer-readable storage medium, characterized in that it stores a computer program adapted to be loaded by a processor for performing the method of processing point cloud data according to any of claims 1 to 12.

15. A decoding apparatus, characterized in that the decoding apparatus comprises a processor and a memory, the memory having stored therein a computer program, the processor being configured to execute the method of processing point cloud data according to any one of claims 1 to 12 by calling the computer program stored in the memory.

16. An encoding apparatus, characterized in that the encoding apparatus comprises a processor and a memory, the memory having stored therein a computer program, the processor being configured to execute the method of processing point cloud data according to any one of claims 1 to 12 by calling the computer program stored in the memory.