Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/62—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding by frequency transforming in three dimensions
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/18—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

• the present disclosure generally relates to the field of point cloud data sources.
• PCC point cloud compression
• the disclosure concerns a method for encoding a point cloud and a corresponding encoder. It also concerns a method for reconstructing a point cloud and a corresponding decoder. It further concerns computer programs implementing the encoding and reconstructing methods of the invention.
• a point cloud consists in a set of points. Each point is defined by its spatial location (x, y, z), i.e. geometry information, and different attributes, which typically include the color information in (R, G, B) or (Y, U, V) or any other color coordinate system. Geometry can be regarded as one of the attribute data. In the rest of this disclosure, both geometry and other attribute data are considered as attributes of points.
• Point cloud data sources are found in many applications. Important applications relying on huge point cloud data sources can be found in geographic information systems, robotics, medical tomography and scientific visualization.
• Scanned 3D point clouds often have thousands of points and occupy large amounts of storage space. Additionally, they can be generated at a high rate when captured live from 3D scanners, increasing the data rate even further. Therefore, point cloud compression is critical for efficient networked distribution and storage.
• the first coding approach is based on the octree based point-cloud representation. It is described, for instance, in J. Peng and C.-C. Jay Kuo, "Geometry-guided progressive lossless 3D mesh coding with octree (OT) decomposition," ACM Trans. Graph, vol. 21 , no. 2, pp. 609-616, July 2005 and in Yan Huang, Jingliang Peng, C.-C. Jay Kuo and M. Gopi, "A Generic Scheme for Progressive Point Cloud Coding," IEEE Transactions on Visualization and Computer Graphics, vol. 14, no. 2, pp. 440-453, 2008.
• An octree is a tree data structure where every branch node represents a certain cube or cuboid bounding volume in space. Starting at the root, every branch has up to eight children, one for each sub-octant of the node's bounding box.
• a single bit is used to mark whether every child of a branch node is empty and then this branch node configuration can be efficiently represented in a single byte, assuming some consistent ordering of the eight octants.
• the point distribution in space can be efficiently encoded.
• the number of bits set in the first byte tells the decoder the number of consecutive bytes that are direct children.
• the bit positions specify the voxel/child in the octree they occupy.
• the right half side of Figure 2 illustrates this byte stream representation for the octree represented in the left half side.
• octree representation Due to its hierarchical nature, the octree representation is very efficient in exploiting the sparsity of point clouds. Thus, octree decomposition based strategy is very efficient for compressing sparse point clouds. However, octree-based representation is inefficient for representing and compressing watertight dense point clouds. Moreover, most of the existing octree-based point cloud representations independently represent geometry and other attributes of points.
• the second coding approach is based on the segmentation based point cloud representation. It is described, for instance, in T. Ochotta and D. Saupe, "Compression of point-based 3d models by shape-adaptive wavelet coding of multi-heightfields," in Proc. Eurographics Symp. on Point- Based Graphics, 2004, and in J. Digne, R. Chaine, S. Valette, et al, "Self-similarity for accurate compression of point sampled surfaces," Computer Graphics Forum, vol. 33, p. 155-164, 2014.
• the segmentation based point cloud representation comprises three steps: plane-like decomposition, plane projection and coding.
• the point cloud data is segmented into plane-like patches.
• the resultant patches are projected onto one or several planes.
• the projected image(s) are compressed. Efficient image/video compression techniques can be used for compressing the projected image(s).
• the present disclosure proposes a solution for improving the situation.
• the disclosure concerns a method for encoding a point cloud according to claim 1 and a corresponding encoder according to claim 2. It also concerns a method for reconstructing a point cloud according to claim 8 and a corresponding decoder according to claim 9. It further concerns computer programs implementing the encoding and reconstructing methods of the invention. It also concerns a signal according to claim 15.
• the present disclosure provides a method for encoding a point cloud comprising a plurality of points, in which each point is defined by attributes, the attributes including the spatial position of the point in a 3D space and at least one feature of the point, wherein the method comprises:
• GFT Graph Fourier Transform
• the coding method of the invention jointly compresses the attributes, which include the geometry i.e. the spatial position of the points in the 3D space and at least one other attribute i.e. a feature such as the color or the texture, and exploits the redundancy among them based on the GFT.
• the graph transform operation is performed within each cluster and thus it is more efficient in the sense of processing time.
• the method comprises, for each cluster, transmitting to a decoder the encoded GFT coefficients and identifiers of the cluster and of the computed GFT.
• identifiers may consist on cluster indices and GFT indices in order to indicate to the decoder which GFT is used for which cluster so that the decoder can perform the correct inverse transform for the final reconstruction of the point cloud.
• the encoding of the GFT coefficients is an entropy encoding.
• the segmentation of the point cloud into clusters uses a normalized cuts technique.
• vertices of the constructed similarity graph consist of the points of the corresponding cluster and the method comprises assigning a weight w between any vertices P, and Pj of the graph.
• the weight w between vertices Pi and Pj is set to 0 if said weight is less than a threshold.
• the small weights are not encoded so as to reduce the overall coding bits while not degrading the coding quality as a small weight means that the corresponding connected vertices are dissimilar to some extent.
• constructing the similarity graph comprises adding an edge e,,j between vertices Pi and Pj if the weight w is equal to or greater than the threshold.
• the present disclosure also provides an encoder for encoding a point cloud comprising a plurality of points, in which each point is defined by attributes, the attributes including the spatial position of the point in a 3D space and at least one feature of the point, wherein the encoder comprises : a segmentation module configured to segment the point cloud into clusters of points on the basis of the attributes of the points;
• a construction module configured to construct, for each cluster, a similarity graph representing the similarity among neighboring points of the cluster in terms of the attributes
• a computation module configured to compute, for each cluster; a Graph Fourier Transform, GFT, based on the constructed similarity graph, the GFT being characterized by its coefficients; and a coding module configured to encode the GFT coefficients.
• the encoder comprises a transmitter configured to transmit to a decoder the encoded GFT coefficients and identifiers of the clusters and of the computed GFTs.
• the present disclosure also provides a method for reconstructing a point cloud comprising a plurality of points, in which each point is defined by attributes, the attributes including the spatial position of the point in a 3D space and at least one feature of the point, wherein the method comprises: receiving data including GFT coefficients associated with cluster indices and GFT indices; decoding the received data; and
• the decoding includes an entropy decoding.
• the present disclosure also provides a decoder for reconstructing a point cloud comprising a plurality of points, in which each point is defined by attributes, the attributes including the spatial position of the point in a 3D space and at least one feature of the point, wherein the decoder comprises: a receiver configured to receive encoded data including GFT coefficients associated with cluster indices and GFT indices;
• a decoding module configured to decode the received data
• a reconstruction module configured to reconstruct clusters by performing for each cluster identified by a received cluster index an inverse GFT.
• the methods according to the disclosure may be implemented in software on a programmable apparatus. They may be implemented solely in hardware or in software, or in a combination thereof. Since these methods can be implemented in software, they can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium.
• a carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like.
• the diagram of figure 3 illustrates an example of the general algorithm for such computer program.
• the disclosure also provides a computer-readable program comprising computer-executable instructions to enable a computer to perform the reconstructing method of the invention.
• the diagram of figure 4 illustrates an example of the general algorithm for such computer program.
• Figure 1 is a schematic view illustrating an octree-based point cloud representation according to the prior art
• Figure 2 already described, is a schematic view illustrating an overview of an octree data structure;
• Figure 3 is a flowchart showing the steps of encoding a point cloud, according to an embodiment of the present invention
• Figure 4 is a flowchart showing the steps of reconstructing a point cloud, according to an embodiment of the present invention.
• Figure 5 is a schematic view illustrating an encoder, according to an embodiment of the invention.
• Figure 6 is a schematic view illustrating a decoder, according to an embodiment of the invention.
• the points of the point cloud PC are characterized by their attributes which include the geometry, i.e. the spatial position of each point and at least another attribute of the point, such as for example the color.
• attributes include the geometry, i.e. the spatial position of each point and at least another attribute of the point, such as for example the color.
• each point of the point cloud has two attributes : the geometry g in the 3D space and the color c in the RGB space.
• the point cloud data is classified into several clusters at step 2. As a result of this clustering, the point cloud data in each cluster can be approximated by a linear function as it has generally been observed that most point cloud data has a piecewise linear behavior.
• This clustering may be performed by any prior art clustering method.
• the clustering is performed using the normalized cuts technique modified, according to the present embodiment, in order to consider similarities between all the attributes of the points of the point cloud.
• multi-lateral relations are considered. That is, both similarities in the geometry and in the other attributes, such as the color, are taken into account to segment the point cluster in order to achieve a clustering result that exploits the piecewise linear property in an optimal way.
• the point cloud PC is clustered into a plurality of clusters identified by corresponding indices C,, wherein 1 ⁇ i ⁇ N, where N is the obtained number of clusters.
• the cluster indices are assigned randomly, while ensuring that a unique index is assigned for each cluster.
• a similarity graph is constructed for each cluster.
• This similarity graph represents the similarity among neighboring vertices, i.e. points, of the cluster and permits to acquire a compact representation of the cluster.
• a graph G (V, E, W) consists of a finite set of vertices V with cardinality
• N, a set of edges E connecting vertices, and a weighted adjacency matrix W .
• (J is a parameter used to control the weight value and is often set empirically.
• the graph is constructed as follows.
• the possible edge weight between it and any other point Pj in the cluster is calculating according to equation (1 ). If j j >— t, an edge 6?j j between Pi and Pj is added and the calculated weight jj is attached to the edge, otherwise, Pi and Pj are disconnected as a small weight means that there's a large discrepancy between them.
• step 6 the GFT is calculated for each cluster based on the constructed similarity graph.
• the weighted adjacency matrix is obtained from the weights of the graph. Then, the graph Laplacian matrix is computed. There exist different variants of Laplacian matrices.
• This Laplacian matrix is employed in one embodiment for two reasons.
• 0 is guaranteed to be an eigenvalue with [l . . . l] T as the corresponding eigenvector.
• This allows a frequency interpretation of the GFT, where the eigenvalues ⁇ 's are the graph frequencies and always have a DC component, which is beneficial for the compression of point cloud data consisting of many smooth regions.
• high-frequency coefficient are reduced, which leads to a compact representation of the point cloud in the GFT domain.
• GFT defaults to the well-known DCT when defined for a line graph (corresponding to the 1 D DCT) or a 4-connectivity graph (2D DCT) with all edge weights equal to 1 . That means that the GFT is at least as good as the DCT in sparse signal representation if the weights are chosen in this way. Due to the above two desirable properties, the unnormalized Laplacian matrix is used for the definition of the GFT as described in the following paragraph.
• the obtained GFT is identified by a GFT index.
• step 6 results in the GFT coefficients and the GFT indices. Then, at step 8, the GFT coefficients are quantized.
• the quantized GFT coefficients are entropy coded, for example by using the CABAC method described in Marpe : "Context-Based Adaptive Binary Arithmetic Coding in the H.264/AVC Video Compression Standard", IEEE Trans. Cir. and Sys. For Video Technol. , vol.13, no.7, pages 620-636, 2003.
• the overhead constituted by the cluster indices and the GFT indices, is entropy coded at step 10 to indicate which GFT is used for which cluster.
• the encoded GFT coefficients, cluster indices and GFT indices are then transmitted, at step 12, to a decoder.
• Figure 4 shows the steps of reconstructing a point cloud, i.e. the decoding steps implemented by the decoder after receiving the encoded GFT coefficients, cluster indices and GFT indices, according to an embodiment of the present disclosure.
• the received GFT coefficients, cluster indices and GFT indices are entropy decoded.
• the decoded GFT coefficients are dequantized.
• an inverse GFT is performed using the dequantized GFT coefficient and the decoded GFT indices which indicate the applied GFT.
• the inverse GFT is given by: where X is the recovered signal representing the point cloud data of each cluster.
• the graph Fourier transform is calculated as follows, where each point in the point cloud is treated as a vertex in a graph. Firstly, each point is connected to its neighbors as long as they are similar. Two points are disconnected if there is a large discrepancy between them.
• L D— W .
• the eigenvectors U of L are the basis vectors of the GFT.
• the attributes of the points of the point cloud are stacked into a column vector, and the GFT and inverse GFT are computed according to equations 3 and 4.
• Figure 5 is a block diagram of an exemplary embodiment of an encoder 30 implementing the encoding method of the present disclosure.
• the encoder 30 includes one or more processors and a memory 32.
• the encoder 30 comprises: a segmentation module 34 configured to segment the point cloud into clusters of points on the basis of the attributes of the points;
• a construction module 36 configured to construct, for each cluster, a similarity graph representing the similarity among neighboring points of the cluster in terms of the attributes;
• a computation module 38 configured to compute, for each cluster; a Graph Fourier Transform, GFT, based on the constructed similarity graph, the GFT being characterized by its coefficients; and
• a coding module 40 configured to encode the GFT coefficients.
• the encoder 30 also comprises a transmitter 42 configured to transmit to a decoder the encoded GFT coefficients and identifiers of the clusters and of the computed GFTs.
• a bus 44 provides a communication path between various elements of the encoder 30.
• Other point-to-point interconnection options e.g. non-bus architecture are also feasible.
• Figure 6 is a block diagram of an exemplary embodiment of a decoder 50 implementing the reconstructing method of the present disclosure.
• the decoder 50 includes one or more processors and a memory 52.
• the decoder 50 comprises : a receiver 54 configured to receive encoded data including GFT coefficients associated with cluster indices and GFT indices;
• decoding module 66 configured to decode the received data
• a reconstruction module 58 configured to reconstruct the clusters by performing for each cluster an inverse GFT.
• a bus 60 provides a communication path between various elements of the decoder 50.
• Other point-to-point interconnection options e.g. non-bus architecture are also feasible.
• graph transform is a GFT
• other graph transforms may also be used such as, for example, wavelets on graphs, as described in D. Hammond, P. Vandergheynst, and R. Gribonval, "Wavelets on graphs via spectral graph theory” in Elsevier: Appplied and Computational Harmonic Analysis, vol. 30, April 2010, pp. 129-150, and lifting transforms on graphs, as described in G. Shen, "Lifting transforms on graphs: Theory and applications” in Ph.D. dissertation, University of Southern California, 2010.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)
• Image Processing (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=59061945

Family Applications (1)

Country Status (6)

Cited By (5)

Families Citing this family (18)

Citations (1)

Family Cites Families (19)

• 2017
  • 2017-05-24 EP EP17305610.2A patent/EP3407607A1/en not_active Withdrawn
• 2018
  • 2018-04-12 MX MX2019014040A patent/MX2019014040A/es unknown
  • 2018-04-12 WO PCT/EP2018/059420 patent/WO2018215134A1/en not_active Ceased
  • 2018-04-12 US US16/616,491 patent/US11627339B2/en active Active
  • 2018-04-12 CN CN201880034063.5A patent/CN110663255B/zh active Active
  • 2018-04-12 EP EP18719779.3A patent/EP3632112A1/en active Pending
  • 2018-04-12 JP JP2019561153A patent/JP7309616B2/ja active Active

Patent Citations (1)

Non-Patent Citations (16)

Also Published As

Similar Documents

Legal Events