WO2018162406A1 - Quality metric for point clouds - Google Patents

Quality metric for point clouds Download PDF

Info

Publication number
WO2018162406A1
WO2018162406A1 PCT/EP2018/055335 EP2018055335W WO2018162406A1 WO 2018162406 A1 WO2018162406 A1 WO 2018162406A1 EP 2018055335 W EP2018055335 W EP 2018055335W WO 2018162406 A1 WO2018162406 A1 WO 2018162406A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
images
point
points
viewpoints
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2018/055335
Other languages
French (fr)
Inventor
Sébastien Lasserre
Kangying Cai
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Thomson Licensing SAS
Original Assignee
Thomson Licensing SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing SAS filed Critical Thomson Licensing SAS
Publication of WO2018162406A1 publication Critical patent/WO2018162406A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

  • the present disclosure generally relates to the field of point cloud data sources.
  • a point cloud processing such as a point cloud distortion, or deformation, as used in compression for example.
  • the disclosure concerns a method for measuring a quality of a result of a point cloud processing and a corresponding quality measuring device. It further concerns an encoder comprising the quality measuring device of the disclosure.
  • the disclosure also concerns a computer program implementing the quality measuring method of the invention.
  • a point cloud consists of a set of points. Each point is defined by its spatial location (x, y, z), i.e. geometry information, and different attributes, which typically include the color information in (R, G, B) or (Y, U, V) or any other color coordinate system.
  • Point cloud data sources are found in many applications. Important applications relying on huge point cloud data sources can be found in geographic information systems, robotics, medical tomography and scientific visualization.
  • RMSE Root Mean Square Error
  • a second option for the measurement is to triangulate the point cloud into a 3D mesh and to use the already existing metrics for 3D meshes, such as the symmetric root mean square or symmetric Hausdorff distance.
  • the triangulation method used would affect the resulting distortion metric.
  • metrics for the colors and attribute quality would not be well defined.
  • a point cloud specific quality or distortion metric is useful for the evaluation of compression algorithms.
  • the geometric and color error of these points are calculated by being compared with their corresponding reference points.
  • the geometric PSNR Peak Signal to Noise Ratio
  • the PSNR per YUV component is calculated in a similar way.
  • the second type of quality metrics is based on projection.
  • the authors assume an orthogonal projection of the point cloud over the six sides of the bounding-box of the point cloud.
  • the distortion metric between two point clouds is then the MSE (or PSNR) between the two corresponding composite images, each with the six orthogonal sides.
  • the projection based metric is a good solution for the unified quality metric of point clouds.
  • projection only on the six sides of the bounding-box is not enough, especially for those point clouds with complex shapes.
  • the present invention proposes a solution for improving the situation.
  • the present invention provides a method for measuring a quality of a processing of a first 3D point cloud comprising a plurality of points, in which each point is defined by its spatial location and by at least one attribute, in which the processing transforms the first point cloud into a second 3D point cloud, wherein the method includes:
  • the method of the invention is based on the comparison of the projection of the first point cloud, i.e. the reference or original point cloud, and the projection of the second point cloud, that is the point cloud of which the quality metric is evaluated, i.e. the processed or reconstructed point cloud, on N viewpoints.
  • the number N is automatically selected according to the shape of the first point cloud.
  • each point in the first point cloud or in the second point cloud is represented by a sphere.
  • each point in the first point cloud or in the second point cloud is represented by a disk.
  • determining the N first images and N second images comprises rendering the N first images and the N second images.
  • determining the N first images and N second images comprises estimating the N first images and N second images without rendering the N first images and N second images. Without rendering the images, the complexity of the method and the requirement of the necessary hardware are reduced.
  • estimating the N first images and N second images includes:
  • these contributions are determined at a point of the second point cloud or at a resampled point chosen from points uniformly distributed on a surface represented by the first point cloud or the second point cloud.
  • the attribute of each point in the first point cloud and of each point in the second point cloud is represented by a Gaussian distribution function.
  • comparing the N first images and the N second images includes comparing the contribution of both the spatial locations and the attributes of all the points of the first point cloud and the contribution of both the spatial locations and the attributes of all the points of the second point cloud.
  • determining a quality metric of the second point cloud uses a symmetric Hausdorff distance and/or a Root-Mean-Square, RMS, distance between the first point cloud and the second point cloud.
  • the at least one attribute includes the color.
  • the present invention also provides a device for measuring a quality of a processing of a first 3D point cloud comprising a plurality of points, in which each point is defined by its spatial location and by at least one attribute, in which the processing transforms the first point cloud into a second 3D point cloud, wherein the device includes:
  • an analyzing module configured to analyze the first point cloud to determine a shape of said first point cloud
  • a selection module configured to select, based on the determined shape, a number N of viewpoints distributed over a set of surfaces centered on the first point cloud;
  • an evaluation module configured to determine N first images corresponding to projection views of the first point cloud using the N viewpoints and to determine N second images corresponding to projection views of the second point cloud using the N viewpoints;
  • a comparison module configured to compare the N first images and the N second images and to determine a quality metric of the second point cloud from the comparison.
  • the evaluation module is configured to render the N first images and the N second images. According to a second embodiment, the evaluation module is configured to estimate the N first images and the N second images without rendering the N first images and the N second images.
  • the present disclosure further provides an encoder comprising the device of the invention.
  • the method according to the invention may be implemented in software on a programmable apparatus. It may be implemented solely in hardware or in software, or in a combination thereof.
  • a carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like.
  • the invention thus provides a computer-readable program comprising computer- executable instructions to enable a computer to perform the quality measurement method of the invention.
  • the diagram of figure 3 illustrates an example of the general algorithm for such computer program.
  • FIG. 1 is a schematic view illustrating the principle of a point cloud quality metric evaluation, according to an embodiment of the invention
  • - Figure 2 is a schematic view illustrating the use of the point cloud quality metric evaluation in an inter coding system of a time varying point cloud, according to an embodiment of the invention
  • - Figure 3 is a flowchart showing the steps of the quality measurement method, according to an embodiment of the present invention.
  • - Figure 4 shows an example of viewpoints distribution, according to an embodiment of the present invention;
  • - Figure 5 shows an example of selection of viewpoints, according to an embodiment of the present invention;
  • FIG. 6 shows an example of a Gaussian distribution of the contribution of the color of a point in a point cloud, according to an embodiment of the present invention.
  • FIG. 7 is a schematic view illustrating a quality measurement device, according to an embodiment of the invention.
  • a point cloud is a set of points in a 3D space, each with associated attributes, e.g. color, material properties, etc.
  • Point clouds can be used to reconstruct an object or a scene as a composition of such points. They can be captured using multiple cameras and depth sensors in various setups and may be made up of thousands up to billions of points in order to realistically represent reconstructed scenes.
  • An illustration of a point cloud referenced PC1 is shown in Figure 4.
  • the point cloud PC1 is defined as a set of points wherein a point v has as a mandatory position in a 3D space (x,y,z) and an optional color attribute c that has components r,g,b or y,u,v and optional other attributes (a 0 , ...a A ) possibly representing normal or texture mappings.
  • point cloud PC1 is then simply a set of K points without a strict ordering:
  • the point cloud PC1 is processed using a processing operation 10 resulting in a second point cloud PC2.
  • This processing is for example a lossy or lossless compression comprising for instance compression of geometry and attributes, scalable/progressive coding, coding of sequences of point clouds captured over time, and random access to subsets of the point cloud.
  • Other examples of the processing operation include point cloud upsampling or downsampling which changes the number of points of the point cloud, point cloud resampling which keeps the same number of points, point cloud deformation, etc.
  • the present invention provides a quality metric 12 for the evaluation of the quality of the result of the processing 10 operated on the original point cloud PC1 , or in other words a quality metric for the second point cloud PC2.
  • the first point cloud PC1 (original) will act as a reference for determining the quality of the second point cloud PC2 (degraded).
  • the second point cloud PC2 consists of M points, where M is not necessarily equal to K.
  • PC2 is a version with a different quality possibly resulting from lossy encoding and decoding of PC1. This processing can result in a different number of points M.
  • the proposed quality metric can be used in various processing tasks of point clouds, multi-view/free-view videos and 3D meshes, such as resampling, compression and transmission.
  • the proposed quality metric is used to compress time-varying point clouds.
  • Time-varying or dynamic point clouds such as humans in motion can now be captured by arrays of color plus depth (or 'RGBD') video cameras, and such data is getting very popular in emerging applications such as animation, gaming, virtual reality, and immersive communications.
  • arrays of color plus depth (or 'RGBD') video cameras and such data is getting very popular in emerging applications such as animation, gaming, virtual reality, and immersive communications.
  • time-varying point clouds As each frame is independently captured, the geometry of time-varying point clouds has little explicit spatio-temporal structure. For example, frames, which are the point clouds captured at a given time instant, may have different numbers of points, and there is no explicit association between points over time. Performing motion estimation, motion compensation, and effective compression of such data is therefore a challenging task.
  • the commonly used inter coding framework for time-varying point clouds is as shown in Figure 2.
  • a template point cloud, PCT is used to facilitate the motion estimation step of the inter coding. For each frame PC, PCT is deformed at 20 to match PC,.
  • the quality metric 12, resulting from the method of quality measuring according to embodiments of the present invention, can be advantageously used to estimate the quality of the deformed template point cloud PCT d i, or in other terms to compare the frame PC, with the deformed template point cloud PCT d i.
  • a user specified reconstruction error threshold T is reached at 22, instead of PC, PCT d i, the deformed template point cloud, is used for the further computation including a motion estimation 24 resulting in a motion vector MV, a prediction 26 and a motion vector and residual encoding 28.
  • the above process can be achieved based on the Iterative Closest Point (ICP) algorithm.
  • ICP Iterative Closest Point
  • the ICP algorithm has been described in the article from Paul J. Besl et al.: "A method for registration of 3-D shapes", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.14, No.2, p. 239-256, 1992 and in the article of T. Tamaki et al.: “Softassign and EM-ICP on GPU", 2010 First International Conference on Networking and Computing, Higashi-Hiroshima, 2010, p. 179-183.
  • the motion vector is represented by R ik and t ik . It can be further represented and signaled by 12 float numbers, 9 for rotation and 3 for translation.
  • the flowchart of Figure 3 details the steps of the method of measure of the quality of the result of the processing 10, providing the second point cloud PC2 from the first point cloud PC1 , according to a preferred embodiment.
  • the point clouds PC1 and PC2 are analyzed in order to determine that the shapes represented by PC1 and PC2 respectively consist of a number of plane-like patches.
  • each point cloud PC1 , PC2 is segmented into plane-like patches and the points on the patch boundaries are regarded as feature points.
  • a number N of viewpoints distributed over a set of surfaces centered on the first point cloud are selected.
  • One possibility for distributing the viewpoints is to distribute them over a set of surfaces, such as spheres for example, that are centered on the point cloud, as shown in Figure 4.
  • the angular step on the same surface, the diameter step between two consecutive surfaces and the number of the surfaces can be determined.
  • This selection is performed as described in the following steps.
  • a first step as illustrated in Figure 5, one viewpoint is put above each plane-like patch and one viewpoint is put above the patch boundary between two patches.
  • these viewpoints are the points A1 , A2, A3.
  • A1 is above P1
  • A2 is above P2
  • A3 is above the patch boundary between P1 and P2.
  • the direction of each viewpoint is normal to the corresponding plane-like patch and pointing to the inside of the point cloud.
  • the plane-like patches can be defined using the method described in the article of Ainala et al. "Plane Projection Approximation and Learning Based Voxel Geometry Compression” Geneva, Switzerland: ISO/IEC JTC1/SC29/WG1 1/m38816, May 2016.
  • One or a user defined number of viewpoints is put along the normal of the corresponding patch using predefined distance between the mean of the patch and the first viewpoint, and the predefined interval value between viewpoints.
  • the normal of a patch can be calculated by the well-known PCA method.
  • each two contiguous patches are combined into one bigger patch and the same process is repeated for each of the new patches.
  • the patches P1 , P2 are combined into the patch P3 and this processing results on a fourth viewpoint A4.
  • the viewpoint pairs which are very close to each other i.e. the distance between the two viewpoints is less than a certain threshold, are preferably combined.
  • N viewpoints are finally generated at the end of step 32.
  • N first images corresponding to rendered images of the first point cloud are evaluated using the N viewpoints and N second images corresponding to rendered images of the second point cloud are evaluated using the N viewpoints.
  • this evaluation comprises rendering the N first images and the N second images.
  • the images can be rendered using common OpenGL or DirectX rendering pipeline.
  • the rendering of each point cloud PC1 , PC2 could be achieved by representing each point by a disk or a sphere.
  • a sphere In the case of a sphere representation, a sphere will be centered on the geometric position of the represented point.
  • the radius of the sphere can be determined by the minimum distance between the point and the other points of the point cloud, or the average of the last n shortest distances. The number n is chosen equal to 6, for example.
  • the sphere center has the same attribute property as the corresponding point.
  • the distribution of the color attribute could be described by one well-known distribution function such as a Gaussian distribution that is centered on the point.
  • the texture coordinates of the other parts of the sphere can be determined by interpolating the texture coordinates of the point and of its neighboring points.
  • a disk representing a point can be defined in a similar way with the additional definition of the normal of the disk. If the point normal is available, it will be used as the normal of the disk. Otherwise, the disk normal can be determined by the locally least-squares polynomial fitting operation as described in the article of Ochotta et al. "Compression of point-based 3D models by shape-adaptive wavelet coding of multi- heightfields", in Proc. Eurographics Symp. on Point-Based Graphics, 2004.
  • the evaluation of the N first images and N second images comprises estimating the N first images and N second images without rendering the N first images and N second images.
  • the distribution of point cloud attribute data such as color can be formulated.
  • the attribute contributed by the first point cloud PC1 can be calculated at any spatial position.
  • Such spatial position could be a point of the second point cloud PC2 or a resampled point from points which are uniformly distributed on the surface represented by the second point cloud PC2.
  • the attribute contributed by the second point cloud PC2 at the same spatial position is calculated.
  • the attributes at this spatial position of the first point cloud PC1 and the second point cloud PC2 are then compared to determine a unified quality metric.
  • the overall quality metric for the second point cloud PC2 can then be determined for example using PSNR, symmetric Hausdorff or symmetric root-mean-square (RMS) distance.
  • PSNR is defined here as the peak signal of the geometry over the symmetric RMS distortion.
  • a similar metric is defined, according to which the color of the first point cloud to the most nearby color in the second point cloud is compared and the PSNR is computed per YUV component in the YUV color space. The advantage of this metric is that it corresponds to PSNR in Video Coding.
  • One example corresponding to the second embodiment is as follows. According to this example, a point is represented by a sphere and color is used as the representation of the attribute data.
  • and ⁇ r / 3 0 where r is the average or maximum distance of all the point pairs whose two points are closest to each other among all the points in the first point cloud PC1.
  • r can be set as a per-point parameter r t which is the minimum distance between the current point p and any other point of the first point cloud PC1 .
  • the color of v determined by the whole first point cloud PC1 can be calculated by the sum of the color distribution of all the points of the first point cloud at v, for example
  • the first and the second step are repeated for the second point cloud PC2. Then, at step 36, the evaluated images are compared. According to the first embodiment, the rendered images of the first point cloud and of the second point cloud are compared.
  • the quality metric of v is determined by comparing Clr(v, PC2) and Clr(v, PCl). For example, this quality metric is defined as:
  • the quality metric of the second point cloud PC2 is determined from the quality metric of v using PSNR, symmetric Hausdorff or symmetric root-mean-square (RMS) distance, v could be a point of the second point cloud PC2 or a resampled point from the points which are uniformly distributed on the surface represented by the second point cloud PC2.
  • Figure 7 illustrates a quality measurement device 40 according to an embodiment of the invention.
  • This quality measurement device is advantageously implemented in an encoder of a 3D point cloud.
  • the quality measurement device 40 includes at least one memory 42 and at least one processor 44 configured to implement the above described steps.
  • the quality measurement device 40 includes an analyzing module 46 configured to analyze the first point cloud PC1 to determine a shape of said first point cloud.
  • the quality measurement device 40 also includes a selection module 48 configured to select, based on the determined shape, a number N of viewpoints distributed over a set of surfaces centered on the first point cloud.
  • the quality measurement device 40 also includes an evaluation module 50 configured to evaluate N first images corresponding to projection views of the first point cloud using the N viewpoints and N second images corresponding to projection views of the second point cloud using the N viewpoints.
  • the quality measurement device 40 also includes a comparison module 52 configured to compare the N first images and the N second images.
  • the modules 44, 46, 48, 50, 52 are advantageously implemented by the processor(s) 44.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Generation (AREA)

Abstract

This method for measuring a quality of a result of a processing (10) of a first point cloud (PC1) comprising a plurality of points, in which each point is defined by its spatial location and by at least one attribute, in which the processing (10) transforms the first point cloud (PC1) into a second point cloud (PC2), includes: -analyzing the first point cloud (PC1) to determine a shape of said first point cloud (PC1); -selecting a number N of viewpoints distributed over a set of surfaces centered on the first point cloud (PC1); -evaluating N first images corresponding to projection views of the first point cloud (PC1) using the N viewpoints and N second images corresponding to projection views of the second point cloud (PC2) using the N viewpoints; -comparing the N first images and the N second images; and -determining a quality metric(12)of the second point cloud (PC2) from the comparison.

Description

QUALITY METRIC FOR POINT CLOUDS
FIELD
The present disclosure generally relates to the field of point cloud data sources.
More particularly, it deals with the evaluation of the performance of a point cloud processing, such as a point cloud distortion, or deformation, as used in compression for example.
Thus, the disclosure concerns a method for measuring a quality of a result of a point cloud processing and a corresponding quality measuring device. It further concerns an encoder comprising the quality measuring device of the disclosure.
The disclosure also concerns a computer program implementing the quality measuring method of the invention.
BACKGROUND
The approaches described in this section could be pursued, but are not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
A point cloud consists of a set of points. Each point is defined by its spatial location (x, y, z), i.e. geometry information, and different attributes, which typically include the color information in (R, G, B) or (Y, U, V) or any other color coordinate system. Point cloud data sources are found in many applications. Important applications relying on huge point cloud data sources can be found in geographic information systems, robotics, medical tomography and scientific visualization.
Beyond these applications that are more industrial and scientifically oriented, the rise in popularity of inexpensive 3D scanners based on time of flight or other depth sensing technologies, 3D capturing on mobile devices and the rise of cloud based 3D printing are creating a huge demand for large scale interoperable compressed 3D point cloud storage and transmission data formats in the consumer market. Scanned 3D point clouds often have thousands of points and occupy large amounts of storage space. Additionally, they can be generated at a high rate when captured live from 3D scanners, increasing the data rate even further. Therefore, point cloud compression is critical for efficient networked distribution and storage. While there has been quite some work on point cloud compression in the 3D graphics and robotics domains, they generally do not address some of the requirements for building efficient interoperable and complete end-to-end 3D multimedia systems.
For example, the coding of color and other attribute values, which are important for high quality rendering, are often ignored or dealt with inefficiently, for example by performing a simple one way predictive coding of colors. Also, compression of time-varying point clouds, i.e. sequences of clouds captured at a fast rate over time, is limited. Furthermore, methods with low complexity and lossy encoding are of interest for systems where hardware/bandwidth limitations apply or where a low delay is critical. Research in point cloud compression still needs to happen to address these challenges. To measure the performance of different media compression algorithms, the bitrate and distortion, i.e. quality reduction compared to the original media frames, need to be evaluated. While assessing the bitrate is straight forward, a method for measuring point cloud distortion is still needed.
A first option for this measurement is to use the Root Mean Square Error (RMSE) between input and output points. Unfortunately, this is not well defined when points are decimated or merged and the correspondence between input and output points is lost. Furthermore, the existing works calculate RMSE for geometry and attributes separately.
A second option for the measurement is to triangulate the point cloud into a 3D mesh and to use the already existing metrics for 3D meshes, such as the symmetric root mean square or symmetric Hausdorff distance. However, in this case, the triangulation method used would affect the resulting distortion metric. Additionally, metrics for the colors and attribute quality would not be well defined. Thus, a point cloud specific quality or distortion metric is useful for the evaluation of compression algorithms.
Most of the existing works independently calculate separate quality metrics for geometry, color and other properties of point clouds. For instance, the articles of Mekuria "Open source framework for benchmarking and quality assessment of point cloud compression algorithms", Geneva, Switzerland: ISO/IEC JTC1/SC29/WG1 1/m35491 , February 2015 and of Mekuria and Cesar :"Document describing the quality metric and its usage in the point cloud compression reference software", Warsaw, Poland: ISO/IEC JTC1/SC29/WG1 1/m36529, June 2015, deploy the symmetric Hausdorff and Root-Mean-Square (RMS) distance to point clouds. For each point in the point cloud to be compared, the most nearby point in the reference point cloud is regarded as its reference. Then, the geometric and color error of these points are calculated by being compared with their corresponding reference points. Finally, the geometric PSNR (Peak Signal to Noise Ratio) is defined as the peak signal of the geometry over the symmetric RMS distance. The PSNR per YUV component is calculated in a similar way.
The article of Ochotta et al. "Compression of point-based 3D models by shape-adaptive wavelet coding of multi-heightfields," in Proc. Eurographics Symp. on Point-Based Graphics, 2004, calculates similar quality metrics for point clouds in a more plausible manner. In this article, the reference point cloud is first approximated by a set of surfaces generated by least-squares polynomial fitting. Then, each point in the point cloud to be compared is projected on those surfaces. Only the quality metrics for geometric information are discussed in this paper.
More recently, the article of Ricardo L. de Queiroz et al.:"Motion-compensated compression of dynamic voxelized point Clouds", Geneva, Switzerland: ISO/IEC JTC1/SC29/WG1 1/m38675, May 2016, proposes two types of quality metrics for point clouds. The first type of quality metrics is based on correspondence. It is similar with the ones proposed by the documents described above. Moreover, this paper proposes a unified quality metric for point clouds by calculating a weighted sum of PSNR geometry and PSNR color. However, it is not a trivial task to select a proper weight for combining PSNR geometry and PSNR color. And a global weight for the whole point cloud is not the optimal solution either. The second type of quality metrics is based on projection. In the projection-based distortion measure, the authors assume an orthogonal projection of the point cloud over the six sides of the bounding-box of the point cloud. The distortion metric between two point clouds is then the MSE (or PSNR) between the two corresponding composite images, each with the six orthogonal sides. The projection based metric is a good solution for the unified quality metric of point clouds. However, projection only on the six sides of the bounding-box is not enough, especially for those point clouds with complex shapes.
SUMMARY
The present invention proposes a solution for improving the situation.
Accordingly, the present invention provides a method for measuring a quality of a processing of a first 3D point cloud comprising a plurality of points, in which each point is defined by its spatial location and by at least one attribute, in which the processing transforms the first point cloud into a second 3D point cloud, wherein the method includes:
- analyzing the first point cloud to determine a shape of said first point cloud;
- selecting, based on the determined shape, a number N of viewpoints distributed over a set of surfaces centered on the first point cloud;
- determining N first images corresponding to projection views of the first point cloud using the N viewpoints and N second images corresponding to projection views of the second point cloud using the N viewpoints;
- comparing the N first images and the N second images; and
- determining a quality metric of the second point cloud from the comparison. Thus, the method of the invention is based on the comparison of the projection of the first point cloud, i.e. the reference or original point cloud, and the projection of the second point cloud, that is the point cloud of which the quality metric is evaluated, i.e. the processed or reconstructed point cloud, on N viewpoints. The number N is automatically selected according to the shape of the first point cloud. Advantageously, each point in the first point cloud or in the second point cloud is represented by a sphere.
Alternatively, each point in the first point cloud or in the second point cloud is represented by a disk. According to a first embodiment, determining the N first images and N second images comprises rendering the N first images and the N second images.
According to a second embodiment, determining the N first images and N second images comprises estimating the N first images and N second images without rendering the N first images and N second images. Without rendering the images, the complexity of the method and the requirement of the necessary hardware are reduced.
Advantageously, estimating the N first images and N second images includes:
- determining a contribution of the at least one attribute of each point in the first point cloud;
- determining the contribution of the attributes of all the points of the first point cloud;
- determining a contribution of the at least one attribute of each point in the second point cloud; and
- determining the contribution of the attributes of all the points of the second point cloud. Advantageously, these contributions are determined at a point of the second point cloud or at a resampled point chosen from points uniformly distributed on a surface represented by the first point cloud or the second point cloud.
According to an embodiment, the attribute of each point in the first point cloud and of each point in the second point cloud is represented by a Gaussian distribution function. Advantageously, comparing the N first images and the N second images includes comparing the contribution of both the spatial locations and the attributes of all the points of the first point cloud and the contribution of both the spatial locations and the attributes of all the points of the second point cloud.
According to an embodiment, determining a quality metric of the second point cloud uses a symmetric Hausdorff distance and/or a Root-Mean-Square, RMS, distance between the first point cloud and the second point cloud.
Preferably, the at least one attribute includes the color.
The present invention also provides a device for measuring a quality of a processing of a first 3D point cloud comprising a plurality of points, in which each point is defined by its spatial location and by at least one attribute, in which the processing transforms the first point cloud into a second 3D point cloud, wherein the device includes:
- an analyzing module configured to analyze the first point cloud to determine a shape of said first point cloud;
- a selection module configured to select, based on the determined shape, a number N of viewpoints distributed over a set of surfaces centered on the first point cloud; and
- an evaluation module configured to determine N first images corresponding to projection views of the first point cloud using the N viewpoints and to determine N second images corresponding to projection views of the second point cloud using the N viewpoints; and
- a comparison module configured to compare the N first images and the N second images and to determine a quality metric of the second point cloud from the comparison.
According to a first embodiment, the evaluation module is configured to render the N first images and the N second images. According to a second embodiment, the evaluation module is configured to estimate the N first images and the N second images without rendering the N first images and the N second images.
The present disclosure further provides an encoder comprising the device of the invention. The method according to the invention may be implemented in software on a programmable apparatus. It may be implemented solely in hardware or in software, or in a combination thereof.
Since the present invention can be implemented in software, the present invention can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like.
The invention thus provides a computer-readable program comprising computer- executable instructions to enable a computer to perform the quality measurement method of the invention.
The diagram of figure 3 illustrates an example of the general algorithm for such computer program.
BRIEF DESCRIPTION OF THE DRAWINGS The present invention is illustrated by way of examples, and not by way of limitation, in the figures of the accompanying drawings, in which like reference numerals refer to similar elements and in which:
- Figure 1 is a schematic view illustrating the principle of a point cloud quality metric evaluation, according to an embodiment of the invention; - Figure 2 is a schematic view illustrating the use of the point cloud quality metric evaluation in an inter coding system of a time varying point cloud, according to an embodiment of the invention;
- Figure 3 is a flowchart showing the steps of the quality measurement method, according to an embodiment of the present invention; - Figure 4 shows an example of viewpoints distribution, according to an embodiment of the present invention; - Figure 5 shows an example of selection of viewpoints, according to an embodiment of the present invention;
- Figure 6 shows an example of a Gaussian distribution of the contribution of the color of a point in a point cloud, according to an embodiment of the present invention; and
- Figure 7 is a schematic view illustrating a quality measurement device, according to an embodiment of the invention.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
A point cloud is a set of points in a 3D space, each with associated attributes, e.g. color, material properties, etc. Point clouds can be used to reconstruct an object or a scene as a composition of such points. They can be captured using multiple cameras and depth sensors in various setups and may be made up of thousands up to billions of points in order to realistically represent reconstructed scenes. An illustration of a point cloud referenced PC1 is shown in Figure 4. The point cloud PC1 is defined as a set of points wherein a point v has as a mandatory position in a 3D space (x,y,z) and an optional color attribute c that has components r,g,b or y,u,v and optional other attributes (a0, ...aA) possibly representing normal or texture mappings.
Thus, a point v in a point cloud can be defined as: point v = (((x, y, z), [c], [a0. . aA]) : x, y, z R , [c (r, g, b) \r, g, b G N], [at ε [0,1]]) The point cloud PC1 is then simply a set of K points without a strict ordering:
PCI = {(vi) i = 0 K - 1}
Let's consider, with reference to Figure 1 , that the point cloud PC1 is processed using a processing operation 10 resulting in a second point cloud PC2. This processing is for example a lossy or lossless compression comprising for instance compression of geometry and attributes, scalable/progressive coding, coding of sequences of point clouds captured over time, and random access to subsets of the point cloud. Other examples of the processing operation include point cloud upsampling or downsampling which changes the number of points of the point cloud, point cloud resampling which keeps the same number of points, point cloud deformation, etc. The present invention provides a quality metric 12 for the evaluation of the quality of the result of the processing 10 operated on the original point cloud PC1 , or in other words a quality metric for the second point cloud PC2.
The first point cloud PC1 (original) will act as a reference for determining the quality of the second point cloud PC2 (degraded). The second point cloud PC2 consists of M points, where M is not necessarily equal to K. PC2 is a version with a different quality possibly resulting from lossy encoding and decoding of PC1. This processing can result in a different number of points M.
The proposed quality metric can be used in various processing tasks of point clouds, multi-view/free-view videos and 3D meshes, such as resampling, compression and transmission.
In the example illustrated in Figure 2, the proposed quality metric is used to compress time-varying point clouds.
Time-varying or dynamic point clouds such as humans in motion can now be captured by arrays of color plus depth (or 'RGBD') video cameras, and such data is getting very popular in emerging applications such as animation, gaming, virtual reality, and immersive communications.
As each frame is independently captured, the geometry of time-varying point clouds has little explicit spatio-temporal structure. For example, frames, which are the point clouds captured at a given time instant, may have different numbers of points, and there is no explicit association between points over time. Performing motion estimation, motion compensation, and effective compression of such data is therefore a challenging task. The commonly used inter coding framework for time-varying point clouds is as shown in Figure 2. A template point cloud, PCT, is used to facilitate the motion estimation step of the inter coding. For each frame PC,, PCT is deformed at 20 to match PC,. The quality metric 12, resulting from the method of quality measuring according to embodiments of the present invention, can be advantageously used to estimate the quality of the deformed template point cloud PCTdi, or in other terms to compare the frame PC, with the deformed template point cloud PCTdi. After a user specified reconstruction error threshold T is reached at 22, instead of PC,, PCTdi, the deformed template point cloud, is used for the further computation including a motion estimation 24 resulting in a motion vector MV, a prediction 26 and a motion vector and residual encoding 28.
According to one embodiment, the above process can be achieved based on the Iterative Closest Point (ICP) algorithm. The ICP algorithm has been described in the article from Paul J. Besl et al.: "A method for registration of 3-D shapes", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.14, No.2, p. 239-256, 1992 and in the article of T. Tamaki et al.: "Softassign and EM-ICP on GPU", 2010 First International Conference on Networking and Computing, Higashi-Hiroshima, 2010, p. 179-183.
Let's set a rotation matrix Ri0 = I and a translation vector ti0 =0, PCT is deformed by PCTdik = Rik * PCT + Tik, where k is the index of iteration. Then the quality of PCTdik is estimated using the embodiments of the present invention. If the error is larger than the threshold T, anther iteration of transformation as follows is performed. For each point, xtj, in PC,, the closest point, yikjt, in PCTdik is determined by ikj *= _ min - yikjj || , where nPCTdik is the number of points in PCT^. Then, an ordered correspondence set ¾ = {yife0„yifel„yife2„ ... ,yifen } is built. The r lj l dik
transformation R* and t* that minimize PC, and Yfk are then determined and at the next iteration k+1 , the rotation matrix and the translation vector Rik' and tik'are updated by Rik' = R* and tik' = t*.
If the error is not larger than T, the motion vector is represented by Rik and tik. It can be further represented and signaled by 12 float numbers, 9 for rotation and 3 for translation. The prediction error is calculated by - yikj* \\ , j = 1, - > nPCTdik - The prediction error can then be signaled into the output bitstream or further entropy coded.
The flowchart of Figure 3 details the steps of the method of measure of the quality of the result of the processing 10, providing the second point cloud PC2 from the first point cloud PC1 , according to a preferred embodiment.
At a first step 30, the point clouds PC1 and PC2 are analyzed in order to determine that the shapes represented by PC1 and PC2 respectively consist of a number of plane-like patches.
According to this analysis, each point cloud PC1 , PC2 is segmented into plane-like patches and the points on the patch boundaries are regarded as feature points.
Then, at step 32, a number N of viewpoints distributed over a set of surfaces centered on the first point cloud are selected.
An example of selected viewpoints is shown in Figure 4.
One possibility for distributing the viewpoints is to distribute them over a set of surfaces, such as spheres for example, that are centered on the point cloud, as shown in Figure 4. According to user specified parameters, the angular step on the same surface, the diameter step between two consecutive surfaces and the number of the surfaces can be determined.
This selection is performed as described in the following steps. At a first step, as illustrated in Figure 5, one viewpoint is put above each plane-like patch and one viewpoint is put above the patch boundary between two patches. For example, considering the patches P1 and P2 in Figure 5, these viewpoints are the points A1 , A2, A3. A1 is above P1 , A2 is above P2 and A3 is above the patch boundary between P1 and P2. The direction of each viewpoint is normal to the corresponding plane-like patch and pointing to the inside of the point cloud.
As an example, the plane-like patches can be defined using the method described in the article of Ainala et al. "Plane Projection Approximation and Learning Based Voxel Geometry Compression" Geneva, Switzerland: ISO/IEC JTC1/SC29/WG1 1/m38816, May 2016. One or a user defined number of viewpoints is put along the normal of the corresponding patch using predefined distance between the mean of the patch and the first viewpoint, and the predefined interval value between viewpoints. The normal of a patch can be calculated by the well-known PCA method.
Then, each two contiguous patches are combined into one bigger patch and the same process is repeated for each of the new patches. In the example of Figure 5, the patches P1 , P2 are combined into the patch P3 and this processing results on a fourth viewpoint A4. At a second step, among all the resultant viewpoints, the viewpoint pairs which are very close to each other, i.e. the distance between the two viewpoints is less than a certain threshold, are preferably combined.
Then, N viewpoints are finally generated at the end of step 32.
At step 34, N first images corresponding to rendered images of the first point cloud are evaluated using the N viewpoints and N second images corresponding to rendered images of the second point cloud are evaluated using the N viewpoints.
According to a first embodiment, this evaluation comprises rendering the N first images and the N second images. The images can be rendered using common OpenGL or DirectX rendering pipeline. The rendering of each point cloud PC1 , PC2 could be achieved by representing each point by a disk or a sphere.
In the case of a sphere representation, a sphere will be centered on the geometric position of the represented point. The radius of the sphere can be determined by the minimum distance between the point and the other points of the point cloud, or the average of the last n shortest distances. The number n is chosen equal to 6, for example. The sphere center has the same attribute property as the corresponding point. The distribution of the color attribute could be described by one well-known distribution function such as a Gaussian distribution that is centered on the point. The texture coordinates of the other parts of the sphere can be determined by interpolating the texture coordinates of the point and of its neighboring points.
In the case of the disk, a disk representing a point can be defined in a similar way with the additional definition of the normal of the disk. If the point normal is available, it will be used as the normal of the disk. Otherwise, the disk normal can be determined by the locally least-squares polynomial fitting operation as described in the article of Ochotta et al. "Compression of point-based 3D models by shape-adaptive wavelet coding of multi- heightfields", in Proc. Eurographics Symp. on Point-Based Graphics, 2004.
According to a second embodiment, the evaluation of the N first images and N second images comprises estimating the N first images and N second images without rendering the N first images and N second images.
According to this second embodiment, it is assumed that the distribution of point cloud attribute data, such as color can be formulated. In this way, the attribute contributed by the first point cloud PC1 can be calculated at any spatial position. Such spatial position could be a point of the second point cloud PC2 or a resampled point from points which are uniformly distributed on the surface represented by the second point cloud PC2.
Then, the attribute contributed by the second point cloud PC2 at the same spatial position is calculated. The attributes at this spatial position of the first point cloud PC1 and the second point cloud PC2 are then compared to determine a unified quality metric. The overall quality metric for the second point cloud PC2 can then be determined for example using PSNR, symmetric Hausdorff or symmetric root-mean-square (RMS) distance.
PSNR is defined here as the peak signal of the geometry over the symmetric RMS distortion. For colors, a similar metric is defined, according to which the color of the first point cloud to the most nearby color in the second point cloud is compared and the PSNR is computed per YUV component in the YUV color space. The advantage of this metric is that it corresponds to PSNR in Video Coding.
One example corresponding to the second embodiment is as follows. According to this example, a point is represented by a sphere and color is used as the representation of the attribute data.
At a first step, it is assumed that the distribution of the color of each point p in the first point cloud PC1 can be described by a Gaussian distribution, as represented in Figure 6. For any spatial position v , its color contributed by p can be calculated by f(v, p =
1
^7 = e _ 2ϊσ12 , where x = l - p|| and σ = r/3 0 where r is the average or maximum distance of all the point pairs whose two points are closest to each other among all the points in the first point cloud PC1. If the first point cloud PC1 is produced from a scanning, r can be set as the scanning resolution if it is available. The reason of setting σ = r/3 Q is that it is reasonable to assume that the influence of one point decreases to almost zero at the position of some other point, as shown in Figure 6. r can be set as a per-point parameter rt which is the minimum distance between the current point p and any other point of the first point cloud PC1 .
At a second step, the color of v determined by the whole first point cloud PC1 can be calculated by the sum of the color distribution of all the points of the first point cloud at v, for example
Figure imgf000015_0001
Or the simplified form
Figure imgf000015_0002
The first and the second step are repeated for the second point cloud PC2. Then, at step 36, the evaluated images are compared. According to the first embodiment, the rendered images of the first point cloud and of the second point cloud are compared.
For the second embodiment in which there is no rendering of images, the quality metric of v is determined by comparing Clr(v, PC2) and Clr(v, PCl). For example, this quality metric is defined as:
QM(v) = \Clr{v, PCl) - Clr{v, PCVj \
Then, the quality metric of the second point cloud PC2 is determined from the quality metric of v using PSNR, symmetric Hausdorff or symmetric root-mean-square (RMS) distance, v could be a point of the second point cloud PC2 or a resampled point from the points which are uniformly distributed on the surface represented by the second point cloud PC2.
Figure 7 illustrates a quality measurement device 40 according to an embodiment of the invention.
This quality measurement device is advantageously implemented in an encoder of a 3D point cloud.
The quality measurement device 40 includes at least one memory 42 and at least one processor 44 configured to implement the above described steps.
More particularly, the quality measurement device 40 includes an analyzing module 46 configured to analyze the first point cloud PC1 to determine a shape of said first point cloud.
The quality measurement device 40 also includes a selection module 48 configured to select, based on the determined shape, a number N of viewpoints distributed over a set of surfaces centered on the first point cloud.
The quality measurement device 40 also includes an evaluation module 50 configured to evaluate N first images corresponding to projection views of the first point cloud using the N viewpoints and N second images corresponding to projection views of the second point cloud using the N viewpoints. The quality measurement device 40 also includes a comparison module 52 configured to compare the N first images and the N second images.
The modules 44, 46, 48, 50, 52 are advantageously implemented by the processor(s) 44.
While there has been illustrated and described what are presently considered to be the preferred embodiments of the present invention, it will be understood by those skilled in the art that various other modifications may be made, and equivalents may be substituted, without departing from the true scope of the present invention. Additionally, many modifications may be made to adapt a particular situation to the teachings of the present invention without departing from the central inventive concept described herein. Furthermore, an embodiment of the present invention may not include all of the features described above. Therefore, it is intended that the present invention is not limited to the particular embodiments disclosed, but that the invention includes all embodiments falling within the scope of the appended claims.
Expressions such as "comprise", "include", "incorporate", "contain", "is" and "have" are to be construed in a non-exclusive manner when interpreting the description and its associated claims, namely construed to allow for other items or components which are not explicitly defined also to be present. Reference to the singular is also to be construed to be a reference to the plural and vice versa.
A person skilled in the art will readily appreciate that various parameters disclosed in the description may be modified and that various embodiments disclosed and/or claimed may be combined without departing from the scope of the invention.

Claims

CLAIMS.
1. Method for measuring a quality of a processing (10) of a first 3D point cloud (PC1 ) comprising a plurality of points, in which each point is defined by its spatial location and by at least one attribute, in which the processing (10) transforms the first point cloud (PC1 ) into a second 3D point cloud (PC2), wherein the method includes:
- analyzing (30) the first point cloud (PC1 ) to determine a shape of said first point cloud (PC1 );
- selecting (32), based on the determined shape, a number N of viewpoints distributed over a set of surfaces centered on the first point cloud (PC1 );
- determining (34) N first images corresponding to projection views of the first point cloud (PC1 ) using the N viewpoints and N second images corresponding to projection views of the second point cloud (PC2) using the N viewpoints;
- comparing (36) the N first images and the N second images; and
- determining a quality metric (12) of the second point cloud (PC2) from the comparison.
2. Method of claim 1 , wherein each point in the first point cloud (PC1 ) or in the second point cloud (PC2) is represented by a sphere.
3. Method of claim 1 , wherein each point in the first point cloud (PC1 ) or in the second point cloud (PC2) is represented by a disk.
4. Method of any one of claims 1 to 3, wherein determining the N first images and N second images comprises rendering the N first images and the N second images.
5. Method of any one of claims 1 to 3, wherein determining the N first images and N second images comprises estimating the N first images and the N second images without rendering the N first images and the N second images.
6. Method of claim 5, wherein estimating the N first images and N second images includes:
- determining a contribution of the at least one attribute of each point in the first point cloud (PC1 );
- determining the contribution of the attributes of all the points of the first point cloud (PC1 );
- determining a contribution of the at least one attribute of each point in the second point cloud (PC2); and
- determining the contribution of the attributes of all the points of the second point cloud (PC2).
7. Method of claim 5 or 6, wherein the attribute of each point in the first point cloud (PC1 ) and of each point in the second point cloud (PC2) is represented by a Gaussian distribution function.
8. Method of any one of claims 5 to 7, wherein comparing the N first images and the N second images includes comparing the contribution of both the spatial locations and the attributes of all the points of the first point cloud (PC1 ) and the contribution of both the spatial locations and the attributes of all the points of the second point cloud (PC2).
9. Method of any one of claims 1 to 8, wherein determining a quality metric of the second point cloud (PC2) uses a symmetric Hausdorff distance and/or a Root-Mean-Square, RMS, distance between the first point cloud (PC1 ) and the second point cloud (PC2).
10. Method of any one of claims 1 to 9, wherein the at least one attribute includes the color.
1 1 . Device (40) for measuring a quality of a processing of a first 3D point cloud (PC1 ) comprising a plurality of points, in which each point is defined by its spatial location and by at least one attribute, in which the processing transforms the first point cloud (PC1 ) into a second 3D point cloud (PC2), wherein the device (40) includes: - an analyzing module (46) configured to analyze the first point cloud (PC1 ) to determine a shape of said first point cloud (PC1 );
- a selection module (48) configured to select, based on the determined shape, a number N of viewpoints distributed over a set of surfaces centered on the first point cloud (PC1 ); and
- an evaluation module (50) configured to determine N first images corresponding to projection views of the first point cloud (PC1 ) using the N viewpoints and to determine N second images corresponding to projection views of the second point cloud (PC2) using the N viewpoints; and - a comparison module (52) configured to compare the N first images and the N second images and to determine a quality metric (12) of the second point cloud (PC2) from the comparison.
12. Device of claim 1 1 , wherein the evaluation module (50) is configured to render the N first images and the N second images.
13. Device of claim 1 1 , wherein the evaluation module is configured to estimate the N first images and N second images without rendering the N first images and N second images.
14. Encoder comprising the device of one of claims 1 1 to 13.
15. A computer-readable program comprising computer-executable instructions to enable a computer to perform the method of any one of claims 1 to 10.
PCT/EP2018/055335 2017-03-07 2018-03-05 Quality metric for point clouds Ceased WO2018162406A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP17305237.4 2017-03-07
EP17305237.4A EP3373583A1 (en) 2017-03-07 2017-03-07 Quality metric for point clouds

Publications (1)

Publication Number Publication Date
WO2018162406A1 true WO2018162406A1 (en) 2018-09-13

Family

ID=58448498

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2018/055335 Ceased WO2018162406A1 (en) 2017-03-07 2018-03-05 Quality metric for point clouds

Country Status (2)

Country Link
EP (1) EP3373583A1 (en)
WO (1) WO2018162406A1 (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020072579A1 (en) * 2018-10-02 2020-04-09 Futurewei Technologies, Inc. Motion estimation using 3d auxiliary data
CN111679288A (en) * 2020-06-19 2020-09-18 中国林业科学研究院资源信息研究所 A method for measuring the spatial distribution of point cloud data
CN112399165A (en) * 2019-08-14 2021-02-23 腾讯美国有限责任公司 Decoding method and device, computer equipment and storage medium
CN113615204A (en) * 2019-03-20 2021-11-05 Lg电子株式会社 Point cloud data transmitting device, point cloud data transmitting method, point cloud data receiving device, and point cloud data receiving method
CN114022621A (en) * 2022-01-07 2022-02-08 清华大学 Method, apparatus and medium for reconstructing mesh surface from point cloud data without normalization
CN114424542A (en) * 2019-09-27 2022-04-29 苹果公司 Video-based point cloud compression with non-canonical smoothing
CN114743075A (en) * 2022-04-11 2022-07-12 北京有竹居网络技术有限公司 Point cloud model detection method and device, electronic equipment and readable storage medium
CN118644440A (en) * 2024-05-21 2024-09-13 北京天数智芯半导体科技有限公司 Point cloud quality assessment method, device and electronic equipment
US12094179B2 (en) 2018-10-05 2024-09-17 Apple Inc. Quantized depths for projection point cloud compression
US12100183B2 (en) 2018-04-10 2024-09-24 Apple Inc. Point cloud attribute transfer algorithm
US12401822B2 (en) 2018-07-12 2025-08-26 Apple Inc. Bit stream structure for compressed point cloud data
US12439083B2 (en) 2019-07-02 2025-10-07 Apple Inc. Point cloud compression with supplemental information messages

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11915456B2 (en) 2018-10-05 2024-02-27 Interdigital Vc Holdings, Inc. Method and device for encoding and reconstructing missing points of a point cloud
CN110246112B (en) * 2019-01-21 2021-04-30 厦门大学 Laser scanning SLAM indoor three-dimensional point cloud quality evaluation method based on deep learning
CN113784129B (en) * 2020-06-10 2024-11-22 Oppo广东移动通信有限公司 Point cloud quality assessment method, encoder, decoder and storage medium
CN112637594B (en) * 2020-12-23 2023-05-26 青岛大学 No-reference 3D point cloud quality assessment method based on bit stream
CN117372325A (en) * 2023-02-17 2024-01-09 中国科学院深圳先进技术研究院 Point cloud quality evaluation method and related devices based on projection and multi-scale features
CN117853484B (en) * 2024-03-05 2024-05-28 湖南建工交建宏特科技有限公司 A vision-based intelligent bridge damage monitoring method and system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6750873B1 (en) * 2000-06-27 2004-06-15 International Business Machines Corporation High quality texture reconstruction from multiple scans
US20120013710A1 (en) * 2009-02-19 2012-01-19 Dimensional Perception Technologies Ltd. System and method for geometric modeling using multiple data acquisition means

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6750873B1 (en) * 2000-06-27 2004-06-15 International Business Machines Corporation High quality texture reconstruction from multiple scans
US20120013710A1 (en) * 2009-02-19 2012-01-19 Dimensional Perception Technologies Ltd. System and method for geometric modeling using multiple data acquisition means

Non-Patent Citations (11)

* Cited by examiner, † Cited by third party
Title
AINALA ET AL., PLANE PROJECTION APPROXIMATION AND LEARNING BASED VOXEL GEOMETRY COMPRESSION, May 2016 (2016-05-01)
DAN-DAN LIU ET AL: "The scattered point cloud denoising method based on surfacelet transform of 3D context", COMPUTER DESIGN AND APPLICATIONS (ICCDA), 2010 INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 25 June 2010 (2010-06-25), pages V5 - 465, XP031726569, ISBN: 978-1-4244-7164-5 *
KARTHIK AINALA (UMKC) ET AL: "Point Cloud Geometry Compression with Plane Projection Approximation and Learning Based Compression", 115. MPEG MEETING; 30-5-2016 - 3-6-2016; GENEVA; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11),, no. m38816, 28 May 2016 (2016-05-28), XP030067168 *
MEKURIA RUFAEL ET AL: "Overview of the MPEG Activity on Point Cloud Compression", 2016 DATA COMPRESSION CONFERENCE (DCC), IEEE, 30 March 2016 (2016-03-30), pages 620, XP033027780, DOI: 10.1109/DCC.2016.91 *
MEKURIA, OPEN SOURCE FRAMEWORK FOR BENCHMARKING AND QUALITY ASSESSMENT OF POINT CLOUD COMPRESSION ALGORITHMS, February 2015 (2015-02-01)
MEKURIA; CESAR, DOCUMENT DESCRIBING THE QUALITY METRIC AND ITS USAGE IN THE POINT CLOUD COMPRESSION REFERENCE SOFTWARE, June 2015 (2015-06-01)
OCHOTTA ET AL.: "Compression of point-based 3D models by shape-adaptive wavelet coding of multi-heightfields", PROC. EUROGRAPHICS SYMP. ON POINT-BASED GRAPHICS, 2004
PAUL J. BESL ET AL.: "A method for registration of 3-D shapes", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 14, no. 2, 1992, pages 239 - 256
RICARDO L. DE QUEIROZ ET AL., MOTION-COMPENSATED COMPRESSION OF DYNAMIC VOXELIZED POINT CLOUDS, May 2016 (2016-05-01)
RUFAEL MEKURIA ET AL: "Proposal for Quality and Performance Metrics Point Cloud Compression", 111. MPEG MEETING; 6-2-2015 - 20-2-2015; GENEVA; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11),, no. m35491, 30 January 2015 (2015-01-30), XP030063859 *
T. TAMAKI ET AL.: "Softassign and EM-ICP on GPU", 2010 FIRST INTERNATIONAL CONFERENCE ON NETWORKING AND COMPUTING, 2010, pages 179 - 183, XP031855301

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12100183B2 (en) 2018-04-10 2024-09-24 Apple Inc. Point cloud attribute transfer algorithm
US12401822B2 (en) 2018-07-12 2025-08-26 Apple Inc. Bit stream structure for compressed point cloud data
KR102537087B1 (en) * 2018-10-02 2023-05-26 후아웨이 테크놀러지 컴퍼니 리미티드 Motion estimation using 3D auxiliary data
KR20210069683A (en) * 2018-10-02 2021-06-11 후아웨이 테크놀러지 컴퍼니 리미티드 Motion estimation using 3D auxiliary data
US20210217202A1 (en) * 2018-10-02 2021-07-15 Futurewei Technologies, Inc. Motion Estimation Using 3D Auxiliary Data
WO2020072579A1 (en) * 2018-10-02 2020-04-09 Futurewei Technologies, Inc. Motion estimation using 3d auxiliary data
US11688104B2 (en) 2018-10-02 2023-06-27 Huawei Technologies Co., Ltd. Motion estimation using 3D auxiliary data
US12094179B2 (en) 2018-10-05 2024-09-17 Apple Inc. Quantized depths for projection point cloud compression
US12688619B2 (en) 2018-10-05 2026-07-21 Apple Inc. Quantized depths for projection point cloud compression
CN113615204B (en) * 2019-03-20 2024-02-20 Lg电子株式会社 Point cloud data sending device, point cloud data sending method, point cloud data receiving device and point cloud data receiving method
CN113615204A (en) * 2019-03-20 2021-11-05 Lg电子株式会社 Point cloud data transmitting device, point cloud data transmitting method, point cloud data receiving device, and point cloud data receiving method
US12069316B2 (en) 2019-03-20 2024-08-20 Lg Electronics Inc. Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method
US12439083B2 (en) 2019-07-02 2025-10-07 Apple Inc. Point cloud compression with supplemental information messages
CN112399165B (en) * 2019-08-14 2022-03-25 腾讯美国有限责任公司 Decoding method and device, computer equipment and storage medium
CN112399165A (en) * 2019-08-14 2021-02-23 腾讯美国有限责任公司 Decoding method and device, computer equipment and storage medium
CN114424542B (en) * 2019-09-27 2024-02-06 苹果公司 Video-based point cloud compression with non-canonical smoothing
CN114424542A (en) * 2019-09-27 2022-04-29 苹果公司 Video-based point cloud compression with non-canonical smoothing
CN111679288B (en) * 2020-06-19 2022-10-21 中国林业科学研究院资源信息研究所 Method for measuring spatial distribution of point cloud data
CN111679288A (en) * 2020-06-19 2020-09-18 中国林业科学研究院资源信息研究所 A method for measuring the spatial distribution of point cloud data
CN114022621A (en) * 2022-01-07 2022-02-08 清华大学 Method, apparatus and medium for reconstructing mesh surface from point cloud data without normalization
CN114022621B (en) * 2022-01-07 2022-05-31 清华大学 Method, apparatus and medium for reconstructing mesh surface from point cloud data without normalization
CN114743075A (en) * 2022-04-11 2022-07-12 北京有竹居网络技术有限公司 Point cloud model detection method and device, electronic equipment and readable storage medium
CN118644440A (en) * 2024-05-21 2024-09-13 北京天数智芯半导体科技有限公司 Point cloud quality assessment method, device and electronic equipment
CN118644440B (en) * 2024-05-21 2025-06-13 北京天数智芯半导体科技有限公司 Point cloud quality assessment method, device and electronic equipment

Also Published As

Publication number Publication date
EP3373583A1 (en) 2018-09-12

Similar Documents

Publication Publication Date Title
EP3373583A1 (en) Quality metric for point clouds
EP3405928B1 (en) Method for compressing point cloud
Würmlin et al. 3D video fragments: Dynamic point samples for real-time free-viewpoint video
EP2850835B1 (en) Estimation, encoding and decoding of motion information in multidimensional signals through motion zones, and of auxiliary information through auxiliary zones
CN100581250C (en) System and method for three-dimensional computer graphics compression
Mihajlovic et al. Splatfields: Neural gaussian splats for sparse 3d and 4d reconstruction
US11317074B2 (en) Method and apparatus for processing data for three-dimensional image
KR102577135B1 (en) A skeleton-based dynamic point cloud estimation system for sequence compression
US20050017968A1 (en) Differential stream of point samples for real-time 3D video
US11989919B2 (en) Method and apparatus for encoding and decoding volumetric video data
Xu et al. Cluster-based point cloud coding with normal weighted graph fourier transform
KR101817140B1 (en) Coding Method and Device for Depth Video Plane Modeling
KR102729399B1 (en) Create arbitrary views
Jin et al. Meshreduce: Scalable and bandwidth efficient 3d scene capture
Perry et al. Study of subjective and objective quality evaluation of 3D point cloud data by the JPEG committee
Sevom et al. Geometry-guided 3D data interpolation for projection-based dynamic point cloud coding
Zhang et al. A 3D Haar wavelet transform for point cloud attribute compression based on local surface analysis
Park et al. A mesh-based disparity representation method for view interpolation and stereo image compression
KR20220069827A (en) Method and Apparatus for transforming 3-dimensional Volume Resolution Using Truncated Signed Distance Function
JP2006527945A (en) Representation method of picture sequence using 3D model, corresponding signal, and corresponding apparatus
US20250247556A1 (en) Point cloud data transmission method, point cloud data transmission device, point cloud data reception method, and point cloud data reception device
Magnor et al. Multiview image coding with depth maps and 3d geometry for prediction
Matsuzaki et al. Point cloud color upsampling with attention-based coarse colorization and refinement
Würmlin et al. Image-space free-viewpoint video
TW202338738A (en) Video signal and processing thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18707390

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18707390

Country of ref document: EP

Kind code of ref document: A1