WO2020256244A1

WO2020256244A1 - Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, point cloud data reception method

Info

Publication number: WO2020256244A1
Application number: PCT/KR2019/018176
Authority: WO
Inventors: 허혜정; 오세진; 박유선
Original assignee: 엘지전자 주식회사
Priority date: 2019-06-21
Filing date: 2019-12-20
Publication date: 2020-12-24

Abstract

A point cloud data transmission method according to embodiments may comprise: a step for acquiring point cloud data; a step for encoding the point cloud data; and/or a step for transmitting the point cloud data. A point cloud data reception method according to embodiments may comprise: a step for receiving point cloud data; a step for decoding the point cloud data; and/or a step for rendering the point cloud data.

Description

Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Embodiments provide Point Cloud content to provide users with various services such as VR (Virtual Reality, Virtual Reality), AR (Augmented Reality, Augmented Reality), MR (Mixed Reality, Mixed Reality), and autonomous driving service. Provide a solution.

A point cloud is a set of points in 3D space. There is a problem in that it is difficult to generate point cloud data because the amount of points in the 3D space is large.

There is a problem that a large amount of processing is required to transmit and receive point cloud data.

The technical problem according to the embodiments is to provide a point cloud data transmission apparatus, a transmission method, a point cloud data reception apparatus, and a reception method for efficiently transmitting and receiving a point cloud in order to solve the above-described problems.

A technical problem according to embodiments is to provide a point cloud data transmission apparatus, a transmission method, a point cloud data reception apparatus, and a reception method for solving latency and encoding/decoding complexity.

However, it is not limited only to the above-described technical problem, and the scope of the rights of the embodiments may be extended to other technical problems that can be inferred by those skilled in the art based on the entire contents of this document.

In order to achieve the above-described object and other advantages, a method for transmitting point cloud data according to embodiments includes: obtaining point cloud data, encoding point cloud data, and/or transmitting point cloud data; It may include.

In addition, the method of receiving point cloud data according to embodiments may include receiving point cloud data, decoding point cloud data, and/or rendering point cloud data.

A point cloud data transmission method, a transmission device, a point cloud data reception method, and a reception device according to the embodiments may provide a point cloud service with high quality.

The point cloud data transmission method, the transmission device, the point cloud data reception method, and the reception device according to the embodiments may achieve various video codec methods.

A point cloud data transmission method, a transmission device, a point cloud data reception method, and a reception device according to embodiments may provide general-purpose point cloud content such as an autonomous driving service.

The drawings are included to further understand the embodiments, and the drawings represent embodiments together with a description related to the embodiments.

1 shows a system for providing point cloud content according to embodiments.

2 shows a process for providing Point Cloud content according to embodiments.

3 shows an arrangement of Point Cloud capture equipment according to embodiments.

4 shows a point cloud encoder according to embodiments.

5 illustrates voxels in a 3D space according to embodiments.

6 shows an example of an octree and an occupancy code according to embodiments.

7 illustrates an example of a neighbor node pattern according to embodiments.

8 shows an example of a point configuration of Point Cloud content for each LOD according to embodiments.

9 shows an example of a point configuration of Point Cloud content for each LOD according to embodiments.

10 shows an example of a block diagram of a point cloud decoder according to embodiments.

11 shows an example of a point cloud decoder according to embodiments.

12 shows components for encoding Point Cloud video of a transmitter according to embodiments.

13 shows components for decoding Point Cloud video of a receiver according to embodiments.

14 shows an architecture for G-PCC-based point cloud data storage and streaming according to embodiments.

15 shows point cloud data storage and transmission according to embodiments.

16 shows a device for receiving point cloud data according to embodiments.

17 shows an example of a structure capable of interworking with a method/device for transmitting and receiving point cloud data according to embodiments.

18 shows an example of rendering point cloud data according to embodiments.

19 shows an example of configuration of point cloud data and LOD according to embodiments.

20 shows an example of a Molton code search range according to embodiments.

21 shows an example of a search range level according to embodiments.

22 shows an example of a process of encoding and/or decoding attribute information according to embodiments.

23 illustrates an example of a property information prediction unit of an encoder according to embodiments.

24 illustrates an example of a property information prediction unit of a decoder according to embodiments.

25 shows an example of a configuration diagram of an attribute information prediction unit according to embodiments.

26 shows an example of a structure of point cloud data according to embodiments.

27 illustrates an example syntax of information related to a neighbor point set generation option according to embodiments.

28 illustrates information related to a neighbor point set generation option according to embodiments.

29 illustrates an example of information related to a neighbor point set generation option according to embodiments.

30 shows a PCC encoder according to embodiments.

31 shows an example of a geometric information encoder according to embodiments.

32 shows an example of an attribute information encoder according to embodiments.

33 shows an example of a PCC decoder according to embodiments.

34 shows an example of a geometric information decoder according to embodiments.

35 shows an example of an attribute information decoder according to embodiments.

36 illustrates an example of a point cloud data transmission apparatus/method and a reception apparatus/method including a neighbor point set generator according to embodiments.

37 illustrates an attribute information prediction unit and/or a neighbor information conversion unit according to embodiments.

38 illustrates an example of a neighbor information inverse change unit and/or an attribute information prediction unit according to embodiments.

39 illustrates an example of information related to a neighbor point set generation option according to embodiments.

40 illustrates an example of information related to a neighbor point set generation option according to embodiments.

41 illustrates an example of information related to a neighbor point set generation option according to embodiments.

42 shows an example of additional information related to a set of neighboring points according to embodiments.

43 illustrates an example of additional information related to a neighboring point set according to embodiments.

44 shows examples of quant values according to embodiments.

45 illustrates an example of additional information related to a neighboring point set according to embodiments.

46 illustrates an example of additional information related to a neighboring point set according to embodiments.

47 illustrates an example flowchart of a method for relating a neighboring point according to embodiments.

48 shows a method for transmitting point cloud data according to embodiments.

49 shows a method of receiving point cloud data according to embodiments.

The preferred embodiments of the embodiments will be described in detail, examples of which are shown in the accompanying drawings. The detailed description below with reference to the accompanying drawings is intended to describe preferred embodiments of the embodiments, rather than showing only embodiments that can be implemented according to the embodiments of the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it is obvious to a person skilled in the art that the embodiments may be practiced without these details.

Most terms used in the embodiments are selected from general ones widely used in the relevant field, but some terms are arbitrarily selected by the applicant, and their meanings will be described in detail in the following description as necessary. Accordingly, the embodiments should be understood based on the intended meaning of the term, not the simple name or meaning of the term.

1 shows an example of a system for providing point cloud content according to embodiments.

The point cloud data transmission device 10000 according to the embodiments includes a point cloud video acquisition unit (Point Cloud Video Acquisition, 10001), a point cloud video encoder (Point Cloud Video Encoder, 10002) and/or a transmitter (Transmitter ( or Communication module), 10003).

A point cloud video acquisition unit (Point Cloud Video Acquisition, 10001) according to embodiments acquires a Point Cloud video through a process of capturing, synthesizing, or generating a Point Cloud video.

A point cloud video encoder 10002 according to embodiments encodes point cloud video data.

A transmitter (or communication module) 10003 according to embodiments transmits the encoded point cloud video data in the form of a bitstream.

The point cloud data receiving device 10004 according to the embodiments includes a receiver 10005, a point cloud video decoder 10006, and/or a renderer 10007.

A receiver 10005 according to embodiments receives a bitstream including point cloud video data. According to embodiments, the receiver 10005 may transmit feedback information to the point cloud data transmission device 10000.

A point cloud video decoder (Point Cloud Decoder, 10006) decodes the received point cloud video data.

The renderer 10007 renders the decoded point cloud video data. According to embodiments, the renderer 10007 may transmit the feedback information acquired at the receiving end to the point cloud video decoder 10006. The point cloud video data according to embodiments may transmit feedback information to the receiver. According to embodiments, the feedback information received by the point cloud transmission device may be provided to the point cloud video encoder.

The embodiments are point cloud content in order to provide various services such as VR (Virtual Reality, Virtual Reality), AR (Augmented Reality, Augmented Reality), MR (Mixed Reality, Mixed Reality), and autonomous driving service. Can provide.

In order to provide a Point Cloud content service, a Point Cloud video may be obtained first. The acquired Point Cloud video is transmitted through a series of processes, and the receiving side can process and render the received data back into the original Point Cloud video. This allows Point Cloud videos to be presented to users. The embodiments provide a method necessary to effectively perform this series of processes.

The overall process (point cloud data transmission method and/or point cloud data reception method) for providing the Point Cloud content service may include an acquisition process, an encoding process, a transmission process, a decoding process, a rendering process, and/or a feedback process. have.

According to embodiments, a process of providing point cloud content (or point cloud data) may be referred to as a point cloud compression process. According to embodiments, the point cloud compression process may mean a geometry-based point cloud compression process.

Each element of the point cloud data transmission device and the point cloud data reception device according to the embodiments may mean hardware, software, a processor, and/or a combination thereof.

A method for transmitting point cloud data according to embodiments includes: obtaining point cloud data, encoding point cloud data; And/or transmitting point cloud data.

The point cloud data transmission apparatus according to the embodiments may include an acquisition unit that acquires point cloud data, an encoder that encodes point cloud data, and/or a transmitter that transmits point cloud data.

A method of receiving point cloud data according to embodiments may include receiving point cloud data, decoding point cloud data, and/or rendering point cloud data.

The point cloud data receiving apparatus according to embodiments may include a receiving unit for receiving point cloud data, a decoder for decoding point cloud data, and/or a renderer for rendering point cloud data.

2 shows a process for providing Point Cloud content according to embodiments.

Point cloud data transmission method and point cloud data reception method according to embodiments include acquisition (20000), encoding (20001), transmission (20002), decoding (20003), rendering (20004), and/or feedback (20005). do.

Acquisition 20000 according to the embodiments is a step of acquiring point cloud data. The point cloud data according to embodiments may be a Ply (Polygon File format or the Stanford Triangle format) file. The Ply file according to the embodiments includes geometry and/or attribute. Geometry according to embodiments represents points in a three-dimensional space. Attributes according to embodiments represent properties such as color and reflection of each point in a 3D space according to geometry.

The encoding 20001 according to embodiments is a step of encoding point cloud data including geometry and/or attributes. The encoded data according to the embodiments may be in the form of a bitstream.

Transmission 20002 according to embodiments is a step of transmitting encoded data. The transmitting device according to the embodiments receives feedback information from the receiving device according to the embodiments. The received feedback information may be delivered to encoding according to embodiments.

The decoding 20003 according to embodiments is a step of receiving a bitstream and decoding point cloud data included in the bitstream. The decoding step may obtain feedback information about a user according to embodiments.

The rendering 20004 according to embodiments is a step of rendering decoded data including geometry and/or attributes.

The feedback 20005 according to the embodiments is a step of obtaining feedback information from a receiving end and/or a user, and providing the obtained feedback information to the point cloud data transmission method and the point cloud data reception method according to the embodiments. Feedback information according to embodiments includes information about a user. For example, the feedback information includes head orientation information related to the user, viewport information related to the user, and the like. Feedback information according to embodiments may be provided to a decoder and/or a transmitter of a receiver according to the embodiments. It is possible to encode/decode point cloud data corresponding to the user's head orientation and/or viewport. There is an effect of efficiently encoding/decoding user-related data without the need to encode/decode data for all viewpoints.

A process for providing a Point Cloud content service according to an embodiment is as follows.

Point cloud compression processing may include a geometry-based point cloud compression process.

The Point Cloud Compression system may include a transmitting device and a receiving device according to embodiments. According to embodiments, the transmission device may be referred to as an encoder, a transmission device, a transmitter, and the like. According to embodiments, the receiving device may be referred to as a decoder, a receiving device, a receiver, or the like. The transmitting device can encode the Point Cloud video and output the bitstream, and can deliver it to the receiving device through a digital storage medium or network in the form of a file or streaming (streaming segment). For example, the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD.

The transmission device may schematically include a Point Cloud video acquisition unit, a Point Cloud video encoder, and a transmission unit. The receiving device may schematically include a receiving unit, a Point Cloud video decoder, and a renderer. The encoder may be referred to as a Point Cloud video/image/picture/frame encoding device, and the decoder may be referred to as a Point Cloud video/image/picture/frame decoding device. The transmitter can be included in the Point Cloud video encoder. The receiver can be included in the Point Cloud video decoder. The renderer may include a display unit, and the renderer and/or display unit may be configured as a separate device or an external component. The transmitting device and the receiving device may further include separate internal or external modules/units/components for a feedback process. Each element included in the transmitting device and the receiving device according to the embodiments may be configured with hardware, software and/or a processor.

The point cloud video acquisition unit may perform a process of acquiring a point cloud video through a process of capturing, synthesizing, or generating a point cloud video. 3D location (x, y, z)/property (color, reflectance, transparency, etc.) data for multiple points, for example, PLY (Polygon File format or the Stanford Triangle format) file, is created by the acquisition process Can be. In the case of a video having multiple frames, one or more files may be obtained. During the capture process, point cloud related metadata (eg, metadata related to capture) may be created.

The Point Cloud Video Encoder can encode the input Point Cloud video. One video may include a plurality of frames, and one frame may correspond to a still image/picture. In this document, a Point Cloud video may include a Point Cloud image/frame/picture, and the Point Cloud video may be used interchangeably with a Point Cloud image/frame/picture. The Point Cloud video encoder can perform a Geometry-based Point Cloud Compression (G-PCC) procedure. The Point Cloud video encoder can perform a series of procedures such as prediction, transform, quantization, and entropy coding for compression and coding efficiency. The encoded data (encoded video/video information) may be output in the form of a bitstream. When based on the G-PCC procedure, the Point Cloud video encoder can encode the Point Cloud video by dividing it into geometry and attributes as described later. In this case, the output bitstream may include a geometry bitstream and/or an attribute bitstream. The attribute may include (color) texture information.

The encapsulation unit may encapsulate the encoded video/video information or data output in the form of a bitstream in the form of a file or streaming. The transmission unit may transmit the point cloud bitstream or the file/segment including the corresponding bitstream to the reception unit of the receiving device through a digital storage medium or a network. Digital storage media may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmission unit may include an element for generating a media file through a predetermined file format, and may include an element for transmission through a broadcast/communication network. The receiver may extract the bitstream and transmit it to a decoding device.

The Point Cloud video decoder may receive the bitstream and perform an operation corresponding to the operation of the Point Cloud video encoder to decode the video/video. In this case, the Point Cloud video decoder can decode the Point Cloud video by dividing it into geometry and attributes, as described later. For example, the Point Cloud video decoder may restore (decode) geometry from the geometry bitstream included in the input bitstream, and restore attributes based on the attribute bitstream included in the input bitstream and the restored geometry. You can (decode) it. A 3D Point Cloud video/image may be reconstructed based on the location information according to the reconstructed geometry and the (color) texture attribute according to the decoded attribute. As described above, the attribute may include (color) texture information.

The renderer can render decoded Point Cloud video/video. The rendered video/image may be displayed through the display unit. The user can view all or part of the rendered result through a VR/AR display or a general display.

The feedback process may include a process of transferring various feedback information that can be obtained during the rendering/display process to a transmitter or a decoder at a receiver. Interactivity can be provided in Point Cloud video consumption through the feedback process. Depending on the embodiment, head orientation information, viewport information indicating an area currently viewed by the user, and the like may be transmitted in the feedback process. Depending on the embodiment, the user may interact with those implemented in the VR/AR/MR/autonomous driving environment.In this case, information related to the interaction may be transmitted to the transmitting side or the service provider side in the feedback process. have. Depending on the embodiment, the feedback process may not be performed.

Embodiments relate to Point Cloud video compression as described above. For example, the method described in the embodiments is based on a point cloud compression or point cloud coding (PCC) standard (ex.G-PCC or V-PCC standard) of MPEG (Moving Picture Experts Group) or a next-generation video/image coding standard. Can be applied.

Point cloud data according to embodiments may be obtained by a camera or the like. The capture method according to embodiments may include, for example, in-word-facing and/or out-of-facing.

In the inword-facing according to the embodiments, one or more cameras may photograph an object of point cloud data from the outside to the inside of the object.

In the outward-facing according to the embodiments, one or more cameras may photograph an object of point cloud data from the inside to the outside of the object. For example, according to embodiments, there may be four cameras.

Point cloud data or point cloud content according to embodiments may be video or still images of objects/environments expressed in various types of 3D space.

1. Point Cloud content acquisition process according to embodiments:

It refers to the process of acquiring a Point Cloud video through the process of capturing, synthesizing or creating a Point Cloud video. 3D location (x, y, z)/property (color, reflectance, transparency, etc.) data for multiple points, for example, PLY (Polygon File format or the Stanford Triangle format) file, is created by the acquisition process Can be. In the case of a video having multiple frames, one or more files may be obtained. During the capture process, metadata related to the capture may be generated.

1.1 Point Cloud video capture through equipment according to embodiments:

It can be composed of a combination of camera equipment (a combination of an infrared pattern projector and an infrared camera) that can acquire depth for capturing Point Cloud content and RGB cameras that can extract color information corresponding to the depth information. Alternatively, depth information can be extracted through LiDAR, which uses a radar system that measures the position coordinates of the reflector by shooting a laser pulse and measuring the return time. A shape of a geometry composed of points in a three-dimensional space can be extracted from depth information, and an attribute representing the color/reflection of each point can be extracted from RGB information. Point Cloud content may be composed of information about the location (x, y, z) and color (YCbCr or RGB) or reflectance (r) of the points.

Point Cloud content may include an outward-facing method for capturing an external environment and an inward-facing method for capturing a central object. In a VR/AR environment, when objects (e.g., key objects such as characters, players, objects, actors, etc.) are composed of Point Cloud contents that users can freely view at 360 degrees, the composition of the capture camera uses the inword-facing method. Can be used. When the current surrounding environment is composed of Point Cloud contents in a car, such as autonomous driving, the configuration of the capture camera may use the outward-facing method. Since Point Cloud content can be captured through multiple cameras, it may be necessary to calibrate the camera before capturing the content in order to set up a global coordinate system between the cameras.

1.2 Out-of-device point cloud video synthesis according to embodiments:

Point cloud content according to embodiments may be a video or still image of an object/environment displayed on various types of 3D space.

In addition, the acquisition method of Point Cloud content can be composed of arbitrary Point Cloud video based on the captured Point Cloud video.

Or, if you want to provide Point Cloud video for a virtual space created by a computer, capture through an actual camera may not be performed. In this case, the capture process may be replaced with a process in which related data is simply generated.

1.3 Point Cloud video post-processing according to embodiments:

The captured Point Cloud video may require post-processing to improve the quality of the content.

During the image capture process, the maximum/minimum depth value can be adjusted within the range provided by the camera equipment, but point data of the unwanted area may be included even after that, removing the unwanted area (eg, background) or recognizing the connected space. Post-treatment of filling the spatial hole can be performed.

In addition, the Point Cloud extracted from the cameras sharing the spatial coordinate system can be integrated into a single content through the conversion process to the global coordinate system for each point based on the position coordinates of each camera acquired through the calibration process. Through this, a wide range of Point Cloud contents can be created, or Point Cloud contents with a high density of points can be obtained.

4 shows a point cloud encoder according to embodiments.

The point cloud encoder according to the embodiments includes a coordinate system transform unit (Transformation Coordinates, 40000), a quantization (Quantize and Remove Points (Voxelize), 40001), an octree analysis unit (Analyze Octree, 40002), and a surface aproxiation analysis unit (Analyze Surface Approximation, 40003), Arithmetic Encode (40004), Reconstruct Geometry (40005), Transform Colors (40006), Transfer Attributes (40007), RATH Transformation A unit 40008, an LOD generation unit (Generated LOD) 40009, a Lifting transform unit (40010), a coefficient quantization unit (Quantize Coefficients, 40011) and/or an Arithmetic Encode (40012) are included.

A coordinate system transformation unit (Transformation Coordinates, 40000) according to embodiments receives positions and transforms them into a coordinate system. For example, positions may be converted into three-dimensional (XYZ) position information. Position information in a 3D space according to embodiments may be referred to as geometry information.

Quantize and Remove Points (Voxelize) 40001 according to embodiments quantizes geometry information. For example, it is possible to quantize by adjusting the position values of the points based on the minimum position values of the points. The quantization 40001 according to embodiments may voxelize points. Voxelization refers to the minimum unit expressing position information in 3D space.

An octree analysis unit 40002 according to embodiments represents a voxel in an octree structure. The octree according to the embodiments represents points based on a tree structure in which voxel space is occupied.

An Analyze Surface Approximation 40003 according to embodiments may analyze and approximate an octree. The octree analysis and approximation according to the embodiments is a process of analyzing to voxelize a region including a plurality of points in order to efficiently provide octree and voxelization.

The Arithmetic Encode 40004 according to embodiments encodes an octree and/or an approximated octree. For example, the encoding method includes an Arithmetic encoding method. As a result of encoding, a geometry bitstream is generated.

The Reconstruct Geometry 40005 according to embodiments reconstructs an octree and/or an approximated octree. The geometry reconstruction unit 40005 reconstructs an octree/voxel based on a result of analyzing the distribution of points.

The color transform unit 40006 according to embodiments transforms color values (or textures) included in attributes. For example, the format of color information may be converted. The color conversion unit (number) according to the embodiments may be optional according to a color value. The color transformation 40006 according to the embodiments is one of the point cloud attribute coding schemes.

The attribute conversion unit (Transfer Attributes 40007) according to embodiments converts attribute information based on positions and/or reconstructed geometry information. For example, the attribute conversion unit (number) may convert an attribute value of a point at that position based on the position value of a point included in the voxel. The attribute transformation 40007 according to the embodiments is one of the point cloud attribute coding schemes.

The RATH transform unit 40008 according to embodiments is an encoding method that predicts attribute information based on reconstructed geometry information. For example, the RATH conversion unit 40008 may predict attribute information of a node at a higher level of the octree based on attribute information associated with a node at a lower level of the octree. The RATH transform 40008 according to embodiments is one of point cloud attribute coding schemes.

The LOD generation unit (Generated LOD 40009) according to the embodiments generates a level of detail (LOD) for points. The LOD according to the embodiments is a unit of a group that distinguishes points. Points can be classified by LOD. An attribute coding scheme using the LOD scheme according to embodiments may be referred to as prediction transformation.

The lifting conversion unit 40010 according to embodiments refers to a method of organizing points for each LOD and converting an attribute value of a point cloud based on a weight. The Lifting transform 40010 according to embodiments is one of point cloud attribute coding schemes.

The point cloud attribute coding method according to the embodiments may use RAHT transformation, LOD generation and lifting transformation, or a method according to a RAHT/LOD/Lifting combination.

The coefficient quantization unit 40011 according to embodiments quantizes attribute-coded point cloud data based on coefficients.

An Arithmetic Encode 40012 according to embodiments encodes the quantized point cloud data based on an Arithmetic coding scheme.

Each component of the point cloud encoder according to the embodiments may be performed by hardware, software, a processor, and/or a combination thereof. Detailed operations of each component of the point cloud encoder according to the embodiments will be described below.

2. Point Cloud data encoding process according to embodiments:

The acquired Point Cloud data is encoded by reconstructing the location/color information of the points in order to adjust the quality of the Point Cloud content (for example, lossless-lossless, loss-lossy, near-lossless) according to network conditions or applications. Can go through.

A process of reconstructing/encoding location information of points may be referred to as geometry coding, and a process of reconstructing/encoding information on attributes (eg, color) associated with each point may be referred to as attribute coding.

2.1 Point Cloud geometry coding according to embodiments:

Each point of the acquired point cloud can be transmitted without loss, but in that case, real-time streaming is not possible because the size of the content data is large. Among the examples, there is Point Cloud content that is 60 Gbps at 30 fps. The content data size may vary depending on the capture device. In order to provide a Point Cloud content service, it is necessary to reconstruct the content according to the maximum target bitrate.

2.1.1 Quantization of the positions of points according to embodiments:

The first step in reconstructing the location information of each point of the entire acquired point cloud is the quantization process for the location information. Find the minimum x, y, z position values of all points, subtract them from the position values of each point, multiply by the set quantization scale value, and lower or increase the nearest integer value.

2.1.2 Voxelization of Points according to embodiments:

In order to reconstruct each point of the point cloud to which the quantization process is applied, octree-based voxelization is performed based on the location information of the points.

In order to store information of points that exist in 3D like pixels, which are the smallest units for 2D image/video information, the 3D space is a unit based on each axis (x, y, z axis). The three-dimensional cubic space that is divided into (unit=1.0) is called a voxel, and the process of matching points existing in the three-dimensional space with a specific voxel is said to be voxelized. Voxel is a hybrid word that combines volume and pixel. A voxel can estimate spatial coordinates in a positional relationship with a voxel group, and like a pixel, can have color or reflectance information.

5 illustrates voxels in a 3D space according to embodiments.

Only one point may not exist in one voxel. One voxel can have multiple point-related information. Alternatively, one voxel can be integrated into one point information to have it. This adjustment can be performed selectively. When one voxel is expressed as one point, the position value of the center point of the voxel can be set based on the position values of points existing in the voxel, and an attribute transform process related thereto needs to be performed. There is. For example, the attribute conversion process may be adjusted to the average value of the points included in the voxel or the center position value of the voxel and the color or reflectance of the neighboring points within a specific radius (refer to Section 2.2.2).

6 shows an example of an octree and an occupancy code according to embodiments.

2.1.3 An octree for Occupied voxel management according to embodiments:

In order to efficiently manage the area/location of these voxels, Point Cloud contents use octrees.

In order to efficiently manage the space of the 2D image, if the entire space is divided by the x-axis and y-axis, 4 spaces are created, and each of the 4 spaces is divided by the x-axis and y-axis. There are 4 spaces. In order to divide the area until the leaf node becomes a pixel, and to efficiently manage the area by size and location, a data structure of a quadtree can be used. Similarly, the same method is applied to efficiently manage the 3D space according to the location and size of the space. However, since the z-axis is added, 8 spaces are created by dividing it based on the x-axis, y-axis, and z-axis, and if each of the eight spaces is divided again based on the x-axis, y-axis, and z-axis, each small space There are 8 spaces. In order to divide a region until a leaf node becomes a voxel, and to efficiently manage each region size and location, an octree data structure capable of managing eight child node regions can be used.

Since the octree is used to manage the voxels reflecting the positions of the points, the total volume of the octree should be set to (0,0,0) ~ (2d, 2d,2d). 2d is set to a value constituting the smallest bounding box surrounding the entire point of the Point Cloud video, and d is the depth of the octree. The formula to find the d value can be as follows. (x_n^int,y_n^int,z_n^int) is the position value of the points to which the quantization process is applied.

The octree can be expressed as an occupancy code. If a point is included in each node, it is expressed as 1, and if there is no point, it is expressed as 0. Each node has an 8-bit bitmap indicating occupancy for 8 child nodes. Entropy coding of occupancy code through arithmetic coder. The generated occupancy code may be directly encoded or may be encoded through an intra/inter coding process to increase compression efficiency. In the receiver, the occupancy code can be used to reconstruct the octree.

2.1.4 Processing scheme for Sparse octree according to embodiments:

Although the voxelization and octree are used to store the location information of the points of the Point Cloud video, in the case of a specific area where there are not many points, it may be inefficient to voxelize all areas. For example, there are few points in a specific area, so it may not be necessary to construct the entire octree. In this case, an early termination scheme is needed. In case of such a case, in the case of a specific area, that is, a specific node of the octree (but the node is not a leaf node), instead of dividing the node into 8 sub-nodes (child nodes), the location of the points directly for that area only. Either by transmitting the s, or by using a surface model, the position of the point in the node area can be reconstructed based on voxels.

In order to enable the case of direct mode, which directly transmits the location of each point to a specific node, it is possible to check whether the condition is satisfied. The option to use direct mode must be enabled, the node must not be a leaf node, and points below the threshold must exist within a specific node, and the total number of points that can directly transmit the point location. Do not exceed the limit of. If all of these cases are satisfied, the position value of the point can be directly entropy-coded with an arithmetic coder for the corresponding node and transmitted.

A trisoup mode that sets a specific level of the octree (if the level is less than the depth d of the octree), and from that level, uses a surface model to reconstruct the position of points in the node area based on voxels. ) Can also be selectively applied. When using the treetop mode, it specifies the level to which the treetop method is applied. For example, if the specified level is the same as the depth of the octree, the trisoup mode is not applied. The specified level must be less than the depth value of the octree to apply the trisoup method. The three-dimensional cube area of nodes of a designated level is called a block. One block may include one or more voxels. The block or voxel may correspond to a brick. Each block may have 12 edges, and it is checked whether each edge is adjacent to an occupied voxel having a point. Each edge can be adjacent to multiple occupied voxels. A specific position of an edge adjacent to the voxel is called a vertex, and when several occupied voxels are adjacent to one edge, the average position of the corresponding positions may be determined as a vertex. When a vertex is present, entropy coding of the starting point (x, y, z) of the edge, direction vectors of the edge (Δx, Δy, Δz), and vertex position values (relative position values within the edge) with an arithmetic coder .

In the case of applying such a method, a geometry restoration process may be performed through a process of triangle reconstruction, up-sampling, and voxelization.

In order to reconstruct a triangle based on the starting point of the edge, the direction vector of the edge, and the position value of the vertex, first, calculate the centroid value of each vertex, and ② add the square to the values subtracting the center value from each vertex. And find the sum of all the values.

The minimum value of the added value is obtained, and the projection process is performed along the axis with the minimum value. For example, when the x element is the minimum, each vertex is projected on the x-axis based on the center of the block, and is projected on the (y, z) plane. If the projected value on the (y, z) plane is (ai, bi), θ is calculated through atan2(bi, ai), and vertices are aligned based on the θ value. The method of composing triangles according to the number of vertices is to create triangles by combining them according to the sorted order as shown in the following table. For example, if there are 4 vertices, you can construct two triangles. The first triangle consists of the first, second, and third vertices from the aligned vertices, and the second triangle consists of the third, fourth and first vertices.

The upsampling process is performed to voxelize by adding points in the middle along the edge of the triangle. Additional points are created based on the upsampling factor and the width of the block. These points are called refined vertices. Refined vertices are voxelized, and attributes (eg, colors) are coded based on the voxelized position value when attribute coding.

7 illustrates an example of a neighbor node pattern according to embodiments.

2.1.5 Geometry coding according to embodiments:

The location/color data of the point of the point cloud content is reconstructed, and entropy coding can be performed using an arithmetic coder. Because the data size of Point Cloud video is large, compression efficiency can be an important factor. Therefore, a method of increasing compression efficiency can be applied by applying context adaptive arithmetic coding.

Geometry coding encodes the occupancy code of each node in the octree. The occupancy code can be entropy coded using the arithmetic code directly, but it may be adaptively encoded based on occupancy of neighboring nodes (intra-based) or based on the occupancy code of the previous frame (inter-based). Here, the frame may refer to a set of point cloud data generated at the same time. The compression efficiency can vary depending on how many neighboring nodes are referenced. The larger the bit, the more complex it is, but the compression efficiency can be increased by making it skewed to one side. For example, if you have a 3-bit context, you have to code by dividing into 23 = 8 types. The divided coding part affects the complexity of the implementation. Therefore, it is necessary to match the appropriate level of compression efficiency and complexity.

In each node of the octree, coding can be adaptively performed through occupancy of neighboring nodes. To this end, first, a value of a neighbor pattern is obtained based on occupancy of the neighboring node. The order of bits for each location of neighboring nodes is shown in the figure. For example, if neighboring nodes corresponding to 1, 2, 4, and 8 are occupied, 15, which is the sum of 1, 2, 4, and 8, becomes the neighboring node pattern value of the corresponding node. In general, it refers to six neighboring nodes that share a plane around the node. The neighboring node pattern obtained according to occupancy of the six neighboring nodes is composed of a value of 26=64, and 64 different codings may be performed. Alternatively, it is possible to reduce complexity by changing the neighboring node pattern value through a table that internally changes 64 to 10 or 6. Compression efficiency can be improved by performing encoding using the occupied code of the current node and the neighboring node pattern value.

According to embodiments, the intra/inter coding process is an optional process and may be omitted.

2.2 Point Cloud attribute coding according to embodiments:

Voxelization is applied, and in the direct mode, the point cloud data is rearranged to the front of the point cloud data, and in the trisoup mode, a triangle reconstruction, upsampling, and voxelization are added to perform the encoding process for related attribute information based on the reconstructed geometry. Can be done. Since attribute information is dependent on geometry, a process of coding attribute information based on the reconstructed geometry may be necessary.

Point Cloud attribute data may be composed of color (YCbCr or RGB) or reflectance (r) information. In both cases, the same method of coding can be applied. However, the difference is that the color has 3 elements and the reflectance has 1 element, and each element can be treated independently.

Attribute coding methods include prediction transform, lifting transform, and region adaptive hierarchical transform (RAHT), and can be selectively applied.

2.2.1 Transform Color according to the embodiments:

You can perform coding by changing the color from RGB to YCbCr. Color conversion refers to such a color format conversion process.

2.2.2 Attribute Transform according to embodiments:

When only one point exists in one voxel, the position values for points existing in the voxel are set as the center point of the voxel in order to integrate and indicate one point information in one voxel, and the associated attribute value accordingly It may be necessary to convert. In addition, the attribute conversion process is performed even when executed in the treetop mode.

The attribute conversion process may be calculated as an average value of attribute values such as the central position value of the voxel and the color or reflectance of neighboring points within a specific radius, or an average value applied with a weight according to the distance from the central position. In this case, each voxel has a position and a calculated attribute value.

When searching for neighboring points existing within a specific location/radius, a K-D tree or Molton code can be used. The K-D tree is a binary search tree and supports a data structure that can manage points based on location so that the Nearest Neighbor Search (NNS) can be quickly performed. The Molton code can be generated by mixing bits of 3D location information (x, y, z) for all points. For example, if the value of (x, y, z) is (5, 9, 1), it becomes (0101, 1001, 0001) when expressed as a bit. When mixed, it becomes 010001000111, which is 1095. 1095 is the Molton code value of (5, 9, 1). Points are sorted based on the Morton code, and shortest neighbor search (NNS) is possible through a depth-first traversal process.

After the attribute transformation process, the shortest neighbor search (NNS) is sometimes required in another transformation process for attribute coding, and for this, a K-D tree or a Molton code may be used.

Point clouds according to embodiments may be classified into groups according to a level of detail (LOD). As shown in the figure, the degree of detail increases from left to right. In other words, the closer the distance between the points to the left is, the sparse it is, and the closer to the right, the closer the distances between the points.

2.2.2.1 Prediction Transformation According to Examples:

Predictive transformation is a method to which the Level Of Detail (LOD) technique is applied. Each point is set by calculating the LOD value based on the set LOD distance value. For example, the composition of points according to the LOD value can be as follows.

When the point clouds according to the embodiments are distributed, the original order indicates, for example, the order of points P0 to P9.

According to embodiments, when the point cloud is separated for each LOD by LOD generation, for example, a group including P0, P5, P4, and P1 belongs to LOD0, and a group including P1, P6, and P3 belongs to LOD1. A group belonging to and including P9, P8, and P7 may belong to LOD2. LOD-based order represents the order of LOD groups.

Each point in the point cloud can be separated by LOD, and the composition of points by LOD includes points belonging to the LOD lower than the corresponding LOD value. For example, if LOD level 2, it corresponds to all points belonging to

LOD level

1 and 2.

For predictive transformation, a predictor is created for each point in the Point Cloud. Therefore, if there are N points, N predictors can be generated. The predictor may calculate and set a weight (=1/distance) value based on the LOD value for each point, the indexing information for the neighboring points existing within the distance set for each LOD, and the distance value with the neighboring points.

The property (color or reflectance) values of neighboring points set in the predictor of each point are multiplied by a weight value calculated based on the distance of each neighboring point. The color or reflectance values multiplied by the weights of neighboring points are averaged and set as the predicted attribute value of the corresponding point. A quantization process is performed on the residual attribute value obtained by subtracting the predicted attribute value from the color or reflectance value of each point. The quantization process for properties is as follows.

If there are no neighboring points in the predictor of each point, entropy coding is performed using an arithmetic coder directly for the color/reflectance value of the current point, and if there are neighboring points, the property value predicted through the neighboring points from the color or reflectance value of the point Entropy coding is performed on the residual attribute value minus the quantization process using an arithmetic coder.

2.2.2.2 Lifting transformation according to embodiments:

The predictive transformation and lifting transformation process reconstruct points into a set of detail levels through a level of detail (LOD) generation process. The reconstruction method was described above.

Lifting transformation generates a predictor for each point, sets the calculated LOD in the predictor, registers the neighboring points, and sets weights according to the distances to the neighboring points. The difference from prediction transformation is a method of accumulating and applying weights to attribute values. The method is as follows.

1) There is a separate array QW (Quantization Wieght) that stores weight values for each point. The initial value of all elements of QW is 1.0. The value obtained by multiplying the weight of the predictor of the current point to the QW value of the predictor index of the neighboring node registered in the predictor is added.

2) In order to calculate the predicted attribute value, the value obtained by multiplying the attribute value of the point by the weight is subtracted from the existing attribute value. This process is the lift prediction process.

3) Create a temporary array called updateweight and update and initialize it to 0.

4) The weight calculated by additionally multiplying the calculated weight for all predictors by the weight stored in the QW corresponding to the predictor index is cumulatively added to the updateweight by the index of the neighboring node, and for update, the value multiplied by the attribute value of the index of the neighboring node. Is accumulated and summed.

5) For all predictors, the attribute value of update is divided by the weight value of the updateweight of the predictor index, and then added to the existing attribute value. This process is a lift update process.

6) For all predictors, the attribute value updated through the lift update process is additionally multiplied by the weight updated through the lift prediction process (stored in QW), and the quantized value is quantized using an arithmetic coder. Entropy coding.

2.2.2.3 RAHT conversion according to embodiments:

RAHT transformation is a method of predicting attribute information of nodes at a higher level using attribute information associated with a node at a lower level of an octree, and is an intra coding method for attribute information through octree backward scan. The voxel is scanned from the voxel to the entire area, and in each step, the voxel is combined into a larger block and performed up to the root node. Since the merging process is performed only for occupied nodes, in the case of an empty node that is not occupied, merging is performed with the node of the higher level immediately.

The gDC value is also quantized like the high-pass coefficient, and entropy coding is performed using an arithmetic coder at the end.

3. Transmission process according to embodiments:

The transmission process may be a process of processing and transmitting the encoded geometry and attribute data and metadata of the Point Cloud content that has undergone an encoding process. For transmission, processing according to any transmission protocol may be performed. Geometry and attribute data of the generated point cloud content, and related metadata bitstreams may be created as one or more track data, or may be encapsulated into segments. . Data processed for transmission may be delivered through a broadcasting network and/or a broadband. These data may be delivered to the receiving side in an on-demand manner. The receiving side can receive the data through various paths. On the other hand, the encoded geometry and attribute data and metadata of the Point Cloud content that has gone through the encoding process may be stored in a digital storage medium in the form of a media file and transmitted to the receiver.

The point cloud decoder according to embodiments receives a bitstream including geometry and/or attributes for point cloud data. The geometry decoder according to the embodiments decodes the geometry, and the attribute decoder according to the embodiments decodes the attribute. The attribute decoder decodes the attribute based on the decoded geometry. The decoder may generate a point cloud based on the decoded geometry and/or decoded attributes.

4. Decoding process according to embodiments:

The decoding process may include a process of reconstructing (decoding) a Point Cloud video/video by receiving a bitstream and performing an operation corresponding to the encoding operation.

11 shows an example of a point cloud decoder according to embodiments.

The point cloud decoder according to the embodiments may perform the reverse process of the point cloud encoder according to the embodiments.

Point cloud decoders according to embodiments include an arithmetic decoder (11000), an octree synthesis unit (synthesize octree, 11001), a surface opoxidation synthesis unit (synthesize surface approximation, 11002), and a geometry reconstruction unit (reconstruct geometry, 11003), inverse transform coordinates (11004), arithmetic decode (11005), inverse quantize (11006), RAHT (11007), generate LOD (generate LOD, 11008) , Inverse lifting unit (11009), and / or color inverse transform unit (inverse transform colors, 11010).

The arithmetic decoder 11000 according to the embodiments decodes the geometry included in the received bitstream based on an arithmetic method.

The octree synthesizer 11001 according to the embodiments may generate an octree from geometry.

The surface opoxidation synthesis unit 11002 according to the embodiments may synthesize a surface based on the decoded geometry and/or octree.

The geometry reconstructing unit 11003 according to embodiments may regenerate a geometry based on a surface and/or decoded geometry.

The inverse transform coordinates 11004 according to the embodiments may obtain positions (positions) by inverse transforming a coordinate system based on geometry.

An arithmetic decoder 11005 according to embodiments decodes an attribute included in a received bitstream based on an arithmetic method.

The inverse quantize (11006) according to the embodiments performs inverse quantization on the decoded attribute.

The RAHT 11007 according to the embodiments, the generate LOD 11008 according to the embodiments, and/or the inverse lifting 11009 according to the embodiments correspond to the encoder according to the embodiments. The reverse process of the operation can be performed.

The inverse transform colors 11010 according to embodiments may obtain an attribute (for example, a color value) by inversely transforming colors.

The decoding process may include a geometry decoding process and an attribute decoding process. The decoder may reconstruct (decode) geometry from the geometry bitstream included in the input bitstream, and reconstruct (decode) attributes based on the attribute bitstream included in the input bitstream and the restored geometry. A 3D Point Cloud video/image may be reconstructed based on the location information according to the reconstructed geometry and the (color) texture attribute according to the decoded attribute.

Specifically, the decoder acquires information about the geometry by decoding the geometry bitstream with an arithmetic coder, creates an occupancy code based on the information about the geometry, and reconstructs the geometry. When the direct mode is applied, the location information value of the point is directly imported and added, and when the trisoup mode is applied, the geometry is restored through triangle reconstruction, up-sampling, and voxelization. (Refer to Section 2.1.3 for details). The restored geometry may include restored (decoded) point cloud picture/frame without (any) attributes.

In addition, the decoder obtains information on the attribute by decoding the received attribute bitstream with an arithmetic coder, and based on the information on the obtained attribute and the associated location information derived from the geometry coding process. You can create a restored (decoded) point cloud picture/frame with attributes. In the attribute decoding process, if necessary, an inverse quantization process is performed, an inverse transformation process for prediction/lifting/RAHT is selectively performed according to a method applied during encoding, and then, if necessary, color conversion may be performed to restore attributes.

5. Rendering process according to embodiments:

The rendering process refers to the process of rendering and displaying Point Cloud content data in 3D space. It can be rendered according to a desired rendering method with the location and property information of the decoded points through the decoding process. Points of the Point Cloud content may be rendered as a vertex with a certain thickness, a cube with a specific minimum size centered on the vertex position, or a circle centered on the vertex position. The user can view all or part of the rendered result through a VR/AR display or a general display.

6. Feedback process according to embodiments:

The feedback process may include a process of transferring various feedback information that can be obtained during the display process to a transmitting side or a receiving side decoding. Through the feedback process, interactivity can be provided in Point Cloud video consumption. Depending on the embodiment, head orientation information, viewport information indicating an area currently viewed by the user, and the like may be transmitted in the feedback process. Depending on the embodiment, the user may interact with those implemented in the VR/AR/MR/autonomous driving environment.In this case, information related to the interaction may be transmitted to the transmitting side or the service provider side in the feedback process. have. Depending on the embodiment, the feedback process may not be performed.

The head orientation information may mean information on the position, angle, and movement of the user's head. Based on this information, information about the area that the user is currently viewing in the Point Cloud video, that is, viewport information can be calculated.

The viewport information may be information on an area currently viewed by the user in the Point Cloud video. Through this, a gaze analysis is performed, which allows you to check how the user consumes the Point Cloud video, which area of the Point Cloud video and how much they gaze at. The gaze analysis may be performed at the receiving side and transmitted to the transmitting side through a feedback channel. A device such as a VR/AR/MR display may extract a viewport area based on the position/direction of the user's head and a vertical or horizontal FOV supported by the device.

Depending on the embodiment, the above-described feedback information is not only transmitted to the transmitting side, but may be consumed by the receiving side. That is, decoding and rendering of the receiver may be performed using the above-described feedback information. For example, using head orientation information and/or viewport information, only a point cloud video for a region currently viewed by the user may be preferentially decoded and rendered.

Here, the viewport or the viewport area may mean an area that the user is viewing in the Point Cloud video. A viewpoint is a point that a user is viewing in a Point Cloud video, and may mean a center point of a viewport area. That is, the viewport is an area centered on the viewpoint, and the size, shape, etc. occupied by the area may be determined by a field of view (FOV).

Components for video encoding of point cloud data according to embodiments include a data input unit 12000, a quantization processing unit 12001, a voxelization processing unit 12002, an octree occupancy code generation unit 12003, and a front surface model processing unit 12004. , Intra/inter coding processing unit (12005), Arithmetic coder (12006), metadata processing unit (12007), color conversion processing unit (12008), attribute conversion processing unit (12009), prediction/lifting/RAHT conversion processing unit 12010, Arithmetic coder (12011) and/or a transmission processing unit 12012 may be included.

The data input unit 12000 according to the embodiments receives or acquires point cloud data. The data input unit 12000 may correspond to the point cloud acquisition unit 10001 of FIG. 1 according to embodiments.

The quantization processing unit 12001 according to embodiments quantizes a geometry of point cloud data, for example, position value information of points.

The voxelization processing unit 12002 according to embodiments voxelsizes position value information of quantized points.

The octree occupancy code generation unit 12003 according to embodiments may display position value information of voxelized points in an octree based on an octree accupancy code.

The front surface model processing unit 12004 according to embodiments may express and process an octree for position value information of points of a point cloud based on a surface model method.

The intra/inter coding processor 12005 according to embodiments may intra/inter code point cloud data.

The Arithmetic coder 12006 according to embodiments may encode point cloud data based on an Arithmetic coding method.

The metadata processing unit 12007 according to embodiments processes metadata about point cloud data, for example, a set value, and provides it to a necessary process such as a geometry encoding process and/or an attribute encoding process.

The color conversion processing unit 12008 according to embodiments may convert a color of the point cloud data based on an attribute of the point cloud data, for example, attribute value information of points and/or a reconstructed position value.

The attribute conversion processing unit 12009 according to embodiments may convert an attribute value of point cloud data.

The prediction/lifting/RAHT conversion processing unit 12010 according to embodiments may attribute-code the point cloud data based on a combination of a prediction method, a lifting method, and/or a RAHT method.

The Arithmetic coder 12011 according to the embodiments may encode point cloud data based on an Arithmetic coding method.

The above processes may correspond to the point cloud encoder 10002 of FIG. 1 according to embodiments.

The transmission processing unit 12012 according to embodiments may transmit the encoded geometry and/or the encoded attribute.

According to embodiments, a process for a position value of points and a process for an attribute value of points may perform each process by sharing data/information of each other.

12 is a diagram illustrating a transmitter equipment for providing a Point Cloud content service according to embodiments.

Embodiments at the transmitting side may be related to the Point Cloud content transmission device. The Point Cloud content transmission device includes a data input unit, a quantization processing unit, a voxelization processing unit, an occupancy code generation unit, a surface model processing unit, an intra/inter coding processing unit, an arithmetic coder, a metadata processing unit, and a reconstructed position value for the point position values. Geometry bits encoded through the color conversion processing unit, the attribute conversion processing unit, the predictive transformation processing unit, the lifting transformation processing unit, the RAHT transformation processing unit, and the arithmetic coder according to the attribute encoding method used to process the attribute values for the point position/voxel value based on The stream and attribute bitstream are transmitted to the receiving side through the transmission processing unit. The function of each component is described in Chapter 2 Encoding Process and Chapter 3 Transmission Process.

Components for video decoding of point cloud data according to embodiments include a receiving unit 13000, a receiving processing unit 13001, an Arithmetic decoder 13002, an octree reconstruction processing unit 13003 based on an occupancy code, and a surface model processing unit (triangle reconstruction, Up-sampling, voxelization) (13004), inverse quantization processing unit (13005), metadata parser (13006), arithmetic decoder (13007), inverse quantization processing unit (13008), prediction/lifting/RAHT inverse transform processing unit (13009), color An inverse transform processing unit 13010 and/or a renderer 13011 may be included. Each component of decoding according to the embodiments may perform a reverse process of the component of encoding according to the embodiments.

The receiving unit 13000 according to the embodiments receives point cloud data. The receiver 13000 may correspond to the receiver 10007 of FIG. 1 according to embodiments.

The reception processing unit 13001 according to the embodiments may obtain a geometry bitstream and/or an attribute bitstream included in the received point cloud data.

The Arithmetic decoder 13002 according to embodiments may decode a geometry bitstream based on an Arithmetic method.

The octree reconstruction processing unit 13003 based on the Occupancy code according to the embodiments may reconstruct the decoded geometry into an octree based on the Occupancy code.

The surface model processing unit (triangle reconstruction, up-sampling, voxelization) 13004 according to the embodiments performs triangular reconstruction, up-sampling, voxelization, and/or a combination thereof for point cloud data based on a surface model method. The following treatment can be performed.

The inverse quantization processing unit 13005 according to embodiments may inverse quantize point cloud data.

The metadata parser 13006 according to embodiments may parse metadata included in the received point cloud data, for example, a setting value. The metadata parser 13006 may deliver metadata to each process of a geometry decoding process and/or an attribute decoding process. Each process according to embodiments may be performed based on necessary metadata.

The arithmetic decoder 13007 according to the embodiments may decode the attribute bitstream of point cloud data based on an arithmetic method based on the reconstructed position value.

The inverse quantization processing unit 13008 according to embodiments may inverse quantize point cloud data.

The prediction/lifting/RAHT inverse transform processing unit 13009 according to embodiments may process point cloud data based on a prediction/lifting/RAHT method and/or a method according to a combination thereof.

The color inverse transform processing unit 13010 according to embodiments may inversely transform a color value of point cloud data.

The above-described processes may correspond to the point cloud decoder 10006 of FIG. 1 according to embodiments.

The renderer 13011 according to embodiments may render point cloud data.

The drawing is a diagram illustrating receiver equipment for providing Point Cloud content services according to embodiments.

Embodiments at the receiving side may be related to the Point Cloud content receiving device. The Point Cloud content receiving device includes a receiving unit, a receiving processing unit, a metadata parser, an arithmetic decoder for the geometry bitstream of a point, an occupancy code-based octree reconstruction processing unit, a surface model processing unit, an inverse quantization processing unit, and a point location based on the reconstructed position value. /In order to decode the attribute values for the voxel value, it is delivered to the renderer through the color inverse transformation processing unit through the arithmetic decoder, inverse quantization processing unit, and inverse predictive transformation processing unit, lifting inverse transformation processing unit, and RAHT inverse transformation processing unit according to the used attribute encoding method. It provides VR/AR/MR/autonomous driving experiences. The functions of each component are described in Chapter 3 Transmission Process, Chapter 4 Decoding Process, and Chapter 5 Rendering Process.

Within the entire architecture for providing point cloud video described above, point cloud data that undergoes a series of processes of acquisition/encoding/transmission/decoding/rendering may be referred to as point cloud content data or point cloud video data. The term Point Cloud Content Data may also be used as a concept including metadata or signaling information related to these Point Cloud data.

The embodiments provide a method for storing and streaming Point Cloud data that supports various services such as VR (Virtual Reality, Virtual Reality), AR (Augmented Reality, Augmented Reality), MR (Mixed Reality, Mixed Reality), and autonomous driving. to provide.

14 is a diagram showing the overall architecture for storing or streaming point cloud data compressed based on Geometry-based Point Cloud Compression (hereinafter, G-PCC). The process of storing and streaming point cloud data may include an acquisition process, an encoding process, a transmission process, a decoding process, a rendering process and/or a feedback process.

Embodiments propose a method of effectively providing point cloud media/contents/data. Point cloud In order to effectively provide media/contents/data, first, a point cloud can be acquired. For example, point cloud data may be acquired through the process of capturing, synthesizing, or creating a point cloud through one or more cameras. Through this acquisition process, point cloud data including the 3D position (x, y, z position values, etc.) of each point (hereinafter referred to as geometry) and the attributes of each point (color, reflectance, transparency, etc.) It can be obtained, and can be created as a PLY (Polygon File format or the Stanford Triangle format) file including the same. In the case of point cloud data having multiple frames, one or more files may be acquired. In this process, point cloud related metadata (eg, metadata related to capture, etc.) can be created.

The Point Cloud encoder performs a Geometry-based Point Cloud Compression (G-PCC) procedure, which performs a series of procedures such as prediction, transformation, quantization, and entropy coding, and the encoded data (encoded video/video information) is a bitstream. It can be output in (bitstream) format. This can be encoded by dividing into geometry and attributes as described later, and point cloud related metadata can be included in the bit stream. In this case, the output bitstream may include a geometry bitstream and/or an attribute bitstream.

Encapsulation (file/segment encapsulation) may encapsulate encoded point cloud data and/or point cloud related metadata in the form of a file or a segment for streaming. Here, the metadata related to the point cloud may be transmitted from a metadata processing unit. The metadata processing unit may be included in the point cloud video encoder, or may be configured as a separate component/module. The encapsulation processing unit may encapsulate the data in a file format such as ISOBMFF, or may process the data in the form of other DASH segments. The encapsulation processor may include point cloud related metadata on a file format according to an embodiment. Point cloud metadata may be included in boxes of various levels in the ISOBMFF file format, for example, or may be included as data in separate tracks within the file. According to an embodiment, the encapsulation processing unit may encapsulate the point cloud related metadata itself as a file.

The transmission processing unit may apply processing for transmission to the encapsulated point cloud data according to the file format. The transmission processing unit may be included in the transmission unit or may be configured as a separate component/module. The transmission processing unit can process point cloud data according to any transmission protocol. The processing for transmission may include processing for transmission through a broadcasting network and processing for transmission through a broadband. According to an embodiment, the transmission processing unit may receive not only the point cloud data, but also the point cloud related metadata from the metadata processing unit, and may apply processing for transmission to this.

The transmission unit may transmit the point cloud bitstream or the file/segment including the corresponding bitstream to the reception unit of the receiving device through a digital storage medium or a network. For transmission, processing according to any transmission protocol can be performed. Data processed for transmission may be delivered through a broadcasting network and/or a broadband. These data may be delivered to the receiving side in an on-demand manner. Digital storage media may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmission unit may include an element for generating a media file through a predetermined file format, and may include an element for transmission through a broadcast/communication network. The receiver may extract the bitstream and transmit it to a decoding device.

The receiver may receive point cloud data transmitted by the point cloud data transmission device according to the embodiments. Depending on the transmitted channel, the receiver may receive point cloud data through a broadcasting network or may receive point cloud data through a broadband. Alternatively, point cloud video data can be received through a digital storage medium. The receiver may include a process of decoding the received data and rendering it according to a user's viewport.

The reception processing unit may perform processing according to a transmission protocol on the received point cloud data. The receiving processing unit may be included in the receiving unit, or may be configured as a separate component/module. The reception processing unit may perform the reverse process of the transmission processing unit described above so as to correspond to the transmission processing performed by the transmission side. The receiving processing unit may transmit the acquired point cloud data to the decapsulation processing unit, and the acquired point cloud related metadata may be transmitted to the metadata parser.

The decapsulation processing unit (file/segment decapsulation) may decapsulate the point cloud data in the form of a file transmitted from the reception processing unit. The decapsulation processor may decapsulate files according to ISOBMFF or the like to obtain a point cloud bitstream or point cloud related metadata (or a separate metadata bitstream). The acquired point cloud bitstream may be transmitted to the point cloud decoder, and the acquired point cloud related metadata (or metadata bitstream) may be transmitted to the metadata processing unit. The point cloud bitstream may include the metadata (metadata bitstream). The metadata processing unit may be included in the point cloud video decoder, or may be configured as a separate component/module. The point cloud related metadata acquired by the decapsulation processing unit may be in the form of a box or track in a file format. If necessary, the decapsulation processing unit may receive metadata required for decapsulation from the metadata processing unit. The point cloud related metadata may be transmitted to the point cloud decoder and used for a point cloud decoding procedure, or may be transmitted to a renderer and used for a point cloud rendering procedure.

The Point Cloud decoder may decode data by receiving the bitstream and performing an operation corresponding to the operation of the Point Cloud encoder. In this case, the Point Cloud decoder can decode the Point Cloud data by dividing it into geometry and attributes, as described later. For example, the Point Cloud decoder can restore (decode) geometry from the geometry bitstream included in the input bitstream, and restore the attribute value based on the attribute bitstream included in the input bitstream and the restored geometry. You can (decode) it. A point cloud may be restored by restoring the position of each point and attribute information of each point in 3D based on the location information according to the restored geometry and the (color) texture attribute according to the decoded attribute value.

The sensing/tracking unit obtains orientation information and/or user viewport information from the user or the receiving side and transmits it to the receiving unit and/or the transmitting unit. The orientation information provides information on the position, angle, and movement of the user's head. It can be displayed or information about the location, angle, and movement of the device that the user is viewing. Based on this information, information on an area that the user is currently viewing in the 3D space, that is, user viewport information may be calculated.

The user viewport information may be information on a region currently viewed by the user through a device or an HMD in a 3D space. A device such as a display may extract a viewport area based on orientation information and a vertical or horizontal FOV supported by the device. Orientation or viewport information can be extracted or calculated at the receiving end. The orientation or viewport information analyzed by the receiving side may be transmitted to the transmitting side through a feedback channel.

The receiving unit uses the orientation information acquired by the sensing/tracking unit and/or the viewport information indicating the area currently being viewed by the user, and efficiently extracts only the media data of the specific area, that is, the area indicated by the orientation information and/or the viewport information. It can be extracted or decoded. In addition, the transmitter can efficiently encode only media data of a specific area, that is, an area indicated by orientation information and/or viewport information, or generate and transmit a file, using orientation information and/or viewport information acquired by the sensing/track unit. .

The renderer can render decoded Point Cloud data in 3D space. The rendered video/image may be displayed through the display unit. The user can view all or part of the rendered result through a VR/AR display or a general display.

The feedback process may include a process of transferring various feedback information that can be obtained during the rendering/display process to a transmitter or a decoder at a receiver. Interactivity in Point Cloud data consumption can be provided through the feedback process. Depending on the embodiment, head orientation information, viewport information indicating an area currently viewed by the user, and the like may be transmitted in the feedback process. Depending on the embodiment, the user may interact with those implemented in the VR/AR/MR/autonomous driving environment.In this case, information related to the interaction may be transmitted to the transmitting side or the service provider side in the feedback process. have. Depending on the embodiment, the feedback process may not be performed.

According to an embodiment, the above-described feedback information is not only transmitted to the transmitting side, but may be consumed by the receiving side. That is, a decapsulation process, decoding, rendering process, etc. of the receiver may be performed using the above-described feedback information. For example, point cloud data for a region currently viewed by a user may be preferentially decapsulated, decoded, and rendered using orientation information and/or viewport information.

15 shows point cloud data storage and transmission according to embodiments.

The drawing shows a point cloud data transmission apparatus according to embodiments.

Point Cloud data storage and transmission device according to the embodiments is a Point Cloud acquisition unit (Point Cloud Acquisition), a Point Cloud encoding unit (Point Cloud Encoding), a file / segment encapsulation unit (File / Segment Encapsulation), and / or It includes a delivery part (Delivery). Each configuration of the transmission device may be a module/unit/component/hardware/software/processor.

Point cloud geometry, attributes, auxiliary data, mesh data, etc. can be configured as separate streams or can be stored in different tracks in the file. Furthermore, it can be included in a separate segment.

The Point Cloud Acquisition acquires a point cloud. For example, point cloud data may be acquired through a process of capturing, synthesizing, or creating a point cloud through one or more cameras. Through this acquisition process, point cloud data including the 3D position (x, y, z position values, etc.) of each point (hereinafter referred to as geometry) and the attributes of each point (color, reflectance, transparency, etc.) It can be obtained, and can be created as a PLY (Polygon File format or the Stanford Triangle format) file including the same. In the case of point cloud data having multiple frames, one or more files may be acquired. In this process, point cloud related metadata (eg, metadata related to capture, etc.) can be created.

Point Cloud Encoding, the Point Cloud Encoder performs a Geometry-based Point Cloud Compression (G-PCC) procedure, which performs a series of procedures such as prediction, transformation, quantization, and entropy coding, and the encoded data ( The encoded video/video information) may be output in the form of a bitstream. This can be encoded by dividing into geometry and attributes as described later, and point cloud related metadata can be included in the bit stream. In this case, the output bitstream may include a geometry bitstream and/or an attribute bitstream. The point cloud encoding unit may receive metadata. Metadata represents metadata related to content for Point Cloud. For example, there may be initial viewing orientation metadata. The metadata indicates whether the point cloud data is data representing the front or the data representing the back. The point cloud encoding unit may receive orientation information and/or viewport information. Point Cloud Encoding Unit Encoding may be performed based on metadata, orientation information, and/or viewport information.

Specifically, the Point Cloud Encoding unit performs geometry compression, attribute compression, Auxiliary data compression, and Mesh data compression.

Geometry compression encodes point cloud geometry information. Geometry represents a point in three-dimensional space.

Attribute compression encodes the attributes of a point cloud. An attribute represents one or more attributes. For example, there may be N attributes including attributes such as color and reflectance.

Auxiliary data compression encodes Auxiliary data associated with a point cloud. Auxiliary data represents metadata about Point Cloud.

Mesh data compression encodes mesh data. Mesh represents connection information between point clouds. For example, it may be triangular data.

The Point Cloud encoding unit encodes the geometry, attributes, auxiliary data, and mesh data of the point, which are information necessary to render the point. The Point Cloud encoding unit may encode geometry, attributes, auxiliary data, and mesh data and deliver them as one bitstream. Alternatively, the point cloud encoding unit may encode geometry, attributes, auxiliary data, and mesh data and transmit them as a plurality of bitstreams. Each operation of the point cloud encoding unit may be performed in parallel.

The file/segment encapsulation unit performs media track encapsulation and/or metadata track encapsulation. The file/segment encapsulation unit creates a track for delivering the encoded geometry, encoded attributes, encoded auxiliary data, and encoded mesh data in a file format. The bitstream including the encoded geometry, the bitstream including the encoded attribute, the bitstream including the encoded auxiliary data, and the bitstream including the encoded mesh data may be included in one or more tracks. . The file/segment encapsulation unit encapsulates geometry, attributes, auxiliary data, and mesh data into one or more media tracks. In addition, the file/segment encapsulation unit includes metadata in a media track or encapsulates the metadata in a separate metadata track. The file/segment encapsulation unit encapsulates the point cloud stream(s) in the form of files and/or segments. When the point cloud stream(s) is encapsulated and delivered in the form of segment(s), it is delivered in the DASH format. The file/segment encapsulation unit delivers the file when encapsulating the point cloud stream(s) in the form of a file.

The delivery unit may deliver a point cloud bitstream or a file/segment including the corresponding bitstream to a receiving unit of a receiving device through a digital storage medium or a network. For transmission, processing according to any transmission protocol can be performed. Data processed for transmission may be delivered through a broadcasting network and/or a broadband. These data may be delivered to the receiving side in an on-demand manner. Digital storage media may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The delivery unit may include an element for generating a media file through a predetermined file format, and may include an element for transmission through a broadcast/communication network. The delivery unit receives orientation information and/or viewport information from the reception unit. The delivery unit may transmit the obtained orientation information and/or viewport information (or information selected by the user) to the file/segment encapsulation unit and/or the point cloud encoding unit. Based on the orientation information and/or the viewport information, the point cloud encoding unit may encode all point cloud data or the point cloud data indicated by the orientation information and/or the viewport information. Based on the orientation information and/or the viewport information, the file/segment encapsulation unit may encapsulate all point cloud data or the point cloud data indicated by the orientation information and/or the viewport information. Based on the orientation information and/or the viewport information, the delivery unit may deliver all point cloud data or the point cloud data indicated by the orientation information and/or the viewport information.

16 shows a device for receiving point cloud data according to embodiments.

The drawing shows a device for receiving point cloud data.

The Point Cloud data receiving device according to the embodiments includes a delivery client, a sensing/tracking unit, a file/segment decapsulation unit, and a point cloud decoding unit. ) And/or a Point Cloud rendering unit (Point Cloud Rendering), and a display. Each configuration of the receiving device may be a module/unit/component/hardware/software/processor.

The delivery client may receive point cloud data, a point cloud bitstream, or a file/segment including the corresponding bitstream, transmitted by the point cloud data transmission device according to the embodiments. Depending on the transmitted channel, the receiver may receive point cloud data through a broadcasting network or may receive point cloud data through a broadband. Alternatively, point cloud video data can be received through a digital storage medium. The receiver may include a process of decoding the received data and rendering it according to a user's viewport. The reception processing unit may perform processing according to a transmission protocol on the received point cloud data. The receiving processing unit may be included in the receiving unit, or may be configured as a separate component/module. The reception processing unit may perform the reverse process of the transmission processing unit described above so as to correspond to the transmission processing performed by the transmission side. The receiving processing unit may transmit the acquired point cloud data to the decapsulation processing unit, and the acquired point cloud related metadata may be transmitted to the metadata parser.

The sensing/tracking unit acquires orientation information and/or viewport information. The sensing/tracking unit may transmit the obtained orientation information and/or viewport information to a delivery client, a file/segment decapsulation unit, and a point cloud decoding unit.

The delivery client may receive all point cloud data or point cloud data indicated by the orientation information and/or the viewport information based on the orientation information and/or the viewport information. The file/segment decapsulation unit may decapsulate all point cloud data or decapsulate point cloud data indicated by orientation information and/or viewport information based on orientation information and/or viewport information. The point cloud decoding unit may decode all point cloud data or decode point cloud data indicated by the orientation information and/or the viewport information, based on the orientation information and/or the viewport information.

The file/segment decapsulation unit performs media track decapsulation and/or metadata track decapsulation. The decapsulation processing unit (file/segment decapsulation) may decapsulate the point cloud data in the form of a file transmitted from the reception processing unit. The decapsulation processor may decapsulate files or segments according to ISOBMFF or the like to obtain a point cloud bitstream or point cloud related metadata (or a separate metadata bitstream). The acquired point cloud bitstream may be transmitted to the point cloud decoder, and the acquired point cloud related metadata (or metadata bitstream) may be transmitted to the metadata processing unit. The point cloud bitstream may include the metadata (metadata bitstream). The metadata processing unit may be included in the point cloud video decoder, or may be configured as a separate component/module. The point cloud related metadata acquired by the decapsulation processing unit may be in the form of a box or track in a file format. If necessary, the decapsulation processing unit may receive metadata required for decapsulation from the metadata processing unit. The point cloud related metadata may be transmitted to the point cloud decoder and used for a point cloud decoding procedure, or may be transmitted to a renderer and used for a point cloud rendering procedure.

The Point Cloud Decoding unit performs geometry decompression, attribute decompression, Auxiliary data decompression, and/or mesh data decompression. . The Point Cloud decoder may decode data by receiving the bitstream and performing an operation corresponding to the operation of the Point Cloud encoder. In this case, the Point Cloud decoder can decode the Point Cloud data by dividing it into geometry and attributes, as described later. For example, the Point Cloud decoder can restore (decode) geometry from the geometry bitstream included in the input bitstream, and restore the attribute value based on the attribute bitstream included in the input bitstream and the restored geometry. You can (decode) it. A mesh may be reconstructed (decoded) based on the mesh bitstream included in the input bitstream and the restored geometry. Position information according to the restored geometry and a (color) texture attribute according to the decoded attribute value Based on this, the point cloud can be restored by restoring the location of each point in 3D and the attribute information of each point. Each operation of the point cloud decoding unit may be performed in parallel.

Geometry decompression decodes geometry data from the point cloud stream(s). Attribute decompression decodes attribute data from the point cloud stream(s). Auxiliary data decompression decodes the auxiliary data from the point cloud stream(s). Mesh data decompression decodes the mesh data from the point cloud stream(s).

Point Cloud Rendering restores the position of each point in the point cloud and the attributes of the point based on the decoded geometry, attributes, auxiliary data, and mesh data, and renders the point cloud data. . The point cloud rendering unit generates and renders mesh (connection) data between point clouds based on the restored geometry, the restored attributes, the restored auxiliary data, and/or the restored mesh data. The point cloud rendering unit receives metadata from the file/segment encapsulation unit and/or the point cloud decoding unit. The point cloud rendering unit may render point cloud data based on metadata according to an orientation or viewport.

The display displays the rendered result on an actual display device.

Referring to FIG. 17, the structure according to the embodiments is a server 1760, a robot 1710, an autonomous vehicle 1720, an XR device 1730, a smartphone 1740, a home appliance 1750 and/or an HMD. At least one or more of 1770 is connected to the cloud network 1710. Here, a robot 1710, an autonomous vehicle 1720, an XR device 1730, a smartphone 1740, or a home appliance 1750 may be referred to as a device. In addition, the XR device 1730 may correspond to a point cloud data (PCC) device according to embodiments or may be interlocked with a PCC device.

The cloud network 1700 may constitute a part of a cloud computing infrastructure or may mean a network that exists in the cloud computing infrastructure. Here, the cloud network 1700 may be configured using a 3G network, a 4G or long term evolution (LTE) network, or a 5G network.

The server 1760 includes at least one of a robot 1710, an autonomous vehicle 1720, an XR device 1730, a smartphone 1740, a home appliance 1750, and/or an HMD 1770, and a cloud network 1700. The connected devices 1710 to 1770 may be connected through, and may help at least part of the processing of the connected devices.

The HMD (Head-Mount Display) 1770 represents one of types in which an XR device and/or a PCC device according to embodiments may be implemented. The HMD type device according to the embodiments includes a communication unit, a control unit, a memory unit, an I/O unit, a sensor unit, and a power supply unit.

Hereinafter, various embodiments of the devices 1710 to 1750 to which the above-described technology is applied will be described. Here, the devices 1710 to 1750 shown in FIG. 17 may be interlocked/coupled with the point cloud data transmission/reception apparatus according to the above-described embodiments.

<PCC+XR>

The XR/PCC device 1730 is applied with PCC and/or XR (AR+VR) technology to provide a head-mount display (HMD), a head-up display (HUD) provided in a vehicle, a television, a mobile phone, a smart phone, It may be implemented as a computer, wearable device, home appliance, digital signage, vehicle, fixed robot or mobile robot.

The XR/PCC device 1730 analyzes 3D point cloud data or image data acquired through various sensors or from an external device to generate positional data and attribute data for 3D points, thereby Information can be obtained, and the XR object to be output can be rendered and output. For example, the XR/PCC device 1730 may output an XR object including additional information on the recognized object in correspondence with the recognized object.

<PCC+Autonomous Driving+XR>

The autonomous vehicle 1720 may be implemented as a mobile robot, a vehicle, or an unmanned aerial vehicle by applying PCC technology and XR technology.

The autonomous driving vehicle 1720 to which the XR/PCC technology is applied may refer to an autonomous driving vehicle having a means for providing an XR image, an autonomous driving vehicle that is an object of control/interaction within the XR image. In particular, the autonomous vehicle 1720, which is the object of control/interaction in the XR image, is distinguished from the XR device 1730 and may be interlocked with each other.

The autonomous vehicle 1720 having a means for providing an XR/PCC image may acquire sensor information from sensors including a camera, and may output an XR/PCC image generated based on the acquired sensor information. For example, the autonomous vehicle 1720 may provide an XR/PCC object corresponding to a real object or an object in a screen to the occupant by outputting an XR/PCC image with a HUD.

In this case, when the XR/PCC object is output to the HUD, at least a part of the XR/PCC object may be output to overlap the actual object facing the occupant's gaze. On the other hand, when the XR/PCC object is output on a display provided inside the autonomous vehicle, at least a part of the XR/PCC object may be output to overlap the object in the screen. For example, the autonomous vehicle 1220 may output XR/PCC objects corresponding to objects such as lanes, other vehicles, traffic lights, traffic signs, motorcycles, pedestrians, and buildings.

VR (Virtual Reality) technology, AR (Augmented Reality) technology, MR (Mixed Reality) technology and/or PCC (Point Cloud Compression) technology according to the embodiments can be applied to various devices.

That is, VR technology is a display technology that provides objects or backgrounds in the real world only as CG images. On the other hand, AR technology refers to a technology that shows a virtually created CG image on a real object image. Furthermore, MR technology is similar to the AR technology described above in that virtual objects are mixed and combined in the real world. However, in AR technology, the distinction between real objects and virtual objects made from CG images is clear, and virtual objects are used in a form that complements the real objects, whereas in MR technology, the virtual objects are regarded as having the same characteristics as the real objects. It is distinct from technology. More specifically, for example, it is a hologram service to which the aforementioned MR technology is applied.

However, recently, VR, AR, and MR technologies are sometimes referred to as XR (extended reality) technology rather than clearly distinguishing between them. Therefore, embodiments of the present invention are applicable to all of VR, AR, MR, and XR technologies. One such technology can be applied to encoding/decoding based on PCC, V-PCC, and G-PCC technologies.

The PCC method/apparatus according to the embodiments may be applied to a vehicle providing an autonomous driving service.

Vehicles providing autonomous driving service are connected to PCC devices to enable wired/wireless communication.

When the point cloud data (PCC) transmission and reception device according to the embodiments is connected to enable wired/wireless communication with the vehicle, the vehicle receives/processes AR/VR/PCC service related content data that can be provided together with the autonomous driving service. Can be transferred to. In addition, when the point cloud data transmission/reception device is mounted on a vehicle, the point cloud transmission/reception device may receive/process AR/VR/PCC service related content data according to a user input signal input through the user interface device and provide it to the user. The vehicle or user interface device according to the embodiments may receive a user input signal. The user input signal according to the embodiments may include a signal indicating an autonomous driving service.

18 illustrates an example of rendering point cloud data according to embodiments.

Point cloud data according to embodiments may be expressed and/or rendered based on a Level of Details (LOD). LOD refers to the level of detail. As the LOD value increases, the distance between points gets closer.

A method/apparatus according to embodiments refers to a method/device for transmitting point cloud data and/or a method/device for receiving point cloud data according to the embodiments.

The method/apparatus according to the embodiments provides a method for increasing the compression efficiency of attributes of Geometry-based Point Cloud Compression (G-PCC) for compressing 3D point cloud data.

According to the embodiments. Point cloud data transmission method / apparatus (or transmission method / apparatus) may be referred to as an encoder, encoder, encoding, etc., and point cloud data reception method / apparatus (or reception method / apparatus) is a decoder, decoder, decoding, etc. May be referred to.

The method/apparatus according to the embodiments proposes a method of generating a neighboring point set based on a similar attribute in order to increase compression efficiency by changing a method of configuring a neighboring point set during the G-PCC attribute encoding/decoding process.

The method/apparatus according to the embodiments proposes a method for generating a neighboring point set based on a similar attribute, a signaling method for supporting generation of a neighboring point set based on a similar attribute, and/or a signaling method for supporting such a method.

A point cloud (or point cloud data or point cloud content, etc.) according to embodiments may be composed of a set of points.

Each point according to the embodiments may include geometry information and attribute information. Geometry information according to embodiments is 3D location (XYZ) information, and attribute information according to embodiments is color (RGB, YUV, etc.) or/and/and reflection (Reflectance) value. The G-PCC encoding process according to the embodiments is composed of a process of compressing geometry and compressing attribute information based on geometry reconstructed with location information changed through compression (reconstructed geometry = decoded geometry). I can. The G-PCC decoding process according to embodiments is a process of receiving an encoded geometry bitstream and an attribute bitstream, decoding the geometry, and decoding attribute information based on the geometry reconstructed through the decoding process. Can be configured.

The attribute information compression process according to embodiments may use a predictive transform technique, a lifting transform technique, or a RAHT technique.

The predictive transformation method and the lifting transformation method may divide and group points by level of detail (hereinafter referred to as LOD).

This is referred to as ①LOD generation process, and hereinafter, groups having different LODs may be referred to as LODl sets.

l represents the LOD and is an integer starting from 0. LOD0 is a set consisting of points with the largest distance between points, and as l increases, the distance between points belonging to LODl decreases.

The method/apparatus according to the embodiments may generate a set of points having the same LOD based on the LOD of the point cloud data. LOD0 includes P0, P5, P4, P2, LOD1 includes LOD0 plus P1, P6, P3, and LOD2 includes LOD0, LOD1 plus P9, P8, P7.

After generating the LODl set, the method/apparatus according to the embodiments selects X (>0) nearest neighbors in a group having the same or smaller LOD (the distance between nodes is large) based on the ②LODl set. It can be found and registered as a set of neighboring points in a predictor. X is the maximum number that can be set as a neighboring point and can be input as a user parameter.

As shown in the figure, a neighboring point of P3 belonging to LOD1 is found in LOD0 and LOD1. The three nearest neighbor nodes can be P2 P4 P6. These three nodes are registered as a set of neighboring points to the predictor of P3.

Every point can have one predictor. The property is predicted from neighboring points registered in the predictor. The predictor may have a set of neighboring points and register it as ½ distance (or 1/distance) = weight based on a distance value from each neighboring point.

When the set of neighboring points of the predictor is set, the weights of each neighboring point can be normalized with the total sum of weights of the neighboring points.

④ The property can be predicted through the predictor. An average of a value obtained by multiplying the properties of registered neighboring points by a weight may be used as a predicted result, or a specific point may be used as a predicted result. Which method to use, after pre-calculating the compressed result value, you can select the method that can generate the smallest stream.

⑤ The attribute value of the point and the residual of the attribute value predicted by the predictor of the point can be signaled to the receiver by encoding together with the method selected by the predictor.

In the decoder, the same process is performed from steps ① to ③, and in ④, the transmitted prediction method is decoded and attribute values can be predicted according to the method. By decoding the residual value transmitted from ⑤ and adding the predicted value through ④, the attribute value can be restored.

The embodiments can be applied to both a transmitter and a receiver by the method for step ② described above, that is, a method for configuring a set of neighboring points. Since the configuration of the neighboring point set predicts the attribute value based on the neighboring points and signals the residual with the predicted value, the predicted value differs depending on the criteria used to configure the neighboring point set, and the size of the residual value may vary. . Therefore, the method of configuring a set of neighboring points can have a great influence on the attribute compression efficiency of the point cloud.

Since the point cloud's geometry-based nearby relationship has a high probability of having similar properties, it is possible to construct a neighboring set based on the distance value when predicted by the predictor, but such a tendency is based on the characteristics of the point cloud content. Can appear a lot differently.

A point cloud captured by a 3D scanner has a relatively small distance between neighboring points and can be captured in a dense form. In this case, there may be a high probability of having similar properties according to distance. But not all. Depending on the characteristics of the captured object, the probability of having similar properties according to distance may vary.

In the case of a point cloud captured through LiDAR, since the distance between neighboring points can be quite large, the actual distance difference may be large even if it is determined that the geometric-based proximity between points within the corresponding content is high. In this case, the meaning of configuring a set of neighboring points based on a distance and predicting an attribute value through the configured points may not be significant.

That is, the probability of having similar properties according to geometry-based adjacency may or may not be high according to the point cloud content characteristics. If a distance-based neighbor point set is configured for content where there is little relationship between geometry-based adjacency and similar attributes, the residual value with the predicted attribute value can be large, and the large residual value is encoded and transmitted as a bitstream. The size of the stream may increase.

Embodiments intend to propose a method of configuring a set of neighboring points that can increase attribute compression efficiency regardless of content characteristics. For example, we propose a method for generating a neighboring point set based on a similar attribute, a signaling scheme for supporting generation of a neighboring point set based on a similar attribute, and/or a signaling scheme for supporting such a method.

Changes and combinations between the embodiments are possible. Terms used in this document may be understood based on the intended meaning of the terms within the range widely used in the relevant field. The generation of the neighboring point set is all performed in the PCC attribute encoding/decoding of the PCC encoder/decoder. When a predictive transformation technique or a lifting transformation technique is used in the attribute encoding/decoding process, a set of LODl can be generated and a set of neighboring points of the predictor can be generated based on the generated LODl set.

The method/device according to the embodiments includes: 1) Molton order-based LOD generation, 2) Similar attribute-based neighbor point set generation method, 2-1) Attribute similarity measurement method, 2-2) Neighbor point search range setting method, 3) A neighboring point selection method, 4) an attribute information predictor and/or 5) a signaling method may be performed. Each process according to the embodiments will be described in detail below.

1) Molton order based LOD generation

Points in the point cloud may be changed to Molton codes based on x, y, and z position values, and may be sorted based on the changed Molton code values. LODs are generated based on the sorted points, and points in the LODl set may be sorted based on Molton order.

2) A method of generating a set of neighboring points based on similar properties

Points belonging to the LOD _l set can be selected as neighboring points from the following two cases to generate a neighboring point set.

1) Points in the LOD ₀ ~ LOD _ㅣ-1 set

2) Among the points in the LOD _l set, the preceding points in the sorted order

It may be necessary to select the neighboring point by what criteria.

Embodiments make it possible to select neighboring point candidates based on similarity properties to generate a neighboring point set. Whether to generate a neighboring point set based on a distance or a similar property may be signaled to the decoder according to a method applied to the encoder.

The transmitter or receiver according to the embodiments of the present document provides an effect of increasing compression efficiency through a method of generating a neighbor point set based on a neighbor attribute.

2-1) Attribute similarity measurement method

To measure attribute similarity, the distance metric CIE94 defined in Euclidean Color Distance, Correlated Color Temperature, or CIE (Commission on Illumination) can be selectively used.

The method of measuring attribute similarity used in the encoder can be signaled to the decoder.

You can change RGB values to CIE(XYZ) values, normalize them to chromatic values, and calculate CCT values.

<CIE94>

It can be calculated using the CIE1994 color difference model.

20 shows an example of a Molton code search range according to embodiments.

The method/apparatus according to the embodiments configures an LOD, which is a set of points of point cloud data. LODs are arranged in Molton code order.

For example, the method/apparatus according to the embodiments may find a point having a Molton code closest to a point included in the I-th LOD among points included in the 0-th LOD to the I-1 th LOD.

If the point with the closest Molton code is Pi, the search range, e.g., points with the highest similarity to Px among 256 neighboring points, e.g., 3 are composed of a set of neighboring points of Px )can do.

2-2) Neighbor Point Search Range Setting Method

When neighboring points belonging to the set of points Px LOD _l generating set, LOD ₀ ~ _l LOD the point and point which is first in the sequence of points that belong to the set LOD _l _-1 that belong to the set (or Molton code Molton code Px It is possible to search for a set of neighboring points in points that are less than or equal to).

Both the LOD _l set and the LOD ₀ ~ LOD _l-1 set are arranged in Molton order. Among points in the LOD ₀ ~ LOD _ㅣ-1 set and points in the LOD _l set, the nearest point with the Molton code can be found from the points in the preceding order. A set of neighboring points may be generated by comparing the points with points corresponding to the number of neighboring point search ranges in front and behind the point as the center.

When the number of LODs is 1, it is possible to compare only as many as the number of the previous neighboring point search ranges.

Since neighboring points are searched for a specific range of points arranged based on the Morton code, the distance between points is not completely considered when generating a neighboring point set. Basically, it can be said that the neighboring point search range is set by reflecting the distance.

The search range of a neighboring point according to embodiments may be adjusted according to the LOD. Other search ranges according to the LOD may be signaled to the decoder. Embodiments may set a change rate of a search range that is changed according to the LOD. Embodiments may signal the rate of change of the search range to the decoder. In this regard, it will be described in detail below.

Due to the above-described embodiments, the method/apparatus according to the embodiments can efficiently search for a neighboring point.

21 shows an example of a search range level according to embodiments.

The method/apparatus according to the embodiments may perform the Molton code-based search range setting method as described above, or the octree-based search range setting method as follows, in relation to the neighbor point search range setting method.

The method/apparatus according to the embodiments may set a search range according to the search range level. For example, if the search range level is 1 based on the octree parent node, a neighboring point search range of up to 8 points may be set based on the octree. If the search range level is 3, a neighboring point search range of up to 8x8x8 points can be set.

<Octree-based search range setting>

Since all LOD ₀ to LOD _l-1 sets according to the embodiments are arranged in a Molton code, the method/apparatus according to the embodiments compares the Molton codes to determine whether points are in the same parent node when the points are configured as octrees. You can check it.

In embodiments, the neighboring point search range may not be determined as a specific point-based +-range, but may be determined as the range of the upper parent node of the octree to which Px belongs based on the Molton code value of the point Px belonging to the LOD _l set. When searching for a set of neighboring points in this manner, embodiments may set how many higher parent nodes of the octree to which the Px point belongs to the node of the search range level, and the value may be signaled to the decoder. Embodiments may adjust the search range level according to the LOD. You can directly set each LOD, or you can set the ratio of the search range level.

According to embodiments, when searching for a neighboring point set in an octree-based neighboring range, the number of comparisons for generating a neighboring point set may vary according to the point distribution of the content. Compared to the Molton code-based search range setting method, the amount of calculation can be increased, but the amount of calculation is not increased (a calculation that checks whether a simple Molton code is within the range), and if the parent node is the same, it is more likely to be a neighbor node. The accuracy of the range may be higher compared to the Molton code-based search range setting method.

3) How to select a neighboring point

Embodiments may select a neighboring point by comparing distances or attribute values of points in a predetermined neighboring point search range to construct a neighboring point set.

<Distance-based neighbor point selection>

Embodiments may generate a neighboring point set by calculating a distance between points in a neighboring point search range and Px, and selecting points having the closest distance as many as the maximum number of neighboring point sets.

The maximum number of neighboring point sets applied to the encoder may be signaled to the decoder.

Embodiments may generate a neighboring point set by measuring the similarity attribute between points in the neighboring point search range and Px by a method selected from the similarity attribute measurement method, and selecting points with the highest similarity as many as the maximum number of neighboring point sets. .

In addition, embodiments may define a minimum attribute similarity threshold that can be registered in a neighboring point set, and according to the value, if it is smaller than the minimum attribute similarity threshold (if similarity is low), it cannot be registered as a neighboring point. .

The minimum attribute similarity threshold applied to the encoder may be signaled to the decoder.

In embodiments, when there is no neighboring point that can be registered when applying a threshold, direct coding may be performed instead of a residual value through prediction.

Embodiments may select N neighboring points based on distance, and among them, as many as the maximum number of neighboring point sets may be selected as neighboring points. The number of neighboring points that are primarily selected based on distance may be signaled to the decoder.

The point cloud data transmission method according to embodiments may generate a neighboring point set by searching for a neighboring point set based on a parent node of an octree for an LOD.

The point cloud data transmission method according to embodiments may generate a neighboring point set based on similar properties of points included in the neighboring point set.

Due to the above-described embodiments, the method/apparatus according to the embodiments may efficiently set a neighbor point search range in consideration of a distance and/or attribute.

The transmitting device (encoder or encoder) and the receiving device (decoder or decoder) according to the embodiments include respective components according to the embodiments. According to embodiments, each of the following components may correspond to hardware, software, a processor, or a combination thereof.

The space divider 22000 (or space divider) receives PCC data (point cloud data) and divides the space of the PCC data.

The geometric information encoding unit 22001 (or geometric information encoder) encodes the geometric information of PCC data to generate a geometric information bitstream and/or reconstructed (restored) geometric information.

The attribute information encoding unit 22002 (or attribute information encoder) generates an attribute information bitstream by encoding attribute information of PCC data based on the reconstructed geometric information.

The spatial divider 22003 (or spatial divider) receives the geometric information bitstream and divides the space of the geometric information.

The geometric information decoding unit 22004 (or a geometric information decoder) decodes the geometric information to generate restored PCC data and/or restored geometric information.

The attribute information decoding unit 22005 (or attribute information decoder) decodes attribute information of the attribute information bitstream based on the restored geometric information.

The attribute information prediction unit according to embodiments will be described.

The figure is an overall block diagram of a PCC (Point Cloud Compression) data encoder and a decoder.

Input data of an encoder (encoder) according to embodiments may be PCC data. The encoder according to embodiments may generate a geometric information bitstream and/or an attribute information bitstream by encoding PCC data (or point cloud data).

The input data of the decoder (decoder) according to the embodiments may be an encoded geometric information bitstream and/or an attribute information bitstream. The decoder according to embodiments may generate reconstructed PCC data.

Each detailed operation of the embodiments will be described below.

The attribute information prediction unit according to embodiments may be included or connected to the attribute information encoding unit of the encoder and the attribute information decoding unit of the decoder.

Encoding of the point cloud data transmission method according to embodiments may include encoding attribute information of the point cloud data, and encoding attribute information may include predicting attribute information.

The attribute information prediction unit of the encoder according to the embodiments includes each component according to the embodiments. According to embodiments, each of the following components may correspond to hardware, software, a processor, or a combination thereof.

The attribute information conversion unit 23000 (or attribute information converter) receives attribute information of point cloud data and converts the attribute information.

The geometric information mapping unit 23001 (or the geometric information mapper) maps geometric information to attribute information based on the restored geometric information.

The residual attribute information conversion unit 23002 (or residual attribute information converter) converts the attribute information and/or the residual attribute information between the predicted attribute information.

The residual attribute information quantization unit 23003 (or residual attribute information quantizer) quantizes the residual attribute information.

The attribute information entropy encoder 23004 (or attribute information entropy encoder) entropy-encodes the attribute information to generate an attribute information bitstream.

The residual attribute information inverse quantization unit 23005 (or residual attribute information inverse quantizer) inverse quantizes the residual attribute information.

The residual attribute information inverse transform unit 23006 (or the residual attribute information inverse transformer) inversely transforms the residual attribute information.

The filtering unit 23007 adds attribute information and/or predicted attribute information, and filters the generated attribute information.

The memory 23008 stores the filtered attribute information.

The attribute information prediction unit 23009 predicts attribute information and generates predicted attribute information. A method of predicting attribute information will be described in detail below.

The drawing shows detailed operations of the attribute information encoding unit according to embodiments. For details according to the embodiments, refer to the PCC encoder according to the following embodiments. The attribute information prediction unit according to embodiments may be included or connected to the attribute information encoding unit.

The attribute information prediction unit of the decoder according to the embodiments includes each component according to the embodiments. According to embodiments, each of the following components may correspond to hardware, software, a processor, or a combination thereof.

The attribute information entropy decoder 24000 (or attribute information entropy decoder) receives the attribute information bitstream and entropy decodes the attribute information.

The geometric information mapping unit 24001 (or geometric information mapper) maps geometric information to attribute information based on the restored geometric information.

The residual attribute information inverse quantization unit 24002 (or residual attribute information inverse quantizer) inverse quantizes the residual attribute information.

The residual attribute information inverse transform unit 24003 (or the residual attribute information inverse transformer) inversely transforms the residual attribute information.

The attribute information predictor 24004 (or attribute information predictor) predicts attribute information based on attribute information stored in the memory.

The filtering unit 24005 (or filter) filters data obtained by adding the inversely transformed residual attribute information and/or predicted attribute information.

The memory 240006 stores the filtered attribute information.

The attribute information inverse transform unit 24007 (or attribute information inverse converter) generates attribute information by inversely transforming attribute information stored in the memory.

The drawing is a detailed block diagram of an attribute information decoding unit (decoder) according to embodiments. For details of each block, refer to the PCC decoder according to the embodiments. The attribute information prediction unit may be included or connected to the attribute information decoding unit.

According to embodiments, the attribute information prediction unit may be included in both an encoder (transmitting device) and/or a decoder (receiving device).

The attribute information prediction unit according to the embodiments includes each component according to the embodiments. According to embodiments, each of the following components may correspond to hardware, software, a processor, or a combination thereof.

The LOD configuration unit 25000 (or LOD configurator) generates (configuration) LODs for points of point cloud data and/or point cloud data based on attribute information and/or reconstructed location information (restored geometric information). )do.

The neighbor point set construction unit 25001 (or neighbor point set construction unit) constructs (generates) a neighbor point set based on the LOD set.

The predictive transform/inverse transform unit 25002 (or predictive transform/inverse transform) performs predictive transform/inverse transform coding (attribute coding) on the neighboring point set.

The lifting transform/inverse transform unit 25003 (or lifting transform/inverse transformer) performs lifting transform/inverse transform coding (attribute coding) of the neighboring point set.

The attribute information prediction unit according to embodiments may include an LOD construction unit, a neighbor point set construction unit, a prediction transformation/inverse transformation unit, and a lifting transformation/inverse transformation unit. The attribute information prediction unit corresponds to a prediction/lifting conversion processing unit according to embodiments, and for details, refer to prediction conversion according to embodiments and lifting conversion according to embodiments.

The method/apparatus according to the embodiments may provide a neighbor point set construction unit and a prediction transform/inverse transform unit of the attribute information prediction unit. Each operation according to the embodiments will be described as follows.

The neighbor point set configuration unit according to the embodiments has a neighbor point search range setting method (neighbour_search_range_type), a range to search for a neighbor point is set according to the selection method, and the applied method may be transmitted to the decoder. The neighbor point search range setting method may include a Molton code-based search range setting method and an octree-based search range setting method.

The neighbor point set configuration unit according to the embodiments has a neighbor point search range (neighbour_search_range), and a range to search for a neighbor point is set according to a method for setting the neighbor point search range. The search range may be set for each LOD (neighbour_search_range[idx]), or may be set to be changed at a specific rate for each LOD (neighbor_search_range_rate_per_lod).

The neighbor point set configuration unit according to the embodiments has a neighbor point selection method (neighbour_selection_type), selects a neighbor point according to the selection method, and the applied method may be transmitted to the decoder. The neighbor point selection method may include a distance-based method, an attribute-based method, and a distance + attribute-based method.

The neighbor point set configuration unit according to the embodiments has an attribute similarity measurement method (neighbour_attr_difference_method) when selecting a neighboring point in an attribute-based or distance+attribute-based method (), and measures attribute similarity between points according to the selected method. The point is selected and the applied method can be transmitted to the decoder. Methods for measuring attribute similarity may include Euclidean Color Distance, Correlated Color Temperature, and CIE94. This method is an example, and a method that performs the same or similar function may be used.

The neighboring point set configuration unit according to embodiments has a minimum attribute similarity threshold (= or maximum attribute difference threshold) (neighbour_attr_min_similarity_threshold) of the neighboring point when selecting a neighboring point in an attribute-based or distance + attribute-based method, and a minimum attribute Only when it is greater than the similarity threshold can be registered as a neighboring node, and the threshold can be transmitted to the decoder.

When selecting a neighbor point based on a distance + attribute method, the neighbor point set configuration unit according to embodiments has the number of neighbor points (neighbour_attr_1st_phase_num_of_points) that are primarily selected based on the distance, and may be transmitted to the decoder.

In addition, for details of the neighbor point set configuration unit, refer to the description of the encoding process according to the embodiments.

For details of the prediction transform/inverse transform unit and the lifting transform/inverse transform unit according to the embodiments, refer to the description of the encoding process according to the embodiments.

In the step of predicting attribute information of the point cloud data transmission method according to embodiments, a level of detail (LOD) is generated based on the attribute information of the point cloud data and the reconstructed geometry information, and a set of neighboring points is determined based on the LOD. It can be generated, and attribute coding for a set of neighboring points can be performed.

Due to the components of FIGS. 22 to 25 described above, the method/apparatus according to the embodiments may improve encoding/decoding performance of point cloud data.

Point cloud data according to embodiments may have a bitstream form as shown in the drawing. The point cloud data may include a sequence parameter set (SPS), a geometry parameter set (GPS), an attribute parameter set (APS), and a tile parameter set (TPS) including signaling information according to embodiments. Point cloud data may include one or more geometry and/or attributes. The point cloud data may include geometry and/or attributes in units of one or more slices. The geometry may have a structure of a geometry slice header and geometry slice data. For example, the TPS including signaling information is Tile(0). It may include tile_bounding_box_xyz0, Tile(0)_tile_bounding_box_whd, and the like. The geometry may include geom_geom_parameter_set_id, geom_tile_id, geom_slice_id, geomBoxOrigin, geom_box_log2_scale, geom_max_node_size_log2, geom_num_points, and the like.

A method/apparatus according to the embodiments may signal related information to add/perform an operation of the embodiments. The signaling information according to embodiments of the present invention may be used at a transmitting end or a receiving end.

An example of a configuration of an encoded point cloud according to embodiments is shown in the drawing.

Each abbreviation means: Each abbreviation may be referred to by another term within the scope of its equivalent meaning. SPS: Sequence Parameter Set, GPS: Geometry Parameter Set, APS: Attribute Parameter Set, TPS: Tile Parameter Set, Geom: Geometry bitstream = geometry slice header+ geometry slice data, Attr: Attrobite bitstream = attribute blick header + attribute brick data.

The method/apparatus according to the embodiments may signal by adding option information related to generation of a neighboring point set and prediction/lifting transformation to the APS.

The method/apparatus according to the embodiments provides a tile or a slice so that the point cloud can be divided and processed by regions.

The method/apparatus according to the embodiments sets a different neighbor point set generation option for each area when dividing by area, so that the complexity is low, and the reliability of the result is slightly lower, or conversely, the selection method with high complexity but high reliability is selected. Can provide. The method/apparatus according to the embodiments may be set differently according to the processing capacity of the receiver.

Accordingly, when the point cloud is divided into tiles, the method/apparatus according to the embodiments may apply a different neighbor point set generation option for each tile.

When the point cloud is divided into slices, the method/apparatus according to the embodiments may apply a different neighbor point set generation option for each slice.

The method/apparatus according to the embodiments may signal by adding option information related to generation of a neighboring point set and prediction/lifting transformation to a TPS or an Attr for each slice.

The method/apparatus according to the embodiments may include information related to the neighbor point set generation option in the attribute parameter set. A description of each parameter (or signaling information) according to embodiments may be as follows.

aps_attr_paremeter_set_id represents the identifier of the APS attribute parameter set.

aps_seq_parameter_set_id represents the identifier of the ASP sequence parameter set.

attr_coding_type represents the type of attribute coding. Prediction (prediction) related parameters may be signaled according to the attribute coding type.

num_pred_nearest_neighbors represents the number of prediction-related near neighbors.

The method/apparatus according to the embodiments may signal by adding a neighboring point set and option information related to prediction/lifting transformation to the APS.

Neighbour_search_range_type according to embodiments: A method of setting a neighbor point search range may be specified. For example, 1= a Molton code-based search range setting method, and 2= an octree-based search range setting method.

Neighbour_selection_type according to embodiments: A neighboring point selection method may be specified. For example, 1= a distance-based neighbor point selection method, 2= an attribute-based neighbor point selection method, and 3= distance + attribute-based neighbor point selection method may be indicated.

Neighbour_search_range according to embodiments: The number of neighboring points to the left or right of the list, or the level difference of the upper parent node for setting the search range in the octree may be specified according to the method of setting the neighboring point search range.

Neighbour_search_range_rate_per_lod according to embodiments: As the LOD decreases, the% of the range that is changed may be specified.

Neighbour_attr_different_method according to embodiments: A method for measuring attribute similarity may be specified. For example, 1= Euclidian color distance, 2= Correlated Color Temperature, and 3= CIE94.

Neighbour_attr_min_similarity_threshold according to embodiments: A minimum attribute similarity threshold of a neighboring point may be specified.

Neighbour_attr_1st_phase_num_of_points according to embodiments: When selecting a neighboring point based on a distance + attribute, the number of neighboring points that are primarily selected based on a distance may be specified.

Aps_attr_parameter_set_id according to embodiments An identifier for an APS for reference by other syntax elements may be provided. The value of aps_attr_parameter_set_id may have a range of 0 to 15 (inclusive) (provides an identifier for the APS for reference by other syntax elements.The value of aps_attr_parameter_set_id shall be in the range of 0 to 15, inclusive).

Aps_seq_parameter_set_id according to embodiments represents a value of sps_seq_parameter_set_id for active SPS. The value of aps_seq_parameter_set_id may have a range of 0 to 15 (inclusive) (specifies the value of sps_seq_parameter_set_id for the active SPS.The value of aps_seq_parameter_set_id shall be in the range of 0 to 15, inclusive).

Attr_coding_type according to embodiments Represents a coding type for an attribute for a given value of attr_coding_type. The value of attr_coding_type may be 0, 1, or 2 in bitstreams. Other values of attr_coding_type may be reserved for future use by ISO/IEC. Decoders can ignore reserved values of attr_coding_type. For example, 0 = Predicting weight lifting, 1 = Region Adaptive Hierarchical Transferm (RAHT), 2 = Fixed weight lifting (indicates that the coding type for the attribute in Table 7 2Table 7 2 for the given value of attr_coding_type .The value of attr_coding_type shall be equal to 0, 1, or 2 in bitstreams conforming to this version of this Specification.Other values of attr_coding_type are reserved for future use by ISO/IEC.Decoders conforming to this version of this Specification shall ignore reserved values of attr_coding_type. 0 = Predicting weight lifting, 1 = Region Adaptive Hierarchical Transferm (RAHT), 2= Fixed weight lifting)

Indicates the maximum number of near neighbors used for num_pred_nearest_neighbors prediction according to embodiments. The value of numberOfNearestNeighboursInPrediction may be in the range of 1 to xx (specifies the maximum number of nearest neighbors to be used for prediction.The value of numberOfNearestNeighboursInPrediction shall be in the range of 1 to xx).

Max_num_direct_predictors according to embodiments represents the maximum number of a predictor used for direct prediction. The value of max_num_direct_predictors may have a range of 0 to num_pred_nearest_neighbours. The value of the variable MaxNumPredictors used in the decoding process may be as follows. For example, MaxNumPredictors = max_num_direct_predicots + 1 (specifies the maximum number of predictor to be used for direct prediction.The value of max_num_direct_predictors shall be range of 0 to num_pred_nearest_neighbours. The value of the variable MaxNumPredictors that is used in the decoding process as follows: MaxNumPredictors = max_num_direct_predicots + 1)

Lifting_search_range according to embodiments indicates a search range for lifting (specifies search range for the lifting).

Represents a quantization step size for a 1 ^st component of the lifting_quant_step_size attribute according to embodiments. The value of quant_step_size may be in the range of 1 to xx (specifies the quantization step size for the 1st component of the attribute.The value of quant_step_size shall be in the range of 1 to xx).

When the lifting_quant_step_size_chroma attribute according to embodiments is a color, it indicates the quantization step size for the chroma component of the attribute. The value of quant_step_size_chroma may have a range of 1 to xx (specifies the quantization step size for the chroma component of the attribute when the attribute is colour.The value of quant_step_size_chroma shall be in the range of 1 to xx).

Quant_step_size_chroma lod_binary_tree_enabled_flag according to embodiments specifies whether binary tree is enable or not for the log generation.

Represents the number of levels of details for num_detail_levels_minus1 attribute coding according to embodiments. The value of num_detail_levels_minus1 may have a range of 0 to xx (specifies the number of levels of detail for the attribute coding.The value of num_detail_levels_minus1 shall be in the range of 0 to xx).

Sampling_distance_squared [idx] represents a square of a sampling distance for idx according to embodiments. The value of sampling_distance_squared[] may have a range of 0 to xx (specifies the square of the sampling distance for idx. The value of sampling_distance_squared[] shall be in the range of 0 to xx).

Adaptive_prediction_threshold according to embodiments specifies the threshold of prediction.

Represents the number of levels of detail for raht_depth RAHT according to embodiments. The value of depthRAHT may have a range of 1 to xx (specifies the number of levels of detail for RAHT. The value of depthRAHT shall be in the range of 1 to xx).

Raht_binarylevel_threshold according to embodiments represents a level of detail for cutting out a RAHT coefficient. The value of binaryLevelThresholdRAHT may be in the range of 0 to xx (specifies the levels of detail to cut out the RAHT coefficient.The value of binaryLevelThresholdRAHT shall be in the range of 0 to xx).

Represents a quantization step size for a 1 ^st component of the raht_quant_step_size attribute according to embodiments. The value of quant_step_size may be in the range of 1 to xx (specifies the quantization step size for the 1st component of the attribute.The value of quant_step_size shall be in the range of 1to xx).

When aps_extension_present_flag according to embodiments is 1, it indicates that the aps_extension_data syntax structure exists in the APS RBSP syntax structure. When aps_extension_present_flag is 0, it indicates that this syntax structure does not exist. If not present, the value of aps_extension_present_flag can be assumed to be equal to 0 (equal to 1 specifies that the aps_extension_data syntax structure is present in the APS RBSP syntax structure.aps_extension_present_flag equal to 0 specifies that this syntax structure is not present. When not present, the value of aps_ extension_present_flag is inferred to be equal to 0).

It can have any value of aps_extension_data_flag according to embodiments. Its presence and value do not affect decoder conformance to profiles. Decoders conforming to a profile.

The method/device according to the embodiments may include information related to the neighbor point set generation option in the Tile parameter set.

The method/apparatus according to the embodiments may signal by adding the neighbor point set and option information related to prediction/lifting transformation to the TPS.

Represents the number of tiles signaled for the num_tiles bitstream according to embodiments. If not present, num_tiles may be 0 (specifies the number of tiles signaled for the bitstream.When not present, num_tiles is inferred to be 0).

Represents the x offset of the i-th tile in the tile_bounding_box_offset_x[i] coordinate system according to embodiments. If not present, the value of tile_bounding_box_offset_x[ 0] may be sps_bounding_box_offset_x (indicates the x offset of the i-th tile in the cartesian coordinates. When not present, the value of tile_bounding_box_offset_x[ 0] is inferred to be sps_bounding_box_offset_x).

Represents the y offset of the i-th tile in the tile_bounding_box_offset_y[i] coordinate system according to embodiments. If not present, the value of tile_bounding_box_offset_y[ 0] may be sps_bounding_box_offset_y (indicates indicates the y offset of the i-th tile in the cartesian coordinates. When not present, the value of tile_bounding_box_offset_y[ 0] is inferred to be sps_bounding_box_offset_y) .

Represents the z offset of the i-th tile in the tile_bounding_box_offset_z[i] coordinate system according to embodiments. If not present, tile_bounding_box_offset_z[ 0] may be sps_bounding_box_offset_z (indicates indicates the z offset of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_offset_z[ 0] is inferred to be sps_bounding_box_offset_z).

Represents a scale factor of the i-th tile in the tile_bounding_box_scale_factor[i] coordinate system according to embodiments. If not present, tile_bounding_box_scale_factor[ 0] may be sps_bounding_box_scale_factor (indicates the scale factor the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_scale_factor[ 0] is inferred to be sps_bounding_box_scale_factor).

Represents the width of the i-th tile in the tile_bounding_box_size_width[i] coordinate system according to embodiments. If not present, tile_bounding_box_size_width[ 0] may be sps_bounding_box_size_width (indicates the width of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_size_width[ 0] is inferred to be sps_bounding_box_size_width).

Represents the height of the i-th tile in the tile_bounding_box_size_height[i] coordinate system according to embodiments. If not present, tile_bounding_box_size_height[ 0] may be sps_bounding_box_size_height (indicates the height of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_size_height[ 0] is inferred to be sps_bounding_box_size_height).

Indicates the depth of the i-th tile in the tile_bounding_box_size_depth[i] coordinate system according to embodiments. If not present, tile_bounding_box_size_depth[ 0] may be sps_bounding_box_size_depth (indicates the depth of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_size_depth[ 0] is inferred to be sps_bounding_box_size_depth).

For the definition and/or effect of other signaling information, refer to the above.

The method/apparatus according to the embodiments may include information related to the neighbor point set generation option in the Attribute slice header.

Option information related to generation of a neighboring point set and prediction/lifting transformation according to embodiments may be added to the slice header of Attr for signaling.

Abh_attr_parameter_set_id according to embodiments indicates the value of aps_attr_parameter_set_id of the active APS (specifies the value of the aps_attr_parameter_set_id of the active APS).

Represents an attribute set in the abh_attr_sps_attr_idx active SPS according to embodiments. The value of abh_attr_sps_attr_idx may have a range of 0 to sps_num_attribute_sets in the active SPS (specifies the attribute set in the active SPS.The value of abh_attr_sps_attr_idx shall be in the range of 0 to sps_num_attribute_sets in the active SPS).

Indicates the value of abh_attr_geom_slice_id geom slice id according to embodiments (specifies the value of geom slice id).

Due to the signaling information (parameter) according to the above-described embodiments, the method/device according to the embodiments indicates information related to encoding/decoding of point cloud data, and based on the signaling information, the method/device according to the embodiments is The operation can be performed.

30 shows a PCC encoder according to embodiments.

The PCC encoder (or point cloud data transmission apparatus, encoder) according to the embodiments may include each component according to the embodiments. Each component may correspond to hardware, software, processor, and/or a combination thereof.

The space divider 30000 (or space divider) divides PCC data (point cloud data) into space.

The geometry information encoding unit 30001 (or geometry information encoder) encodes the geometry information to generate a geometry information bitstream and/or reconstructed geometry information.

The attribute information encoder 30002 (or attribute information encoder) encodes attribute information to generate an attribute information bitstream.

PCC encoding (point cloud data encoder, encoder, and/or transmission method/apparatus) according to embodiments may perform a process similar to the diagram of the diagram in the PCC encoder.

The PCC encoder according to the embodiments may be composed of a geometric information encoder and/and an attribute information encoder.

The geometry coding according to the embodiments corresponds to the geometric information encoder according to the embodiments, and the attribute coding according to the embodiments corresponds to the attribute information encoder according to the embodiments.

According to embodiments, both geometry and geometric information are referred to as geometric information below.

The PCC data according to embodiments may be composed of geometric information or/and attribute information of a point.

Attribute information is obtained from one or more sensors, such as a vector representing the color of the point (R, G, B) or/and the brightness value or/and the reflection coefficient of the lidar or/and the temperature value obtained from the thermal imaging camera. It can be a vector of one value.

The spatial division unit according to embodiments may divide the input PCC data into at least one 3D block. In this case, the block may mean a tile group, a tile, a slice, or a coding unit (CU), a prediction unit (PU), or a transformation unit (TU). The partitioning may be performed based on at least one of an octree, a quadtree, a binary tree, a triple tree, and a k-d tree. Alternatively, it can be divided into blocks of a predetermined horizontal and vertical height. Alternatively, it can be divided by selectively determining various positions and sizes of blocks. Corresponding information may be entropy-encoded and transmitted to a decoder.

The geometric information encoder according to embodiments generates an encoded geometric information bitstream and reconstructed geometric information from the received geometric information. The generated bitstream may be transmitted to the PCC decoder. In addition, the generated reconstructed geometric information may be input to the attribute information encoding unit.

The attribute information encoding unit according to embodiments receives the received attribute information and generates an attribute information bitstream. The generated attribute information bitstream may be transmitted to the PCC decoder.

The geometric information encoder according to the embodiments includes each component according to the embodiments. Each component may correspond to hardware, software, processor, and/or a combination thereof.

The coordinate system conversion unit 31000 (or coordinate system converter) receives geometric information and converts the coordinate system of the geometric information.

The geometric information conversion quantization unit 31001 (or the geometric information conversion quantizer) converts and quantizes the geometric information.

The residual geometric information quantization unit 3102 (or residual geometric information quantizer) quantizes the geometric information and/or the residual geometric information between the predicted geometric information.

The geometric information entropy encoder 31003 (or the geometric information entropy encoder) entropy-encodes the geometric information to generate a geometric information bitstream.

The residual geometric information inverse quantization unit 3104 (or residual geometric information inverse quantizer) inverse quantizes the residual geometric information.

The filtering unit 31005 (or filter) filters data obtained by summing residual geometric information and/or predicted geometric information.

The memory 31006 stores the filtered geometric information and generates reconstructed geometric information.

The geometric information predictor 31007 (or geometric information predictor) predicts geometric information based on the geometric information stored in the memory.

The PCC encoder (point cloud data encoder, encoder and/or transmission method/apparatus) according to the embodiments may include a geometric information encoder and a property information encoder. The geometric information encoder can generate a geometric information bitstream and reconstructed (reconstructed = reconstructed) geometric information by performing the same process as in the diagram of the drawing.

The geometric information encoding unit according to embodiments may include a coordinate system transforming unit, a geometric information transforming quantization unit, a residual geometric information quantizing unit, a geometric information entropy encoding unit, a residual geometric information inverse quantizing unit, a memory, and a geometric information predicting unit.

The coordinate conversion unit according to the embodiments corresponds to the coordinate system conversion unit of the geometric information encoder according to the embodiments, and a quantization processing unit, a voxelization processing unit, an octree code generation unit, and a surface model processing unit are combined to transform geometric information according to the embodiments. It corresponds to wealth. The intra/inter coding processing unit according to the embodiments corresponds to the geometric information prediction unit according to the embodiments, and the Arithmetic coder corresponds to the geometric information entropy coding unit according to the embodiments. Specific operations according to the embodiments will be described below.

The coordinate system conversion unit according to the embodiments may receive geometric information as an input and convert it into a coordinate system different from the existing coordinate system. Alternatively, the coordinate system transformation may not be performed. The geometric information converted by the coordinate system may be input to the geometric information conversion quantization unit.

Whether the coordinate system is transformed and the coordinate system information according to embodiments may be signaled in units such as a sequence, frame, tile, slice, or block, or whether the coordinate system of neighboring blocks is transformed or not , It can be derived using the location of the unit, and the distance between the unit and the origin.

If the coordinate system information to be converted according to the embodiments is converted to the coordinate system after checking whether the coordinate system is converted, the coordinate system information may be signaled in units such as sequence, frame, tile, slice, block, etc. It can be derived using the size, number of points, quantization value, block division depth, unit location, and distance between the unit and the origin.

The geometric information transform quantization unit according to the embodiments receives geometric information as input, applies one or more transforms such as position transform or/and rotation transform, divides the geometric information by a quantization value, and quantizes the transformed quantized geometric information. . The transformed quantized geometric information may be input to a geometric information entropy encoding unit and a residual geometric information quantizing unit.

The geometric information prediction unit according to embodiments predicts geometric information through geometric information of points in a memory and generates predicted geometric information. The prediction information used for prediction may be encoded by performing entropy encoding.

The residual geometric information quantization unit according to embodiments receives residual geometric information obtained by differentiating the transformed-quantized geometric information and the predicted geometric information, and quantizes it into a quantized value to generate quantized residual geometric information. Quantized residual geometric information may be input to a geometric information entropy encoding unit and a residual geometric information inverse quantization unit.

The geometric information entropy encoding unit according to embodiments may receive quantized residual geometric information and perform entropy encoding. Entropy coding may use various coding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

The residual geometric information inverse quantization unit according to embodiments receives the quantized residual geometric information and restores the residual geometric information by scaling the quantized value. The restored residual geometric information may be restored as geometric information in addition to the predicted geometric information and stored in a memory.

The filtering unit according to embodiments may perform filtering on the reconstructed geometric information. The filtering unit may include a deblocking filter, an offset correction unit, and an ALF.

The memory according to embodiments may store geometric information calculated through a filtering unit. The stored geometric information may be provided to the geometric information prediction unit when performing prediction.

The detailed operation of the attribute information encoder according to the embodiments is as described above.

The attribute information encoder according to the embodiments may include each component according to the embodiments. Each component may correspond to hardware, software, processor, and/or a combination thereof.

The attribute characteristic conversion unit 32000 (or attribute characteristic converter) receives attribute information and converts the attribute of the attribute information.

The geometric information mapping unit 32001 (or geometric information mapper) maps geometric information to attribute information based on the restored geometric information.

The attribute information conversion unit 32002 (or attribute information converter) converts attribute information.

The attribute information quantization unit 32003 (or attribute information quantizer) quantizes attribute information.

The attribute information entropy encoding unit 32004 (or attribute information entropy encoder) entropy-encodes attribute information to generate an attribute information bitstream.

The PCC encoder (point cloud data encoder, encoding and/or transmission method/apparatus) according to the embodiments may include a geometry information encoder and a property information encoder. The attribute information encoder according to the embodiments may generate the attribute information bitstream by performing a process similar to that of the diagram in the drawing.

The attribute information encoder according to the embodiments includes an attribute characteristic transformation unit, a geometric information mapping unit, a transformation unit, a quantization unit, an entropy encoding unit, an inverse quantization unit, an inverse transformation unit, a memory, an attribute information prediction unit, etc. Can include.

The color conversion processing unit according to the embodiments corresponds to the attribute information conversion unit of the attribute information encoder according to the embodiments, and the attribute conversion processing unit corresponds to the geometric information mapping unit according to the embodiments. The prediction/lifting/RAHT conversion processing unit according to the embodiments separates and expresses the attribute information prediction unit, the vehicle attribute information conversion unit, and the residual attribute information quantization unit according to the embodiments. The Arithmetic coder according to the embodiments corresponds to the attribute information entropy encoding unit according to the embodiments. Details will be described below.

The attribute characteristic conversion unit according to embodiments may convert a characteristic of the received attribute information. For example, if the attribute information includes color information, the attribute characteristic conversion unit may convert the color space of the attribute information. The converted attribute information may be input to the geometric information mapping unit. Alternatively, it may be input to the geometric information mapping unit without conversion.

The geometric information mapping unit according to embodiments maps the attribute information received from the attribute information conversion unit and the received restored geometric information to reconstruct attribute information. The attribute information reconstruction may derive an attribute value based on attribute information of one or a plurality of points based on the restored geometric information. The reconstructed attribute information may be input to the residual attribute information conversion unit by being differentiated from the predicted attribute information generated by the attribute information prediction unit.

The residual attribute information conversion unit according to embodiments may convert a residual 3D block including the received residual attribute information using a transformation type such as DCT, DST, DST, SADCT, RAHT, or the like. The converted residual attribute information may be input to the residual attribute information quantization unit. Alternatively, the residual attribute information may be input to the quantization unit without performing transformation. The transformation type may be transmitted to a decoder by performing entropy encoding in an entropy encoder.

The residual attribute information quantization unit according to embodiments generates transform quantized residual attribute information based on the quantized value of the received transformed residual attribute information. The transform quantized residual attribute information may be input to the attribute information entropy encoding unit and the residual attribute inverse quantization unit.

The attribute information entropy encoder according to embodiments may receive transform quantized residual attribute information and perform entropy encoding. Entropy coding may use various coding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

The residual attribute inverse quantization unit according to embodiments receives transformed quantized residual attribute information and generates transformed residual attribute information based on a quantization value. The generated transform residual attribute information may be input to a residual attribute inverse transform unit.

The inverse residual attribute transform unit according to the embodiments may inverse transform a residual 3D block including the received transform residual attribute information using a transform type such as DCT, DST, DST, SADCT, RAHT, or the like. The inversely transformed residual attribute information may be combined with predicted attribute information input from the attribute information predictor to generate restored attribute information. Alternatively, the reconstructed attribute information can be generated by directly adding the predicted attribute information without performing inverse transformation.

The filtering unit according to embodiments may include a deblocking filter, an offset correction unit, an adaptive loop filter (ALF), and the like. The filtering unit may perform filtering on the restored attribute information. Filtering is filtering on geometric information (XYZ) instead of attribute information (RGB, etc.). The filtering algorithm can be used as it is, only the input is different.

The memory according to the embodiments may store attribute information calculated through the filtering unit. The stored attribute information may be provided to the attribute information predictor when performing prediction.

The attribute information predictor according to embodiments generates predicted attribute information based on attribute information of points in a memory. The prediction information may be encoded by performing entropy encoding.

33 shows an example of a PCC decoder according to embodiments.

The PCC decoder according to the embodiments includes each component according to the embodiments. Each component may correspond to hardware, software, processor, and/or a combination thereof.

The geometry information decoding unit 33000 (or geometry information decoder) receives the geometry information bitstream, decodes the geometry information, and generates recovered PCC data and/or recovered geometry information.

The attribute information decoding unit 33001 (or attribute information decoder) receives the geometry information bitstream, decodes attribute information based on the restored geometry information, and generates recovered PCC data.

PCC decoding (point cloud data decoder, decoding, reception method/apparatus) according to embodiments may perform a process similar to the diagram of the diagram in the PCC decoder.

The PCC decoder according to embodiments may include a geometric information decoder and an attribute information decoder.

The spatial division unit according to embodiments may divide a space based on division information provided from an encoder or derived from a decoder.

The geometry information decoder according to embodiments restores geometry information by decoding an input geometry information bitstream. The restored geometric information may be input to the attribute information decoder.

The attribute information decoder according to embodiments receives the received attribute information bitstream and restored geometric information received from the geometry information decoder and restores attribute information. The restored attribute information may consist of restored PCC data together with the restored geometric information.

The geometric information decoder according to the embodiments includes each component according to the embodiments. Each component may correspond to hardware, software, processor, and/or a combination thereof.

A geometric information entropy decoder 34000 (or a geometric information entropy decoder) receives a geometric information bitstream and entropy decodes the geometric information.

The residual geometric information inverse quantization unit 34001 (or the residual geometric information inverse quantizer) inverse quantizes the residual geometric information.

The geometric information predictor 3402 (or geometric information predictor) predicts geometric information based on the geometric information stored in the memory.

The filtering unit 34403 (or filter) filters data obtained by summing the predicted geometric information and residual geometric information.

The memory 34004 stores geometric information.

The coordinate system inverse transform unit 3405 (or coordinate system inverse transform unit) generates geometric information by inversely transforming the coordinate system of the geometric information.

The PCC decoder according to embodiments may include a geometry information decoder and an attribute information decoder. The geometry information decoder can receive the encoded geometry information bitstream and perform the same process as in the diagram in the drawing to restore the geometry information.

The geometry information decoder according to embodiments may include a geometry information entropy decoding unit, a residual geometry information inverse quantization unit, a geometry information prediction unit, and an inverse coordinate system transform unit.

The Arithmetic decoder according to the embodiments corresponds to the geometric information decoder according to the embodiments, the geometric information entropy decoding unit, and the oct group reconstruction processing unit based on the occupancy code, the surface model processing unit, and the inverse quantization processing unit It corresponds to the inverse quantization part.

The geometric information entropy decoder according to embodiments may perform entropy decoding on an input bitstream. For example, for entropy decoding, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied. The geometric information entropy decoder may decode information related to geometric information prediction performed by the encoding apparatus. Quantized residual geometric information generated through entropy decoding may be input to the residual geometric information inverse quantization unit.

The residual geometric information inverse quantization unit according to embodiments may generate residual geometric information by performing inverse quantization based on a quantization parameter and the received quantized residual geometric information.

The geometric information prediction unit according to embodiments may generate predicted geometric information based on information related to generation of predicted geometric information provided from the geometric information entropy decoder and previously decoded geometric information provided from a memory. The geometric information prediction unit may include an inter prediction unit and an intra prediction unit. The inter prediction unit uses information required for inter prediction of the current prediction unit provided by the encoding device, and determines the current prediction unit based on information included in at least one of a space before or after the current space including the current prediction unit. Inter prediction can be performed. The intra prediction unit may generate predicted geometric information based on geometric information of a point in the current space. When the prediction unit performs intra prediction, intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the encoding device. The reconstructed geometric information may be generated by adding the reconstructed residual geometric information to the predicted geometric information.

The reconstructed geometric information according to the embodiments may be provided to the filtering unit. The filtering unit may perform filtering based on the filtering-related information provided from the decoder or the characteristics of the reconstructed geometric information derived from the decoder.

The memory according to embodiments may store the reconstructed geometric information calculated through the filtering unit.

The inverse coordinate system transform unit according to embodiments may perform inverse coordinate system transformation based on information related to coordinate system transformation provided from the geometric information entropy decoding unit and restored geometric information stored in a memory.

Each component according to embodiments of the attribute information decoder according to the embodiments is as follows. Each component may correspond to hardware, software, processor, and/or a combination thereof.

The attribute information entropy decoder 35000 (or attribute information entropy decoder) receives the attribute information bitstream and entropy decodes the attribute information.

The geometric information mapping unit 35001 (or geometric information mapper) maps geometric information to attribute information based on the restored geometric information.

The residual attribute information inverse quantization unit 35002 (or residual attribute information inverse quantizer) inverse quantizes the residual attribute information.

The residual attribute information inverse transform unit 35003 (or residual attribute information inverse transform unit) inversely transforms the residual attribute information.

The attribute information prediction unit 35004 (or attribute information Yetsugi) predicts attribute information based on the memory.

The memory 35005 stores attribute information, attribute information, and/or data obtained by adding up predicted attribute information.

The attribute information inverse transform unit 35006 (or attribute information inverse transform unit) generates attribute information by inversely transforming attribute information.

The PCC decoder (point cloud decoder, decoding, reception method/apparatus) according to the embodiments may include a geometry information decoder and an attribute information decoder. The attribute information decoder may receive the encoded attribute information bitstream and restore the attribute information by performing a process similar to the diagram of the drawing.

The attribute information decoder according to the embodiments may include an attribute information entropy decoding unit, a geometric information mapping unit, a residual attribute information inverse quantization unit, a residual attribute information inverse transformation unit, an attribute information prediction unit, a memory, and an attribute information inverse transformation unit.

The Arithmetic decoder according to the embodiments corresponds to the attribute information decoder attribute information entropy decoding unit according to the embodiments, and the inverse quantization processor corresponds to the residual attribute information inverse quantization unit according to the embodiments. The prediction/lifting/RAHT inverse transform processing unit according to the embodiments is divided into a residual attribute information inverse transform unit and an attribute information prediction unit, and the color inverse transform processing unit corresponds to the attribute information inverse transform unit according to the embodiments.

The attribute information entropy decoding unit according to the embodiments may entropy-decode the received attribute information bitstream to generate transformed quantized attribute information. The generated transformed quantized attribute information may be input to the geometric information mapping unit.

The geometric information mapping unit according to embodiments maps the transformed quantized attribute information received from the attribute information entropy decoding unit and the received restored geometric information. The attribute information mapped to the geometric information may be input to the residual attribute information inverse quantization unit.

The residual attribute information inverse quantization unit according to embodiments performs inverse quantization on the received transformed quantized attribute information based on a quantization value. The inverse quantized transform residual attribute information may be input to the residual attribute information inverse transform unit.

The residual attribute information inverse transform unit according to embodiments may inversely transform a residual 3D block including the received transform residual attribute information using a transform type such as DCT, DST, DST, SADCT, RAHT, and the like. The inversely transformed residual attribute information may be combined with predicted attribute information generated from the attribute information prediction unit and stored in a memory. Alternatively, it may be stored in a memory by adding prediction attribute information without performing inverse transformation.

The attribute information predictor according to embodiments generates predicted attribute information based on attribute information of points in a memory. The prediction information can be obtained by performing entropy decoding.

The attribute information inverse transform unit according to embodiments may receive a type of attribute information and transformation information from the entropy decoder and perform various color space inverse transformations such as RGB-YUV and RGB-YUV.

The configuration of the attribute information encoder and the attribute information decoder according to the embodiments are shown, and detailed operations are as described above.

The attribute information encoder and/or the attribute information decoder according to embodiments may include an attribute information predictor, and detailed operations of the attribute information predictor are as described above.

The neighboring point set generation unit according to the embodiments is included in the attribute information prediction unit, and may be included in both the transmitting end and the receiving end.

Geometry-based Point Cloud Compression (G-PCC) encoder (encoder)/decoder (decoder) (or transmission method/device and receiving method/device) for compressing 3D point cloud data according to embodiments ), a method of generating a set of neighboring points based on similar properties is provided as a way to increase the compression efficiency of the properties of ).

Accordingly, the embodiments provide a higher recovery rate by increasing the attribute compression efficiency of the encoder (encoder)/decoder (decoder) of Geometry-based Point Cloud Compression (G-PCC) for compressing 3D point cloud data. Point cloud content streams can be provided.

A PCC encoder (transmission method/apparatus) or a PCC decoder (receiving method/apparatus) according to the embodiments of the present document may generate a set of neighboring points and increase attribute compression efficiency by using signaling information for this.

The attribute information prediction unit 37000 and/or the neighbor information conversion unit 37001 according to the embodiments may be combined with a method/device according to the embodiments.

The embodiments relate to a method for increasing compression efficiency of attributes of Geometry-based Point Cloud Compression (G-PCC) for compressing 3D point cloud data.

Hereinafter, an encoder and an encoder are referred to as an encoder, and a decoder and a decoder are referred to as a decoder. The method/device according to the embodiments may refer to a method/device for transmitting point cloud data and a method/device for receiving data.

Embodiments propose a method of predicting an attribute and encoding/decoding when applying a neighboring point set generation method based on a similar attribute to increase attribute compression efficiency in a G-PCC attribute encoding/decoding process.

Examples include the following method. For example, a method for predicting an attribute from a set of similar attributes-based neighbor points, a method for signaling additional information values for neighbor points based on similar attributes applied to prediction, and a signaling method for supporting the above two methods may be included.

In the embodiments, when the configuration of a set of neighboring points is generated based on similar properties (for example, not based on distance) in step ② described above, steps ③ and ④ to be changed, for example, weights and predictors are also based on distance. It is a method of changing according to the attribute base).

Steps

③ and ④ can be applied to both the transmitter and the receiver as a normalization method according to the attribute weights and weights of neighboring points and an attribute prediction method through a predictor.

Since the configuration of the neighboring point set predicts attribute values based on neighboring points and signals the residuals with the predicted values, the predicted values may vary depending on the criteria used to configure the neighboring point set and the size of the residual values may vary. . Therefore, the method of configuring a set of neighboring points can have a great influence on the attribute compression efficiency of the point cloud. For example, since residuals are calculated based on similar properties, there may be an advantage that only smaller values can be encoded.

In the embodiments, in the case of generating a neighboring point set based on a similar attribute to improve the case in which the distance-based neighboring point set configuration cannot extract optimal neighboring points according to the characteristics of the content, it is changed and additionally considered. This is a proposal for a signaling scheme.

The embodiments include a proposal for the following method. For example, there may be an attribute prediction method from a similar attribute-based neighbor point set, an additional information value signaling method for a similar attribute-based neighbor point applied to prediction, and/or a signaling method for supporting the above two methods.

Changes and combinations between the embodiments are possible. Terms used in the description of the embodiments may be understood based on the intended meaning of the term within a range widely used in the relevant field.

The method/apparatus according to the embodiments predicts an attribute based on a set of neighboring points selected based on a similar attribute in PCC attribute encoding of a PCC encoder, calculates and encodes a residual based on the predicted information, and encodes additional information of the selected neighboring point. Can be encoded. The method/apparatus according to the embodiments generates a neighboring point set based on additional information of a neighboring point decoded in PCC attribute decoding of a PCC decoder, and calculates the decoded residual value and the attribute value of the previously decoded neighboring point set. In addition, the attribute value of the point can be restored. For example, generation of neighboring points according to embodiments may be applied to both encoders/decoders according to embodiments.

The method/apparatus according to the embodiments may generate an LOD _l set and generate a neighboring point set of the predictor based on the generated LOD _l set when a predictive transform technique or a lifting transform technique is used in an attribute encoding/decoding process.

The method/device according to the embodiments includes: 1) Molton order-based LOD generation, 2) Similar attribute-based neighbor point set generation, 3) Attribute prediction method from neighbor point set, 3-1) weight setting method, 4) neighbor point set Method, 5) additional information signaling method for neighboring points applied to prediction, 5-1) number of neighboring points, 5-2) additional information for neighboring points, 6) additional information encoding method for neighboring points, 7) attribute information prediction unit, 8) neighbor information change unit/neighbor information inverse transform unit, and 9) signaling scheme for this method may be included/performed. Each detailed operation according to the embodiments will be described below.

1) Molton order based LOD generation

The method/apparatus according to the embodiments may change points of a point cloud into a Molton code based on x, y, and z position values, and sort (Ahn ascending order) based on the changed Molton code value. LODs are generated based on the sorted points, and points in the LODl set may be sorted based on Molton order. For example, a method of determining a neighboring point based on similar attribute information rather than distance based may be used.

2) Create a set of neighboring points based on similar properties

In the method/apparatus according to the embodiments, the points Px belonging to the LOD _l set are 1) points belonging to the LOD ₀ to LOD _l-1 set, and 2) points in the ordered order among the points belonging to the LOD _l set. It is possible to select a neighboring point with similar attributes from, and set it as a neighboring point set of Px.

3) Property prediction method from a set of neighboring points

When encoding Px, the method/apparatus according to the embodiments predicts the attribute value of Px based on the attribute values of the neighboring points in the set, and the predicted attribute value and the Px attribute value when a set of neighboring points of Px is configured. The residual of is obtained, the residual value is encoded and transmitted to a decoder. The decoder constructs a set of neighboring points of Px based on the generation of distance-based LOD and additional information received, predicts the attribute value of Px based on the attribute values of the neighboring points in the neighboring set, and adds the received residual value to the Px. Restore attribute values. A method for predicting a property from a set of neighboring points can be applied in different ways from when a set of neighboring points is set based on distance and when a set of neighboring points is generated based on similar properties.

3-1) Weight setting method

The method/apparatus according to the embodiments applies a weight to the attribute value of each neighboring point based on the distance, index, or Molton code difference from the neighboring point in order to predict the Px attribute value through the attributes of the neighboring points of Px. , It can be applied to predict the attribute value of Px.

1 / You can set the distance value to the neighboring point as a weight.

1 / The difference between the Molton code and the neighboring point can be applied as a weight.

<Index-based weight>

1 / The index difference value with the neighboring point can be applied as a weight. The index may be the order when all points are rearranged according to LOD.

For example, it may be the best mode based on similar attributes. If distance-based weights are included, there may be room for distortion in similar properties. Therefore, weights may not be applied according to embodiments.

This weight selection method may be differently selected/applied/transmitted to the decoder according to content characteristics.

4) Property prediction method from neighboring point set

Describes how to predict attribute values based on attribute values of neighboring points.

This attribute prediction method may be signaled to the decoder.

The difference values of the properties of neighboring points can be compared and the property values of the neighboring points having the smallest value can be applied as predicted values. Alternatively, the similarity value of the attribute may be compared, and the attribute value of the neighboring point having the highest similarity value may be applied as a predicted value.

The Molton code difference value with neighboring points can be compared and the attribute value of the neighboring point having the smallest value can be applied as a predicted value. For example, the method/apparatus according to the embodiments may select one point having the smallest Molton code difference value among the three. If it is based on a similarity attribute, when selecting a similar point within a range, additional information indicating the selected point may be required, and thus the bit stream may increase. The method/apparatus according to the embodiments may minimize additional information.

The method/apparatus according to the embodiments may compare the difference value of the index corresponding to the sorted order and apply the attribute value of the neighboring point having the smallest value as the predicted value. For example, if you select 3 from the middle value of the search range, you can send a small difference in the index for 3.

The method/device according to the embodiments may provide a method of selecting the smallest index difference in addition to distance-based attribute prediction.

A value obtained by averaging the attribute values of neighboring points can be applied as a predicted value.

A value obtained by multiplying the attribute value of the neighboring point by the weight can be applied as a predicted value.

The method/device according to the embodiments may find the best method among the above methods, apply the method, and signal the applied method to the decoder. Alternatively, you can selectively set and apply a method to one content.

5) Additional information signaling method for neighboring points applied to prediction

The method/apparatus according to the embodiments may signal information on a neighboring point applied to prediction in order to transmit less additional information.

When the encoder according to the embodiments selects a neighboring point based on the similarity attribute, the decoder according to the embodiments cannot know which point has a property similar to Px belonging to the set. Since the attribute value of Px is not in a restored state, which point among the points belonging to LOD ₀ ~ LOD _ㅣ-1 and the points belonging to LOD _l from which the attribute value has already been restored, the points before Px in the sorted order It is not known whether it is similar to the property value of Px.

Therefore, when selecting a neighboring point based on similar properties, additional information on the selected neighboring point may need to be transmitted to the decoder. The encoder according to the embodiments may transmit additional information to the decoder according to the embodiments.

5-1) Number of neighboring points

The number of neighboring points applied for prediction may be transmitted to the decoder, or the number of neighboring points may not be transmitted from the neighboring point set according to an attribute prediction method. For example, since the smallest attribute difference selection method transmits a residual value for one neighboring point, it may not be necessary to transmit the number of neighboring points. However, the average value selection method requires information on how many neighboring points are used for prediction. The number of neighboring points applied to prediction may be changed according to the minimum attribute similarity threshold. For example, the transmission apparatus/method according to the embodiments may transmit the number of points used for the average value as additional information to the reception apparatus/method according to the embodiments.

5-2) Additional information on neighboring points

The encoder according to the embodiments may transmit additional information on the neighboring point applied to the prediction to the decoder according to the embodiments. For additional information, the following method can be selectively applied.

The difference between values of the Molton code and the neighboring points applied to the property prediction may be transmitted to the decoder.

After the LOD is configured, the difference in index values with neighboring points applied to property prediction may be transmitted in the order of the rearranged point cloud arrangement. For example, you can combine the LODs to sort them, and the number of times to indicate your choice.

The method/apparatus according to the embodiments may sort by Molton code prior to constructing the LOD. Differences in index values with neighboring points applied to property prediction in the order of the point cloud arrangement arranged based on the Molton code can be transmitted. For example, it may represent the difference in the index value before the LOD.

Accordingly, the method/apparatus according to the embodiments can signal while effectively reducing the amount of additional information.

If it is a neighboring point of point Px in LOD _l , a difference in index value from neighboring points applied to attribute prediction may be transmitted around a first position having a Molton code value greater than or equal to Pxmc in LOD ₀ to LOD _l-1 .

When neighboring candidate nodes are sorted based on distance, index values of neighboring points applied to property prediction may be transmitted to the decoder.

The method/apparatus according to the embodiments may find the best method among the above-described methods, apply a corresponding method, and signal the applied method to the decoder. Alternatively, you can selectively set and apply a method to one content.

6) Additional information encoding method for neighboring points

The method/apparatus according to the embodiments may directly encode additional information on all neighboring points to transmit additional information on the neighboring points, or to other neighboring points or neighboring points of another point. A residual value for additional information may be calculated and encoded. This additional information prediction method may be signaled to the decoder. For example, there is an effect of reducing the amount of data losslessly by signaling a method of encoding additional information.

Lossless compression can be performed on additional information of neighboring points.

When sending additional information of several neighboring points, the additional information of the first neighboring point (the most similar attribute) is directly encoded, and the other neighboring points calculate a residual value with the additional information of the first neighboring point, and lossless compression can be performed. have. For example, there may be many cases in which a residual value of 0 appears according to embodiments.

For example, the method/apparatus according to the embodiments may additionally signal a residual value between neighboring points and/or a residual value between additional information.

A method of determining a reference point to be directly encoded for each specific number can be used. For example, an additional information value for the first neighboring point of the first point per four points is determined as a reference value, and additional information of all neighboring points existing before the next reference point is calculated as a residual value from the reference value, thereby performing lossless compression. The number for selection of reference points may be signaled to the decoder.

7) Attribute information prediction unit

An attribute information predictor according to embodiments will be described with reference to FIG. 21.

Referring to FIG. 21, an overall block diagram of a PCC data encoder and a decoder is shown.

PCC data may be entered and encoded as an input of the encoding device according to the sub-embodiments to output a geometric information bitstream and an attribute information bitstream. A geometric information bitstream and an attribute information bitstream encoded as inputs of a decoder according to embodiments may be input, decoded, and reconstructed PCC data may be output. The attribute information prediction unit belongs to the attribute information encoding unit of the encoder and the attribute information decoding unit of the decoder.

The structure related to the attribute information encoding unit according to the embodiments has been described in FIG. 22.

In the aspect of the attribute information encoding unit, a structure related to the attribute information prediction unit according to embodiments has been described in FIG. 23.

In terms of the attribute information decoding unit, a structure related to the attribute information prediction unit according to the embodiments has been described with reference to FIG. 24.

The attribute information prediction unit and/or the predicting step according to the embodiments may be present in all of the encoder (encoder), the decoder (decoder), the encoding step and/or the decoding step according to the embodiments.

The attribute information predictor according to embodiments may include an LOD constructing unit, a neighboring point set constructing unit, a predictive transform/inverse transform unit, and a lifting transform/inverse transform unit. The attribute information prediction unit corresponds to a prediction/lifting conversion processing unit according to embodiments, and reference will be made to prediction conversion according to embodiments, lifting conversion according to embodiments, and explanations related to FIG. 25.

The method/apparatus according to the embodiments may efficiently encode/decode point cloud data through operations of a neighbor point set construction unit and a prediction transform/inverse transform unit of the attribute information prediction unit. An example of each configuration according to the embodiments will be described as follows.

The neighbor point set configuration unit has a neighbor point selection method (neighbour_selection_type), selects a neighbor point according to the selection method, and the applied method may be transmitted to the decoder. The neighbor point selection method may include a distance-based method, an attribute-based method, and a distance + attribute-based method.

The neighbor point set configuration unit has a method of measuring attribute similarity when selecting a neighboring point in an attribute-based or distance + attribute-based method, and selects a neighboring point by measuring attribute similarity between points according to the selected method, and the applied method is to the decoder. Can be transmitted.

In addition, details of the neighbor point set configuration unit may refer to the description of other embodiments.

The predictive transform/inverse transform unit and the lifting transform/inverse transform unit have a weight application method (neighbour_predict_weight_type) to generate predicted values from points in the configured neighboring point set. I can. The applied weight application method may be transmitted to the decoder. The weight application method may include distance-based weighting, Molton code-based weighting, index-based weighting, and weight not being applied.

The predictive transform/inverse transform unit and the lifting transform/inverse transform unit have an attribute prediction method (neighbour_predict_method) from points in the configured neighboring point set, and may predict attribute values according to the selected method and transmit them to the decoder. Attribute prediction methods include selecting the attribute of the neighboring point with the smallest attribute difference, the method of selecting the attribute of the neighboring point with the smallest Molton code difference, the method of selecting the attribute of the neighboring point with the smallest index difference, There may be a method of selecting an average attribute value of neighboring points, and a method of selecting an averaged value by applying a weight to attribute values of neighboring points.

In addition, details of the predictive transform/inverse transform unit and the lifting transform/inverse transform unit may refer to descriptions of other embodiments.

8) Neighbor information change unit/neighbor information inverse transform unit

When generating a neighboring point set based on a similar attribute, the receiving device can restore the neighboring point set only when additional information on the selected neighboring point is transmitted to the decoder. Accordingly, the method/apparatus according to the embodiments may add a neighbor information transforming unit/inverse transforming unit to the PCC encoder/decoder.

Referring to FIG. 37, with respect to the attribute information encoding unit, the neighbor information conversion unit belongs to the attribute information encoding unit.

The point cloud transmission apparatus according to embodiments may include an attribute information encoder.

The attribute information encoder according to embodiments may include an attribute information predictor.

The point cloud receiving apparatus according to embodiments may include an attribute information decoder.

The attribute information decoder according to embodiments may include an attribute information predictor.

The attribute information predictor according to embodiments may predict an attribute value (or attribute information) of a point (eg, a first point) based on attribute information of neighboring points. The attribute information prediction method may include at least one of a distance-based weight, a Molton code-based weight, an index-based weight, and/or no weight. The weight according to the embodiments is applied to the attribute value of each neighboring point.

Regarding the distance-based weight, a weight may be generated based on a distance or 1/distance between the first point and each neighboring point.

In relation to the Molton code-based weight, a weight may be generated based on a Molton code difference value or a 1/Molton code difference value of the first point and each neighboring point.

In relation to the index-based weight, the weight may be generated based on the index difference value or 1/index difference value of the first point and each neighboring point.

The attribute information predictor according to embodiments may predict attribute information of a point based on attribute information of neighboring points.

Regarding the method of selecting the smallest attribute difference, the attribute predictor generates (determines) the attribute value of the second point having the smallest difference value as a predicted value based on attribute difference values between neighboring points.

Regarding the method of selecting the smallest Molton code difference, the attribute prediction unit uses the attribute value of the second point having the smallest difference value as a predicted value based on the Molton code difference value between each neighboring point and/or between the first point and each neighboring point. Create (determine).

Regarding the method of selecting the smallest index difference, the attribute predictor generates (determines) the attribute value of the second point having the smallest difference value as a predicted value based on the index difference value between the first point or each neighboring point.

Regarding the attribute average value selection method, the attribute predictor generates (determines) an average value of attribute values of each neighboring point as a predicted value of the first point.

Regarding the weighted average value selection method, the attribute predictor multiplies an attribute value of each neighboring point by a weight, and generates (determines) an average value of the multiplied values as a predicted value of the first point.

The point cloud data according to the embodiments includes the number of neighboring points (the number of neighboring points applied to the prediction), additional information about the neighboring points (information about the neighboring points applied to the prediction), for example, a difference value of the Molton code value, and an LOD. The index difference value in the sorted order, the index difference value in the order sorted by the Molton code, the index difference value in the first position equal to or equal to the Molton code of the first point in a plurality of LOD sets, and/or the neighboring candidate nodes are distance-based. It includes at least one of the index values in the order sorted by.

The encoder of the point cloud transmission apparatus according to the embodiments may perform at least one of direct encoding of additional information (or parameter, signaling information) about a neighboring point, encoding a residual value between neighboring points, and/or encoding a residual value between additional information. I can.

The neighbor information conversion unit of the point cloud data transmission apparatus according to embodiments may include information related to neighbor information in a bitstream of the point cloud data and transmit it.

The neighbor information inverse transform unit 38000 and/or the attribute information predictor 38001 according to the embodiments may be combined with a method/device according to the embodiments.

38 illustrates an example of an attribute information decoding unit according to embodiments. The neighbor information inverse transform unit according to embodiments may be included in the attribute information decoding unit.

The neighbor information transform unit/inverse transform unit according to embodiments may exist in the encoder and decoder according to the embodiments. Each configuration according to the embodiments will be described as follows.

The neighbor information conversion unit may operate when selecting a neighboring point in an attribute-based or distance + attribute-based method.

The neighbor information conversion unit has a type of additional information on the neighboring point (neighbor_property_type) applied to prediction, configures an additional information bitstream for the neighboring point according to the selection method, transmits the additional information type to the decoder, and the configured data bitstream is The residual may be quantized and included in the attribute information bitstream together with the encoded value and transmitted to the decoder. The type of additional information about the neighboring points is the method of configuring the bitstream by the difference of the Molton code value, the method of configuring the bitstream by the difference of the index value in the state of being sorted by LOD, and the method of configuring the bitstream by the difference of index values when sorting based on the distance of neighboring candidate nodes. It can be a way.

The neighbor information transform unit may transmit a method (neighbour_property_encoding_type) used to encode additional information about a neighbor point applied to prediction to the decoder. When applying the residual value encoding method between additional information, a reference point sampling rate (neighbour_property_ref_point_sampling_rate) that determines a reference point to be directly encoded has a value, and the value can be transmitted to the decoder.

The neighbor information converter may encode the number of neighbor points applied to the prediction, add an attribute bitstream, and transmit it to the decoder.

The neighbor information transform unit may configure additional location information on the neighboring point applied to the prediction, encode it through the attribute information entropy encoder, add it to the attribute information bitstream, and transmit it.

When the encoder selects a neighboring point in an attribute-based or distance + attribute-based method, the neighbor information inverse transform unit receives neighbor information in the attribute bitstream and is transmitted, and when the received neighbor information is decoded and registered as a neighbor point, it can operate. have.

The neighbor information inverse transform unit configures the bitstream with the difference between the number of neighboring points in the received bitstream, the additional information of the neighboring points (a method in which the bitstream is formed by a difference in the Morton code value, a method in which the index value difference in the state of being sorted by LOD, or neighbor candidate nodes). When sorting based on distance, information according to the method of configuring the bitstream with an index value), additional information encoding method (direct encoding, residual encoding between neighboring points, or encoding residual values between additional information), selected through reference point sampling rate values A set of neighboring points can be constructed by finding neighboring points. The attribute information prediction unit may restore the attribute value through the configured set of neighboring points.

9) Signaling scheme for the above-described method

The method/apparatus according to the embodiments may signal related information to perform the above-described operations. The signaling information according to embodiments may be used in a transmission method/device or a reception method/device.

The bitstream configuration of point cloud data according to the embodiments is as described in FIG. 26.

Neighbor point set generation and prediction/lifting conversion-related option information may be added to the APS and signaled.

Tiles or slices are provided so that point clouds (point cloud data) can be divided and processed by area.

When dividing by region, by setting a different neighbor point set generation option for each region, the complexity is low, and the reliability of the result is slightly lower, or conversely, a selection method with high complexity but high reliability can be provided. It can be set differently according to the processing capacity of the receiver.

Therefore, when the point cloud is divided into tiles, a different neighbor point set generation option may be applied for each tile.

When the point cloud is divided into slices, a different neighbor point set generation option may be applied for each slice.

The neighbor point set generation and prediction/lifting conversion related option information may be added to the TPS or the Attr for each slice for signaling.

When generating a set of neighboring points based on similar properties, additional information related to the set of neighboring points may be added and transmitted to the Predicting weight lifting bitstream syntax and the Fixed weight lifting bitsream syntax.

The neighbor information inverse transform unit of the point cloud data receiving apparatus according to the embodiments may obtain neighbor information related information from the received bitstream.

Due to the above-described embodiments of FIGS. 37-38, the method/apparatus according to the embodiments may process an attribute prediction method in consideration of characteristics of point cloud data.

The method/apparatus according to the embodiments may include information related to the neighbor point set generation option in the attribute parameter set.

The method/apparatus according to the embodiments may signal by adding a neighboring point set and option information related to prediction/lifting transformation to the APS. The description of signaling information according to embodiments is as follows.

neighbour_selection_type: specifies how to select neighbor points

1= Distance-based neighbor point selection method

2= attribute-based neighbor point selection method

neighbour_predict_weight_type: Specifies how to apply weight to generate prediction values from neighboring points

0= no weight applied

1= distance based weight

2= Molton code based weight

3= index-based weight

neighbour_predict_method: Specify the property prediction method from neighboring points

1= Method to select the property of the neighboring point with the smallest property difference

2= Method to select the property of the neighboring point with the smallest Molton code difference

3= Method to select the property of the neighboring point with the smallest index difference

4= Method to select the average value of the attributes of neighboring points

5= Method of selecting the averaged value by applying weight to the attribute values of neighboring points

neighbour_property_type: Specifies the type of additional information about the neighboring point

1= A method of constructing a bitstream with a difference in Molton code values

2= A method of composing a bitstream by index value difference in the state of being sorted by LOD

3= A method of constructing a bitstream with an index value at the time of sorting neighbor candidate nodes based on distance

neighbour_property_encoding_type: Specifies the method of encoding additional information about neighboring points

1= Direct coding method

2= encoding method for residual values between neighboring points

3= Coding of residual values between additional information, a method of determining reference points to be directly encoded for each specific number

neighbour_property_ref_point_sampling_rate: When applying the residual value encoding method between additional information, specify the reference point sampling rate that determines the reference point to be directly encoded.

According to embodiments, when neighbour_property_encoding_type is 3, neighbour_property_ref_point_sampling_rate may exist.

aps_attr_parameter_set_id represents an identifier for an APS for reference by other syntax elements. The value of aps_attr_parameter_set_id may have a range of 0 to 15 (inclusive) (provides an identifier for the APS for reference by other syntax elements.The value of aps_attr_parameter_set_id shall be in the range of 0 to 15, inclusive).

aps_seq_parameter_set_id represents the value of sps_seq_parameter_set_id for active SPS. The value of aps_seq_parameter_set_id may have a range of 0 to 15 (inclusive) (specifies the value of sps_seq_parameter_set_id for the active SPS.The value of aps_seq_parameter_set_id shall be in the range of 0 to 15, inclusive).

attr_coding_type represents a coding type for an attribute for a given value of attr_coding_type. The value of attr_coding_type may be 0, 1, or 2 in bitstreams according to embodiments. Other values of attr_coding_type may be reserved for future use by ISO/IEC. Decoders according to embodiments may ignore reserved values of attr_coding_type. 0= Predicting Weight Lifting, 1= Region Adaptive Haraquical Transfer (RAHT), 2= Fix Weight Lifting (indicates that the coding type for the attribute in Table 7 2 Table 7 2 for the given value of attr_coding_type.The value of attr_coding_type. shall be equal to 0, 1, or 2 in bitstreams conforming to this version of this Specification.Other values of attr_coding_type are reserved for future use by ISO/IEC.Decoders conforming to this version of this Specification shall ignore reserved values of attr_coding_type. 0 = Predicting weight lifting, 1 = Region Adaptive Hierarchical Transferm (RAHT), 2= Fixed weight lifting).

num_pred_nearest_neighbours represents the maximum number of near neighbors used for prediction. The value of numberOfNearestNeighboursInPrediction shall be in the range of 1 to xx (specifies the maximum number of nearest neighbors to be used for prediction.

max_num_direct_predictors represents the maximum number of predictors used for direct prediction. The value of max_num_direct_predictors may have a range of 0 to num_pred_nearest_neighbours. The value of the variable MaxNumPredictors can be used in the decoding process as follows: MaxNumPredictors = max_num_direct_predicots + 1 (specifies the maximum number of predictor to be used for direct prediction.The value of max_num_direct_predictors shall be range of 0 to num_pred_nearest_neighbours. The value of the variable MaxNumPredictors that is used in the decoding process as follows: MaxNumPredictors = max_num_direct_predicots + 1).

lifting_search_range specifies search range for the lifting.

lifting_quant_step_size represents the quantization step size for 1 ^st component of the attribute. The value of quant_step_size shall be in the range of 1 to xx (specifies the quantization step size for the 1st component of the attribute.

lifting_quant_step_size_chroma indicates the quantization step size for the chroma component of the attribute when the attribute is color. The value of quant_step_size_chroma may be in the range of 1 to xx (specifies the quantization step size for the chroma component of the attribute when the attribute is colour.The value of quant_step_size_chroma shall be in the range of 1 to xx).

lod_binary_tree_enabled_flag specifies whether binary tree is enable or not for the log generation.

num_detail_levels_minus1 represents the number of levels of details for attribute coding. The value of num_detail_levels_minus1 may have a range of 0 to xx (specifies the number of levels of detail for the attribute coding.The value of num_detail_levels_minus1 shall be in the range of 0 to xx).

sampling_distance_squared [idx] represents the square of the sampling distance for idx. The value of sampling_distance_squared[] may have a range of 0 to xx (specifies the square of the sampling distance for idx. The value of sampling_distance_squared[] shall be in the range of 0 to xx).

adaptive_prediction_threshold specifies the threshold of prediction.

raht_depth represents the number of levels of details for RAHT. The value of depthRAHT may have a range of 1 to xx (specifies the number of levels of detail for RAHT. The value of depthRAHT shall be in the range of 1 to xx).

raht_binarylevel_threshold represents a level of detail for cutting-out RAHT coefficients. The value of binaryLevelThresholdRAHT may be in the range of 0 to xx (specifies the levels of detail to cut out the RAHT coefficient.The value of binaryLevelThresholdRAHT shall be in the range of 0 to xx).

raht_quant_step_size represents the quantization step size for the 1 ^st component of the attribute. The value of quant_step_size may be in the range of 1 to xx (specifies the quantization step size for the 1st component of the attribute.The value of quant_step_size shall be in the range of 1to xx).

When aps_extension_present_flag is 1, it indicates that the aps_extension_data syntax structure is present in the APS RBSP syntax structure. When aps_extension_present_flag is 0, this syntax structure does not exist. If not present, the value of aps_extension_present_flag can mean 0 (equal to 1 specifies that the aps_extension_data syntax structure is present in the APS RBSP syntax structure.aps_extension_present_flag equal to 0 specifies that this syntax structure is not present.When not present. , the value of aps_ extension_present_flag is inferred to be equal to 0).

aps_extension_data_flag can have any value. The presence and value of this value does not affect the decoder following the profile. Decoders may have any value. Its presence and value do not affect decoder conformance to profiles. Decoders conforming to a profile.

The method/apparatus according to the embodiments may include information related to the neighbor point set generation option in the tile parameter set.

The TPS according to the embodiments may include information syntax related to the neighbor point set generation option.

The method/apparatus according to the embodiments may signal by adding the neighbor point set according to the embodiments and option information related to prediction/lifting transformation to the TPS. A description of each signaling information according to embodiments is as follows.

num_tiles represents the number of tiles signaled for the bitstream. If not present, num_tiles may be 0 (specifies the number of tiles signaled for the bitstream.When not present, num_tiles is inferred to be 0).

tile_bounding_box_offset_x[ i] represents the x offset of the i-th tile in cartesian coordinates. If not present, the value of tile_bounding_box_offset_x[ 0] may be sps_bounding_box_offset_x (indicates the x offset of the i-th tile in the cartesian coordinates. When not present, the value of tile_bounding_box_offset_x[ 0] is inferred to be sps_bounding_box_offset_x).

tile_bounding_box_offset_y[ i] represents the y offset of the i-th tile in cartesian coordinates. If not present, the value of tile_bounding_box_offset_y[ 0] may be sps_bounding_box_offset_y (indicates indicates the y offset of the i-th tile in the cartesian coordinates. When not present, the value of tile_bounding_box_offset_y[ 0] is inferred to be sps_bounding_box_offset_y) .

tile_bounding_box_offset_z[i] represents the z offset of the i-th tile in Cartesian coordinates. If not present, the value of tile_bounding_box_offset_z[ 0] may be sps_bounding_box_offset_z (indicates indicates the z offset of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_offset_z[ 0] is inferred to be sps_bounding_box_offset) .

tile_bounding_box_scale_factor[ i] represents the scale factor of the i-th tile in Cartesian coordinates. If not present, the value of tile_bounding_box_scale_factor[ 0] may be sps_bounding_box_scale_factor (indicates the scale factor the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_scale_factor[ 0] is inferred to be sps_bounding_box_scale_factor).

tile_bounding_box_size_width[ i] represents the width of the i-th tile in Cartesian coordinates. If not present, the value of tile_bounding_box_size_width[ 0] can be sps_bounding_box_size_width (indicates the width of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_size_width[ 0] is inferred to be sps_bounding_box_size_width).

tile_bounding_box_size_height[ i] represents the height of the i-th tile in Cartesian coordinates. If not present, the value of tile_bounding_box_size_height[ 0] may be sps_bounding_box_size_height (indicates the height of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_size_height[ 0] is inferred to be sps_bounding_box_size_height).

tile_bounding_box_size_depth[ i] represents the depth of the i-th tile in Cartesian coordinates. If not present, tile_bounding_box_size_depth[ 0] may be sps_bounding_box_size_depth (indicates the depth of the i-th tile in the Cartesian coordinates. When not present, the value of tile_bounding_box_size_depth[ 0] is inferred to be sps_bounding_box_size_depth).

Slice header of Attr according to embodiments may include information related to a neighbor point set generation option.

The method/apparatus according to the embodiments may signal by adding option information related to generation of a neighboring point set and prediction/lifting transformation according to the embodiments to a slice header of Attr. A description of each signaling information according to embodiments is as follows.

abh_attr_parameter_set_id represents the value of aps_attr_parameter_set_id of the active APS (specifies the value of the aps_attr_parameter_set_id of the active APS).

abh_attr_sps_attr_idx represents an attribute set in the active SPS. The value of abh_attr_sps_attr_idx may have a range of 0 to sps_num_attribute_sets in the active SPS (specifies the attribute set in the active SPS.The value of abh_attr_sps_attr_idx shall be in the range of 0 to sps_num_attribute_sets in the active SPS).

abh_attr_geom_slice_id specifies the value of geom slice id.

The method/device according to the embodiments may include information related to the neighbor point set generation option in the predicting weight lifting bitstream.

Predicting weight lifting bitstream syntax of attribute slice data according to embodiments may include additional information related to a neighboring point set.

When generating a neighboring point set based on a similar attribute, the method/apparatus according to the embodiments may signal by adding additional information related to the neighboring point set according to the embodiments to the prediction weight lifting bitstream.

When neighbor_selection_type is an attribute-based or distance+attribute-based method for selecting a neighboring point (>1), additional information of the neighboring point may be added to the attribute information bitstream as follows.

The following scheme may be a syntax corresponding to a case where the neighbour_predict_method value is less than 4, that is, the attribute prediction method based on neighboring points is selected not the average value but the smallest attribute difference, the smallest Molton code difference, and the smallest index difference. A description of each signaling information according to embodiments is as follows.

neighbour_point_info: Specify the value of the neighboring point additional information. The additional information value may be the difference in Molton code value depending on the neighbor_property_type, the difference in the index value in the state sorted by LOD, or the difference in the index value in the state in the state in which the Molton code is sorted, and from LOD0 ~ LOD The Molton code may be a difference between the index values at the first position having the same or greater value, and may be an index value when neighboring candidate nodes are aligned based on distance.

neighbour_point_residual_info: Specify the residual value of the reference neighboring point of the neighboring point additional information value

predIndex[ i ]: predIndex[ i] represents a predicator index for decoding the i-th point value of the attribute. The value of predIndex[ i] may range from 0 to max_num_direct_predictors. Predictor index for decoding the ith point value of the attribute. The value of predIndex[ i] may have a range of 0 to max_num_direct_predictors (specifies the predictor index to decode the i-th point value of the attribute. The value of predIndex[ i] shall be range of 0 to max_num_direct_predictors. The predictor index to decode the i-th point value of the attribute.The value of predIndex[ i] shall be range of 0 to max_num_direct_predictors).

The following scheme may be a syntax corresponding to a case where the value of neighbour_predict_method is greater than 3, that is, the attribute prediction method based on neighbor points is selected as an average value or a weighted average value.

Neighbour_num_of_points according to embodiments: specifying the number of neighboring points applied to attribute prediction. When the attribute prediction method is an average value or a weighted average value, it is necessary to specify how many neighboring points are applied, and additional information about each neighboring point may be required.

44 shows examples of quant values according to embodiments.

The method/apparatus according to the embodiments may include information related to the neighbor point set generation option in quantvalues.

A description of each signaling information according to embodiments is as follows.

When isZero is 1, it indicates that residual value[k][i] is 0. When isZero is 0, residual value[k][i] is not 0 (equal to 1 indicates that residual value[k][i] is equal to 0. isZero equal to 0 indicates that residual value[k] [i] is not equal to 0).

values[k][i] represents the k-th dimension and the i-th point value of the attribute (describes the k-th dimension and the i-th point value of the attribute).

remaining_values[k][i] represents the k-th dimension and i-th point remaining values of the attribute. If not present, the value of remaining_value[k][i] can be 0 (describes the k-th dimension and the i-th point remaining value of the attribute. When not present, the value of remaining_value[k][ i] is inferred to be 0).

The method/device according to the embodiments may include information related to the neighbor point set generation option in the fixed weight lifting bitstream.

Fixed weight lifting bitstream syntax of attribute slice data according to embodiments may include additional information syntax related to a neighboring point set.

The method/apparatus according to the embodiments may signal by adding additional information related to the neighbor point set according to the embodiments to the fixed weight lifting bitstream when generating a neighboring point set based on a similar attribute.

When Neighbor_selection_type is an attribute-based or distance+attribute-based method to select a neighboring point (>1), additional information of the neighboring point can be added to the attribute information bitstream as follows.

The following scheme may be a syntax corresponding to a case where the neighbour_predict_method value is less than 4, that is, the attribute prediction method based on neighboring points is selected not the average value but the smallest attribute difference, the smallest Molton code difference, and the smallest index difference.

The apparatus/method for transmitting and receiving point cloud data according to embodiments compresses properties of an encoder (encoder)/decoder (decoder) of Geometry-based Point Cloud Compression (G-PCC) for compressing 3D point cloud data. As a way to increase the efficiency, after generating a set of neighboring points based on similar attributes, there is an effect of providing a method of predicting, encoding, and decoding attributes based on the generated neighboring point set.

For this reason, the embodiments increase the property compression efficiency of the encoder (encoder)/decoder (decoder) of Geometry-based Point Cloud Compression (G-PCC) for compressing 3D point cloud data, resulting in a higher recovery rate. It is possible to provide a provided point cloud content stream.

Due to the parameters of FIGS. 39 to 47 described above, the method/apparatus according to the embodiments can efficiently signal encoding/decoding of point cloud data.

The method/device according to the embodiments may perform S47000 and S47001, perform S47002, S47003, and S47004, or perform S47000 to S47005. The following can be combined/modified with the above-described embodiments for each step.

S47000, the method/apparatus according to the embodiments sets a search range for a neighboring point. For example, an octree-based search range can be set according to the search range type. A detailed octree-based search range setting method is as described in FIG. 21 and the like.

S47001, the method/apparatus according to the embodiments selects a neighboring point based on the neighboring point search range. For example, a neighboring point can be selected based on the similarity attribute. A specific method for selecting a neighboring point based on similarity attributes is as described above.

S47002, the method/device according to the embodiments sets a weight for a set of neighboring points. For example, there are distance-based, Molton code-based, index-based and/or weightless methods, and the like, and specific methods are as described above.

S47003, the method/apparatus according to the embodiments predicts attribute information on a neighboring point set. For example, there are a method of selecting the smallest Molton code difference, selecting the smallest index difference, selecting an attribute average value, and selecting a weighted average value, and the specific method is as described above.

S47004, The method/apparatus according to the embodiments encodes the residual attribute information based on the prediction attribute information.

S47005, the method/device according to the embodiments transmits residual attribute information.

48 shows a method for transmitting point cloud data according to embodiments.

The point cloud data transmission method according to the embodiments includes the following method.

S48000, the method according to the embodiments acquires point cloud data. A method of acquiring point cloud data according to embodiments is as described with reference to FIGS. 1, 2, 3, and the like.

S48001, the method according to the embodiments encodes point cloud data. The encoding method according to the embodiments is as described in FIGS. 4-9, 12, 14-16, 18-22, 23, 25, 37, and the like.

S48002, the method according to the embodiments transmits point cloud data. Point cloud data according to embodiments may be transmitted as described in FIGS. 1-3 and 14-16.

49 shows a method of receiving point cloud data according to embodiments.

A method of receiving point cloud data according to embodiments includes the following method.

S49000, the method according to the embodiments receives point cloud data. Point cloud data reception methods according to embodiments are as described in FIGS. 1-2, 4, 11, 13, 14-16, etc.

S49001, the method according to the embodiments decodes the point cloud data. Point cloud data decoding methods according to embodiments are as described in FIGS. 24, 25, 33, 34, 35, 38, and the like.

S49002, the method according to the embodiments renders the point cloud data. The point cloud data rendering (or decoding) method according to embodiments may be processed based on the signaling information described in FIGS. 27-29 and 39-46.

Each of the above-described parts, modules or units may be software, processor, or hardware parts that execute successive processes stored in a memory (or storage unit). Each of the steps described in the above-described embodiment may be performed by processor, software, and hardware parts. Each module/block/unit described in the above-described embodiment may operate as a processor, software, or hardware. In addition, the methods suggested by the embodiments may be executed as code. This code can be written to a storage medium that can be read by the processor, and thus can be read by a processor provided by the apparatus.

For convenience of explanation, each drawing has been described separately, but it is also possible to design a new embodiment by merging the embodiments described in each drawing. In addition, designing a computer-readable recording medium in which a program for executing the previously described embodiments is recorded is also within the scope of the rights of the embodiments according to the needs of the skilled person.

The apparatus and method according to the embodiments are not limitedly applicable to the configuration and method of the described embodiments as described above, but the above-described embodiments are all or part of each of the embodiments so that various modifications can be made. It may be configured in combination.

On the other hand, it is possible to implement the method proposed by the embodiments as a code readable by the processor on a recording medium readable by the processor provided in the network device. The processor-readable recording medium includes all types of recording devices that store data that can be read by the processor. Examples of recording media that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and also include those implemented in the form of carrier waves such as transmission through the Internet. . Further, the processor-readable recording medium is distributed over a computer system connected through a network, so that the processor-readable code can be stored and executed in a distributed manner.

In addition, although preferred embodiments of the embodiments have been illustrated and described above, the embodiments are not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the embodiments claimed in the claims. In addition, various modifications can be implemented by those of ordinary skill in the art, and these modifications should not be understood individually from the technical idea or prospect of the embodiments.

It is understood by those skilled in the art that various changes and modifications are possible in the embodiments without departing from the spirit or scope of the embodiments. Accordingly, the embodiments are intended to cover variations and modifications of the embodiments provided within the appended claims and their equivalents.

In the present specification, both apparatus and method inventions are mentioned, and descriptions of both apparatus and method inventions may be applied to complement each other.

In this document, “/” and “,” are interpreted as “and/or”. For example, “A/B” is interpreted as “A and/or B”, and “A, B” is interpreted as “A and/or B”. Additionally, “A/B/C” means “at least one of A, B and/or C”. In addition, “A, B, C” also means “at least one of A, B and/or C”.

(In this document, the term “/”and “,” should be interpreted to indicate“and/or”. For instance, the expression “A/B” may mean “A and/or B.” Further, “A, B" may mean“A and/or B.” Further, “A/B/C may mean“at least one of A, B, and/or C.”Also,“A/B/C” may mean "at least one of A, B, and/or C.”

Additionally, in this document “or” is to be interpreted as “and/or”. For example, “A or B” may mean 1) only “A”, 2) only “B”, or 3) “A and B”. In other words, “or” in this document may mean “additionally or alternatively”. (Further, in the document, the term “or" should be interpreted to indicate “and/or”. For instance, the expression "AorB" in this document should be interpreted to indicaste "additionally or alternatively ".

Various elements of the embodiments may be performed by hardware, software, firmware, or a combination thereof. Various elements of the embodiments may be implemented on a single chip such as a hardware circuit. Depending on the embodiments, the embodiments may optionally be performed on individual needles. Depending on the embodiments, at least one of the elements of the embodiments may be executed in one or more processors including instructions for performing operations according to the embodiments.

Terms such as first and second are used to describe various elements of the embodiments. These terms do not limit the interpretation of the elements of the embodiments. These terms are used to distinguish between one element and another. For example, a first user input signal may be referred to as a second user input signal. Similarly, the second user input signal may be referred to as a first user input signal. These terms can be interpreted within the scope of the embodiments. Both the first user input signal and the second user input signal are user input signals, and do not mean the same user input signals unless clearly indicated in context.

The terms used to describe the embodiments are used for the purpose of describing specific embodiments, and are not intended to limit the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly indicates. And/or the expression is used in a sense including all possible combinations between terms. The include expression describes the existence of features, numbers, steps, elements, and/or components, and does not imply that no additional features, numbers, steps, elements, and/or components are included. .

Conditional expressions such as when, when, and when used to describe the embodiments are not limited to an optional case. When a specific condition is satisfied, it is intended to perform a related operation in response to a specific condition or to interpret the related definition.

As described above, related contents have been described in the best mode for carrying out the embodiments.

As described above, the embodiments may be applied wholly or partially to the point cloud data transmission/reception apparatus and system.

Those skilled in the art may variously change or modify the embodiments within the scope of the embodiments.

Embodiments may include changes/modifications, and changes/modifications do not depart from the scope of the claims and the same.

Claims

Obtaining point cloud data;

Encoding the point cloud data; And

Transmitting the point cloud data; Containing,

Point cloud data transmission method.
The method of claim 1,

The encoding step includes encoding attribute information of the point cloud data,

The step of encoding the attribute information includes predicting the attribute information,

Point cloud data transmission method.
The method of claim 2,

Predicting the attribute information

Generates a level of detail (LOD) based on attribute information of the point cloud data and reconstructed geometry information,

Generate a set of neighboring points based on the LOD,

Performing attribute coding on the neighboring point set,

Point cloud data transmission method.
The method of claim 3,

Generating the neighboring point set by searching for the neighboring point set based on the parent node of the octree for the LOD,

Point cloud data transmission method.
The method of claim 4,

Generating the neighboring point set based on the similarity property of points included in the neighboring point set,

Point cloud data transmission method.
An acquisition unit for acquiring point cloud data;

An encoder that encodes the point cloud data; And

A transmitter for transmitting the point cloud data; Containing,

Point cloud data transmission device.
The method of claim 6,

The encoder includes an attribute information encoder for encoding attribute information of the point cloud data,

The attribute information encoder predicts the attribute information,

Point cloud data transmission device.
The method of claim 7,

Generates a level of detail (LOD) based on attribute information of the point cloud data and reconstructed geometry information,

Generate a set of neighboring points based on the LOD,

Predicting the attribute information by performing attribute coding on the neighboring point set,

Point cloud data transmission device.
The method of claim 8,

Generating the neighboring point set by searching for the neighboring point set based on the parent node of the octree for the LOD,

Point cloud data transmission device.
The method of claim 9,

Generating the neighboring point set based on the similarity attribute of points included in the neighboring point set,

Point cloud data transmission device.
Receiving point cloud data;

Decoding the point cloud data; And

Rendering the point cloud data; Containing,

How to receive point cloud data.
The method of claim 11,

The decoding step includes decoding attribute information of the point cloud data,

The step of decoding the attribute information includes predicting the attribute information,

How to receive point cloud data.
The method of claim 12,

Predicting the attribute information

Generates a level of detail (LOD) based on attribute information of the point cloud data and reconstructed geometry information,

Generate a set of neighboring points based on the LOD,

Performing attribute coding on the neighboring point set,

How to receive point cloud data.
The method of claim 13,

Generating the neighboring point set by searching for the neighboring point set based on the parent node of the octree for the LOD,

How to receive point cloud data.
The method of claim 14,

Generating the neighboring point set based on the similarity property of points included in the neighboring point set,

How to receive point cloud data.
A receiver for receiving point cloud data;

A decoder for decoding the point cloud data; And

A renderer for rendering the point cloud data; Containing,

Point cloud data receiving device.
The method of claim 16,

The decoder includes an attribute information decoder for decoding attribute information of the point cloud data,

The attribute information decoder predicts the attribute information,

Point cloud data receiving device.
The method of claim 17,

Generates a level of detail (LOD) based on attribute information of the point cloud data and reconstructed geometry information,

Generate a set of neighboring points based on the LOD,

Predicting the attribute information by performing attribute coding on the neighboring point set,

Point cloud data receiving device.
The method of claim 18,

Generating the neighboring point set by searching for the neighboring point set based on the parent node of the octree for the LOD,

Point cloud data receiving device.
The method of claim 19,

Generating the neighboring point set based on the similarity property of points included in the neighboring point set,

Point cloud data receiving device.