JP7112600B2 - Method for point cloud decompression, method and apparatus for point cloud compression - Google Patents

Description

This application claims priority from U.S. patent application Ser. claims priority to U.S. Provisional Application Serial No. 62/812,964, entitled "TECHNEQUES AND APPARATUS FOR SELECTIVE GEOMETRY SMOOTHING INSIDE PATCHES FOR POINT CLOUD COMPRESSION," filed March 1, 2019, and all of the above applications. The contents are incorporated herein by reference.

This disclosure generally describes embodiments related to point cloud compression.

The background art provided herein is for the purpose of generally presenting the context of the present disclosure. In view of the extent of the work described in the background section, the work of the presently signed inventors, and aspects not otherwise qualified as prior art statements at the time of submission, clearly and implicitly precludes the subject matter of this disclosure. not recognized as technology.

Various technologies are developed and grasped to express the world, such as world objects and world environments, in a three-dimensional (3D) space. A 3D representation of the world can enable more immersive interaction and communication. A point cloud can be used as a 3D representation of the world. A point cloud is a set of points in 3D space, each point having associated attributes such as color, material properties, texture information, intensity attributes, reflectance attributes, motion-related attributes, modality attributes, and others. It has various attributes. Such point clouds can contain large amounts of data and can be costly and time consuming to store and transmit.

Aspects of the present disclosure provide methods and apparatus for point cloud compression and decompression. In some examples, the point cloud compression/decompression apparatus includes processing circuitry.

According to some aspects of the present disclosure, an apparatus for point cloud decompression includes processing circuitry. The processing circuitry decodes the point cloud prediction information from the encoded bitstream and reconstructs a geometry reconstruction cloud based on the point cloud geometry image decoded from the encoded bitstream. The processing circuitry also filters and smoothes at least geometry samples within blocks in addition to boundary samples of blocks of the geometry reconstruction cloud to produce a smoothed geometry reconstruction cloud. Reconstruct the points of the point cloud based on the geometric reconstruction cloud.

In some embodiments, the processing circuitry selects regions of high frequency components having levels above the threshold level within the block. In some examples, the processing circuitry detects edges within the block based on depth values of the geometry reconstruction cloud.

In some embodiments, the processing circuitry selects regions of motion content having levels higher than a threshold level within the block. In some examples, the processing circuitry selects points within the block based on motion information of corresponding pixels within the geometry image.

In some embodiments, the prediction information includes flags that indicate to apply selective smoothing within blocks of the point cloud. In some examples, the prediction information dictates a particular algorithm for selecting points within the block. Predictive information also includes parameters for use in specific algorithms.

According to some aspects of the present disclosure, an apparatus for point cloud compression includes processing circuitry. Processing circuitry compresses the geometry image associated with the point cloud and reconstructs a geometry reconstruction cloud based on the compressed geometry image of the point cloud. Then, processing circuitry filters and smoothes at least the geometry samples within the blocks in addition to the boundary samples of the blocks of the geometry reconstruction cloud to produce a smoothed geometry reconstruction cloud. Generate a texture image of the point cloud based on the reconstructed geometry cloud.

In some embodiments, the processing circuitry selects regions of high frequency components having levels above the threshold level within the block. For example, the processing circuitry detects edges within blocks based on depth values in the geometry reconstruction cloud.

In some embodiments, the processing circuitry selects regions of motion content having levels higher than a threshold level within the block. For example, the processing circuitry selects points within the block based on motion information for corresponding pixels within the geometry image.

In some embodiments, the processing circuitry includes a flag in the encoded bitstream of the compressed point cloud that indicates to apply selective smoothing within blocks of the point cloud. In some examples, the processing circuitry includes in the compressed point cloud encoded bitstream an indicator that directs a particular algorithm for selecting points within the block to which to apply selective smoothing.

Aspects of the present disclosure further provide a non-transitory computer-readable medium having stored thereon instructions that, when executed by a computer for point cloud compression/decompression, perform a method for point cloud compression/decompression. let the computer do it.

Further features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and drawings. In the drawing:
1 is a schematic representation of a simplified block diagram of a communication system (100) according to an embodiment; FIG. 2 is a schematic representation of a simplified block diagram of a streaming system (200) according to an embodiment; FIG. FIG. 3 shows a block diagram of an encoder (300) for encoding point cloud frames according to some embodiments. FIG. 4 illustrates a block diagram of a decoder decoding a compressed bitstream corresponding to point cloud frames according to some embodiments; FIG. 3 is a schematic diagram of a simplified block diagram of a video decoder according to an embodiment; 2 is a schematic illustration of a simplified block diagram of a video encoder according to an embodiment; FIG. FIG. 4 illustrates geometry and texture images of point clouds according to some embodiments of the present disclosure; FIG. 4 illustrates example syntax according to some embodiments of the present disclosure. 4 depicts a flow chart outlining an example process according to some embodiments of the present disclosure. 4 depicts a flow chart outlining an example process according to some embodiments of the present disclosure. 1 is a schematic diagram of a computer system according to an embodiment; FIG.

Aspects of the present disclosure provide point cloud coding techniques, particularly using video-coding for point cloud compression (V-PCC). V-PCC can utilize a versatile video codec to perform point cloud compression. The point cloud coding technique in this disclosure can improve both lossless and lossy compression with V-PCC.

A point cloud is a set of points in 3D space, each point having associated attributes such as color, material properties, texture information, intensity attributes, reflectance attributes, motion-related attributes, modality attributes, and others. It has various attributes. Point clouds are used to reconstruct an object or scene as a combination of such points. These points can be captured using multiple cameras and depth sensors in various installations and consist of thousands to billions of points to realistically represent the reconstructed scene. be.

Compression techniques are needed to reduce the amount of data needed to represent the point cloud. Techniques are therefore needed for lossy compression of point clouds for use in real-time communication and six Degrees of Freedom (6 DoF) virtual reality. In addition, technology for reversible point group compression is required in the background of dynamic mapping such as automatic driving and application of cultural heritage. The moving picture experts group (MPEG) has begun to focus on compression of geometry and attributes such as color and reflectance, scalable/progressive coding, time encoding a sequence of point clouds captured over the course of , random access to a subset of the point cloud.

According to one aspect of the present disclosure, the main principle behind V-PCC is to leverage existing video codecs to capture dynamic point cloud geometry shape, occupancy, and texture as three separate video sequences. Compress. Additional metadata required to interpret the three video sequences are compressed separately. A small portion of the overall bitstream is metadata, which can be efficiently encoded/decoded using software implementations. Most of the information is processed by video codecs.

FIG. 1 is a simplified block diagram of a communication system (100) according to embodiments of the present disclosure. A communication system (100) includes a plurality of terminal devices that can communicate with each other, eg, via a network (150). For example, a communication system (100) comprises a pair of terminal devices (110), (120) connected together via a network (150). In the example of FIG. 1, a first pair of terminal devices (110), (120) perform unidirectional transmission of point cloud data. For example, the terminal device (110) can compress a cloud of points (eg, points representing structures) captured by the sensor 105 connected to the terminal device (110). The compressed point cloud is transmitted, for example in the form of a bitstream, over the network (150) to other terminal devices (120). The terminal device (120) can receive the compressed point cloud from the network (150), decompress the bitstream to reconstruct the point cloud, and appropriately display the reconstructed point cloud accordingly. . Unidirectional data transmission is common in media serving applications and the like.

In the example of FIG. 1, terminals 110 and 120 may be illustrated as servers and personal computers, but the principles of the present disclosure are not so limited. Embodiments of the present disclosure apply to laptop computers, tablet computers, smart phones, gaming consoles, media players and/or dedicated three-dimensional (3D) equipment. Network (150) represents any number of networks that transmit compressed point clouds between terminal devices (110) and (120). Network (150) may include, for example, wired (cable) and/or wireless communication networks. The network (150) may exchange data over circuit-switched and/or packet-switched channels. Typical networks include telecommunications networks, local area networks, wide area networks, and/or the Internet. For the purposes of this discussion, the architecture and topology of network (150) may not be critical to the operation of the present disclosure unless described herein below.

By way of example, FIG. 2 shows an application of the disclosed subject matter for point clouds. The disclosed subject matter is equally applicable to other point cloud supporting applications, including 3D telepresence applications, virtual reality.

Streaming system 200 may include a capture subsystem (213). The capture subsystem (213) uses a point cloud source (201), such as a light detection and ranging (LIDAR) system, a 3D camera, a 3D scanner, a graphics generation component that generates an uncompressed point cloud in software, such as a compressed It may also include a similar graphics generation component that generates a point cloud (202) that does not exist. In one example, the point cloud (202) includes points captured by a 3D camera. Compared to the compressed point cloud (204) (compressed point cloud bitstream), the point cloud (202) is drawn with a thicker line to emphasize the large amount of data. The compressed point cloud (204) may be generated by electronics (220) including an encoder (203) coupled to the point cloud source (201). Encoder (203) may include hardware, software, or a combination thereof to implement or implement aspects of the disclosed subject matter, as described in more detail below. Compressed point cloud (204) (or bitstream (204) of compressed point cloud (204)) drawn with thin lines to emphasize the small amount of data compared to stream of point cloud (202) , may be stored on the streaming server (205) for future use. One or more streaming client subsystems, such as the client subsystems (206) and (208) of FIG. 2, access the streaming server (205) to generate a copy (207) of the compressed point cloud (204). and (209) can be retrieved. The client subsystem (206) may include, for example, a decoder (210) within the electronic device (230). The decoder (210) decodes the incoming copy (207) of the compressed point cloud and produces an outgoing stream of reconstructed point cloud (211) that can be rendered on the rendering device (212). Some streaming systems may compress the compressed point clouds (204), (207) and (209) (eg, compressed point cloud bitstreams) according to a particular standard. In some examples, video coding standards are used for point cloud compression. Examples of these standards include High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), and others.

Note that electronics (220) and (230) may include other components (not shown). For example, electronics (220) may include a decoder (not shown) and electronics (230) may include an encoder (not shown).

FIG. 3 shows a block diagram of a V-PCC encoder (300) for encoding point cloud frames according to some embodiments. In some embodiments, the V-PCC encoder (300) may be used in the communication system (100) and the streaming system (200). For example, the encoder (203) can be constructed and operated in a manner similar to the V-PCC encoder (300).

The V-PCC encoder (300) receives uncompressed input point cloud frames and produces a bitstream corresponding to the compressed point cloud frames. In some embodiments, the V-PCC encoder (300) may receive point cloud frames from a point cloud source, such as, for example, the point cloud source (201).

In the example of FIG. 3, the V-PCC encoder (300) includes block generation module 306, block packing module 308, geometry image generation module 310, texture image generation module 312, coupled together as shown in FIG. block information module 304, occupancy map module 314, smoothing module 336, image padding modules 316 and 318, group expansion module 320, video compression modules 322, 323 and 332, auxiliary block information compression module 338, entropy compression module 334, and a multiplexer. 324 included.

According to one aspect of the present disclosure, the V-PCC encoder (300) provides an image-based representation of the 3D point cloud frame, and some metadata necessary to convert the compressed point cloud back to the decompressed point cloud. Convert to data (eg, occupancy map and block information). In some examples, the V-PCC encoder (300) converts 3D point cloud frames into geometry images, texture images, and occupancy maps, and then uses video coding techniques to convert geometry images, texture images , and the occupancy map can be encoded into the bitstream. In general, a geometry image is a 2D image with pixels padded with geometry values associated with points projected onto the pixels, and the pixels padded with geometry values are called geometry samples. A texture image is a 2D image with pixels padded with texture values associated with points projected onto the pixels, and the pixels padded with texture values are called texture samples. An occupancy map is a 2D image with pixels padded with values that indicate whether they are occupied by a block.

The block generation module (306) partitions the point cloud into a set of blocks (eg, a block is defined as a contiguous subset of the surface described by the point cloud), which may or may not be stacked. Each block can be described by a depth field for a plane in 2D space. In some embodiments, the block generation module (306) aims to minimize reconstruction errors while decomposing the point cloud into a minimum number of blocks with smooth boundaries.

A block information module (304) can collect block information indicating the size and shape of the block. In some examples, the block information may be packed into image frames and then encoded by the auxiliary block information compression module 338 to generate compressed auxiliary block information.

A block packing module 308 maps the extracted blocks to a two-dimensional (2D) grid while minimizing unused space, ensuring that each M x M (e.g., 16x16) block of the grid is associated with a unique block. are arranged to ensure Efficient block packing may directly impact compression efficiency by minimizing unused space or ensuring temporal consistency.

A geometry image generation module (310) can generate a 2D geometry image associated with the geometric shape of the point cloud at a given block location. A texture image generation module (312) can generate a 2D texture image associated with the texture of the point cloud at a given block location. Geometry image generation module 310 and texture image generation module (312) utilize the 3D to 2D mapping computed during the packing process to store the geometry shape and texture of the point cloud as an image. To better handle the case of projecting multiple points onto the same sample, each block is projected onto two images called layers. In the example, the geometry image is represented by a WxH monochromatic frame in YUV42Q-8 bit format. To generate a texture image, the texture generation process utilizes the reconstructed/smoothed geometry shape to compute the colors associated with the resampled points (also called color transfer).

Occupancy map module 314 can generate an occupancy map that describes padding information at each unit. For example, the occupancy map contains a binary map indicating for each cell of the grid whether this cell belongs to an empty space or to a point cloud. In one example, the occupancy map uses binary information for each pixel that describes whether this pixel is padded. In another example, the occupancy map uses binary information describing for each block of pixels whether the block of pixels is padded.

The occupancy map generated by the occupancy map module 314 can be compressed using lossless or lossy encoding. If lossless encoding is used, entropy compression module 334 is used to compress the occupancy map. If lossy encoding is used, the video compression module 332 is used to compress the occupancy map.

Note that the block packing module 308 may leave some empty space between 2D blocks packed into image frames. Image padding modules 316 and 318 can pad empty spaces (referred to as padding) to produce image frames that may be suitable for 2D video and image codecs. Image padding, also known as background filling, allows unused space to be padded with redundant information. In some instances, good background filling increases bitrate minimally, but does not introduce obvious coding distortion around block boundaries.

Video compression modules 322, 323 and 332 may encode 2D images such as padded geometry images, padded texture images, and occupancy maps based on a suitable video coding standard such as HEVC, VVC. can. In one example, the video compression modules 322, 323, and 332 are individual components that operate separately. Note that in another example, the video compression modules 322, 323, and 332 can be implemented as a single component.

In some examples, smoothing module 336 is arranged to generate a smoothed image of the reconstructed geometry image. The smoothed image information can be provided to texture image generator 312 . The texture image generator 312 can then adjust the texture image generation based on the reconstructed geometry image. For example, if the block shape (eg, geometry shape) is slightly distorted during encoding and decoding, the distortion can be taken into account to correct for block shape distortion when generating the texture image.

In some embodiments, group extension 320 is arranged to pad pixels around object boundaries with redundant low-frequency content to improve the coding gain and visual quality of the reconstructed point cloud. It is

A multiplexer 324 can multiplex the compressed geometry image, the compressed texture image, the compressed occupancy map, and the compressed auxiliary block information into the compressed bitstream.

FIG. 4 shows a block diagram of a V-PCC decoder (400) for decoding compressed bitstreams corresponding to point cloud frames according to some embodiments. In some embodiments, the V-PCC decoder (400) may be used in the communication system (100) and the streaming system (200). For example, the decoder (210) may be arranged and operate similarly to the V-PCC decoder (400). The V-PCC decoder (400) receives the compressed bitstream and produces a reconstructed point cloud based on the compressed bitstream.

In the example of FIG. 4, the V-PCC decoder (400) includes, as shown in FIG. ), an auxiliary block information decompression module (442), a geometry reconstruction module (444), a smoothing module (446), a texture reconstruction module (448) and a color smoothing module (452).

A demultiplexer (432) can receive the compressed bitstream and partition it into a compressed texture image, a compressed shape image, a compressed occupancy map, and compressed auxiliary block information.

Video decompression modules (434) and (436) can decode compressed images according to a suitable standard (eg, HEVC, VVC, etc.) and output decompressed images. For example, the video decompression module (434) decodes compressed texture images and outputs decompressed texture images, and the video decompression module (436) decodes compressed geometry images and outputs decompressed geometry images. can do.

The occupancy map decompression module (438) can decode the compressed occupancy map according to a suitable standard (eg, HEVC, VVC, etc.) and output the decompressed occupancy map.

Auxiliary block information decompression module (442) can decode auxiliary block information compressed according to a suitable standard (eg, HEVC, VVC, etc.) and output decompressed auxiliary block information.

A geometry reconstruction module (444) can receive the decompressed geometry image and generate a reconstructed point cloud geometry shape based on the decompressed occupancy map and the decompressed auxiliary block information.

A smoothing module (446) can smooth discrepancies at the edges of blocks. The smoothing process aims to mitigate potential discontinuities that can occur at block boundaries due to compression artifacts. In some embodiments, a smoothing filter can be applied to pixels located at block boundaries to reduce distortion that can be caused by compression/decompression.

A texture reconstruction module (448) can determine texture information for the points of the point cloud based on the decompressed texture image and the smoothed geometry shape.

A color smoothing module (452) can smooth out coloration discrepancies. Non-adjacent blocks in 3D space are often packed next to each other in 2D video. In some instances, pixel values from non-adjacent blocks can be confused by block-based video codecs. The purpose of color smoothing is to reduce visible artifacts that appear at block boundaries.

FIG. 5 shows a block diagram of a video decoder (510) according to an embodiment of the disclosure. The video decoder (510) can be used with the V-PCC decoder (400). For example, the video decompression modules (434) and (436), the occupancy map decompression module (438) can be similarly arranged as the video decoder (510).

The video decoder (510) may include a parser (520) for reconstructing symbols (521) from compressed images, eg, encoded video sequences. These symbol categories contain information for managing the operation of the video decoder (510). A parser (520) can perform parsing/entropy decoding on the received encoded video sequence. The encoding of the encoded video sequence can follow a video coding technique or standard and can follow various principles including variable length coding, Huffman coding, arithmetic coding with or without context dependence, etc. can. A parser (520) may extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. . A subgroup may include a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), and so on. The parser (520) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and the like.

A parser (520) can build symbols (521) by performing entropy decoding/parsing operations on the video sequence received from the buffer memory.

The reconstruction of the symbols (521) depends on the type of coded video picture or some coded video pictures (e.g., inter-pictures and intra-pictures, inter-blocks and intra-blocks) and other factors, It may also relate to a plurality of different units. Which units are involved and how they are controlled may be controlled by subgroup control information parsed from the encoded video sequence by the parser (520). For the sake of brevity, we do not describe such subgroup control information flow between the parser (520) and the following units.

In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into multiple functional units described below. In a practical implementation operating under commercial constraints, many of these units will interact closely with each other and can be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, conceptually it is appropriate to break it down into the following functional units.

The first unit is the scaler/inverse transform unit (551). The scaler/inverse transform unit (541) receives the quantized transform coefficients in symbols (521) and control information from the parser (520) and specifies the transform method to use, block size, quantized coefficients, quantized scaling matrix, etc. include. The scaler/inverse transform unit (551) may output blocks containing sample values, which may be input to the aggregator (555).

In some cases, the output samples of the scaler/inverse transform unit (551) are intra-coded blocks, i.e., they do not use prediction information from previously reconstructed pictures, but are reconstructed prior to the current picture. It may belong to a block that can use the prediction information from the predicted part. Such prediction information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) uses already-reconstructed surrounding information extracted from the current picture buffer (558) to predict the same size and shape as the block it is reconstructing. generates a block of A current picture buffer (558) buffers, for example, a partially reconstructed current picture and/or a fully reconstructed current picture. In some cases, the aggregator (555) adds the prediction information generated by the intra prediction unit (552) to the output sample information provided from the scaler/inverse transform unit (551) based on each sample.

In other cases, the output samples of the scaler/inverse transform unit (551) may belong to blocks that may be inter-coded and potentially motion compensated. In such cases, the motion compensated prediction unit (553) may access the reference picture memory (557) to obtain samples for prediction. After performing motion compensation on the obtained samples according to the symbols (521) belonging to the block, these samples are converted by an aggregator (555) to the output of the scaler/inverse transform unit (551) (in this case , called residual samples or residual signals) to produce output sample information. The address in the reference picture memory (457) from which the motion compensation unit (553) extracts the prediction samples may be controlled by a motion vector, said motion vector being sent to the motion compensation unit (553) in the form of symbols (521). A symbol (521) may, for example, comprise X, Y, and reference picture components. Motion compensation may include interpolation of sample values obtained from the reference picture memory (557) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, and the like.

The output samples of the aggregator (555) may be processed with various loop filtering techniques in a loop filter unit (556). The video compression technique may include an in-loop filter technique that includes symbols ( 521) is controlled by parameters available to the loop filter unit (556). However, it may also respond to meta-information obtained during decoding of earlier parts (in decoding order) of the encoded picture or encoded video sequence, or may be reconstructed earlier and loop filtered. It may be responsive to processed sample values.

The output of the loop filter unit (556) may be a sample stream, which can be output to the rendering device and stored in the reference picture memory (557) for use in future inter-picture prediction.

Once fully reconstructed, a particular coded picture can be used as a reference picture for future prediction. For example, the encoded picture corresponding to the current picture is completely reconstructed, and once the encoded picture is identified as a reference picture (eg, by the parser (520)), the current picture buffer (558 ) can become part of the reference picture memory (557) and can reallocate a new current picture buffer before starting reconstruction of subsequent coded pictures.

The video decoder (510) is, for example, ITU-T H. The decoding operation may be performed according to a standard, such as certain video compression techniques in the H.265 proposal. An encoded video sequence conforms to both the syntax of the video compression technology or standard and the profile recorded in the video compression technology or standard, in the sense that the encoded video sequence conforms to the video compression technology or standard being used. It can conform to the syntax specified in the standard. Specifically, a profile may select a tool from all available tools for a video compression technology or standard as a tool that may only be used with the profile. Compliance also requires that the complexity of the encoded video sequences be within limits set at the level of the video compression technology or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstructed sample rate (eg, measured in megasamples per second), maximum reference picture size, and the like. In some cases, the limit placed by the level may be further constrained by the specification of the Hypothetical Reference Decoder (HRD) and metadata managed by the HRD buffer signaled in the encoded video sequence.

FIG. 6 shows a block diagram of a video encoder (603) according to embodiments of the present disclosure. The video encoder (603) can be used for the V-PCC encoder (300) to compress the point cloud. In the example, video compression modules (322) and (323) and video compression module (332) are arranged similarly to encoder (603).

A video encoder (603) can receive images, such as padded geometry images, padded texture images, and produce compressed images.

According to an embodiment, the video encoder (603) converts the pictures of the source video sequence into an encoded video sequence (compressed image ) and compressed. Running at an appropriate encoding rate is one of the functions of the controller (650). In some embodiments, controller (650) controls and is operatively coupled to other functional units described below. Couplings are not illustrated for clarity. The parameters set by the controller (650) are rate control related parameters (picture skip, quantizer, λ value of rate-distortion optimization technique...), picture size, group of pictures (GOP) layout, maximum motion vector search. It may also include ranges and the like. Controller (650) may be arranged to have other suitable functions pertaining to video encoder (603) optimized for a particular system design.

In some embodiments, the video encoder (603) is arranged to operate in an encoding loop. As a very simplified explanation, in the example the encoding loop is responsible for creating a symbol like symbol stream based on the source encoder (630) (e.g. the input picture to be encoded and the reference picture ), which may include a (local) decoder (533) embedded in the video encoder (603). The decoder (633) reconstructs the symbols (compression between the symbols and the encoded video bitstream is not disclosed) to create samples similar to how the (remote) decoder creates sample data. (because it is lossless for the video compression techniques considered in the subject matter). Input the reconstructed sample stream (sample data) into the reference picture memory (634). The contents of the reference picture memory (634) are also bit-accurate between the local and remote encoders, since the decoding of the symbol stream results in bit-accurate regardless of the decoder's position (local or remote). Accurate. That is, the prediction part of the encoder sees as reference picture samples exactly the same sample values that the decoder "sees" when using prediction during decoding. This basic principle of reference picture synchrony (and the resulting drift if synchrony cannot be maintained, e.g. due to channel error) is also used in some related techniques.

The operation of the 'local' decoder (633) may be the same as that of a 'remote' decoder, such as the video decoder (510), and was described in detail above with reference to FIG. But also, referring briefly to FIG. 5, the symbols are available and the entropy encoder (645) and parser (520) losslessly encode/decode the symbols into an encoded video sequence. The entropy decoding portion of the video decoder (510), including the parser (520), may not be fully implemented in the local decoder (633) if it can be parsed.

In this case, it can be observed that any decoder technique other than analytic/entropy decoding present in the decoder is necessarily present in the corresponding encoder in essentially the same functional form. As such, the disclosed subject matter focuses on the operation of the decoder. The description of the encoder technique can be simplified because the encoder technique is the inverse of the generically described decoder technique. Further explanation is necessary only in certain areas and is provided below.

In operation, in some embodiments, the source encoder (630) may perform motion-compensated predictive encoding, using one or more previous pictures designated as "reference pictures" from a video sequence. Predictive coding is performed on the input picture by referring to the previously coded picture. In this way, the encoding engine (632) may encode differences between pixel blocks of an input picture and pixel blocks of a reference picture, which reference picture may be selected as a predictive reference to the input picture. .

A local video decoder (633) may decode encoded video data for pictures that may be designated as reference pictures based on the symbols produced by the source encoder (630). The operation of the encoding engine (632) may advantageously be a lossless process. If the encoded video data can be decoded by a video decoder (FIG. 5, not shown), the reconstructed video sequence will typically be a replica of the source video sequence with some errors. The local video decoder (633) may copy the decoding process that may be performed by the video decoder on the reference pictures and store the reconstructed reference pictures in the reference picture buffer (634). In this way, the encoder (603) can locally store replicas with common content of reconstructed reference pictures as reconstructed reference pictures obtained by remote video decoders (where the transmission error is do not have).

The predictor (635) can perform predictive searches for the encoding engine (632). That is, for a new picture to be encoded, the predictor (635) stores in the reference picture memory (634) sample data (referred to as candidate reference pixel blocks) that can be used as suitable prediction references for the new picture, or For example, specific metadata such as motion vectors of reference pictures, block shapes, etc. may be retrieved. The predictor (635) can operate pixel block by pixel block based on the sample block so that an appropriate prediction reference can be found. In some cases, the input picture is a prediction reference obtained from multiple reference pictures stored in the reference picture memory (634), e.g., as identified by the search results obtained by the predictor (635). may have

The controller (650) can manage encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters for encoding video data.

The outputs of all the above functional units may be entropy encoded in an entropy encoder (645). An entropy encoder (645) encodes the symbols by performing lossless compression on the symbols generated by each functional unit, eg, based on techniques such as Huffman coding, variable length coding, arithmetic coding. converted to a coded video sequence.

A controller (650) may manage the operation of the video encoder (603). The controller (650) may assign each coded picture a particular coded picture type that may affect the coding techniques that may be applied to the corresponding picture. For example, pictures can typically be assigned as one of the following picture types.

Intra pictures (I pictures) may be pictures that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video video codecs allow different types of intra pictures, including independent decoder refresh (“IDR”) pictures, for example. Those skilled in the art are aware of these variations and corresponding applications and features of I-pictures.

Predicted pictures (P-pictures) may be pictures that are coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index. to predict the sample values of each block.

A bi-predictive picture (B-picture) may be a picture that is encoded and decoded using intra-prediction or inter-prediction, which uses at most two motion vectors and a reference index. , to predict the sample values for each block. Similarly, multiple predicted pictures can reconstruct a single block using more than two reference pictures and associated metadata.

A source picture is typically spatially subdivided into multiple sample blocks (e.g., each being a 4x4, 8x8, 4x8 or 16x16 sample block) and coded block by block. may be A block can be predictively coded with reference to other (already coded) blocks specified by the corresponding picture coding assignments applied to the block. For example, blocks of an I picture may be coded non-predictively, or they may be coded predictively with reference to coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of P pictures may be predictively coded via spatial prediction or temporal prediction with reference to one previously coded reference picture. Blocks of B pictures may be predictively coded via spatial prediction or temporal prediction with reference to one or two previously coded reference pictures.

The video encoder (603) is, for example, ITU-T H. The encoding operations may be performed according to a predetermined video encoding technique or standard of the H.265 proposal. The video encoder (603) may perform various compression operations during its operation, including predictive encoding operations that take advantage of temporal and spatial redundancies in the input video sequence. The encoded video data may thus conform to the syntax specified by the video encoding technology or standard being used.

The video may be in the form of multiple source pictures (video) in chronological order. Intra-picture prediction (usually abbreviated to intra-prediction) exploits spatial relationships in a given picture, while inter-picture prediction exploits (temporal or other) relationships between pictures. In the example, a particular picture, called the current picture being encoded/decoded, is partitioned into blocks. A block in the current picture may be coded by a vector called a motion vector if the block in the current picture is similar to a reference block in a previously encoded and still buffered reference picture in the video. good. A motion vector points to a reference block in a reference picture, and when using multiple reference pictures, the motion vector may have a third dimension that identifies the reference picture.

In some embodiments, bi-prediction techniques may be used for inter-picture prediction. According to bi-predictive techniques, for example, the first and second reference pictures that precede the current picture in the video in decoding order (but can be in the past and future in display order, respectively). Use two reference pictures. A block in a current picture may be coded with a first motion vector pointing to a first reference block in a first reference picture and a second motion vector pointing to a second reference block in a second reference picture. . The block may be predicted by a combination of the first reference block and the second reference block.

Merge mode techniques can also be used for inter-picture prediction to improve coding efficiency.

According to some embodiments of the present disclosure, prediction, eg, inter-picture prediction and intra-picture prediction, is performed on a block-by-block basis. For example, depending on the HEVC standard, pictures in a video picture sequence are partitioned into compression coding tree units (CTUs), where a CTU in a picture is e.g. 64x64 pixels, 32x32 pixels or 16x16 pixels. have the same size of In general, a CTU contains three coding treeblocks (CTBs): one luma CTB and two chrominance CTBs. Each CTU may be recursively quadtree partitioned into one or more coding units (CUs). For example, a 64×64 pixel CTU may be partitioned into one 64×64 pixel CU, or four 32×32 pixel CUs, or 16 16×16 pixel CUs. In an example, each CU is analyzed to identify the prediction type used for that CU, eg, inter-prediction type or intra-prediction type. Depending on temporal and/or spatial predictability, a CU is partitioned into one or more prediction units (PUs). Each PU typically contains a luminance prediction block (PB) and two chrominance PBs. In an embodiment, prediction operations during encoding (encoding/decoding) are performed per prediction block. In examples where a luminance prediction block is used as the prediction block, the prediction block includes a matrix of pixel values (e.g., luminance values), e.g., 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc. .

According to some aspects of this disclosure, geometry smoothing may be performed by both the encoder side (used for point cloud compression) and the decoder side (used for point cloud reconstruction). In one example, at the encoder side, after compressing the geometry video, the compressed geometry video and the corresponding occupancy map are used to reconstruct the geometry part of the point cloud, and the reconstructed point cloud (geometry part) is the geometry Called Rebuild Cloud. Geometry reconstruction clouds are used to generate texture images. For example, the texture image generator 312 can identify the colors associated with the resampled points in the geometry reconstruction cloud (also called color shift) and generate the texture image accordingly.

In some examples, geometry smoothing is applied to the geometry reconstruction cloud before color transfer. For example, the smoothing module 336 can apply smoothing (eg, a smoothing filter) to the geometry reconstruction cloud generated based on the reconstructed geometry image. In some embodiments of the present disclosure, the smoothing module 336 is arranged to not only recover shape distortions at block boundaries, but also to recover shape distortions within blocks.

On the decoder side, using the V-PCC decoder 400 of FIG. 4, as an example, the smoothing module 446 can apply smoothing to the geometry reconstruction cloud to produce a smoothed geometry reconstruction cloud. . A texture reconstruction module 448 can then identify texture information for points within the point cloud based on the decompressed texture image and the smoothed geometry reconstruction cloud.

According to some aspects of this disclosure, distortion may be caused by quantization errors during geometry compression and/or during conversion from high resolution occupancy maps to low resolution maps. Quantization errors can affect block boundaries, which can affect reconstructed depth values (point geometry information) within blocks, and that reconstructed planes are not smooth may lead to This disclosure provides techniques for smoothing reconstructed depth values within blocks.

The proposed methods can be used separately or combined in any order. Further, each of the methods (or embodiments), encoders and decoders may be implemented by processing circuitry (eg, one or more processors or one or more integrated circuits), in one example one or more processors is stored on a non-transitory computer-readable medium.

FIG. 7 shows a geometry image 710 and a texture image 750 used for the point cloud. The point cloud is decomposed into blocks. In some related examples, smoothing is applied only to block boundaries, such as the boundaries indicated by 711 in FIG. In this disclosure, smoothing can be applied at specific locations within a block, as indicated by 721 . Locations can be selected based on certain criteria. Additional computational complexity is minimized because smoothing is applied selectively within blocks. In some embodiments, the candidate points whose reconstructed depth values differ the most compared to the uncompressed depth values can be identified and added to the list. The list can also include boundary points. Smoothing can then be applied to the points in the list by smoothing module 336, smoothing module 446, or the like.

In some embodiments, a set of candidate points within a block to be smoothed by a smoothing filter can be obtained based on reconstructed depth values. In some embodiments, suitable algorithms are used on both the encoder and decoder sides to select candidate points whose reconstructed depth values are most different from the original uncompressed values, e.g. The uncompressed value of is not available at the encoder side. In some examples, candidate points are selected for which the reconstructed depth values are believed to have relatively large quantization errors. In one example, regions with high frequency components (high spatial frequency components) in the depth map (eg, reconstructed geometry image) may be selected. For example, if the ratio of the intensity of high spatial frequency components to the intensity of low spatial frequency components in a region is higher than a threshold, then the region is a high frequency region with relatively high levels of high spatial frequency components, and the smoothing filter may be selected to apply the In another example, regions with high motion content of the depth map (eg, reconstructed geometry image) can be selected. For example, regions can be selected based on motion vector information commonly used in video codecs.

In some embodiments, edge detection can be applied to a depth map (e.g., a reconstructed geometry image) to identify points corresponding to edges within blocks, and points corresponding to edges within blocks. Smoothing can be applied to the points. In general, edge regions have relatively high spatial frequency content.

In some embodiments, candidate points may be obtained based on information implicitly provided by the video compression tool (eg, HEVC) used by V-PCC to compress/decompress the depth map. In one example, pixels with large motion vectors can be selected, and points corresponding to pixels with large motion vectors can be selected as candidate points and added to a list for applying smoothing. In another example, pixels with a relatively large response to sample adaptive offset (SAO) can be selected, points corresponding to pixels with a large response to SAO are selected as candidate points, and smoothing is applied to the list can be added to

According to some aspects of this disclosure, the encoder side and decoder side use the same algorithm to identify points (or regions) to apply smoothing within a block. In some embodiments, flags and parameters can be included in the encoded bitstream so that the decoder side specifies algorithms and parameters for selecting points within the block at which the encoder applies smoothing. , and then the decoder side can use the same algorithm and parameters to select points to apply smoothing within the block.

FIG. 8 illustrates example syntax according to some embodiments of the present disclosure. In the example of FIG. 8, selective_smoothing_inside_patches_present_flag is used to indicate whether selective smoothing is used within the block. In the example, if selective_smoothing_inside_patches_present_flag is true, then direct the algorithm with the parameters indicated, for example, by algorithm_to_find_candidates_inside_patches.

Also in the example, if the algorithm is an edge detection algorithm, the parameters used for the edge detection algorithm, e.g., the size of the kernel of the edge detection algorithm indicated by kernel_size, kernel[i], i =0...kernel_size*kernel_size can indicate a value in the kernel, and so on.

In FIG. 8, XYZ indicates another suitable algorithm for selecting candidate points to apply smoothing within the block, and XYZ_parameters indicates the parameter values used for algorithm XYZ.

FIG. 9 shows a flowchart outlining a process (900) according to an embodiment of the present disclosure. The process (900) can be used to encode the point cloud during the encoding process. In various embodiments, the process (900) performs the functions of the processing circuitry, e.g., the processing circuitry in the terminal device (110), the encoder (203), the V-PCC encoder (300). Executed by a processing circuit or the like. In some embodiments, the process (900) is implemented by software instructions such that when the processing circuitry executes the software instructions, the processing circuitry performs the process (900). The process starts at (S901) and proceeds to (S910).

At (S910), the geometry image associated with the point cloud is compressed. In an example, block generation module 306 can generate blocks of point clouds. The geometry image generation module 310 also stores geometry information, such as depth values of points, as a geometry image. Video compression module 322 can compress the geometry image associated with the group.

At (S920), generate a geometry reconstruction cloud according to the compressed geometry image. In an example, the video compression module 322 can generate a reconstructed geometry image in response to the compressed geometry image. A reconstructed geometry image can be used to generate a geometry reconstruction cloud.

At (S930), a smoothing filter is applied to at least the geometry samples within the blocks in addition to the boundary samples of the blocks of the geometry reconstruction cloud. In some examples, the smoothing module 336 can apply a smoothing filter to the boundary points of the blocks. A smoothing module 336 also selectively applies a smoothing filter to some points within the block. In some embodiments, the points at which the estimated reconstructed depth values are most likely to differ from the original uncompressed values can be selected. For example, points with high levels of high spatial frequency content within the region can be selected. In another example, points within the depth map that have high motion content (eg, identified based on motion vector information provided by video compression module 322) may be selected.

At (S940), a texture image is generated based on the smoothed geometry reconstruction cloud. In an example, the texture image generation module 312 can identify colors (also called color shifts) to associate with resampled points in the smoothed geometry reconstruction cloud and generate texture images accordingly.

At (S950), the texture image is compressed. In an example, the video compression module 323 can generate compressed texture images. The compressed geometry image, compressed texture image and other appropriate information are then multiplexed to produce an encoded bitstream. In some examples, flags and parameters associated with selective geometry smoothing within blocks can be included in the encoded bitstream. The process then proceeds to (S999) and terminates.

FIG. 10 shows a flowchart outlining a process (1000) according to an embodiment of the present disclosure. The point cloud can be reconstructed using the process (1000) during the decoding process. In various embodiments, the process (1000) performs the functions of the processing circuitry, e.g., the processing circuitry in the terminal device (120), the processing circuitry performing the functions of the decoder (210), the V-PCC decoder (400). Executed by a processing circuit or the like. In some embodiments, the process (1000) is implemented by software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry performs the process (1000). The process starts at (S1001) and proceeds to (S1010).

In (S1010), the prediction information of the point cloud is decoded from the encoded bitstream corresponding to the point cloud. In some examples, the prediction information includes flags and parameters associated with selective geometry smoothing within blocks.

At (S1020), generate a geometry reconstruction cloud based on the geometry image decoded from the encoded bitstream. In an example, the video decompression module 436 can decode the geometry information to generate a decompressed geometry image. A geometry reconstruction module 444 can generate a geometry reconstruction cloud based on the decompressed geometry image.

At (S1030), a smoothing filter is applied to at least the geometry samples within the blocks in addition to the boundary samples of the blocks of the geometry reconstruction cloud. In some examples, the smoothing module 446 may apply a smoothing filter to the geometry samples of the boundary points of the block. A smoothing module 446 also selectively applies a smoothing filter to some geometry samples within the block. In some embodiments, the points at which the estimated reconstructed depth values are most likely to differ from the original uncompressed values can be selected. For example, points with high levels of high spatial frequency content within the region can be selected. In another example, points within the depth map that have high motion content (eg, identified based on motion vector information provided by the video decompression module 436) may be selected.

At (S1040), the points of the point cloud are reconstructed based on the smoothed geometry reconstruction cloud. For example, the texture reconstruction module (448) can determine texture information for the points of the point cloud based on the decompressed texture image and the smoothed geometry reconstruction cloud. A color smoothing module (452) can then smooth the coloration discrepancies. The process then proceeds to (S1099) and terminates.

The techniques described above are implemented as computer software by computer readable instructions and physically stored on one or more computer readable media. For example, FIG. 11 illustrates a computer system (1100) suitable for implementing some embodiments of the disclosed subject matter.

Computer software can be encoded using any suitable machine code or computer language, which can be assemble, compile, link, or otherwise produce code containing instructions. , such instructions may be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or by interpretation, microcode execution, or the like.

The instructions may be executed on various types of computers or components thereof including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, and the like.

The components associated with the computer system (1100) shown in FIG. 11 are exemplary in nature and are not intended to limit the scope of use or functionality of the computer software for implementing embodiments of the present disclosure. The arrangement of components should not be interpreted as having any dependency or requirement relating to any or combination of components illustrated in the exemplary embodiment of computer system (1100).

Computer system (1100) may include a number of human-machine interface input devices. Such human-machine interface input devices include, for example, tactile input (e.g. keystrokes, swipes, movement of data gloves), audio input (e.g. voice, clapping), visual input (e.g. posture), olfactory input (e.g. (not shown). Human-machine interface devices include, for example, audio (e.g., voice, music, ambient sounds), pictures (e.g., scanned images, photographic images obtained from static image capture devices), video (e.g., two-dimensional video, stereo It may also be used to capture certain media that are not necessarily directly related to human conscious input, such as 3D motion pictures (including motion pictures).

Human-machine interface input devices include keyboard (1101), mouse (1102), trackpad (1103), touch screen (1110), data glove (not shown), joystick (1105), microphone (1106), scanner ( 1107), may include one or more of the cameras (1108) (only one of each is shown).

Computer system (1100) may also include a number of human-machine interface output devices. Such human-machine interface output devices are capable of stimulating one or more of the human user's senses through, for example, haptic output, sound, light, and smell/taste. Such human-machine interface output devices include haptic output devices such as touch panel (1110), data glove (not shown), or joystick (1105) haptic feedback devices that do not function as input devices. ), audio output devices (e.g. speakers (1109), headphones (not shown)), visual output devices (e.g. screen (1110), CRT screens, LCD screens, plasma screens, OLED screens , each with or without touchscreen input capability and with or without haptic feedback capability, some of which are, for example, stereo image output, virtual reality glasses (not shown), Holo 2D visual output or 3D or higher dimensional output via a graphic display and smoke tank (not shown), and a printer (not shown).

Computer system 1100 may further include human-accessible storage devices and their associated media, such as CD/DVD ROM/RW (1120) having or similar to media (1121). Optical media, thumb drives (1122), removable hard or solid state drives (1123), legacy magnetic media (e.g. magnetic tapes and floppy disks (not shown)), dedicated ROM/ASIC/PLD based devices (e.g. security dongle (not shown)) and the like.

Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not include transmission media, carrier waves, or other transitory signals. be.

Computer system (1100) may further include network interfaces to one or more communication networks. Networks can be, for example, wireless, wired, or optical. Networks may also be local area networks, wide area networks, metropolitan networks, vehicle and industrial networks, real-time networks, delay tolerant networks, and the like. Examples of networks include, for example, local area networks (e.g., Ethernet, wireless LAN), cellular networks including GSM, 3G, 4G, 5G, LTE, etc., wired or wireless including cable TV, satellite TV, and broadcast TV. Wide area digital networks, vehicle and industrial networks including CANBus, and the like. A specific network usually needs to be connected to a specific general purpose data port or an external network interface adapter on a peripheral bus (1149) (eg, a computer system's USB port). Other networks are typically integrated into the core of the computer system (1100) by connecting to the system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). be. Computer system (1100) can use any of these networks to communicate with other entities. Such communication may be unidirectional receive only (e.g., television broadcast), unidirectional transmit only (e.g., onto a CAN bus to some CAN bus device), or bidirectional (e.g., local or wide area digital network). to other computer systems using ). Specific protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

The human-machine interface devices, human-accessible storage devices, and network interfaces mentioned above can be attached to the core (1140) of the computer system (1100).

The core (1140) contains dedicated programmable processing in the form of one or more Central Processing Units (CPUs) (1141), Graphics Processing Units (GPUs) (1142), Field Programmable Gate Arrays (FPGAs) (1143). Units, such as hardware accelerators (1144) used for specific tasks. These devices, read only memory (ROM) (1145), random access memory (RAM) (1146), e.g. internal hard drives that are not user accessible, internal mass storage such as SSDs (1147) connect to the system bus (1148). may be connected via In some computer systems, the system bus (1148) may be accessed in the form of one or more physical plugs to allow for expansion with additional CPUs, GPUs, and the like. Peripherals can be connected to the core's system bus (1148) either directly or through a peripheral bus (1149). Peripheral bus architectures include PCI, USB, and the like.

CPU (1141), GPU (1142), FPGA (1143), and accelerator (1144) may execute specific instructions that, in combination, may constitute the computer code described above. The computer code may be stored in ROM (1145) or RAM (1146). Temporary data may also be stored in RAM (1146) and permanent data may be stored, for example, in internal mass storage (1147). A buffer memory can provide fast storage and retrieval to any of the storage devices, and may be used by one or more CPUs (1141), GPUs (1142), mass storage devices (1147). ), ROM (1145), RAM (1146), etc.

The computer-readable medium may have computer code for performing various computer-implemented operations. The media and computer code may be media and computer code specially designed and constructed for the purposes of the present disclosure, or they may be of the type known and available to those skilled in the art of computer software. may be of

By way of example and not limitation, a computer system (1100) having an architecture, particularly a core (1040), is a processor(s) (including CPUs, GPUs, FPGAs, accelerators, etc.) that is connected to one or more tangible computers. Functionality provided as a result of executing software embodied in a readable medium may be provided. Such computer-readable media may include user-accessible mass storage as described above, and storage having some non-transitory nature of core (1140), such as core internal mass storage (1147) or It may be a medium associated with ROM (1145). Software implementing various embodiments of the present disclosure may be stored in such devices and executed by core (1140). A computer-readable medium may include one or more memories or chips, depending on particular needs. The software causes the cores (1140), and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute particular processes or particular portions of particular processes described herein, and defining data structures to be stored in RAM (1146) by the process, and modifying such data structures. Additionally or alternatively, the computer system may provide functionality provided by logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1144)), where the circuitry is implemented in software. Alternatively, or in conjunction with software, certain processes or certain portions of certain processes described herein can be performed.Where appropriate, references to software include logic and, conversely, logic. References to may include software References to computer readable media may include circuits (e.g., integrated circuits (ICs), etc.) that store executable software, circuits that implement executable logic, as appropriate. This disclosure encompasses any suitable combination of hardware and software.
Appendix A: Acronyms JEM: Joint Exploration Model VVC: Versatile Video Coding BMS: Benchmark Set MV: Motion Vectors HEVC: High Efficiency Video Coding SEI: Auxiliary Extension Information VUI: Video Usability Information GOP: Group of Pictures TU: Transform Unit PU: prediction unit CTU: coding tree unit CTB: coding tree region PB: prediction block HRD: virtual reference decoder SNR: signal to noise ratio CPU: central processing unit GPU: graphics processing unit CRT: cathode ray tube LCD: liquid crystal display OLED: Organic Light Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read Only Memory RAM: Random Access Memory ASIC: Application Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Network GSM: Global System of Mobile Communications LTE: Long Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Array SSD: Solid State Drive IC: Integrated Circuit CU: Coding Unit Having described exemplary embodiments, there are alterations, permutations, and various alternative equivalents that fall within the scope of this disclosure. Accordingly, one skilled in the art could devise numerous systems and methods not expressly shown or described herein, but which embody the teachings of the present disclosure and thus fall within the spirit and scope of the present disclosure. recognized that it can be done.

Claims (17)

A method for point cloud decompression, comprising:
a processor decoding the point cloud prediction information from the encoded bitstream;
the processor reconstructing a geometry reconstruction cloud based on a geometry image of the point cloud decoded from the encoded bitstream;
said processor applying a filter to at least geometry samples within said blocks in addition to block boundary samples of said geometry reconstruction cloud to produce a smoothed geometry reconstruction cloud;
said processor reconstructing points of said point cloud based on said smoothed geometry reconstruction cloud. 2. The method of claim 1, further comprising the step of selecting regions of high frequency content within the block having levels higher than a threshold level. 2. The method of claim 1, further comprising the processor selecting regions of motion content having a level higher than a threshold level within the block. 3. The method of claim 2, further comprising the processor detecting edges within the block based on depth values of the geometry reconstruction cloud. 4. The method of claim 3, further comprising the processor selecting points within the block based on motion information of corresponding pixels within the geometry image. A method according to any one of claims 1 to 5, wherein said prediction information includes a flag indicating to apply selective smoothing within blocks of said point cloud. 7. The method of claim 6, wherein said prediction information dictates a particular algorithm for selecting points within said block. 8. The method of claim 7, wherein the predictive information includes parameters for use in the particular algorithm. A method for point cloud compression, comprising:
a processor compressing the geometric image associated with the point cloud;
said processor reconstructing a geometry reconstruction cloud based on a compressed point cloud geometry image;
said processor applying a filter to at least geometry samples within said blocks in addition to block boundary samples of said geometry reconstruction cloud to produce a smoothed geometry reconstruction cloud;
said processor generating a texture image of said point cloud based on said smoothed geometry reconstruction cloud. 10. The method of claim 9, further comprising the processor selecting regions of high frequency content having levels above a threshold level within the block. 10. The method of claim 9, further comprising the processor selecting regions of motion content having a level higher than a threshold level within the block. 11. The method of claim 10, further comprising the processor detecting edges within the block based on depth values of the geometry reconstruction cloud. 12. The method of claim 11, further comprising the processor selecting points within the block based on motion information of corresponding pixels within the geometry image. A method according to any one of claims 9 to 13, wherein the compressed point cloud coded bitstream includes a flag indicating to apply selective smoothing within blocks of the point cloud. 15. The method of claim 14, wherein the compressed point cloud coded bitstream includes an indicator that indicates a particular algorithm for selecting points within the block to which the selective smoothing is applied. . Apparatus comprising processing circuitry for performing the method of any one of claims 1-8. Apparatus comprising processing circuitry for performing the method of any one of claims 9-15.

Images