WO2023277128A1 - Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme - Google Patents

Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme Download PDF

Info

Publication number
WO2023277128A1
WO2023277128A1 PCT/JP2022/026209 JP2022026209W WO2023277128A1 WO 2023277128 A1 WO2023277128 A1 WO 2023277128A1 JP 2022026209 W JP2022026209 W JP 2022026209W WO 2023277128 A1 WO2023277128 A1 WO 2023277128A1
Authority
WO
WIPO (PCT)
Prior art keywords
unit
point cloud
trisoup
point
upsampling
Prior art date
Application number
PCT/JP2022/026209
Other languages
English (en)
Japanese (ja)
Inventor
恭平 海野
圭 河村
Original Assignee
Kddi株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kddi株式会社 filed Critical Kddi株式会社
Publication of WO2023277128A1 publication Critical patent/WO2023277128A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present invention relates to a point cloud decoding device, a point cloud decoding method, and a program.
  • Non-Patent Document 1 discloses an inter-prediction technique for geometric information.
  • Non-Patent Document 1 based on the distribution of points in the motion-compensated reference point group, the presence or absence of points in each subspace (node) of the point group to be encoded is estimated. At this time, especially when both the reference point group and the encoding target point group are sparse, there is a problem that if there is even a slight deviation in the result of motion compensation, the correlation in the time direction cannot be fully utilized.
  • the present invention has been made in view of the above-mentioned problems, and a point cloud decoding device and a point cloud decoding method that can increase the encoding efficiency by inter prediction of position information by upsampling the reference point cloud. and to provide programs.
  • a first feature of the present invention is a point cloud decoding device, a geometric information reconstruction unit configured to reconstruct position information of the point cloud, and a geometric information reconstructed by the geometric information reconstruction unit an upsampling unit configured to upsample a point cloud; and a tree synthesis unit configured to perform inter prediction with reference to the upsampled point cloud, the upsampling unit is configured to calculate an offset vector from the coordinates of each point in the reconstructed point group, and perform upsampling using the offset vector.
  • a second feature of the present invention is a point cloud decoding device comprising a geometric information decoding unit configured to decode a first flag that controls whether or not Trisoup at multiple levels is permitted. is the gist.
  • a third feature of the present invention is a point group decoding method, which includes a geometric information reconstruction unit configured to reconstruct position information of the point cloud, and A step A of upsampling a point cloud, and a step B of performing inter prediction with reference to the upsampled point cloud, wherein in the step A, from the coordinates of each point in the reconstructed point cloud
  • the gist is to calculate an offset vector and perform upsampling using the offset vector.
  • a fourth feature of the present invention is a program for causing a computer to function as a point cloud decoding device, wherein the point cloud decoding device is configured to reconstruct position information of a point cloud for geometric information reconstruction.
  • an upsampling unit configured to upsample the point cloud reconstructed by the geometric information reconstruction unit; and an inter prediction with reference to the upsampled point cloud.
  • a tree synthesizing unit configured to calculate an offset vector from the coordinates of each point in the reconstructed point cloud, and perform upsampling using the offset vector.
  • a point cloud decoding device it is possible to provide a point cloud decoding device, a point cloud decoding method, and a program that can improve the encoding efficiency of position information based on inter prediction by upsampling the reference point cloud.
  • FIG. 1 is a diagram showing an example of the configuration of a point cloud processing system 10 according to one embodiment.
  • FIG. 2 is a diagram showing an example of functional blocks of the point cloud decoding device 200 according to one embodiment.
  • FIG. 3 is a diagram showing an example of the configuration of encoded data (bitstream) received by the geometric information decoding unit 2010 of the point group decoding device 200 according to one embodiment.
  • FIG. 4 is a diagram showing an example of the syntax configuration of GPS2011.
  • FIG. 5 is a flowchart showing an example of upsampling in the upsampling unit 2120 of the point cloud decoding device 200 according to one embodiment.
  • FIG. 6A is a diagram for explaining the operation of the upsampling unit 2120 of the point cloud decoding device 200 according to one embodiment.
  • FIG. 6B is a diagram for explaining the operation of the upsampling unit 2120 of the point cloud decoding device 200 according to one embodiment.
  • FIG. 7 is a flowchart showing an example of processing in the tree synthesizing unit 2020 of the point cloud decoding device 200 according to one embodiment.
  • FIG. 8 is a flowchart showing an example of decoding processing of Trisoup information by the tree synthesizing unit 2020 of the point cloud decoding device 200 according to one embodiment.
  • FIG. 9 is a diagram showing an example of functional blocks of the point cloud encoding device 100 according to one embodiment.
  • FIG. 10 is a diagram showing an example of functional blocks of the tree synthesizing unit 2020 of the point cloud decoding device 200 according to one embodiment.
  • FIG. 1 is a diagram showing a point cloud processing system 10 according to an embodiment according to this embodiment.
  • the point cloud processing system 10 has a point cloud encoding device 100 and a point cloud decoding device 200.
  • the point cloud encoding device 100 is configured to generate encoded data (bitstream) by encoding an input point cloud signal.
  • the point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bitstream.
  • the input point cloud signal and the output point cloud signal are composed of position information and attribute information of each point in the point cloud.
  • the attribute information is, for example, color information and reflectance of each point.
  • such a bitstream may be transmitted from the point cloud encoding device 100 to the point cloud decoding device 200 via a transmission channel.
  • the bitstream may be stored in a storage medium and then provided from the point cloud encoding device 100 to the point cloud decoding device 200 .
  • FIG. 2 is a diagram showing an example of functional blocks of the point group decoding device 200 according to this embodiment.
  • the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute It has information decoding section 2060 , inverse quantization section 2070 , RAHT section 2080 , LoD calculation section 2090 , inverse lifting section 2100 , inverse color conversion section 2110 , upsampling section 2120 and frame buffer 2130 .
  • the geometric information decoding unit 2010 is configured to receive as input a bitstream related to geometric information (geometric information bitstream) among the bitstreams output from the point cloud encoding device 100 and decode the syntax.
  • the decoding process is, for example, context adaptive binary arithmetic decoding process.
  • the syntax includes control data (flags and parameters) for controlling decoding processing of position information.
  • the tree synthesizing unit 2020 receives the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating in which node in the tree described later the point group exists, and determines in which region in the decoding target space the point is located. It is configured to generate tree information that exists.
  • the decoding process of the occupancy code may be configured to be performed inside the tree synthesizing unit 2020 .
  • This process divides the decoding target space into rectangular parallelepipeds, refers to the occupancy code to determine whether points exist in each rectangular parallelepiped, divides the rectangular parallelepiped in which the points exist into multiple rectangular parallelepipeds, and refers to the occupancy code.
  • Tree information can be generated by recursively repeating the process.
  • inter prediction which will be described later, may be used when decoding such an occupancy code.
  • a method called "Octree” that recursively divides the octree with the above-mentioned rectangular parallelepiped always as a cube, and "QtBt” that performs quadtree division and binary tree division in addition to octree division can be used. Whether or not to use "QtBt" is transmitted from the point group encoding device 100 side as control data.
  • the tree synthesis unit 2020 is configured to decode the coordinates of each point based on any tree configuration determined in the point cloud encoding device 100. ing.
  • the approximate surface synthesizing unit 2030 is configured to generate approximate surface information using the tree information generated by the tree synthesizing unit 2020, and decode the point group based on the approximate surface information.
  • the approximate surface information is obtained by decoding the points instead of decoding individual point clouds. It is an approximation of the existence area of the group with a small plane.
  • the approximate surface synthesizing unit 2030 can generate approximate surface information and decode the point group by, for example, a technique called "Trisoup". A specific processing example of "Trisoup” will be described later. Also, when decoding a sparse point group acquired by Lidar or the like, this process can be omitted.
  • the geometric information reconstruction unit 2040 reconstructs geometric information (decoding process position information in the coordinate system assumed by ).
  • the inverse coordinate transformation unit 2050 receives as input the geometric information reconstructed by the geometric information reconstruction unit 2040, transforms the coordinate system assumed by the decoding process into the coordinate system of the output point group signal, and converts the position information. configured to output.
  • the upsampling unit 2120 is configured to receive the geometric information reconstructed by the geometric information reconstructing unit 2040 and perform upsampling processing as described later.
  • the frame buffer 2130 is configured to receive the point cloud upsampled by the upsampling unit 2120 and store it as a reference frame.
  • the stored reference frames are read from the frame buffer 2130 and used as reference frames when the tree synthesis unit 2020 inter-predicts temporally different frames.
  • whether or not to use a reference frame at which time for each frame may be determined, for example, based on control data transmitted as a bitstream from the point group encoding device 100.
  • the geometric information output from the geometric information reconstruction unit 2040 is stored in the frame buffer 2130 as a reference frame, and then the reference frame stored in the frame buffer 2130 is used when inter prediction is performed by the tree synthesis unit 2020. may be up-sampled by the up-sampling unit 2120 and then input to the tree synthesis unit 2020 .
  • the upsampling unit 2120 may be configured to be included inside the tree synthesis unit 2020, as shown in FIG.
  • the configuration example shown in FIG. 10 will be described below.
  • the tree synthesizing unit 2020 acquires from the frame buffer 2130 an already-decoded point cloud that is different in time from the point cloud to be decoded, and inputs it to the motion compensation unit 2021 .
  • the motion compensation unit 2021 is configured to perform motion compensation using motion information (motion vectors, etc.) decoded from control data. Since a known method can be applied to a specific motion compensation method, details thereof are omitted.
  • the motion compensation unit 2021 may be configured to perform class classification processing on each point in the decoded point group and apply different motion compensation processing to each class.
  • the motion compensation unit 2021 may be configured to assign a class of either "ground” or "object” to each point, and apply motion compensation only to points considered to be "object”.
  • the upsampling unit 2021 is configured to input the motion-compensated point group and perform upsampling processing as described later.
  • the upsampling unit 2021 may be configured to perform upsampling processing only for a specific class.
  • the upsampling unit 2021 may be configured to perform upsampling processing only on the "ground” points, or to perform upsampling processing only on the "object” points.
  • the upsampling unit 2021 when the above-described class classification is performed based on the value of the z-axis coordinate (height component) of each point, the upsampling unit 2021 performs motion compensation based on the z-coordinate of each point in the point group. Alternatively, it may be configured to control whether or not to execute the upsampling process.
  • the sampler 2021 may be configured to perform upsampling.
  • the upsampler 2021 may be configured to perform upsampling.
  • the second threshold is a value greater than the first threshold.
  • the probability estimation unit 2022 is configured to estimate the probability that the value of each bit of the occurrence code is 0 or that the value of each bit of the occurrence code is 1, using the point group after upsampling as input. ing.
  • the occupation information decoding unit 2023 is configured to decode the occupation code by entropy decoding based on the probability estimated by the probability estimation unit 2022 .
  • the tree configuration unit 2024 is configured to configure a tree based on the above-mentioned Occupancy Code value.
  • the attribute information decoding unit 2060 is configured to receive a bitstream related to attribute information (attribute information bitstream) among the bitstreams output from the point cloud encoding device 100 and decode the syntax.
  • the decoding process is, for example, context adaptive binary arithmetic decoding process.
  • the syntax includes control data (flags and parameters) for controlling attribute information decoding processing.
  • the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.
  • the inverse quantization unit 2070 performs inverse quantization based on the quantized residual information decoded by the attribute information decoding unit 2060 and the quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060. and configured to generate dequantized residual information.
  • the inverse quantized residual information is output to either the RAHT section 2080 or the LoD calculation section 2090 according to the features of the point cloud to be decoded. Which one is output is specified by control data decoded by attribute information decoding section 2060 .
  • the RAHT unit 2080 receives as input the inverse quantized residual information generated by the inverse quantization unit 2070 and the geometric information generated by the geometric information reconstruction unit 2040, and performs a Haar transform called RAHT (Region Adaptive Hierarchical Transform) ( In the decoding process, a kind of inverse Haar transform is used to decode the attribute information of each point.
  • RAHT Region Adaptive Hierarchical Transform
  • a kind of inverse Haar transform is used to decode the attribute information of each point.
  • the LoD calculation unit 2090 is configured to receive the geometric information generated by the geometric information reconstruction unit 2040 and generate LoD (Level of Detail).
  • LoD is a reference relationship (referred point and referenced point ) is information for defining
  • LoD classifies each point included in geometric information into a plurality of levels, and encodes or decodes the attributes of points belonging to lower levels using attribute information of points belonging to higher levels. This is information that defines the structure.
  • Non-Patent Document 1 As a specific method for determining LoD, for example, the method described in Non-Patent Document 1 may be used.
  • the inverse lifting unit 2100 uses the LoD generated by the LoD calculation unit 2090 and the inverse quantized residual information generated by the inverse quantization unit 2070 to determine the attribute information of each point based on the hierarchical structure defined by the LoD. is configured to decode the As a specific reverse lifting process, for example, the method described in Non-Patent Document 1 can be used.
  • the inverse color conversion unit 2110 uses the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100 when the attribute information to be decoded is color information and color conversion has been performed on the side of the point cloud encoding device 100. is configured to perform inverse color conversion processing. Whether or not to execute the inverse color conversion process is determined by the control data decoded by the attribute information decoding unit 2060 .
  • the point cloud decoding device 200 is configured to decode and output the attribute information of each point in the point cloud through the above processing.
  • FIG. 3 is an example of the structure of encoded data (bitstream) received by the geometric information decoding unit 2010.
  • FIG. 3 is an example of the structure of encoded data (bitstream) received by the geometric information decoding unit 2010.
  • the bitstream may contain GPS2011.
  • GPS2011 is also called a geometry parameter set, and is a set of control data related to decoding of geometric information. A specific example will be described later.
  • Each GPS 2011 includes at least GPS id information for individual identification when multiple GPS 2011 exist.
  • the bitstream may contain GSH2012A/2012B.
  • GSH2012A/2012B is also called geometry slice header or geometry data unit header, and is a set of control data corresponding to slices described later.
  • the term “slice” will be used, but the term “slice” can also be read as a data unit. A specific example will be described later.
  • GSH 2012A/2012B includes at least GPS id information for designating GPS 2011 corresponding to each GSH 2012A/2012B.
  • the bitstream may contain slice data 2013A/2013B next to GSH 2012A/2012B.
  • the slice data 2013A/2013B includes data obtained by encoding geometric information.
  • An example of the slice data 2013A/2013B is an occupancy code, which will be described later.
  • the bitstream has a configuration in which one GSH 2012A/2012B and one GPS 2011 correspond to each slice data 2013A/2013B.
  • GSH2012A/2012B which GPS 2011 to refer to is designated by GPS id information, so a common GPS 2011 can be used for multiple slice data 2013A/2013B.
  • GPS2011 does not necessarily need to transmit for each slice.
  • FIG. 3 it is possible to configure a bitstream such that GPS 2011 is not encoded immediately before GSH 2012B and slice data 2013B.
  • FIG. 3 is just an example. As long as GSH 2012A/2012B and GPS 2011 correspond to each slice data 2013A/2013B, elements other than those described above may be added as components of the bitstream.
  • the bitstream may include a sequence parameter set (SPS) 2001, as shown in FIG. Also, in the same way, it may be shaped into a configuration different from that in FIG. 3 at the time of transmission. Furthermore, it may be combined with a bitstream decoded by the attribute information decoding unit 2060, which will be described later, and transmitted as a single bitstream.
  • SPS sequence parameter set
  • FIG. 4 is an example of the syntax configuration of GPS2011.
  • syntax name explained below is just an example. As long as the functionality of the syntaxes described below is similar, the syntax names can be different.
  • the GPS 2011 may include GPS id information (gps_geom_parameter_set_id) for identifying each GPS 2011.
  • the Descriptor column in FIG. 4 means how each syntax is encoded.
  • ue(v) means an unsigned zero-order exponential Golomb code, and u(1) means a 1-bit flag.
  • the GPS 2011 may include a flag (interprediction_enabled_flag) that controls whether or not the tree synthesis unit 2020 performs inter prediction.
  • inter prediction is not performed when the value of interpretation_enabled_flag is "0", and that inter prediction is performed when the value of interpretation_enabled_flag is "1".
  • interpretation_enabled_flag may be included in the SPS2001 instead of the GPS2011.
  • the geometric information decoding unit 2020 may be configured to additionally decode the following syntax when performing inter prediction, that is, when the value of the interpolation_enabled_flag is "1".
  • the GPS 2011 may include a flag (reference_upsampling_enabled_flag) that controls whether to apply upsampling by the upsampling unit 2120 when performing inter prediction.
  • a flag reference_upsampling_enabled_flag
  • upsampling is not applied when the value of reference_upsampling_enabled_flag is "0", and that upsampling is applied when the value of reference_upsampling_enabled_flag is "1".
  • the geometric information decoding unit 2020 may be configured to additionally decode the following syntax when upsampling is applied, that is, when the value of reference_upsampling_enabled_flag is "1".
  • the GPS 2011 may include syntax (upsampling_number_div2) that controls the number of upsampling points. How to specifically use the value of the syntax will be described later.
  • the GPS 2011 may include a syntax (upsampling_interval) that controls the interval between upsampling points. How to specifically use the value of the syntax will be described later.
  • upsampling_interval a syntax that controls the interval between upsampling points. How to specifically use the value of the syntax will be described later.
  • the GPS 2011 may include a flag (trisoup_enabled_flag) that controls whether Trisoup is used in the approximate surface synthesis unit 2030.
  • Trisoup is not used when the value of trisoup_enabled_flag is "0", and that Trisoup is used when the value of trisoup_enabled_flag is "1".
  • the geometric information decoding unit 2020 may be configured to additionally decode the following syntax when using Trisoup, that is, when the value of trisoup_enabled_flag is "1".
  • trisoup_enabled_flag may be included in SPS2001 instead of GPS2011.
  • the GPS 2011 may include a flag (trisoup_multilevel_enabled_flag, first flag) that controls whether to allow Trisoup at multiple levels.
  • Trisoup_multilevel_enabled_flag when the value of trisoup_multilevel_enabled_flag is '0', Trisoup at multiple levels is not permitted. It may be defined that Trisoup is allowed.
  • the value of the syntax may be regarded as the value for Trisoup at a single level, that is, "0".
  • trisoup_multilevel_enabled_flag may be defined to be included in SPS2001 instead of GPS2011.
  • the value of the syntax may be regarded as the value for Trisoup at a single level, ie, "0".
  • the geometric information decoding unit 2020 may be configured to additionally decode the following syntax when trisoup at multiple levels is permitted, that is, when the value of trisoup_multilevel_enabled_flag is "1".
  • the GPS 2011 may include syntax (log2_trisoup_max_node_size_minus3) that defines the maximum value of the Trisoup node size when allowing Trisoup at multiple levels.
  • the syntax may be expressed as a logarithmic value with base 2 for the maximum value of the actual Trisoup node size. Further, the syntax may express the maximum value of the actual Trisoup node size as a value obtained by subtracting 3 after converting to base 2 logarithm.
  • the GPS 2011 may include syntax (log2_trisoup_min_node_size_minus3) that defines the minimum value of the Trisoup node size when allowing Trisoup at multiple levels.
  • the syntax may be expressed as a logarithmic value with base 2 for the minimum value of the actual Trisoup node size. Furthermore, the syntax may express the minimum value of the actual Trisoup node size as a value after converting to base 2 logarithm and then subtracting 3.
  • the value of the syntax may be constrained to always be 0 or more and log2_trisoup_max_node_size_minus3 or less.
  • the geometric information decoding unit 2020 may be configured to additionally decode the following syntax when Trisoup at multiple levels is not permitted, that is, when the value of trisoup_multilevel_enabled_flag is "0".
  • the GPS 2011 may include syntax (log2_trisoup_node_size_minus3) to specify the Trisoup node size when not allowing Trisoup at multiple levels.
  • the syntax may be expressed as a logarithmic value with base 2 for the actual Trisoup node size. Further, the syntax may express the actual Trisoup node size as a value after converting to base 2 logarithm and then subtracting 3.
  • Up-sampling unit 2120 Upsampling in the upsampling section 2120 will be described below with reference to FIGS. 5 and 6. FIG.
  • FIG. 5 is a flowchart showing an example of upsampling in the upsampling section 2120.
  • step S501 the upsampling unit 2120 performs upsampling on all points included in the point group reconstructed from the geometric information output from the geometric information reconstructing unit, that is, the positional information. Determine whether it is finished.
  • the upsampling unit 2120 ends the process in step S504. On the other hand, if the processing of all points has not been completed, the upsampling unit 2120 proceeds to step S502.
  • step S502 the upsampling unit 2120 calculates an offset vector.
  • the offset vector vu may be calculated as normalized so that the L2 norm of vu is 1, for example. That is, the upsampling section 2120 may calculate the offset vector vu based on the following equation.
  • vu vp/(X2+Y2+Z2)1/2
  • the upsampling section 2120 may use approximate calculations to perform integer operations for the square root calculation and division described above. Also, in the approximate calculation described above, if the absolute value of vp is too small, the error due to the integer calculation becomes too large, so i ⁇ vu, which will be described later, may be obtained instead of vu.
  • the offset vector vu is calculated as a vector that points in the same direction as the vector vp that points to the point P starting from the origin O and that has a predetermined size (for example, L2 norm) for the point P to be up-sampled. For example, it may be calculated by a method other than the above.
  • the offset vector vu is calculated so as to be normal to the spherical surface, it can be calculated by a method other than the above. I don't mind.
  • the upsampling section 2120 may derive the offset vector vu by a method that includes some approximation errors with respect to the direction and magnitude described above.
  • the upsampling unit 2120 moves to step S503.
  • step S503 the upsampling unit 2120 performs upsampling based on the offset vector vu calculated in step S502.
  • FIG. 6A shows an example of how the point P looks after upsampling.
  • i shown in FIG. 6B is a parameter that controls the upsampling interval, and a value decoded as the syntax (upsampling_interval) that controls the interval of the upsampling points included in GPS2011 is used.
  • N shown in FIG. 6B is a parameter that controls the number of points generated by upsampling for one input point, and the value decoded as the syntax (upsampling_number_div2) that controls the number of upsampling points included in GPS2011 is use.
  • Upsampling can be realized, for example, by adding a point having a coordinate value vs as follows, where vp is the coordinate of point P.
  • 2N points are added to one input point by upsampling.
  • the upsampling unit 2120 After performing upsampling as described above, the upsampling unit 2120 returns to step S501 and performs the following processing.
  • FIG. 7 is a flowchart showing an example of processing in the tree synthesizing unit 2020. As shown in FIG. An example of synthesizing a tree using "Octree" will be described below.
  • step S701 the tree synthesizing unit 2020 confirms whether or not all depth processes have been completed.
  • the depth number may be included as control data in a bitstream transmitted from the point cloud encoding device 100 to the point cloud decoding device 200 .
  • the tree synthesizing unit 2020 calculates the node size of the depth.
  • the initial Depth node size may be defined as '2 raised to the power of Depth'. That is, when the depth number is N, the node size of the first depth may be defined as 2 to the Nth power.
  • the node size at the second and subsequent Depths may be defined by decreasing the number of N by one. That is, the node size of the second depth is defined as "2 to the (N-1) power", the node size of the third depth is defined as "2 to the (N-2) power", . may be defined as
  • node size is always defined as a power of 2
  • the value of the exponent part (N, N ⁇ 1, N ⁇ 2, etc.) may be simply considered as the node size.
  • node size refers to the value of the exponent part.
  • step S709 the tree synthesis unit 2020 proceeds to step S702.
  • the tree synthesizing unit 2020 processes The depth number may be varied based on the value of the syntax that defines the minimum value of the Trisoup node size (log2_trisoup_min_node_size_minus3) or the syntax that defines the Trisoup node size (log2_trisoup_node_size_minus3). In such a case, for example, it may be defined as follows.
  • Trisoup node size can be defined by, for example, (log2_trisoup_min_node_size_minus3+3).
  • the Trisoup node size can be defined as (log2_trisoup_node_size_minus3+3).
  • step S709 if the processing for all the processing depth numbers has been completed, the tree synthesizing unit 2020 proceeds to step S709; otherwise, the tree synthesizing unit 2020 proceeds to step S702.
  • the tree synthesizing unit 2020 may determine that Trisoup is applied to all nodes having a node size (N-number of processing depths) when proceeding to step S709.
  • step S702 the tree synthesizing unit 2020 determines whether it is necessary to decode Trisoup_applied_flag, which will be described later, at the relevant depth.
  • the tree synthesizing unit 2020 may determine that "decoding of Trisoup_applied_flag is required”.
  • the tree synthesis unit 2020 may determine that "decoding of Trisoup_applied_flag is unnecessary" when the above condition is not satisfied.
  • the maximum Trisoup node size can be defined, for example, by (log2_trisoup_max_node_size_minus3+3).
  • step S703 the tree synthesizing unit 2020 determines whether or not all nodes included in the depth have been processed.
  • the tree synthesizing unit 2020 moves to step S701 and performs the processing of the next depth.
  • step S704 the tree synthesizing unit 2020 confirms whether decoding of the Trisoup_applied_flag determined in step S702 is necessary.
  • step S705 If it is determined that decoding of Trisoup_applied_flag is necessary, the tree synthesizing unit 2020 proceeds to step S705, and if it is determined that decoding of Trisoup_applied_flag is not required, the tree synthesizing unit 2020 proceeds to step S708. .
  • step S705 the tree synthesizing unit 2020 decodes Trisoup_applied_flag.
  • Trisoup_applied_flag is a 1-bit flag (second flag) that indicates whether or not Trisoup is applied to the target node. For example, it may be defined that Trisoup is applied to the target node when the flag value is "1", and that Trisoup is not applied to the target node when the flag value is "0".
  • the tree synthesizing unit 2020 moves to step S706.
  • step S706 the tree synthesizing unit 2020 checks the value of Trisoup_applied_flag decoded in step S705.
  • step S707 the tree synthesizing unit 2020 stores the target node as a node to which Trisoup is applied, that is, as a Trisoup node. It is assumed that the node division by "Octree" is no longer applied to such a target node. After that, the tree synthesizing unit 2020 advances to step S703 to process the next node.
  • step S708 the tree synthesizing unit 2020 decodes the information called the occpancy code.
  • the occpancy code is divided into 8 nodes (called child nodes) by dividing the target node in half along the x, y, and z axes. This is information indicating whether or not a point to be decoded is included.
  • the occupation code assigns 1-bit information to each child node, and if the 1-bit information is "1", it is defined that the point to be decoded is included in the child node. If the bit information is '0', it may be defined that the point to be decoded is not included in the child node.
  • the tree synthesizing unit 2020 When decoding such an occurrence code, the tree synthesizing unit 2020 pre-estimates the probability that each child node has a point to be decoded, and entropy-decodes the bits corresponding to each child node based on the probability. good.
  • the point group encoding device 100 may perform entropy encoding.
  • inter-prediction may be used to estimate such probabilities.
  • the method described in Non-Patent Document 1 can be applied.
  • a point cloud upsampled by the upsampling unit 2120 may be used as a reference point cloud for inter prediction.
  • the tree synthesizing unit 2020 advances to step S703 to process the next node.
  • step S709 the tree synthesizing unit 2020 decodes the Trisoup information.
  • the tree synthesizing unit 2020 executes the process of step S709 only when Trisoup is used, that is, when the value of trisoup_enabled_flag is "1". That is, if Trisoup is not used, tree synthesizing section 2020 advances to step S710 and terminates the process.
  • FIG. 8 is a flowchart showing an example of decoding processing for Trisoup information.
  • step S801 the tree synthesizing unit 2020 decodes syntax that controls sampling intervals of decoding points.
  • step S802 the tree synthesizing unit 2020 determines whether the processing of all Trisoup hierarchies has been completed.
  • the total number of Trisoup hierarchies can be defined as follows.
  • the total number of Trisoup layers can be defined as (maximum Trisoup node size - minimum Trisoup node size + 1).
  • the total number of Trisoup hierarchies can be defined by (log2_trisoup_max_node_size_minus3 ⁇ log2_trisoup_min_node_size_minus3+1).
  • Trisoup at multiple levels that is, when the value of trisoup_multilevel_enabled_flag is "0", the total number of Trisoup layers is 1.
  • the tree synthesizing unit 2020 proceeds to step S807 and ends the processing. If the processing of all Trisoup hierarchies has not been completed, the tree synthesizing unit 2020 moves to step S803.
  • step S803 the tree synthesizing unit 2020 decodes the number of unique segments in the Trisoup hierarchy.
  • the number of unique segments is the number of sides forming a Trisoup node belonging to the Trisoup hierarchy.
  • step S803 instead of decoding the number of unique segments, the tree synthesizing unit 2020 may decode a flag indicating whether (one or more) unique segments exist in the Trisoup hierarchy as described above. .
  • Trisoup at multiple levels is not permitted, that is, when the value of trisoup_multilevel_enabled_flag is "0", it is obvious that a unique segment exists in the Trisoup hierarchy. may omit the decoding of the flag indicating whether there is (one or more) unique segments in .
  • the tree synthesizing unit 2020 moves to step S804.
  • step S804 the tree synthesizing unit 2020 confirms the number of unique segments belonging to the Trisoup hierarchy.
  • the tree synthesizing unit 2020 proceeds to step S802 to proceed to the processing of the next Trisoup hierarchy.
  • step S803 when the flag indicating whether (one or more) unique segments exist in the Trisoup hierarchy is decoded in step S803, the tree synthesizing unit 2020 determines that the Trisoup hierarchy contains unique segments. After decoding the number of unique segments in the layer, the process proceeds to step S805, and if there is no unique segment in the Trisoup layer, the process proceeds to step S802.
  • step S805 the tree synthesizing unit 2020 decodes whether each unique segment includes a vertex used for Trisoup processing.
  • the number of vertices that can exist for each unique segment may be limited to only one point.
  • the tree synthesizing unit 2020 moves to step S806.
  • step S806 the tree synthesizing unit 2020 decodes position information indicating where the vertex exists on each unique segment for each unique segment determined to have a vertex in step S805.
  • the position information may be encoded with an L-bit equal-length code.
  • the tree synthesizing unit 2020 After decoding the vertex positions for all unique segments having vertices in the Trisoup hierarchy, the tree synthesizing unit 2020 proceeds to step S802 to proceed to the processing of the next Trisoup hierarchy.
  • FIG. 9 is a diagram showing an example of functional blocks of the point group encoding device 100 according to this embodiment.
  • the point group encoding device 100 includes a coordinate transformation unit 1010, a geometric information quantization unit 1020, a tree analysis unit 1030, an approximate surface analysis unit 1040, a geometric information encoding unit 1050, Geometric information reconstruction unit 1060, color conversion unit 1070, attribute transfer unit 1080, RAHT unit 1090, LoD calculation unit 1100, lifting unit 1110, attribute information quantization unit 1120, attribute information encoding unit 1130 , an up-sampling unit 1140 and a frame buffer 1150 .
  • the coordinate transformation unit 1010 is configured to perform transformation processing from the three-dimensional coordinate system of the input point group to any different coordinate system. Coordinate transformation may transform the x, y, z coordinates of the input point cloud to arbitrary s, t, u coordinates, for example, by rotating the input point cloud. Also, as one variation of transformation, the coordinate system of the input point cloud may be used as it is.
  • the geometric information quantization unit 1020 is configured to quantize the position information of the input point group after coordinate transformation and remove points with overlapping coordinates. Note that when the quantization step size is 1, the position information of the input point group matches the position information after quantization. That is, when the quantization step size is 1, it is equivalent to not performing quantization.
  • the tree analysis unit 1030 is configured to receive the position information of the quantized point group as input and generate an occupancy code indicating at which node in the encoding target space the point exists based on the tree structure described later. It is
  • the tree analysis unit 1030 is configured to generate a tree structure in this process by recursively partitioning the encoding target space into rectangular parallelepipeds.
  • a tree structure can be generated by recursively executing the process of dividing the rectangular parallelepiped into multiple rectangular parallelepipeds until the rectangular parallelepipeds reach a predetermined size.
  • Each rectangular parallelepiped is called a node.
  • each rectangular parallelepiped generated by dividing a node is called a child node, and an occupancy code is expressed by 0 or 1 as to whether or not a point is included in the child node.
  • the tree analysis unit 1030 is configured to generate an occupancy code while recursively dividing a node until it reaches a predetermined size.
  • the tree analysis unit 1030 determines the tree structure, and the determined tree structure is transmitted to the point cloud decoding device 200 as control data.
  • tree-structured control data may be configured so that it can be decoded according to the procedure described in FIGS.
  • the approximate surface analysis unit 1040 is configured to use the tree information generated by the tree analysis unit 1030 to generate approximate surface information.
  • the approximate surface information is obtained by decoding the points instead of decoding individual point clouds. It is an approximation of the existence area of the group with a small plane.
  • the approximate surface analysis unit 1040 may be configured to generate approximate surface information by, for example, a technique called “Trisoup”. Also, when decoding a sparse point group acquired by Lidar or the like, this process can be omitted.
  • the geometric information encoding unit 1050 encodes syntax such as the occupancy code generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040 to generate a bitstream (geometric information bitstream).
  • the bitstream may contain, for example, the syntax described in FIG.
  • the encoding process is, for example, context adaptive binary arithmetic encoding process.
  • the syntax includes control data (flags and parameters) for controlling decoding processing of position information.
  • the geometric information reconstruction unit 1060 Based on the tree information generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040, the geometric information reconstruction unit 1060 generates geometric information (code It is configured to reconstruct the coordinate system assumed by the transformation processing, that is, the position information after the coordinate transformation in the coordinate transformation unit 1010).
  • the upsampling unit 1140 is configured to receive the geometric information reconstructed by the geometric information reconstructing unit 1060 and perform the upsampling described with reference to FIGS. 5 and 6, for example.
  • the frame buffer 1150 is configured to receive the point cloud upsampled by the upsampling unit 1140 and store it as a reference frame.
  • the stored reference frames are read from the frame buffer 1150 and used as reference frames when the tree analysis unit 1030 inter-predicts temporally different frames.
  • whether or not the reference frame at which time is to be used for each frame is determined, for example, based on the value of the cost function representing the coding efficiency, and the information of the reference frame to be used is used as control data for the point group decoding device. 200.
  • the geometric information output from the geometric information reconstruction unit 1060 is stored in the frame buffer 1150 as a reference frame, and then the reference frame stored in the frame buffer 1150 is used when inter prediction is performed by the tree analysis unit 1030. may be up-sampled by up-sampling section 1140 and then input to tree analysis section 1030 .
  • the upsampling unit 1140 may be configured to be included inside the tree analysis unit 1030 .
  • the color conversion unit 1070 is configured to perform color conversion when the input attribute information is color information. Color conversion does not necessarily need to be executed, and whether or not to execute color conversion processing is encoded as part of control data and transmitted to the point cloud decoding device 200 .
  • the attribute transfer unit 1080 performs distortion of attribute information based on the position information of the input point group, the position information of the point group after reconstruction by the geometric information reconstruction unit 1060, and the attribute information after color change by the color conversion unit 1070. is configured to correct the attribute value so that is minimized.
  • a specific correction method for example, the method described in Non-Patent Document 2 can be applied.
  • the RAHT unit 1090 receives as input the attribute information transferred by the attribute transfer unit 1080 and the geometric information generated by the geometric information reconstruction unit 1060, and uses a type of Haar transformation called RAHT (Region Adaptive Hierarchical Transform) to transform each configured to generate point residual information;
  • RAHT Region Adaptive Hierarchical Transform
  • the LoD calculation unit 1100 is configured to receive the geometric information generated by the geometric information reconstruction unit 1060 and generate LoD (Level of Detail).
  • LoD is a reference relationship (referred point and referenced point ) is information for defining
  • LoD classifies each point included in geometric information into a plurality of levels, and encodes or decodes the attributes of points belonging to lower levels using attribute information of points belonging to higher levels. This is information that defines the structure.
  • Non-Patent Document 2 As a specific method for determining LoD, for example, the method described in Non-Patent Document 2 may be used.
  • the lifting unit 1110 is configured to generate residual information through a lifting process using the LoD generated by the LoD calculation unit 1100 and the attribute information after the attribute transfer by the attribute transfer unit 1080.
  • Non-Patent Document 2 As a specific lifting process, for example, the method described in Non-Patent Document 2 may be used.
  • the attribute information quantization section 1120 is configured to quantize the residual information output from the RAHT section 1090 or the lifting section 1110 .
  • the case where the quantization step size is 1 is equivalent to the case where no quantization is performed.
  • the attribute information encoding unit 1130 performs encoding processing using the quantized residual information output from the attribute information quantization unit 1120 as a syntax, and generates a bitstream related to attribute information (attribute information bitstream). is configured as
  • the encoding process is, for example, context adaptive binary arithmetic encoding process.
  • the syntax includes control data (flags and parameters) for controlling attribute information decoding processing.
  • the point cloud encoding device 100 is configured to input position information and attribute information of each point in the point cloud, perform encoding processing, and output a geometric information bitstream and an attribute information bitstream. ing.
  • the point cloud encoding device 100 and the point cloud decoding device 200 described above may be implemented as a program that causes a computer to execute each function (each process).
  • the present invention is applied to the point group encoding device 100 and the point group decoding device 200 as examples, but the present invention is not limited to such examples.
  • a point cloud encoding/decoding system having the functions of the point cloud encoding device 100 and the point cloud decoding device 200 can be similarly applied.
  • the United Nations-led Sustainable Development Goals (SDGs) Goal 9 "Develop resilient infrastructure, It will be possible to contribute to the promotion of sustainable industrialization and the expansion of innovation.
  • Point group processing system 100 Point group encoding device 1010... Coordinate transformation unit 1020... Geometric information quantization unit 1030... Tree analysis unit 1040... Approximate surface analysis unit 1050... Geometric information encoding unit 1060... Geometric information reconstruction unit 1070... Color conversion unit 1080... Attribute transfer unit 1090... RAHT unit 1100... LoD calculation unit 1110... Lifting unit 1120... Attribute information quantization unit 1130... Attribute information encoding unit 1140... Up-sampling unit 1150... Frame buffer 200...
  • Point group Decoding device 2010 Geometric information decoding unit 2020 Tree synthesis unit 2030 Approximate surface synthesis unit 2040
  • Geometric information reconstruction unit 2050 Inverse coordinate transformation unit 2060
  • Attribute information decoding unit 2070 Inverse quantization unit 2080
  • RAHT unit 2090
  • LoD calculator 2100 Inverse lifting unit 2110
  • Inverse color conversion unit 2120 Upsampling unit 2130 Frame buffer

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)

Abstract

Un dispositif 200 de décodage de nuage de points selon la présente invention comprend : une unité 2040 de reconstruction d'informations géométriques pour reconstruire des informations de position d'un nuage de points ; une unité de suréchantillonnage 2120 pour suréchantillonner le nuage de points reconstruit par l'unité 2040 de reconstruction d'informations géométriques ; et une unité de synthèse d'arbre 2020 pour effectuer une inter-prédiction en référence au nuage de points suréchantillonné. L'unité de suréchantillonnage 2120 calcule un vecteur de décalage à partir des coordonnées de chaque point dans le nuage de points reconstruit et effectue le suréchantillonnage à l'aide des vecteurs de décalage.
PCT/JP2022/026209 2021-07-02 2022-06-30 Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme WO2023277128A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021111088A JP2023007934A (ja) 2021-07-02 2021-07-02 点群復号装置、点群復号方法及びプログラム
JP2021-111088 2021-07-02

Publications (1)

Publication Number Publication Date
WO2023277128A1 true WO2023277128A1 (fr) 2023-01-05

Family

ID=84692757

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/026209 WO2023277128A1 (fr) 2021-07-02 2022-06-30 Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme

Country Status (2)

Country Link
JP (1) JP2023007934A (fr)
WO (1) WO2023277128A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6732152B1 (ja) * 2017-10-12 2020-07-29 三菱電機株式会社 点群エンコーダ
US20210029187A1 (en) * 2019-07-03 2021-01-28 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus and point cloud data reception method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6732152B1 (ja) * 2017-10-12 2020-07-29 三菱電機株式会社 点群エンコーダ
US20210029187A1 (en) * 2019-07-03 2021-01-28 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus and point cloud data reception method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
DRICOT ANTOINE; ASCENSO JOAO: "Adaptive Multi-level Triangle Soup for Geometry-based Point Cloud Coding", 2019 IEEE 21ST INTERNATIONAL WORKSHOP ON MULTIMEDIA SIGNAL PROCESSING (MMSP), IEEE, 27 September 2019 (2019-09-27), pages 1 - 6, XP033660072, DOI: 10.1109/MMSP.2019.8901791 *

Also Published As

Publication number Publication date
JP2023007934A (ja) 2023-01-19

Similar Documents

Publication Publication Date Title
CN113615181B (zh) 用于点云编解码的方法、装置
US12008712B2 (en) Inter-component residual prediction for attributes in geometry point cloud compression coding
US20210314616A1 (en) Predictor index signaling for predicting transform in geometry-based point cloud compression
CN113632142A (zh) 点云压缩的方法和装置
US12002244B2 (en) Global scaling for geometry-based point cloud coding
WO2022070469A1 (fr) Dispositif de décodage de groupe de points, procédé de décodage de groupe de points, et programme
JP2022102806A (ja) 点群復号装置、点群復号方法及びプログラム
CN115885317A (zh) 点云解码装置、点云解码方法及程序
WO2021256486A1 (fr) Dispositif de décodage de groupe de points, procédé de décodage de groupe de points et programme
WO2022071284A1 (fr) Dispositif de décodage de groupe de points, procédé de décodage de groupe de points, et programme
WO2023277128A1 (fr) Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme
WO2022071285A1 (fr) Dispositif de décodage de groupe de points, procédé de décodage de groupe de points, et programme
CN116057933A (zh) 点云解码装置、点云解码方法和程序
EP4172943A1 (fr) Facteur d'échelle pour valeurs de paramètre de quantification dans la compression de nuage de points basée sur la géométrie
JP2023053827A (ja) 点群復号装置、点群復号方法及びプログラム
WO2023132330A1 (fr) Dispositif de décodage de nuage de points, procédé de décodage de nuage de points, et programme
WO2023132331A1 (fr) Dispositif de décodage de nuage de points, procédé de décodage de nuage de points, et programme
WO2023132329A1 (fr) Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme
WO2024009675A1 (fr) Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme
WO2024009676A1 (fr) Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme
JP7505954B2 (ja) 点群復号装置、点群復号方法及びプログラム
JP7499665B2 (ja) 点群復号装置、点群復号方法及びプログラム
WO2024009562A1 (fr) Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme
WO2024009561A1 (fr) Dispositif de décodage de nuage de points, procédé de décodage de nuage de points et programme
US20230342987A1 (en) Occupancy coding using inter prediction with octree occupancy coding based on dynamic optimal binary coder with update on the fly (obuf) in geometry-based point cloud compression

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22833274

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE