WO2022260115A1

WO2022260115A1 - Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

Info

Publication number: WO2022260115A1
Application number: PCT/JP2022/023234
Authority: WO
Inventors: 敏康杉尾; 賀敬井口; 孝啓西
Original assignee: パナソニックインテレクチュアルプロパティコーポレーションオブアメリカ
Priority date: 2021-06-11
Filing date: 2022-06-09
Publication date: 2022-12-15
Also published as: JPWO2022260115A1; CN117396920A

Abstract

This three-dimensional data encoding method includes (S161) calculating a first residual between an angle component of a first three-dimensional point among a plurality of three-dimensional points each represented by a distance component and an angle component, and a predicted value of the angle component of the first three-dimensional point, calculated by inter prediction, and (S162) using a first quantization step to quantize the first residual, wherein, in the first quantization step, a calculation is performed in accordance with at least either the distance component of the first three-dimensional point, or a distance component of one or more second three-dimensional points that are included in the plurality of three-dimensional points and are positioned in the vicinity of the first three-dimensional point.

Description

Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

The present disclosure relates to a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, and a three-dimensional data decoding device.

In the future, it is expected that devices or services that utilize 3D data will spread in a wide range of fields, such as computer vision, map information, monitoring, infrastructure inspection, or video distribution for autonomous operation of automobiles or robots. Three-dimensional data is acquired in various ways, such as range sensors such as range finders, stereo cameras, or a combination of multiple monocular cameras.

As one of the three-dimensional data representation methods, there is a representation method called a point cloud that expresses the shape of a three-dimensional structure using a point group in a three-dimensional space. A point cloud stores the position and color of the point cloud. Point clouds are expected to become mainstream as a method of expressing three-dimensional data, but point clouds have a very large amount of data. Therefore, in the storage or transmission of 3D data, it is essential to compress the amount of data by encoding, as with 2D moving images (one example is MPEG-4 AVC or HEVC standardized by MPEG). Become.

In addition, point cloud compression is partially supported by a public library (Point Cloud Library) that performs point cloud-related processing.

Also, there is known a technique of searching for and displaying facilities located around a vehicle using three-dimensional map data (see Patent Document 1, for example).

WO2014/020663

It is desired that the encoding efficiency can be improved in the 3D data encoding process and the 3D data decoding process.

An object of the present disclosure is to provide a three-dimensional data encoding method, a three-dimensional data decoding method, a three-dimensional data encoding device, or a three-dimensional data decoding device that can improve encoding efficiency.

A three-dimensional data encoding method according to one aspect of the present disclosure is calculated by the angle component of a first three-dimensional point among a plurality of three-dimensional points each represented by a distance component and an angle component, and inter prediction. Further, calculating a first residual between the first three-dimensional point and the predicted value of the angular component, and quantizing the first residual using a first quantization step, wherein the first quantization step includes: according to at least one of a distance component of the first three-dimensional point and a distance component of one or more second three-dimensional points included in the plurality of three-dimensional points and located around the first three-dimensional point Calculated.

The present disclosure can provide a three-dimensional data encoding method and the like that can improve encoding efficiency.

FIG. 1 is a block diagram of a 3D data encoding device according to an embodiment. FIG. 2 is a diagram illustrating an example of a prediction tree according to the embodiment; FIG. 3 is a block diagram of a three-dimensional data decoding device according to an embodiment. FIG. 4 is a flowchart showing an example of a procedure for encoding each 3D point of a prediction tree (Predtree) in a 3D data encoding device. FIG. 5 is a flowchart showing an example of adaptive quantization processing for a residual signal in encoding processing. FIG. 6 is a flowchart showing an example of adaptive inverse quantization processing for quantized values generated by adaptive quantization in encoding processing. FIG. 7 is a flowchart showing an example of a procedure for decoding each 3D point of a prediction tree (Predtree) in the 3D data decoding device. FIG. 8 is a flowchart showing an example of adaptive inverse quantization processing for quantized values in decoding processing. FIG. 9 is a block diagram of a 3D data encoding device according to a modification of the embodiment. FIG. 10 is a block diagram of a three-dimensional data decoding device according to a modification of the embodiment. FIG. 11 is a flowchart showing an example of 3D data encoding processing according to the embodiment. FIG. 12 is a flowchart illustrating an example of 3D data decoding processing according to the embodiment. FIG. 13 is a flowchart showing an example of 3D data encoding processing according to the modification. FIG. 14 is a flowchart showing an example of 3D data decoding processing according to the modification.

Depending on the 3D point, the likelihood of inter-prediction of the angle component may change depending on at least one of the distance component and the distance components of surrounding 3D points. For example, in inter-prediction, there are cases where angle components are more likely to be predicted at distant positions (or near positions), resulting in smaller residuals. In other words, the prediction of the angle component is likely to go wrong at a near position (or a far position), and the residual error may become large. Therefore, quantization using the first quantization step calculated according to at least one of the distance component of the first three-dimensional point and the distance components of one or more surrounding second three-dimensional points reduces the quantization error. Browsing and redundant quantized values can be reduced. Therefore, coding efficiency can be improved.

Also, the one or more second three-dimensional points may be encoded before the first three-dimensional points in the encoding order of the plurality of three-dimensional points.

Further, a second residual between an angle component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angle component of the third three-dimensional point calculated by intra prediction is and quantizing the second residual using a second quantization step different from the first quantization step.

Depending on the 3D point, the residual error of the angle component may increase or decrease depending on the prediction method. Therefore, when the residual of the angle component changes according to the prediction method in addition to the distance component, the residual is quantized using a quantization step that is adaptively calculated according to the distance component and the prediction method. By doing so, the coding efficiency can be improved.

Further, the second quantization step includes distance components of the third three-dimensional point and one or more fourth three-dimensional points included in the plurality of three-dimensional points and located around the third three-dimensional point. It may be calculated according to at least one of the distance components of the point.

Intra prediction can also improve coding efficiency by adaptively quantizing according to the distance component.

Also, the first quantization step may be calculated so as to increase as the first prediction accuracy of the predicted value of the angular component of the first three-dimensional point decreases.

The higher the first prediction accuracy is, the smaller the residual error is because a more accurate prediction value is generated. Conversely, the smaller the first prediction accuracy, the larger the residual, because an inaccurate prediction is generated. Therefore, by calculating the first quantization step so that the smaller the first prediction accuracy, the larger the first quantization step, the larger the quantization error and the redundant quantization value can be reduced. Therefore, coding efficiency can be improved.

Further, the first quantization step is calculated according to at least one of the distance component of the first three-dimensional point and the distance component of the one or more second three-dimensional points and a first reference value, The first reference value is a reference value for calculating the first quantization step so that the lower the first prediction accuracy is, the higher the first quantization step is. A bitstream may be generated that includes the first residual and the first reference value.

The first reference value is a reference value for calculating the first quantization step so that the smaller the first prediction accuracy, the larger the first quantization step. can be calculated. Therefore, it is possible to reduce the increase in quantization error and redundant quantization values. Therefore, coding efficiency can be improved. Also, by including the first reference value in the bitstream, the decoding side can perform the same adaptive quantization as the encoding side.

Further, a second residual between an angle component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angle component of the third three-dimensional point calculated by intra prediction is and quantizing the second residual using a second quantization step different from the first quantization step, the second quantization step being the distance component of the third three-dimensional point and the one It is calculated according to at least one of the distance components of the fourth three-dimensional point and a second reference value, and the second reference value is the second value of the predicted value of the angle component of the third three-dimensional point. A reference value for calculating the second quantization step so that the smaller the prediction accuracy is, the higher the second quantization step is, and the bitstream further includes the quantized second residual and the second reference value. It's okay.

The second reference value is a reference value for calculating the second quantization step so that the smaller the second prediction accuracy, the larger the second quantization step. can be calculated. Therefore, it is possible to reduce the increase in quantization error and redundant quantization values. Therefore, coding efficiency can be improved.

Also, the first quantization step may be calculated so as to decrease as at least one of the distance component of the first three-dimensional point and the distance component of the one or more second three-dimensional points increases.

Depending on the 3D point, the larger at least one of the distance component and the distance components of surrounding 3D points is, the more difficult it is to inter-predict the angle component. Quantizing the first residual using one quantization step can effectively reduce a large quantization error and redundant quantization values. Therefore, coding efficiency can be improved.

Further, a second residual between an angle component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angle component of the third three-dimensional point calculated by intra prediction is and quantizing the second residual using a second quantization step different from the first quantization step, the second quantization step being the distance component of the third three-dimensional point and the one It may be calculated so as to decrease as at least one of the distance components of the fourth three-dimensional point increases.

Depending on the 3D point, the larger at least one of the distance component and the distance components of surrounding 3D points, the more difficult it is to inter-predict the angle component. Even in this case, by adaptively quantizing according to the distance component, it is possible to effectively reduce the increase in quantization error and the redundancy of quantized values. Therefore, coding efficiency can be improved.

Further, a three-dimensional data encoding method according to another aspect of the present disclosure calculates a predicted value of a position of a three-dimensional point using one of inter prediction and intra prediction, and compares the predicted value and the position. and quantize the residual using a quantization step according to the prediction scheme.

Depending on the 3D point, the positional residual may be larger or smaller depending on the prediction method. Therefore, when the position residual varies according to the prediction method, the coding efficiency can be improved by quantizing the residual using the quantization step calculated according to the prediction method.

A three-dimensional data decoding method according to one aspect of the present disclosure is calculated by the angle component of a first three-dimensional point among a plurality of three-dimensional points each represented by a distance component and an angle component, and inter prediction. Obtaining a first residual between the first three-dimensional point and the predicted value of the angular component, and inversely quantizing the first residual using a first quantization step, wherein the first quantization step is , the distance component of the first three-dimensional point, and the distance component of one or more second three-dimensional points included in the plurality of three-dimensional points and located around the first three-dimensional point. calculated as

Depending on the 3D point, the likelihood of inter-prediction of the angle component may change depending on at least one of the distance component and the distance components of surrounding 3D points. For example, in inter-prediction, there are cases where angle components are more likely to be predicted at distant positions (or near positions), resulting in smaller residuals. In other words, the prediction of the angle component is likely to go wrong at a near position (or a far position), and the residual error may become large. Therefore, quantization is performed using the first quantization step calculated according to at least one of the distance component of the first three-dimensional point and the distance component of one or more surrounding second three-dimensional points, thereby improving the coding efficiency. It is possible to properly decode the first residual that may have been modified.

Also, the one or more second three-dimensional points may be decoded before the first three-dimensional points in the decoding order of the plurality of three-dimensional points.

Further, a second residual between an angle component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angle component of the third three-dimensional point calculated by intra prediction is and inverse quantizing the second residual using a second quantization step different from the first quantization step.

Depending on the 3D point, the residual error of the angle component may increase or decrease depending on the prediction method. Therefore, when the residual of the angle component changes according to the prediction method in addition to the distance component, it is quantized and encoded using a quantization step adaptively calculated according to the distance component and the prediction method. It is possible to properly decode the first residual, which may have improved coding efficiency.

Also in intra prediction, it is possible to appropriately decode the second residual, which may have been adaptively quantized according to the distance component to improve the coding efficiency.

The higher the first prediction accuracy is, the smaller the residual error is because a more accurate prediction value is generated. Conversely, the smaller the first prediction accuracy, the larger the residual, because an inaccurate prediction is generated. Therefore, by calculating the first quantization step so that the smaller the first prediction accuracy, the larger the quantization error, and the redundant quantization value may be reduced. The first residual can be properly decoded.

Further, a first reference value is obtained, and the first quantization step includes at least one of the distance component of the first three-dimensional point and the distance component of the one or more second three-dimensional points and the first The first reference value may be a reference value for calculating the first quantization step so that the smaller the first prediction accuracy, the larger the first quantization step.

The first reference value is a reference value for calculating the first quantization step so that the smaller the first prediction accuracy, the larger the first quantization step. By calculating , it is possible to appropriately decode the first residual that may have reduced the increase in quantization error and redundant quantization values.

a second residual between an angle component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angle component of the third three-dimensional point calculated by intra prediction; obtaining a second reference value, and inverse quantizing the second residual using a second quantization step different from the first quantization step, wherein the second quantization step is for the third three-dimensional point; calculated according to at least one of the distance component and the distance component of the one or more fourth three-dimensional points, and a second reference value, wherein the second reference value is the angular component of the third three-dimensional point The second quantization step may be a reference value for calculating the second quantization step so that the smaller the second prediction accuracy of is, the larger it becomes.

The second reference value is a reference value for calculating the second quantization step so that the smaller the second prediction accuracy, the larger the second quantization step. can be calculated. Therefore, it is possible to appropriately decode the second residual that may have reduced the increase in quantization error and redundant quantization values.

Depending on the 3D point, the larger at least one of the distance component and the distance components of surrounding 3D points is, the more difficult it is to inter-predict the angle component. The first residual, which may have been quantized using one quantization step to improve coding efficiency, can be properly decoded.

Further, a second residual between an angle component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angle component of the third three-dimensional point calculated by intra prediction is and inverse quantizing the second residual using a second quantization step different from the first quantization step, the second quantization step comprising the distance component of the third 3D point and the It may be calculated so as to decrease as at least one of the distance components of the one or more fourth three-dimensional points increases.

Depending on the 3D point, the larger at least one of the distance component and the distance components of surrounding 3D points, the more difficult it is to inter-predict the angle component. Even in this case, it is possible to appropriately decode the second residual, which may have been adaptively quantized according to the distance component and thus improved in coding efficiency.

A three-dimensional data decoding method according to another aspect of the present disclosure acquires a residual between a position of a three-dimensional point and a prediction value calculated by one of inter prediction and intra prediction, and Inverse quantize the residual using a quantization step according to .

Depending on the 3D point, the positional residual may be larger or smaller depending on the prediction method. For this reason, when the position residual changes according to the prediction method, the residual that may have been quantized using the quantization step calculated according to the prediction method to improve the coding efficiency is can be decrypted properly.

A three-dimensional data encoding device according to one aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to generate a plurality of three-dimensional data each represented by a distance component and an angle component. calculating a first residual between an angle component of a first three-dimensional point among points and a predicted value of the angle component of the first three-dimensional point calculated by inter prediction, and performing a first quantization step; quantizing the first residual using It is calculated according to at least one of the distance components of one or more positioned second three-dimensional points.

A 3D data encoding device according to another aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to perform 3D data encoding using one of inter prediction and intra prediction. A predicted value of the position of the point is calculated, a residual between the predicted value and the position is calculated, and the residual is quantized using a quantization step according to the prediction scheme.

A three-dimensional data decoding device according to one aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to generate a plurality of three-dimensional points each represented by a distance component and an angle component. obtain a first residual between the angle component of the first three-dimensional point and the predicted value of the angle component of the first three-dimensional point calculated by inter prediction, and perform a first quantization step inverse quantizing the first residual using is calculated according to at least one of the distance components of one or more second three-dimensional points located at .

A 3D data decoding device according to another aspect of the present disclosure includes a processor and a memory, and the processor uses the memory to determine the position of a 3D point and one of inter prediction and intra prediction. A residual from the predicted value calculated by the prediction method is acquired, and the residual is inversely quantized using a quantization step according to the prediction method.

In addition, these general or specific aspects may be realized by a system, method, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. and any combination of recording media.

Hereinafter, embodiments will be specifically described with reference to the drawings. It should be noted that each of the embodiments described below is a specific example of the present disclosure. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are examples, and are not intended to limit the present disclosure. Further, among the constituent elements in the following embodiments, constituent elements not described in independent claims will be described as optional constituent elements.

(Embodiment)
In the present embodiment, a case will be described in which either inter prediction or intra prediction is switched and executed with respect to position information of a point group (point cloud).

FIG. 1 is a block diagram of a three-dimensional data encoding device 100 according to this embodiment. Although FIG. 1 shows a processing unit for encoding position information (geometry) of a point group, the three-dimensional data encoding device 100 includes a processing unit for encoding attribute information of a point group, etc. may be provided with other processing units. In inter-prediction and intra-prediction, a point group to be encoded is encoded while referring to an already-encoded point group.

Here, inter prediction refers to a second reference 3D point group (second frame) belonging to a second 3D point group (second frame) different from the first 3D point group (first frame) to which the 3D point to be encoded or decoded belongs. This is a prediction method that uses points to calculate predicted values. Intra prediction is a prediction method that calculates a prediction value using a first reference 3D point belonging to a first 3D point group (first frame) to which a 3D point to be encoded or decoded belongs.

3D data encoding apparatus 100 includes grouping section 101, buffer 102, adaptive quantization section 103, adaptive inverse quantization section 104, buffer 105, intra prediction section 106, buffer 107, and motion compensation. It includes a section 108 , an inter prediction section 109 , a switching section 110 and an entropy coding section 111 .

The grouping unit 101 extracts a point group to be a prediction tree (Predtree), which is a unit of encoding, from the target point group, which is the data of the input point group to be encoded, and sets it as one group. . In the input target point group, the position of the point group is represented by, for example, three-dimensional polar coordinates.

FIG. 2 is a diagram showing an example of a prediction tree according to this embodiment.

A prediction tree 200 indicates a reference relationship in predicting the positions of a plurality of three-dimensional points forming a point cloud set in one group. The prediction tree 200 includes, for example, a first node 201 , a second node 211 and a third node 212 that are child nodes of the first node, and a fourth node 221 that is a child node of the second node 211 . A first node 201, a second node 211, a third node 212, and a fourth node 221 represent different 3D points. The first node 201 is higher than the second node 211 and the third node 212 , and the second node 211 is higher than the fourth node 221 . The first node 201 is a node referred to in calculating the predicted value of the second node 211 and calculating the predicted value of the third node 212 . The first node 201 may be referenced in calculating the predicted value of the fourth node 221 . The second node 211 is a node referred to in calculating the predicted value of the fourth node 221 . In this way, one or more nodes higher than the lower node are referred to in calculating the predicted value of the lower node. Which node is used and how the predicted value is calculated may be determined according to the prediction mode.

The encoding order and decoding order of the prediction tree 200 may be depth-first order or breadth-first order. The prediction tree 200 may be generated by sequentially processing a plurality of 3D points arranged in Morton's order from the leading 3D point. For example, the prediction tree 200 may be generated such that the closest 3D point to the 3D point to be processed in the process of generating the prediction tree 200 is a child node of the 3D point to be processed.

Depth-priority order is that the root node is first, then the lower nodes (child nodes) of the previous node are arranged in order, and then the remaining multiple nodes are arranged in order of increasing depth. The order is determined by repeating the order of the highest node in the list, and then the lower nodes (child nodes) of the previous node, up to the leaf node. In FIG. 2, the depth priority order is, for example, the first node 201, the second node 211, the fourth node 221, and the third node 212. In FIG.

Breadth-first order is the order determined by arranging the nodes included in each layer of the prediction tree in ascending order, starting with the root node. In FIG. 2, the breadth-first order is, for example, the order of the first node 201, the second node 211, the third node 212, and the fourth node 221. In FIG.

The polar coordinates of the point group to be encoded may be obtained by converting the orthogonal coordinates of the input point group into polar coordinates, or the input point group itself may be polar coordinates. The polar coordinates may be expressed as polar coordinates (d, Φ, θ) using these three elements, for example, the distance d from the sensor position to the target point, the horizontal angle Φ, and the elevation angle θ. Further, instead of the elevation angle θ, sensor information such as laser_id of Lidar may be used. For example, the polar coordinates may be expressed as (d, Φ, laser_id) by using the laser_id having the scan angle closest to the elevation angle θ instead of the elevation angle. That is, in this way, laser_id may be used in polar coordinates as a value indicating elevation angle θ. By using laser_id instead of elevation angle θ, the amount of information can be reduced, and coding efficiency can be improved.

The buffer 102 holds the generated prediction tree. For example, the buffer 102 may initialize data held for each prediction tree. A plurality of three-dimensional points included in the prediction tree (Predtree) held in the buffer 102 are sequentially subjected to encoding processing one by one.

Then, the difference (first residual signal) between each of the plurality of three-dimensional points included in the prediction tree (Predtree) and the selected prediction point is calculated. This first residual signal is also called a prediction residual. Also, the first residual signal is an example of the first residual.

Adaptive quantization section 103 adaptively quantizes the first residual signal. The entropy coding unit 111 entropy-codes the adaptively quantized first residual signal to generate coded data, and outputs (generates) a bitstream containing the coded data.

Adaptive inverse quantization section 104 adaptively inversely quantizes the first residual signal adaptively quantized by adaptive quantization section 103 . The adaptively inverse quantized first residual signal is added to the prediction value based on the selected prediction points (one or more candidate points) to obtain three-dimensional points (reference points) used for intra prediction and inter prediction. is decoded as Note that the predicted value is calculated based on position information of one or more candidate points. The buffer 105 holds the decoded intra-prediction reference point group. For example, the buffer 105 may initialize data held for each prediction tree (target point group). The buffer 107 also holds reference point groups for inter prediction. For example, the buffer 107 may initialize data held for each prediction tree (target point group).

The intra prediction unit 106 selects intra prediction points to be used for prediction, such as a plurality of 3D points (reference point group for intra prediction) included in a prediction tree (Predtree) including a 3D point to be encoded. The information in the prediction tree (Predtree) is referred to and determined by a predetermined method. For example, the intra prediction unit 106 uses two 3D points (decoding points) adaptively dequantized immediately before the 3D point to be encoded (for example, an ancestor node such as a parent node of a prediction tree). Intra prediction points may be determined by extrapolation or the like.

A motion compensation unit 108 reproduces an encoded point group based on a plurality of 3D points (a plurality of decoding points) included in a prediction tree (Predtree) including a 3D point to be encoded, and performs encoding. By detecting the displacement between the coded point cloud and the point cloud to be encoded (motion detection) and correcting the coded point cloud based on the detected displacement (motion compensation), Generate an inter-prediction point cloud, which is a reference point cloud for prediction.

The inter prediction unit 109 determines inter prediction points to be used for prediction by a predetermined method based on the motion-compensated inter prediction point group. For example, the inter prediction unit 109 may select a point closest to the intra prediction point from the inter prediction point group as the inter prediction point, or may select the point that was encoded immediately before (last) without referring to the intra prediction point. A 3D point near a 3D point (e.g., a 3D point corresponding to an ancestor node such as a parent node of a prediction tree) (e.g., the 3D point closest to the previously encoded 3D point) as an inter prediction point You may choose.

Also, the inter prediction unit 109 may determine an inter prediction point using the decoded horizontal angle Φp and laser_idp of the parent node of the prediction tree (Predtree). For example, the inter prediction unit 109 selects from among the inter prediction point group, the value Φp+offset 3D points with Φ close in value to may be selected as inter-prediction points. Here, offset is a value of 0 or more, and the value may be determined according to, for example, the angular velocity of horizontal scanning of lidar. As a result, the 3D data encoding apparatus 100 can generate prediction values close to each value of the polar coordinates (d, Φ, laser_id) of the encoding target point from the inter-prediction point cloud, and can improve the encoding efficiency. . A point to be encoded is an example of a first three-dimensional point. Also, the encoding target point may be an example of a second three-dimensional point.

The switching unit 110 selects either intra prediction points or inter prediction points to determine prediction points to be used for prediction. In this way, in the three-dimensional data encoding device 100, position information of one or more candidate points among a plurality of encoded three-dimensional points is determined as a predicted point, and a predicted value is calculated based on the predicted point. be. The intra prediction unit 106 and the inter prediction unit 109 determine a prediction point (intra prediction point or inter prediction point) based on the 3D point encoded immediately before the 3D point to be encoded. That is, 3D data encoding apparatus 100 determines one or more candidate points for calculating a predicted value based on one or more reference points among a plurality of encoded 3D points. One or more reference points are 3D points encoded immediately before the 3D point to be encoded, and correspond to, for example, parent nodes (ancestor nodes) of one 3D point to be encoded in the prediction tree. It may be a three-dimensional point that

Note that the 3D data encoding device 100 may select either one of the intra prediction points and the inter prediction points as the prediction point according to the procedure of the flowchart of FIG. 4, which will be described later. Information about prediction (prediction information) for selecting which of intra prediction points and inter prediction points to use as prediction points may be entropy-encoded and described in the header of each three-dimensional point. Alternatively, it may be described interleaved with each 3D point. Information related to motion compensation such as motion vectors may be described in the header of a frame or prediction tree (Predtree), or may be entropy-encoded and described in the header of each 3D point, It may be described interleaved with each 3D point.

For example, when the three-dimensional data encoding device 100 performs encoding by switching frames of an inter prediction point group to be referenced for each node of a prediction tree (Predtree), the identification number of the reference frame selected for inter prediction (hereinafter referred to as frame number) may be added to each node of the prediction tree (Predtree). Thereby, the 3D data encoding apparatus 100 can improve encoding efficiency by selecting the optimum reference frame number for each node and performing inter prediction. Also, the 3D data decoding device 120 (see later) can determine from which reference frame to generate an inter prediction point by decoding the reference frame number added to each node of the bitstream. A bitstream with improved coding efficiency can be appropriately decoded by selecting the optimum reference frame number and inter-predicting it.

In addition, the reference point group for inter prediction may be a point group included in an encoded frame different from the encoding target frame, or may be a point group included in the same frame as the encoding target frame. It can be a group.

In this way, the three-dimensional data encoding apparatus 100 predicts the encoding target point using inter prediction in addition to intra prediction, thereby reducing the amount of information in the first residual signal to be entropy encoded. can improve the coding efficiency.

Note that the 3D data encoding apparatus 100 does not always need to refer to the inter prediction point, and can be set at a predetermined time interval (for example, every 1 second), a predetermined frame interval (for example, every 30 frames), or a 3D The buffer 107 storing the inter-prediction reference point cloud is initialized at an arbitrary timing for notification to the data decoding device, so that encoding is performed based only on the information of the point cloud to be encoded. may As a result, in the corresponding three-dimensional data decoding device, it is possible to start jumping playback from a point group that is not the head of a bitstream that does not refer to an inter prediction point, and improve the random access property or error resistance of the bitstream. there is a possibility.

When the input point group to be encoded has coordinates represented by orthogonal coordinates as position information, the three-dimensional data encoding device 100 encodes the position information indicated by the coordinates represented by orthogonal coordinates. When the input point group to be encoded has coordinates expressed in polar coordinates as position information, the three-dimensional data encoding device 100 encodes the position information indicated by the coordinates expressed in polar coordinates.

FIG. 3 is a block diagram of the three-dimensional data decoding device 120 according to this embodiment. Note that FIG. 3 shows a processing unit related to decoding position information (geometry) of a point cloud, but the three-dimensional data decoding device 120 includes other processing units such as a processing unit that decodes attribute information of a point cloud. A processing unit may be provided. The three-dimensional data decoding device 120 performs inter-prediction decoding for decoding a point group to be decoded while referring to a decoded point group. For example, the 3D data decoding device 120 decodes the bitstream generated by the 3D data encoding device 100 shown in FIG.

The three-dimensional data decoding device 120 includes an entropy decoding unit 121, an adaptive inverse quantization unit 122, a buffer 123, an intra prediction unit 124, a buffer 125, a motion compensation unit 126, an inter prediction unit 127, and a switching unit. 128.

The 3D data decoding device 120 acquires the bitstream generated by the 3D data encoding device 100 .

The entropy decoding unit 121 generates an adaptively quantized first residual signal by entropy decoding the input bitstream for each three-dimensional point of a prediction tree (Predtree). Adaptive inverse quantization section 122 adaptively inverse quantizes the adaptively quantized first residual signal to reproduce the first residual signal. The first residual signal of each 3D point is added to the prediction value based on the prediction point corresponding to each 3D point, and then generated (output) as a decoded point. In other words, 3D data decoding device 120 calculates the position information of one 3D point to be decoded by adding the prediction value and the prediction residual.

The buffer 123 holds the generated decoding points as a reference point group for intra prediction. For example, the buffer 123 may initialize data held for each prediction tree (target point group). The buffer 125 also holds the generated decoding points as a reference point group for inter prediction. For example, the buffer 125 may initialize data held for each prediction tree (target point group).

The intra prediction unit 124 selects intra prediction points to be used for prediction, such as a plurality of 3D points (reference point group for intra prediction) included in a prediction tree (Predtree) including a 3D point to be decoded. The information in the tree (Predtree) is referred to and determined by a predetermined method. For example, the intra prediction unit 124 uses two 3D points (decoding points) adaptively dequantized immediately before the 3D point to be decoded (for example, an ancestor node such as a parent node of a prediction tree). Intra prediction points may be determined by inserting.

The motion compensation unit 126 reproduces the decoded point group based on a plurality of 3D points (a plurality of decoding points) included in a prediction tree (Predtree) including the 3D point to be decoded, and reproduces the decoded point group. An inter-prediction point group, which is a reference point group for inter-prediction after alignment, is generated by correcting (motion compensation) the displacement between the group and the point group to be decoded.

The inter prediction unit 127 determines inter prediction points to be used for prediction by a predetermined method based on the motion-compensated inter prediction point group. For example, the inter prediction unit 127 may select a point closest to the intra prediction point from the group of inter prediction points as the inter prediction point, or may select the point closest to the intra prediction point as the inter prediction point. A 3D point (e.g., the closest 3D point to the previously decoded 3D point) near the original point (e.g., the 3D point corresponding to an ancestor node such as the parent node of the prediction tree) is selected as the inter prediction point. may

Also, the inter prediction unit 127 may determine an inter prediction point using the decoded horizontal angle Φp and laser_idp of the parent node of the prediction tree (Predtree). For example, the inter prediction unit 127 selects from among the inter prediction point group, the value Φp+offset 3D points with Φ close in value to may be selected as inter-prediction points. Here, offset is a value of 0 or more, and the value may be determined according to, for example, the angular velocity of horizontal scanning of lidar. As a result, the 3D data decoding device 120 generates a prediction value close to each value of the polar coordinates (d, Φ, laser_id) of the decoding target point from the inter-prediction point cloud to appropriately generate a bitstream with improved coding efficiency. can be decrypted. A decoding target point is an example of a first three-dimensional point. Also, the decoding target point may be an example of the second three-dimensional point.

The switching unit 128 selects either intra prediction points or inter prediction points to determine prediction points to be used for prediction. In this way, in the 3D data decoding device 120, the position information of one or more candidate points among the plurality of decoded 3D points is determined as the prediction point, and the prediction value is calculated based on the prediction point. The intra prediction unit 124 and the inter prediction unit 127 determine a prediction point (intra prediction point or inter prediction point) based on the 3D point decoded immediately before the 3D point to be decoded. That is, 3D data decoding device 120 determines one or more candidate points for calculating the predicted value based on one or more reference points among the plurality of decoded 3D points. The one or more reference points are 3D points encoded immediately before the 3D point to be decoded. It may be the origin point.

Note that the 3D data decoding device 120 may select either one of the intra prediction points and the inter prediction points as the prediction point according to the procedure of the flowchart of FIG. 5, which will be described later. In addition, three-dimensional data decoding device 120 selects between intra prediction points and inter prediction points based on prediction information (prediction information) for selecting intra prediction points and inter prediction points. You may select any one of as a prediction point. The prediction information may be entropy-encoded and described in the header of each 3D point, or may be described interleaved with each 3D point. Information related to motion compensation such as motion vectors may be described in the header of a frame or prediction tree (Predtree), entropy-encoded and described in the header of each point, or may be described in the header of each point. It may be described interleaved with the origin point. In this way, prediction information or information about motion compensation may be notified to the 3D data decoding device 120 from the corresponding 3D data encoding device 100 . In addition, the reference point group for inter prediction may be a point group included in an encoded frame different from the encoding target frame, or may be a point group included in the same frame as the encoding target frame. It can be a group.

In this way, the 3D data decoding device 120 predicts the decoding target points using not only intra prediction but also inter prediction, so that a bit stream (for example, A point cloud can be decoded from the bitstream output from the 3D data encoding apparatus 100 of FIG.

Note that the 3D data decoding device 120 does not always need to refer to the inter prediction point, but rather at a predetermined time interval (for example, every 1 second), a predetermined frame interval (for example, every 30 frames), or a corresponding cubic Decoding is performed based only on the information of the point cloud to be decoded by initializing the buffer 125 storing the inter-prediction reference point cloud at an arbitrary timing notified from the original data encoding device 100 or the like. You may do so. As a result, the three-dimensional data decoding device 120 can start jumping playback from a point group that is not the head of a bitstream that does not refer to an inter-prediction point, and can improve random access and error resistance of the bitstream. have a nature.

The three-dimensional data decoding device 120 decodes the position information indicated by the coordinates expressed by the orthogonal coordinates when the bitstream has encoded data in which the position information indicated by the coordinates expressed by the orthogonal coordinates is encoded. . The 3D data decoding device 120 decodes the position information indicated by the coordinates expressed in polar coordinates when the bitstream has encoded data in which the position information indicated by the coordinates expressed in polar coordinates is encoded.

FIG. 4 is a flowchart showing an example of a procedure for encoding each 3D point of a prediction tree (Predtree) in the 3D data encoding device 100. FIG.

In this example, the 3D data encoding device 100 first determines intra prediction points from a reference point group for intra prediction (S101). The 3D data encoding apparatus 100 may determine intra prediction points using, for example, the method of determining prediction points using a prediction tree, disclosed in the embodiments already described. For example, 3D data encoding apparatus 100 generates a prediction tree using a plurality of encoded 3D points, and selects one or more candidate points from the plurality of encoded 3D points based on the prediction tree. You may choose. The 3D data encoding apparatus 100 may determine, as an intra prediction point, a prediction point with the smallest code amount among at least one intra prediction point determined by at least one of the above methods. In addition, the 3D data encoding apparatus 100 uses at least one intra-prediction point determined by at least one of the above methods, the prediction point having the smallest absolute value sum (or square sum) of coordinate residuals may be determined as intra prediction points.

Next, the 3D data encoding device 100 outputs intra-prediction-related parameters (S102). If there are two or more candidates for the intra prediction point determination method determined in step S101, the 3D data encoding apparatus 100 outputs information indicating the selected candidate determination method to the bitstream as an intra prediction related parameter. may

Next, the 3D data encoding device 100 determines inter prediction points by referring to at least one candidate point extracted from the inter prediction point group. For example, the 3D data encoding apparatus 100 may determine one candidate point as an inter prediction point, or determine a prediction point having coordinates that are average values of coordinates of a plurality of candidate points as an inter prediction point. good too. Alternatively, the 3D data encoding apparatus 100 may determine a prediction point having, as coordinates, an average value of coordinates of an intra prediction point and at least one candidate point, as an inter prediction point.

Here, the 3D data encoding device 100 may search for a point located near the intra prediction point as at least one candidate point (S103).

Next, the 3D data encoding device 100 may assign a smaller index value to each of the determined at least one inter prediction point in descending order of proximity to the intra prediction point (S104).

Next, the three-dimensional data encoding apparatus 100 determines whether or not the search has ended (S105), and if the search has ended (Yes in S105), the process proceeds to the next step S106, where the search has not ended. If so (No in S105), the process returns to step S103. The end of the search may be determined by finding a predetermined number of inter-prediction points, by searching for all points in a predetermined range, or by searching for a predetermined number of inter-prediction points. or searching all of the point cloud in the predetermined range.

Next, the 3D data encoding device 100 determines a prediction method (S106). Specifically, the 3D data encoding apparatus 100 determines whether the method for determining prediction points is intra prediction or inter prediction. That is, the 3D data encoding apparatus 100 determines whether to determine the intra prediction point as the prediction point or the inter prediction point as the prediction point. For example, the 3D data encoding apparatus 100 may determine the prediction method for the prediction point with the smallest code amount among the intra prediction points and the inter prediction points. Also, the 3D data encoding apparatus 100 may determine the prediction method for the prediction point with the smallest absolute value sum (or square sum) of the coordinate residuals among the intra prediction points and the inter prediction points.

Three-dimensional data encoding apparatus 100 determines whether the determined prediction method mode is an inter mode indicating that the prediction method is inter prediction or an intra prediction mode indicating that the prediction method is intra prediction. Determine (S107).

When the determined prediction method is inter prediction (inter mode in S107), the 3D data encoding apparatus 100 adds identification information (e.g., a flag) indicating that the inter prediction point is determined as the prediction point to the bitstream. Output (S108).

Next, the 3D data encoding apparatus 100 uses bit Output to a stream (S109). Index values may be assigned to one or more candidate points that are used to determine predicted values.

Further, when the determined prediction method is intra prediction (intra mode in S107), 3D data encoding apparatus 100 sets identification information (for example, a flag) indicating that intra prediction points have been determined as prediction points by bit Output to a stream (S111). The identification information in steps S108 and S111 is information indicating whether an inter prediction point or an intra prediction point is determined as a prediction point.

After step S109 or step S111, the 3D data encoding device 100 references the prediction points obtained by the determined prediction method and encodes the coordinate information of the 3D points to be encoded (S110).

In this way, the 3D data encoding apparatus 100 refers to the inter prediction point group and the intra prediction points to determine at least one inter prediction point, and obtains a prediction point from these intra prediction points and inter prediction points. A method is determined, and the position information (coordinate information) of the 3D point to be encoded is encoded with reference to the predicted point.

Note that in step S103, without referring to the intra prediction point, the 3D point encoded immediately before (for example, the 3D point corresponding to the ancestor node such as the parent node of the prediction tree) depends on the intra prediction related parameter. Inter-prediction points may be searched for in the vicinity of three-dimensional points that are uniquely determined independently of each other. In this case, step S102 may not be performed immediately after step S101, and step S102 may be performed immediately after step S111.

FIG. 5 is a flowchart showing an example of adaptive quantization processing for residual signals in encoding processing.

Adaptive quantization is performed in step S110.

The adaptive quantization unit 103 determines whether or not the residual of the coordinate components of the 3D point to be adaptively quantized next is the residual of the horizontal angle Φ (S121).

If the residual of the coordinate components of the next three-dimensional point to be adaptively quantized is the residual of the horizontal angle Φ (Yes in S121), adaptive quantization section 103 performs A distance d in polar coordinates is obtained (S122).

Next, the adaptive quantization unit 103 uses the acquired distance d to calculate the quantization step QS for the residual of the horizontal angle Φ (S123). The quantization step QS is specifically a quantization step (that is, a quantization width).

Next, the adaptive quantization unit 103 quantizes the residual of the horizontal angle Φ at the quantization step QS, and calculates the quantized value of the horizontal angle Φ after quantization (S124). Note that the quantized value of the horizontal angle Φ may be calculated using Equation 1 below.

Quantized value of residual signal of horizontal angle Φ = residual signal of horizontal angle Φ/QS (Equation 1)

If the residual of the coordinate components of the three-dimensional point to be adaptively quantized next is not the residual of the horizontal angle Φ (No in S121), adaptive quantization section 103 performs cubic If the residual of the coordinate component of the original point is the residual of the distance d or the residual of the elevation angle θ, the residual of the distance d or the residual of the elevation angle θ is quantized (S125). For example, the adaptive quantization section 103 may perform adaptive quantization on the residual of the distance d or the residual of the elevation angle θ in the same manner as the quantization to the residual of the horizontal angle Φ. Quantization may be performed in the quantization step.

Note that the quantization step QS may be calculated so as to decrease as the distance d increases (that is, increase as the distance d decreases). Specifically, the quantization step QS may be calculated using Equation 2 below.

　QS=Φprecision/d (Equation 2)

As shown in Equation 2, the quantization step QS may be calculated by dividing a predetermined reference value Φprecision by the distance d. Although the distance d used in Equation 2 is the distance in the polar coordinates of the three-dimensional point to be encoded, it is not limited to this. The distance d may be the distance in polar coordinates of the 3D point encoded in the encoding order prior to the 3D point to be encoded. Further, the distance d may be a distance calculated using distances of polar coordinates of a plurality of three-dimensional points encoded in an encoding order prior to the encoding target three-dimensional point. It may be the average, median, maximum value, minimum value, etc. of the distances of the polar coordinates of a plurality of 3D points encoded in the encoding order prior to the target 3D point. Further, the distance d is calculated using the distance of the three-dimensional point to be encoded and the distance of the polar coordinates of one or more three-dimensional points encoded in the encoding order prior to the three-dimensional point to be encoded. It may be a distance, for example, the average of the distance of the 3D point to be encoded and the distance of the polar coordinates of one or more 3D points encoded in the encoding order before the 3D point to be encoded, It may be a median value, maximum value, minimum value, or the like.

Further, the distance d may be a distance calculated using the polar coordinate distances of a plurality of three-dimensional points encoded in the encoding order after the encoding target three-dimensional point. It may be the average, median, maximum value, minimum value, etc. of the polar coordinate distances of a plurality of 3D points encoded in the encoding order after the target 3D point. Further, the distance d is calculated using the distance of the three-dimensional point to be encoded and the distance of the polar coordinates of one or more three-dimensional points encoded in the encoding order after the three-dimensional point to be encoded. It may be a distance, for example, the average of the distance of the 3D point to be encoded and the distance of the polar coordinates of one or more 3D points encoded in the encoding order after the 3D point to be encoded, It may be a median value, maximum value, minimum value, or the like.

It should be noted that the one or more 3D points encoded in the encoding order prior to the 3D point to be encoded is an example of the one or more second 3D points or the one or more fourth 3D points. In addition, the one or more 3D points encoded in the encoding order after the 3D point to be encoded is another example of the one or more second 3D points or the one or more fourth 3D points. be. The one or more second 3D points are 3D points that are included in the plurality of 3D points and located around the 3D point to be encoded. A 3D point located in the periphery means that the spatial position is close to the 3D point to be encoded, or the temporal position is close to the 3D point to be encoded. A spatial position close to the 3D point to be encoded means a 3D point located at a position where the distance from the 3D point to be encoded is less than a predetermined distance, or a 3D point to be encoded are three-dimensional points in an order whose difference from the coding order of is less than a predetermined difference. A temporal position close to a 3D point to be encoded is a 3D point included in a frame whose generation order difference is less than a predetermined difference in the generation order of a plurality of frames. Note that the quantization step QS is not limited to being calculated by Equation 2. For example, the quantization step QS may be calculated by referring to a table in which values are associated with distances d in multiple stages. That is, the quantization step QS is determined to be the first value when the distance d is included in the first distance range, and is determined to be the second distance range where the distance d is greater than the maximum value of the first distance range. If included, a second value that is less than the first value may be determined. That is, the quantization step QS may be determined to be a large first value for a short first distance range, and may be determined to a small second value for a long second distance range. The relationship between the quantization step QS and the distance d does not need to be a linear function, and may satisfy a monotonically decreasing relationship in which the quantization step QS decreases as the distance d increases. In this case, the table may associate the first distance range and the first value, and may associate the second distance range and the second value. Further, the table is not limited to dividing the distance range into two stages, and may be divided into three or more stages. Even in this case, the distance indicated by the divided distance range is large. A quantization step QS having a smaller value should be associated.

Thus, the quantization step QS is set so that the value of the quantization step QS decreases as the value of the distance d increases, and the value of the quantization step QS increases as the value of the distance d decreases. calculated to As a result, the larger the distance d, the smaller the quantization step QS, so that quantization errors can be suppressed, and the smaller the distance d, the larger the quantization step QS, so more information can be reduced. .

This is because the farther the distance to the subject, the more difficult it is to predict the horizontal angle Φ of the 3D point on the subject, and the larger the residual signal value. Therefore, by decreasing the quantization step QS as the distance d increases, the quantization error when the residual signal is quantized can be suppressed, and the coding efficiency can be improved. Conversely, the closer the distance to the object is, the easier it is to predict Φ of the three-dimensional point on the object, and the smaller the value of the residual signal. Therefore, the coding efficiency can be improved by quantizing the residual signal by increasing the quantization step QS as the distance d decreases.

Note that Φprecision used in Equation 2 is a parameter representing the bit precision of the horizontal angle Φ. The greater the bit precision, the more accurate the predicted value can be generated, and hence the smaller the calculated residual. Conversely, the smaller the bit precision value, the larger the calculated residual error because the more inaccurate predicted value is calculated. Therefore, the smaller the bit precision, the smaller the quantization step to suppress the quantization error, and the higher the bit precision, the larger the quantization step, thereby improving the coding rate.

Also, Φprecision may be a parameter representing the prediction accuracy of the horizontal angle Φ. The larger the prediction accuracy of the horizontal angle Φ, the more accurate the prediction value can be generated, and the smaller the calculated residual. Conversely, the smaller the prediction accuracy value, the larger the calculated residual error, because an inaccurate prediction value is calculated. Therefore, the smaller the prediction accuracy, the larger the quantization step and the more the encoding efficiency can be improved. The prediction accuracy is, for example, the bit accuracy of predicted values, the bit accuracy of interpolation, and the like.

The three-dimensional data encoding device 100 may add Φprecision to the header of GPS (geometry parameter set), slice, etc. and include it in the bitstream. Here, the bit precision represents the number of bits expressing the horizontal angle Φ. For example, a bit precision of 32 bits indicates that the horizontal angle Φ is represented by a 32-bit numerical value. Note that the 3D data encoding apparatus 100 may add a parameter for calculating Φprecision to the header of GPS, slice, or the like. By adding Φprecision to the bitstream, the 3D data decoding device 120 can know the bit precision of the horizontal angle Φ used in the 3D data encoding device 100 by decoding the bitstream. I can decode the bitstream properly.

Note that Φprecision may have different values for intra prediction and inter prediction. For example, ΦprecisionIntra for intra prediction and ΦprecisionInter for inter prediction may be prepared, and the value of Φprecision may be switched according to intra prediction or inter prediction. As a result, the quantization step QS can be switched depending on whether intra prediction or inter prediction, and coding efficiency can be improved. For example, if intra prediction tends to result in a larger residual signal after prediction of Φ compared to inter prediction, ΦprecisionIntra is set smaller than ΦprecisionInter so that the quantization step QS of intra prediction is inter-predicted. The quantization step QS may be smaller than the prediction quantization step, and the coding efficiency may be improved by suppressing the quantization error of the intra-prediction residual signal.

Thus, since different Φprecision is used for inter prediction and intra prediction, the calculated quantization step QS can also be calculated to have different values for inter prediction and intra prediction. That is, in quantization, the first residual between the horizontal angle Φ and the inter prediction value predicted by inter prediction is calculated using the first quantization step QS1. Also, in quantization, a second residual between the horizontal angle Φ and an intra prediction value predicted by intra prediction is calculated using a second quantization step QS2 different from the first quantization step QS1. Specifically, the first quantization step QS1 and the second quantization step QS2 may be calculated using Equations 3 and 4 below, respectively.

　QS1=ΦprecisionInter/d (Equation 3)

　QS2=ΦprecisionIntra/d (Formula 4)

Thus, the first quantization step QS1 may be calculated using ΦprecisionInter as the first reference value and the distance d. Also, the second quantization step QS2 may be calculated using ΦprecisionIntra and the distance d as the second reference value.

Note that the three-dimensional data encoding device 100 may add ΦprecisionIntra and ΦprecisionInter to headers such as GPS and slices and include them in the bitstream. That is, the 3D data encoding apparatus 100 may generate a bitstream including the first reference value and the second reference value. ΦprecisionIntra is an example of a second reference value in intra prediction. ΦprecisionIntra may be a parameter representing the bit precision of the horizontal angle Φ, or may be a parameter representing the prediction accuracy of the horizontal angle Φ. Also, ΦprecisionInter is an example of a first reference value in inter prediction. ΦprecisionInter may be a parameter representing the bit precision of the horizontal angle Φ, or may be a parameter representing the prediction accuracy of the horizontal angle Φ. Thus, the reference value may be a value that indicates the accuracy of predictive values that can be generated by intra-prediction and inter-prediction. The three-dimensional data encoding device 100 can improve encoding efficiency while suppressing quantization errors by switching the quantization step according to the reference value.

Also, the 3D data encoding device 100 may add parameters for calculating ΦprecisionIntra and ΦprecisionInter to headers such as GPS and slices. By adding ΦprecisionIntra and ΦprecisionInter to the bitstream, the 3D data decoding device 120 converts the bit precision of the horizontal angle Φ of each of the intra prediction and the inter prediction used in the 3D data encoding device 100 into the bitstream. can be known by decoding , and the bitstream can be decoded appropriately.

An example of switching Φprecision, which is the bit precision of the horizontal angle Φ, between intra prediction and inter prediction is shown, but this is not necessarily the case. For example, between an intra prediction frame or slice and an inter prediction frame or slice. The bit precision of the horizontal angle Φ may be switched. Also, Φprecision may be switched in frame units, slice units, or stream units. That is, Φprecision may be switched even if intra prediction is continued when the frame or slice of the 3D point to be encoded is switched in intra prediction. Also, Φprecision may be switched even if inter prediction is continued when the frame or slice of the 3D point to be encoded is switched in inter prediction. In this case, the 3D data encoding apparatus 100 may add the bit precision of the horizontal angle Φ or information for generating it to the header of each frame or slice. In this way, Φprecision may have the same value in units of a predetermined point group including one or more 3D points, that is, in units of frames, units of slices, units of streams, and the like. Similarly, the same value may be used for ΦprecisionInter in units of predetermined point groups including one or more three-dimensional points, that is, in units of frames, units of slices, units of streams, and the like. For ΦprecisionIntra, the same value may be used in units of a predetermined point group including one or more 3D points, that is, in units of frames, units of slices, units of streams, and the like.

FIG. 6 is a flowchart showing an example of adaptive inverse quantization processing for quantized values generated by adaptive quantization in encoding processing.

Adaptive inverse quantization is performed after the adaptive quantization described in FIG. 5, for example, in step S110.

Adaptive inverse quantization section 104 determines whether or not the quantized value of the residual of the coordinate component of the 3D point to be adaptively inverse quantized next is the quantized value of the residual of horizontal angle Φ. (S131).

Adaptive inverse quantization section 104 performs adaptive inverse quantization next when the quantized value of the residual of the coordinate component of the encoding target 3D point is the quantized value of the residual of horizontal angle Φ (Yes in S131 ), the distance d in the polar coordinates of the three-dimensional point to be encoded is obtained (S132).

Next, the adaptive inverse quantization unit 104 uses the acquired distance d to calculate the inverse quantization step iQS for the quantized value of the residual of the horizontal angle Φ (S133). The inverse quantization step iQS is also called an inverse quantization width. Note that if the same three-dimensional point is processed, the quantization step QS calculated in step S123 and the inverse quantization step iQS calculated in step S133 have the same value. That is, the inverse quantization step iQS is calculated in the same manner as the quantization step QS.

Next, the adaptive inverse quantization unit 104 inversely quantizes the quantized value of the residual of the horizontal angle Φ at the inverse quantization step iQS to calculate the residual of the horizontal angle Φ (S134). Note that the residual signal of the horizontal angle Φ may be calculated using Equation 5 below.

　Horizontal angle Φ residual signal = horizontal angle Φ residual signal quantized value * iQS (Equation 5)

Note that adaptive inverse quantization section 104 performs adaptive inverse quantization next when the quantized value of the residual of the coordinate component of the encoding target 3D point is not the quantized value of the residual of horizontal angle Φ (in S131 No), that is, if the quantized value of the residual of the coordinate component of the three-dimensional point to be encoded is the quantized value of the residual of the distance d or the quantized value of the residual of the elevation angle θ, then the residual of the distance d The quantized value of the difference or the quantized value of the residual of the elevation angle θ is inversely quantized (S135). For example, the adaptive inverse quantization unit 104 performs inverse quantization to the quantized value of the residual of the horizontal angle Φ with respect to the quantized value of the residual of the distance d or the quantized value of the residual of the elevation angle θ. Adaptive inverse quantization may be performed, or quantization may be performed with a fixed inverse quantization step.

In the adaptive quantization described using FIG. 5, quantization is adaptively executed according to the distance d of the encoding target point for the residual signal of the horizontal angle Φ in the polar coordinates of the encoding target point. In the adaptive inverse quantization described with reference to FIG. 6, the quantized value of the residual signal of the horizontal angle Φ in the polar coordinates of the encoding target point is adaptively inversely quantized according to the distance d of the encoding target point. is executed. Coding efficiency can be improved by adaptively controlling the quantization of the residual signal of the horizontal angle Φ according to the value of the distance d. Note that the residual signal of the horizontal angle Φ here is generated from the difference between the intra prediction value calculated by the intra prediction and the input value to be encoded when the encoding target point is encoded by intra prediction. is the residual signal. Adaptive quantization may be performed on the residual signal. In addition, when the encoding target point is encoded by inter prediction, the residual signal of the horizontal angle Φ is the residual signal generated from the difference between the inter prediction value calculated by inter prediction and the input value to be encoded. and adaptive quantization may be performed on the residual signal. By applying common adaptive quantization to the residual signal generated by at least one of intra prediction and inter prediction in this way, while suppressing the circuit scale of additional hardware and the amount of software code , the coding efficiency can be improved.

It should be noted that although an example of applying adaptive quantization to intra-prediction and inter-prediction residual signals has been shown in the present embodiment, the present invention is not necessarily limited to this. For example, the 3D data encoding apparatus 100 may apply adaptive quantization to intra-prediction residual signals and not apply adaptive quantization to inter-prediction residual signals. This makes it possible to improve the encoding efficiency of intra prediction while suppressing the amount of processing. Further, for example, the 3D data encoding apparatus 100 may apply adaptive quantization to the inter-prediction residual signal and may not apply adaptive quantization to the intra-prediction residual signal. This makes it possible to improve the inter-prediction encoding efficiency while suppressing the amount of processing.

In addition, in this embodiment, an example of adaptive quantization and adaptive inverse quantization with respect to the horizontal angle Φ of the polar coordinates is shown, but it is not necessarily limited to this. For example, when the elevation angle θ is used instead of the laser_id as the polar coordinates of the three-dimensional point, adaptive quantization and adaptive inverse quantization are applied to the elevation angle θ according to the distance d using the same method as for the horizontal angle Φ. I don't mind. Thereby, the encoding efficiency of the elevation angle θ can be improved.

Although Φprecision is used as a parameter for controlling the quantization step QS in this embodiment, any information may be used as long as it is a quantization parameter that can control the value of the quantization step QS. For example, a quantization parameter for the horizontal angle Φ may be added to a header or the like, and a value calculated from the quantization parameter may be used instead of Φprecision. Also, in this case, three-dimensional data encoding apparatus 100 prepares a first quantization parameter for inter prediction and a second quantization parameter for intra prediction of horizontal angle Φ, respectively. A 2-quantization parameter may be added to a header such as GPS or slice. As a result, the quantization parameter for the horizontal angle Φ can be appropriately set for intra prediction and inter prediction, and coding efficiency can be improved.

Note that when the laser_id is changed from the previously encoded 3D point in the polar coordinates of the 3D point, the 3D data encoding apparatus 100 performs quantization on the 3D point without adaptive quantization. Ordinary quantization may be performed.

FIG. 7 is a flowchart showing an example of a procedure for decoding each 3D point of a prediction tree (Predtree) in the 3D data decoding device 120. FIG. FIG. 7 corresponds to the decoding of the bitstream generated according to the encoding procedure of FIG. That is, the bitstream includes an encoded first residual signal (prediction residual) and index values assigned to candidate points used to calculate the prediction value.

In this example, the 3D data decoding device 120 first acquires intra-prediction-related parameters from the bitstream (S141).

Next, the 3D data decoding device 120 determines intra prediction points based on the acquired intra prediction related parameters (S142). Specifically, the 3D data decoding device 120 determines intra prediction points in the same manner as in step S101 of FIG. The three-dimensional data decoding device 120 is notified of intra-prediction-related parameters from the corresponding three-dimensional data encoding device 100, and determines intra-prediction points according to the intra-prediction-related parameters. The intra-prediction-related parameters are obtained in step S141 and include information specifying at least one method of determining intra-prediction points and parameters accompanying this information.

Next, the 3D data decoding device 120 acquires identification information indicating the prediction method mode from the bitstream (S143).

Next, three-dimensional data decoding device 120 determines whether the acquired identification information indicates an inter prediction mode indicating that the prediction method is inter prediction or an intra prediction mode indicating that the prediction method is intra prediction. Determine (S144).

When the prediction method is inter prediction (inter mode in step S144), the 3D data decoding device 120 acquires inter prediction related parameters from the bitstream (S145).

Next, the 3D data decoding device 120 executes processing (S146 to S149) for determining inter prediction points. Specifically, the 3D data decoding device 120 determines inter prediction points by the same method as steps S103 to S105 in FIG. For example, the 3D data decoding device 120 determines inter prediction points by referring to at least one candidate point extracted from the inter prediction point group. For example, 3D data decoding device 120 may determine one candidate point as an inter prediction point, or may determine a prediction point having coordinates that are average values of coordinates of a plurality of candidate points as an inter prediction point. good. Alternatively, the 3D data decoding device 120 may determine, as an inter prediction point, a prediction point having, as coordinates, an average value of coordinates of an intra prediction point and at least one candidate point.

Here, the 3D data decoding device 120 may search for a point located near the intra prediction point as at least one candidate point (S146).

Next, the 3D data decoding device 120 may give each of the determined at least one inter prediction point a smaller index value in descending order of proximity to the intra prediction point (S147).

Next, the three-dimensional data decoding device 120 determines whether or not the search has ended (S148). If the search has ended (Yes in S148), the process proceeds to the next step S149. (No in S148), the process returns to step S146. The end of the search may be determined by finding a predetermined number of inter-prediction points, by searching for all points in a predetermined range, or by searching for a predetermined number of inter-prediction points. or searching all of the point cloud in the predetermined range.

Next, the 3D data decoding device 120 determines inter prediction points based on inter prediction-related parameters while referring to the inter prediction point group and intra prediction points (S149). For example, the 3D data decoding device 120 uses information about the number of candidate points used to determine the coordinates of the inter prediction points, which are included in the inter prediction related parameters, and an index value assigned to each candidate point used. Inter prediction points are determined by identifying candidate points used for determining the coordinates of inter prediction points based on and determining the coordinates of inter prediction points using the identified candidate points. That is, the 3D data decoding device 120 selects one candidate point from a plurality of decoded 3D points based on the index value included in the bitstream.

After step S149, or in the case of intra mode in step S144, the 3D data decoding device 120 refers to the prediction points obtained by the designated prediction method, and determines the position information (coordinate information) of the 3D point to be decoded. ) is decoded (S150).

In this way, when the prediction method is inter prediction, 3D data decoding device 120 refers to the inter prediction point to decode the coordinate information of the point to be decoded, and when the prediction method is intra prediction, the intra prediction point to decode the coordinate information of the decoding target point.

In addition, in step S146, without referring to the intra prediction point, the 3D point decoded immediately before (for example, the 3D point corresponding to the ancestor node such as the parent node of the prediction tree), regardless of the intra prediction related parameter An inter-prediction point may be searched for in the neighborhood of a three-dimensional point uniquely determined by . In this case, steps S141 and S142 may not be performed immediately before step S143, and steps S141 and S142 may be performed when the intra mode is determined in step S144.

FIG. 8 is a flowchart showing an example of adaptive inverse quantization processing for quantized values in decoding processing.

Adaptive inverse quantization is performed, for example, in step S150.

The adaptive inverse quantization unit 122 determines whether the quantized value of the residual of the coordinate component of the decoding target 3D point to be adaptively inverse quantized next is the quantized value of the residual of the horizontal angle Φ. (S151).

Adaptive inverse quantization section 122 performs adaptive inverse quantization next when the quantized value of the residual of the coordinate component of the decoding target 3D point is the quantized value of the residual of horizontal angle Φ (Yes in S151). , the distance d in the polar coordinates of the three-dimensional point to be decoded is obtained (S152).

Next, the adaptive inverse quantization unit 122 uses the acquired distance d to calculate the inverse quantization step iQS for the quantized value of the residual of the horizontal angle Φ (S153).

Next, the adaptive inverse quantization unit 122 inversely quantizes the quantized value of the residual of the horizontal angle Φ at the inverse quantization step iQS to calculate the residual of the horizontal angle Φ (S154). Note that the residual signal of the horizontal angle Φ may be calculated using Equation 6 below.

　Horizontal angle Φ residual signal = horizontal angle Φ residual signal quantized value * iQS (Equation 6)

Note that adaptive inverse quantization section 104 performs adaptive inverse quantization next when the quantized value of the residual of the coordinate component of the encoding target 3D point is not the quantized value of the residual of horizontal angle Φ (in S151 No), that is, if the quantized value of the residual of the coordinate component of the three-dimensional point to be encoded is the quantized value of the residual of the distance d or the quantized value of the residual of the elevation angle θ, then the residual of the distance d The quantized value of the difference or the quantized value of the residual of the elevation angle θ is inversely quantized (S155). For example, the adaptive inverse quantization unit 104 performs inverse quantization to the quantized value of the residual of the horizontal angle Φ with respect to the quantized value of the residual of the distance d or the quantized value of the residual of the elevation angle θ. Adaptive inverse quantization may be performed, or quantization may be performed with a fixed inverse quantization step.

Note that the inverse quantization step iQS may be calculated so as to decrease as the distance d increases (that is, increase as the distance d decreases) in the same manner as the quantization step QS. Specifically, the inverse quantization step iQS may be calculated using Equation 7 below.

iQS = Φprecision/d (Equation 7)

As shown in Equation 7, the inverse quantization step iQS may be calculated by dividing a predetermined reference value Φprecision by the distance d. Although the distance d used in Equation 7 is the distance in the polar coordinates of the three-dimensional point to be decoded, it is not limited to this. The distance d may be the polar coordinate distance of the 3D point decoded in the decoding order prior to the 3D point to be decoded. Further, the distance d may be a distance obtained from the distance of the polar coordinates of a plurality of three-dimensional points decoded in the decoding order prior to the three-dimensional point to be decoded. It may be the average, median, maximum value, minimum value, etc. of the polar coordinate distances of a plurality of 3D points decoded in the previous decoding order. Further, the distance d may be a distance obtained from the distance of the 3D point to be decoded and the distance of the polar coordinates of one or more 3D points decoded in decoding order prior to the 3D point to be decoded. , for example, the average, median, maximum value, minimum value, etc. of the distance of the 3D point to be decoded and the distance of the polar coordinates of one or more 3D points decoded in the decoding order prior to the 3D point to be decoded There may be.

Further, the distance d may be a distance calculated using distances of polar coordinates of a plurality of three-dimensional points decoded in decoding order after the three-dimensional point to be decoded. It may be the average, median, maximum value, minimum value, etc. of distances of polar coordinates of a plurality of three-dimensional points decoded in decoding order after the point. Further, the distance d is a distance calculated using the distance of the three-dimensional point to be decoded and the distance of the polar coordinates of one or more three-dimensional points decoded in the decoding order after the three-dimensional point to be decoded, For example, the average, median, maximum and minimum values of the distance of the 3D point to be decoded and the distance of the polar coordinates of one or more 3D points decoded in the decoding order after the 3D point to be decoded and so on.

It should be noted that the one or more 3D points decoded in the decoding order before the 3D point to be decoded is an example of the one or more second 3D points or the one or more fourth 3D points. Also, the one or more 3D points decoded in decoding order after the 3D point to be decoded is another example of the one or more second 3D points or the one or more fourth 3D points. The one or more second 3D points are 3D points that are included in the plurality of 3D points and located around the 3D point to be decoded. A three-dimensional point located in the vicinity means that the spatial position is close to the decoding target three-dimensional point, or the temporal position is close to the decoding target three-dimensional point. A spatial position close to a 3D point to be decoded means a 3D point located at a position where the distance from the 3D point to be decoded is less than a predetermined distance, or a decoding order of the 3D points to be decoded. are the three-dimensional points in the order that the difference between is less than the predetermined difference. A temporal position close to a 3D point to be decoded means a 3D point included in a frame whose generation order difference is less than a predetermined difference in the generation order of a plurality of frames.

Note that Φprecision used in Equation 7 is a parameter representing the bit precision of the horizontal angle Φ. The greater the bit precision, the more accurate the predicted value can be generated, and hence the smaller the calculated residual. Conversely, the smaller the bit precision value, the larger the calculated residual error because the more inaccurate predicted value is calculated. Therefore, the smaller the bit precision, the smaller the quantization step to suppress the quantization error, and the higher the bit precision, the larger the quantization step, thereby improving the coding rate.

The three-dimensional data decoding device 120 may acquire Φprecision by decoding what is added to a bitstream such as a header of GPS, slice, or the like. Also, the three-dimensional data decoding device 120 may acquire parameters for calculating Φprecision by decoding from headers such as GPS and slices. By decoding Φprecision from the bitstream, the 3D data decoding device 120 can know the bit precision of the horizontal angle Φ used in the 3D data encoding device 100 by decoding the bitstream. I can decode the stream properly.

Note that Φprecision may have different values for intra prediction and inter prediction. For example, ΦprecisionIntra for intra prediction and ΦprecisionInter for inter prediction may be respectively decoded from a bitstream, and the value of Φprecision used for decoding may be switched according to intra prediction or inter prediction. Accordingly, by switching the quantization step QS depending on whether intra-prediction or inter-prediction, a bitstream with improved coding efficiency can be appropriately decoded. Note that the three-dimensional data decoding device 120 may acquire parameters for calculating ΦprecisionIntra and ΦprecisionInter by decoding information added to headers such as GPS and slice.

Thus, since different Φprecision is used for inter prediction and intra prediction, the calculated inverse quantization step iQS can also be calculated to have different values for inter prediction and intra prediction. That is, in the inverse quantization, the first residual between the horizontal angle Φ and the inter prediction value predicted by the inter prediction is calculated using the first inverse quantization step iQS1. Also, in quantization, a second residual between the horizontal angle Φ and the intra prediction value predicted by intra prediction is calculated using a second inverse quantization step iQS2 different from the first inverse quantization step iQS1. Specifically, the first inverse quantization step iQS1 and the second inverse quantization step iQS2 may be calculated using Equations 8 and 9 below, respectively.

　QS1=ΦprecisionInter/d (Equation 8)

　QS2=ΦprecisionIntra/d (Equation 9)

Thus, the first inverse quantization step iQS1 may be calculated using ΦprecisionInter as the first reference value and the distance d. Also, the second inverse quantization step iQS2 may be calculated using ΦprecisionIntra and the distance d as the second reference value.

ΦprecisionIntra is an example of a second reference value in intra prediction. ΦprecisionIntra may be a parameter representing the bit precision of the horizontal angle Φ, or may be a parameter representing the prediction accuracy of the horizontal angle Φ. Also, ΦprecisionInter is an example of a first reference value in inter prediction. ΦprecisionInter may be a parameter representing the bit precision of the horizontal angle Φ, or may be a parameter representing the prediction accuracy of the horizontal angle Φ. Thus, the reference value may be a value that indicates the accuracy of predictive values that can be generated by intra-prediction and inter-prediction. The 3D data decoding device 120 can decode the first residual or the second residual with improved coding efficiency by switching the quantization step according to the reference value.

Since the inverse quantization step iQS is calculated in the same manner as the quantization step QS, it can also be said to be the quantization step QS. Also, since the first inverse quantization step iQS1 is calculated in the same manner as the first quantization step QS1, it can also be said to be the first quantization step QS1. Also, since the second inverse quantization step iQS2 is calculated in the same manner as the second quantization step QS2, it can also be said to be the second quantization step QS2.

Although an example in which Φprecision, which is the bit precision of the horizontal angle Φ, is switched between intra prediction and inter prediction and added to the bitstream is shown, this is not necessarily the case. For example, an intra prediction frame or slice, The bit precision of the horizontal angle Φ may be switched between inter-predicted frames or slices and added to the bitstream. Also, Φprecision added to the bitstream may be switched on a frame-by-frame basis, a slice-by-slice basis or a stream-by-stream basis. That is, Φprecision added to the bitstream may be switched even if intra prediction is continued when the frame or slice of the 3D point to be encoded is switched in intra prediction. Also, Φprecision added to the bitstream may be switched even if inter prediction is continued when the frame or slice of the 3D point to be encoded is switched in inter prediction. In this case, the 3D data decoding device 120 may decode the bit precision of the horizontal angle Φ or the information for generating it from the header for each frame or slice. In this way, Φprecision may have the same value in units of a predetermined point group including one or more 3D points, that is, in units of frames, units of slices, units of streams, and the like. Similarly, the same value may be used for ΦprecisionInter in units of predetermined point groups including one or more three-dimensional points, that is, in units of frames, units of slices, units of streams, and the like. For ΦprecisionIntra, the same value may be used in units of a predetermined point group including one or more 3D points, that is, in units of frames, units of slices, units of streams, and the like.

In the adaptive inverse quantization described with reference to FIG. 8, inverse quantization is adaptively executed according to the distance d of the decoding target point for the quantized value of the residual signal of the horizontal angle Φ in the polar coordinates of the decoding target point. be done. As a result, the quantization of the residual signal of the horizontal angle Φ can be adaptively controlled by the value of the distance d to appropriately decode the bitstream with improved coding efficiency. When decoding the decoding target point by intra prediction, three-dimensional data decoding device 120 combines the intra prediction value calculated by intra prediction with the residual signal of horizontal angle Φ obtained by adaptive inverse quantization. is added to calculate the decoded value (horizontal angle Φ) of the decoding target point. Further, when decoding the decoding target point by inter prediction, three-dimensional data decoding apparatus 120 combines the inter prediction value calculated by inter prediction and the residual signal of horizontal angle Φ obtained by adaptive inverse quantization. By adding, the decoded value (horizontal angle Φ) of the decoding target point is calculated. To obtain the decoded value of the decoding target point by adding the residual signal obtained by applying common adaptive inverse quantization to the prediction value generated by at least one of intra prediction and inter prediction in this way , the bitstream can be appropriately decoded while suppressing the circuit scale of additional hardware and the amount of software code.

Although this embodiment has shown an example of appropriately decoding a bitstream to which adaptive quantization is applied to intra-prediction and inter-prediction residual signals, the present invention is not necessarily limited to this. For example, the three-dimensional data decoding device 120 applies adaptive quantization to the intra-prediction residual signal and does not apply adaptive quantization to the inter-prediction residual signal. The bitstream may be decoded with adaptive dequantization applied for prediction residual signals and without adaptive dequantization applied for inter-prediction residual signals. As a result, it is possible to appropriately decode a bitstream whose encoding efficiency is improved by intra prediction while suppressing the amount of processing. Further, for example, the three-dimensional data decoding device 120 applies adaptive quantization to the inter-prediction residual signal and does not apply adaptive quantization to the intra-prediction residual signal. , the bitstream may be decoded with adaptive dequantization applied for inter-prediction residual signals and without applying adaptive dequantization for intra-prediction residual signals. As a result, the 3D data decoding device 120 can appropriately decode a bitstream in which inter-prediction encoding efficiency is improved while suppressing the amount of processing.

FIG. 9 is a block diagram of a three-dimensional data encoding device 130 according to a modification of the present embodiment. Note that FIG. 9 shows a processing unit for encoding the position information (geometry) of the point cloud, but the three-dimensional data encoding device 130 includes a processing unit for encoding the attribute information of the point cloud, etc. may be provided with other processing units. In inter-prediction and intra-prediction, a point group to be encoded is encoded while referring to an already-encoded point group. The three-dimensional data encoding device 130 differs from the three-dimensional data encoding device 100 of FIG. 1 in its configuration and operation by converting point groups having position information expressed in rectangular coordinates into position information expressed in polar coordinates. and quantizing the second residual signal of the orthogonal coordinates corresponding to the error generated in the transformation between the orthogonal coordinates and the polar coordinates. . On the other hand, the three-dimensional data encoding device 130 is the same as the three-dimensional data encoding device 100 in terms of its configuration and operation, except for the above differences.

The three-dimensional data encoding device 130 includes a coordinate transformation unit 131, a grouping unit 132, a buffer 133, an adaptive quantization unit 134, an adaptive inverse quantization unit 135, a buffer 136, an intra prediction unit 137, A buffer 138 , a motion compensation unit 139 , an inter prediction unit 140 , a switching unit 141 , a coordinate transformation unit 142 , a buffer 143 , a quantization unit 144 and an entropy coding unit 145 are provided.

The coordinate conversion unit 131 converts the coordinate system of the position information of the target point group, which is the input point group data to be encoded, from the orthogonal coordinate system to the polar coordinate system. That is, the coordinate transformation unit 131 generates position information in the polar coordinate system by transforming the coordinate system of the position information in the orthogonal coordinate system of one three-dimensional point to be encoded. The point group to be encoded converted into polar coordinates is output to the grouping unit 132 .

The grouping unit 132 extracts a point group to be a prediction tree (Predtree), which is a unit of encoding, from the target point group, which is a point group to be encoded converted into polar coordinates, and sets them as one group. . A buffer 133 holds the generated prediction tree. For example, the buffer 133 may initialize data held for each prediction tree. A plurality of three-dimensional points included in the prediction tree (Predtree) held in the buffer 133 are sequentially subjected to encoding processing one by one.

The difference (first residual difference signal) is calculated. This first residual signal is a residual signal of positional information expressed in polar coordinates. The first residual signal is also called prediction residual. This first residual signal is an example of the first residual. Since the positional information of the plurality of three-dimensional points held in the buffer 133 has been converted into the polar coordinate system, the first residual is the difference between the positional information in the polar coordinate system after conversion and the predicted value.

The adaptive quantization section 134 adaptively quantizes the first residual signal. The entropy coding unit 111 entropy-codes the adaptively quantized first residual signal to generate coded data, and outputs (generates) a bitstream containing the coded data.

The adaptive inverse quantization section 135 adaptively inversely quantizes the first residual signal adaptively quantized by the adaptive quantization section 134 . The adaptively inverse quantized first residual signal is added to the prediction value based on the selected prediction points (one or more candidate points) to obtain three-dimensional points (reference points) used for intra prediction and inter prediction. is decoded as Note that the predicted value is calculated based on position information of one or more candidate points. The buffer 136 holds the decoded intra-prediction reference point group. For example, the buffer 136 may initialize data held for each prediction tree (target point group). The buffer 138 also holds reference point groups for inter prediction. For example, the buffer 138 may initialize data held for each prediction tree (target point group). The position information of the decoding points held in the

buffers

136 and 138 is represented by polar coordinates.

The coordinate transformation unit 142 acquires the same decoding points as the decoding points held in the

buffers

136 and 138, and transforms the coordinate system of the position information of the acquired decoding points from the polar coordinate system to the rectangular coordinate system. That is, the coordinate transformation unit 142 inversely transforms the coordinate system of the position information in the polar coordinate system after transformation by the coordinate transformation unit 131 to generate the position information in the orthogonal coordinate system.

The buffer 143 holds the positional information of the three-dimensional points expressed in orthogonal coordinates input to the three-dimensional data encoding device 130 .

Then, the difference (second residual signal) between the input positional information in the rectangular coordinate system and the positional information in the rectangular coordinate system whose coordinate system is converted from the polar coordinate system to the rectangular coordinate system in the coordinate conversion unit 142 is calculated. be. This second residual signal is an example of the second residual. That is, the second residual signal is the position information in the orthogonal coordinate system that has not been coordinate-transformed by the coordinate transforming unit 131, and the position information that has been transformed once into the polar coordinates and then inversely transformed back into the orthogonal coordinate system. , and is a transformation error generated by coordinate transformation.

The quantization unit 144 quantizes the second residual signal.

The entropy encoding unit 145 entropy-encodes the first residual signal and the quantized second residual signal to generate encoded data, and outputs a bitstream containing the encoded data.

In this way, the 3D data encoding device 130 converts the coordinate system of the position information of the 3D points from the orthogonal coordinate system to the polar coordinate system, and encodes the position information in the polar coordinate system. As a result, it is possible to improve the prediction accuracy of the points to be encoded when encoding the point cloud generated by acquiring the 3D position of the surrounding object around the sensor position, such as LiDAR. , which may improve the coding efficiency.

FIG. 10 is a block diagram of a 3D data decoding device 150 according to a modification of the present embodiment. Note that FIG. 10 shows a processing unit related to decoding of point cloud position information (geometry), but the three-dimensional data decoding device 150 includes other processing units such as a processing unit that decodes point cloud attribute information. A processing unit may be provided. The three-dimensional data decoding device 150 performs inter-prediction decoding for decoding a point group to be decoded while referring to a decoded point group. For example, the 3D data decoding device 150 decodes the bitstream generated by the 3D data encoding device 130 shown in FIG. 3D data decoding device 150 differs from 3D data decoding device 120 in FIG. 3 in its basic configuration and operation. 2 The difference is that the transformation error obtained by entropy decoding and inverse quantization of the residual signal is added to the decoding point transformed from the corresponding polar coordinate decoding point to the orthogonal coordinate decoding point, and is output as the orthogonal coordinate decoding point. . On the other hand, the three-dimensional data decoding device 150 is the same as the three-dimensional data decoding device 120 except for the above differences.

The three-dimensional data decoding device 150 includes an entropy decoding unit 151, an adaptive inverse quantization unit 152, a buffer 153, an intra prediction unit 154, a buffer 155, a motion compensation unit 156, an inter prediction unit 157, and a switching unit. 158 , a coordinate transformation unit 159 and an inverse quantization unit 160 .

The entropy decoding unit 151 entropy-decodes the input bitstream for each three-dimensional point of a prediction tree (Predtree) to generate an adaptively quantized first residual signal and a quantized second residual signal. and

The adaptive inverse quantization unit 152 adaptively inversely quantizes the adaptively quantized first residual signal to reproduce the first residual signal. The first residual signal of each 3D point is added to the prediction value based on the prediction point corresponding to each 3D point, and then generated (output) as a decoded point expressed in polar coordinates.

The buffer 153 holds the generated decoding points as a reference point group for intra prediction. For example, the buffer 153 may initialize data held for each prediction tree (target point group). Also, the buffer 155 holds the generated decoding points as a reference point group for inter prediction. For example, the buffer 155 may initialize data held for each prediction tree (target point group). The position information of the decoding points held in the

buffers

153 and 155 is represented by polar coordinates.

Also, the intra prediction unit 154, the motion compensation unit 156, the inter prediction unit 157, and the switching unit 158 have the same functions as the intra prediction unit 124, the motion compensation unit 126, the inter prediction unit 127, and the switching unit 128, respectively. be.

The coordinate transformation unit 159 acquires the same decoding points as the decoding points held in the

buffers

153 and 155, and transforms the coordinate system of the position information of the acquired decoding points from the polar coordinate system to the rectangular coordinate system.

The inverse quantization unit 160 inversely quantizes the quantized second residual signal to reproduce the second residual signal.

The position information in the orthogonal coordinate system obtained by coordinate transformation by the coordinate transformation unit 159 and the second residual signal inversely quantized and reproduced by the inverse quantization unit 160 are added, and then converted to the orthogonal coordinate system. is generated (output) as a decoding point including the position information of .

In this way, the three-dimensional data decoding device 150 converts the coordinate system of the decoding point having the position information of the polar coordinate system from the polar coordinate system to the orthogonal coordinate system, and converts the position information of the orthogonal coordinate system and the position information of the polar coordinate system. Means are provided for adding to a second residual signal in rectangular coordinates corresponding to the error resulting from the transformation between the two. As a result, the three-dimensional data decoding device 150 extracts from the encoded bitstream (for example, the bitstream output from the three-dimensional data encoding device 130 in FIG. 9) while referring to the encoded point group in the polar coordinates. A point cloud can be decoded.

As described above, the three-dimensional

data encoding devices

100 and 130 according to the present embodiment perform the processing (three-dimensional data encoding method) shown in FIG. Three-dimensional

data encoding devices

100 and 130 each generate an angle component of a first three-dimensional point among a plurality of three-dimensional points each represented by a distance component and an angle component, and a first A first residual from the predicted value of the angle component of the 3D point is calculated (S161). Next, 3D

data encoding apparatuses

100 and 130 quantize the first residual using the first quantization step (S162). The first quantization step is calculated according to the first distance obtained from at least one of the distance component of the first three-dimensional point and the distance component of one or more encoded second three-dimensional points.

For example, one or more second 3D points are encoded before the first 3D point in the encoding order of the plurality of 3D points.

For example, the 3D

data encoding devices

100 and 130 further predict the angle component of the third 3D point among the plurality of 3D points and the angle component of the third 3D point calculated by intra prediction. Calculate the second residual with the value. The 3D

data encoding apparatuses

100 and 130 quantize the second residual using a second quantization step different from the first quantization step.

For example, the second quantization step is calculated according to the second distance obtained from at least one of the distance component of the third three-dimensional point and the distance component of one or more encoded fourth three-dimensional points. .

For example, the first quantization step is calculated so as to increase as the first prediction accuracy of the angle component of the first three-dimensional point decreases.

For example, the first quantization step is calculated according to the first distance and the first reference value. The first reference value is a reference value for calculating the first quantization step such that the smaller the first prediction accuracy, the larger the first quantization step. The 3D

data encoding devices

100, 130 further generate a bitstream containing the quantized first residual and the first reference value.

For example, the second quantization step is calculated according to the second distance and the second reference value. The second reference value is a reference value for calculating the second quantization step such that the smaller the second prediction accuracy of the angular component of the third three-dimensional point, the larger the second quantization step. The bitstream further includes a quantized second residual and a second reference value.

For example, the first quantization step is calculated so as to decrease as the first distance increases.

The second quantization step is calculated so as to decrease as the second distance increases.

For example, the 3D

data encoding devices

100 and 130 each include a processor and memory, and the processor uses the memory to perform the above processing.

Also, the three-dimensional

data decoding devices

120 and 150 according to the present embodiment perform the processing (three-dimensional data decoding method) shown in FIG. Three-dimensional

data decoding devices

120 and 150 each generate an angle component of a first three-dimensional point among a plurality of three-dimensional points each represented by a distance component and an angle component, and a first cubic A first residual from the predicted value of the angle component of the original point is acquired (S171). Next, the 3D

data decoding devices

120 and 150 inversely quantize the first residual using the first quantization step (S172). The first quantization step is calculated according to the first distance obtained from at least one of the distance component of the first three-dimensional point and the distance component of one or more decoded second three-dimensional points.

For example, one or more second 3D points are decoded before the first 3D point in the decoding order of the plurality of 3D points.

For example, the 3D

data decoding devices

120 and 150 further include the angle component of the third 3D point among the plurality of 3D points and the predicted value of the angle component of the third 3D point calculated by intra prediction. Obtain the second residual with Next, the 3D

data decoding devices

120 and 150 inversely quantize the second residual using a second quantization step different from the first quantization step.

For example, the second quantization step is calculated according to the second distance obtained from at least one of the distance component of the third three-dimensional point and the distance component of one or more decoded fourth three-dimensional points.

For example, the 3D

data decoding devices

120 and 150 acquire the first reference value. A first quantization step is calculated according to the first distance and the first reference value. The first reference value is a reference value for calculation such that the smaller the first prediction accuracy, the larger the first quantization step.

For example, the 3D

data decoding devices

120 and 150 further acquire a second reference value. A second quantization step is calculated according to the second distance and the second reference value. The second reference value is a reference value for calculating the second quantization step such that the smaller the second prediction accuracy of the angular component of the third three-dimensional point, the larger the second quantization step.

For example, the second quantization step is calculated so as to decrease as the second distance increases.

For example, the 3D

data decoding devices

120 and 150 include a processor and memory, and the processor uses the memory to perform the above processing.

(Modification)
In the above embodiments, three-dimensional

data encoding apparatuses

100 and 130 perform adaptive quantization and switch the quantization step used between inter prediction and intra prediction in adaptive quantization. , the quantization step used in inter prediction and intra prediction in quantization may be switched regardless of whether adaptive quantization is performed. Similarly, the three-dimensional

data decoding devices

120 and 150 perform inverse adaptive quantization, and in the inverse adaptive quantization, switch the quantization step used between inter prediction and intra prediction. Regardless of executing inverse adaptive quantization, the quantization step used in inter prediction and intra prediction in inverse quantization may be switched.

The three-dimensional

data encoding devices

100 and 130 according to the modified examples perform the processing (three-dimensional data encoding method) shown in FIG. The 3D

data encoding devices

100 and 130 calculate the predicted value of the position of the 3D point using one of inter prediction and intra prediction (S181). Next, the 3D

data encoding device

100, 130 calculates the residual between the predicted value and the position (S182). The 3D

data encoding apparatuses

100 and 130 quantize the residual using a quantization step according to the prediction method (S183-S185). That is, the 3D

data encoding apparatuses

100 and 130 quantize the residual using the first quantization step when the prediction value is calculated by inter prediction. Further, when the prediction value is calculated by intra prediction (intra prediction in S173), the 3D

data encoding apparatuses

100 and 130 quantize the residual using a second quantization step different from the first quantization step. (S185). The positions of the three-dimensional points may be represented by orthogonal coordinates or polar coordinates.

For example, the 3D

data encoding devices

The 3D

data decoding devices

120 and 150 according to the modification perform the processing (3D data decoding method) shown in FIG. The 3D

data decoding devices

120 and 150 obtain residuals between the position of the 3D point and the predicted value calculated by one of the inter prediction and intra prediction (S191). The 3D

data decoding devices

120 and 150 inversely quantize the residual using a quantization step according to the prediction method (S192-S194). That is, when the prediction value is calculated by inter prediction (inter prediction in S192), 3D

data decoding apparatuses

120 and 150 inversely quantize the residual using the first quantization step (S193). Further, when the prediction value is calculated by intra prediction (intra prediction in S192), 3D

data decoding devices

120 and 150 inversely quantize the residual using a second quantization step different from the first quantization step. (S194). The positions of the three-dimensional points may be represented by orthogonal coordinates or polar coordinates.

For example, the 3D

data decoding devices

Although the three-dimensional data encoding device and the three-dimensional data decoding device according to the embodiment and modifications of the present disclosure have been described above, the present disclosure is not limited to this embodiment.

Also, each processing unit included in the three-dimensional data encoding device, the three-dimensional data decoding device, etc. according to the above embodiments is typically realized as an LSI, which is an integrated circuit. These may be made into one chip individually, or may be made into one chip so as to include part or all of them.

In addition, circuit integration is not limited to LSIs, and may be realized with dedicated circuits or general-purpose processors. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of the circuit cells inside the LSI may be used.

In addition, in each of the above embodiments, each component may be configured with dedicated hardware or realized by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor.

Also, the present disclosure may be implemented as a three-dimensional data encoding method, a three-dimensional data decoding method, or the like executed by a three-dimensional data encoding device, a three-dimensional data decoding device, or the like.

Also, the division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, one functional block can be divided into a plurality of functional blocks, and some functions can be moved to other functional blocks. may Moreover, single hardware or software may process the functions of a plurality of functional blocks having similar functions in parallel or in a time-sharing manner.

Also, the order in which each step in the flowchart is executed is for illustrative purposes in order to specifically describe the present disclosure, and orders other than the above may be used. Also, some of the above steps may be executed concurrently (in parallel) with other steps.

The three-dimensional data encoding device and the three-dimensional data decoding device according to one or more aspects have been described above based on the embodiments, but the present disclosure is not limited to these embodiments. . As long as it does not deviate from the spirit of the present disclosure, various modifications that a person skilled in the art can think of are applied to this embodiment, and a form constructed by combining the components of different embodiments is also within the scope of one or more aspects may be included within

The present disclosure can be applied to a 3D data encoding device and a 3D data decoding device.

100, 130 three-dimensional

data encoding device

101, 132

grouping section

102, 105, 107, 123, 125, 133, 136, 138, 143, 153, 155

buffer

103, 134

adaptive quantization section

104, 122, 135, 152 adaptive

inverse quantization section

106, 124, 137, 154

intra prediction section

108, 139

motion compensation section

109, 127, 140, 157

inter prediction section

110, 128, 141, 158

switching section

111, 145

entropy coding section

120, 150 three-dimensional

data decoding device

121, 151

entropy decoding unit

126, 156

motion compensation unit

131, 142, 159 coordinate transformation unit 144 quantization unit

Claims

An angle component of a first three-dimensional point among a plurality of three-dimensional points each represented by a distance component and an angle component, and a predicted value of the angle component of the first three-dimensional point calculated by inter prediction. Calculate the first residual with
quantizing the first residual using a first quantization step;
In the first quantization step, a distance component of the first three-dimensional point and one or more second three-dimensional points included in the plurality of three-dimensional points and located around the first three-dimensional point. A three-dimensional data encoding method calculated according to at least one of the distance components.
The 3D data encoding method of claim 1, wherein the one or more second 3D points are encoded before the first 3D points in the encoding order of the plurality of 3D points.
moreover,
calculating a second residual between an angular component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angular component of the third three-dimensional point calculated by intra prediction;
The three-dimensional data encoding method according to claim 1, wherein the second residual is quantized using a second quantization step different from the first quantization step.
The second quantization step includes calculating the distance component of the third three-dimensional point and one or more fourth three-dimensional points included in the plurality of three-dimensional points and located around the third three-dimensional point. The three-dimensional data encoding method according to claim 3, wherein the distance component is calculated according to at least one of the distance components.
The three-dimensional data encoding method according to claim 1, wherein the first quantization step is calculated so as to increase as the first prediction accuracy of the predicted value of the angular component of the first three-dimensional point decreases.
The first quantization step is calculated according to at least one of the distance component of the first three-dimensional point and the distance component of the one or more second three-dimensional points and a first reference value,
The first reference value is a reference value for calculating the first quantization step such that the smaller the first prediction accuracy, the larger the first quantization step;
6. The 3D data encoding method of claim 5, wherein the 3D data encoding method further generates a bitstream including the quantized first residual and the first reference value.
moreover,
calculating a second residual between an angular component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angular component of the third three-dimensional point calculated by intra prediction;
quantizing the second residual using a second quantization step different from the first quantization step;
The second quantization step includes calculating the distance component of the third three-dimensional point and one or more fourth three-dimensional points included in the plurality of three-dimensional points and located around the third three-dimensional point. Calculated according to at least one of the distance components and the second reference value,
the second reference value is a reference value for calculating the second quantization step such that the second prediction accuracy of the predicted value of the angle component of the third three-dimensional point is smaller, the second prediction accuracy is higher;
7. The method of claim 6, wherein the bitstream further comprises the quantized second residual and the second reference value.
The first quantization step is calculated so as to decrease as at least one of the distance component of the first three-dimensional point and the distance component of the one or more second three-dimensional points increases. three-dimensional data encoding method.
moreover,
calculating a second residual between an angular component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angular component of the third three-dimensional point calculated by intra prediction;
quantizing the second residual using a second quantization step different from the first quantization step;
The second quantization step includes calculating the distance component of the third three-dimensional point and one or more fourth three-dimensional points included in the plurality of three-dimensional points and located around the third three-dimensional point. 9. The three-dimensional data encoding method according to claim 8, wherein the distance component is calculated so as to be smaller as at least one of the distance components is larger.
Calculate the predicted value of the position of the three-dimensional point by one of the inter-prediction and intra-prediction prediction methods,
calculating a residual between the predicted value and the position;
A three-dimensional data encoding method, wherein the residual is quantized using a quantization step according to the prediction scheme.
The angle component of a first three-dimensional point among a plurality of three-dimensional points each represented by a distance component and an angle component, and prediction of the angle component of the first three-dimensional point calculated by inter prediction. get the first residual with the value
inverse quantizing the first residual using a first quantization step;
In the first quantization step, a distance component of the first three-dimensional point and one or more second three-dimensional points included in the plurality of three-dimensional points and located around the first three-dimensional point. A three-dimensional data decoding method calculated according to at least one of distance components.
12. The method of decoding 3D data according to claim 11, wherein the one or more second 3D points are decoded before the first 3D point in the decoding order of the plurality of 3D points.
moreover,
Obtaining a second residual between an angular component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angular component of the third three-dimensional point calculated by intra prediction;
12. The three-dimensional data decoding method according to claim 11, wherein the second residual is inversely quantized using a second quantization step different from the first quantization step.
The second quantization step includes calculating the distance component of the third three-dimensional point and one or more fourth three-dimensional points included in the plurality of three-dimensional points and located around the third three-dimensional point. 14. The three-dimensional data decoding method according to claim 13, calculated according to at least one of the distance components.
The three-dimensional data decoding method according to claim 11, wherein the first quantization step is calculated so as to increase as the first prediction accuracy of the predicted value of the angle component of the first three-dimensional point decreases.
moreover,
obtain the first reference value;
The first quantization step is calculated according to at least one of the distance component of the first three-dimensional point and the distance component of the one or more second three-dimensional points and a first reference value,
The three-dimensional data decoding method according to claim 15, wherein the first reference value is a reference value for calculation so that the first quantization step increases as the first prediction accuracy decreases.
moreover,
A second residual error and a second reference value between an angular component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angular component of the third three-dimensional point calculated by intra prediction. and get
inverse quantizing the second residual using a second quantization step different from the first quantization step;
The second quantization step includes calculating the distance component of the third three-dimensional point and one or more fourth three-dimensional points included in the plurality of three-dimensional points and located around the third three-dimensional point. Calculated according to at least one of the distance components and the second reference value,
17. The second reference value according to claim 16, wherein the second reference value is a reference value for calculating the second quantization step such that the smaller the second prediction accuracy of the angle component of the third three-dimensional point, the larger the second prediction accuracy. three-dimensional data decoding method.
12. The first quantization step is calculated so as to decrease as at least one of the distance component of the first three-dimensional point and the distance component of the one or more second three-dimensional points increases. three-dimensional data decoding method.
moreover,
Obtaining a second residual between an angular component of a third three-dimensional point among the plurality of three-dimensional points and a predicted value of the angular component of the third three-dimensional point calculated by intra prediction;
inverse quantizing the second residual using a second quantization step different from the first quantization step;
The second quantization step includes calculating the distance component of the third three-dimensional point and one or more fourth three-dimensional points included in the plurality of three-dimensional points and located around the third three-dimensional point. The three-dimensional data decoding method according to claim 18, wherein the larger at least one of the distance components is calculated, the smaller the distance component.
Obtaining the residual between the position of the three-dimensional point and the predicted value calculated by one of the inter prediction and intra prediction,
A three-dimensional data decoding method, wherein the residual is inversely quantized using a quantization step according to the prediction scheme.
a processor;
with memory and
The processor, using the memory,
An angle component of a first three-dimensional point among a plurality of three-dimensional points each represented by a distance component and an angle component, and a predicted value of the angle component of the first three-dimensional point calculated by inter prediction. Calculate the first residual with
quantizing the first residual using a first quantization step;
In the first quantization step, a distance component of the first three-dimensional point and one or more second three-dimensional points included in the plurality of three-dimensional points and located around the first three-dimensional point. A three-dimensional data encoding device calculated according to at least one of distance components.
a processor;
with memory and
The processor, using the memory,
Calculate the predicted value of the position of the three-dimensional point by one of the inter-prediction and intra-prediction prediction methods,
calculating a residual between the predicted value and the position;
A three-dimensional data encoding device that quantizes the residual using a quantization step according to the prediction method.
a processor;
with memory and
The processor, using the memory,
The angle component of a first three-dimensional point among a plurality of three-dimensional points each represented by a distance component and an angle component, and prediction of the angle component of the first three-dimensional point calculated by inter prediction. get the first residual with the value
inverse quantizing the first residual using a first quantization step;
In the first quantization step, a distance component of the first three-dimensional point and one or more second three-dimensional points included in the plurality of three-dimensional points and located around the first three-dimensional point. A three-dimensional data decoding device calculated according to at least one of distance components.
a processor;
with memory and
The processor, using the memory,
Obtaining the residual between the position of the three-dimensional point and the predicted value calculated by one of the inter prediction and intra prediction,
A three-dimensional data decoding device that inversely quantizes the residual using a quantization step according to the prediction method.