RU2791241C2 - Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device and three-dimensional data decoding device - Google Patents

Abstract

FIELD: encoding technique.

SUBSTANCE: invention relates to the field of encoding and decoding three-dimensional data and can find application in a wide range of fields of technology, such as machine vision, which provides autonomous operation of cars or robots, cartographic information, monitoring, infrastructure inspection and video distribution. The method for encoding three-dimensional data includes dividing three-dimensional points included in three-dimensional data into subclouds, including the first subcloud of three-dimensional points and the second subcloud of three-dimensional points (S2631); attaching the first information indicating the space of the first subcloud of three-dimensional points to the header of the first subcloud and attaching the second information indicating the space of the second subcloud to the header of the second subcloud of three-dimensional points (S2632). Fragments of information of attributes of the first three-dimensional points and fragments of information of attributes of the second three-dimensional points are encoded so that these fragments of information are decoded sequentially or in parallel.

EFFECT: reducing the processing time when encoding or decoding three-dimensional data.

76 cl, 218 dwg

Description

The technical field to which the invention belongs

[0001] The present disclosure relates to a 3D data encoding method, a 3D data decoding method, a 3D data encoding apparatus, and a 3D data decoding apparatus.

State of the art

[0002] Devices or services using three-dimensional data are expected to find wide application in a wide range of areas of technology, such as machine vision, which provides autonomous operation of cars or robots, mapping information, monitoring, infrastructure survey and video distribution. Three-dimensional data is obtained through various means, including a distance sensor such as a range finder, as well as a stereo camera and a combination of multiple monocular cameras.

[0003] Techniques for representing 3D data include a technique known as point cloud schema, which represents the shape of a 3D structure by a point group in 3D space. In a point cloud based scheme, the positions and colors of the point group are preserved. Although the point cloud is supposed to be the main way to represent 3D data, the huge amount of point group data requires compressing the amount of 3D data through coding for accumulation and transmission, as in the case of a 2D moving picture (examples include MPEG-4 AVC and HEVC, a standardized via MPEG).

[0004] Meanwhile, point cloud compression is partially supported, for example, by an open source library (Point Cloud Library) for point cloud related processing.

[0005] In addition, a technology is known for searching and displaying an infrastructure object located in the vicinity of a vehicle (for example, see Patent Document (PTL) 1).

List of bibliographic references

Patent Documents

[0006] PTL 1. International Publication WO 2014/020663

The essence of the invention

Technical task

[0007] There is a need to reduce processing time when encoding or decoding three-dimensional data.

[0008] The present disclosure aims to provide a 3D data encoding method, a 3D data decoding method, a 3D data encoding apparatus, or a 3D data decoding apparatus that can reduce processing time.

The solution of the problem

[0009] A method for encoding 3D data according to one aspect of the present disclosure includes: dividing 3D points included in the 3D data into 3D point subclouds including a first 3D point subcloud and a second 3D point subcloud; attaching first information indicating the space of the first 3D point subcloud to a header of the first 3D point subcloud; attaching second information indicating the space of the second 3D point subcloud to a header of the second 3D point subcloud; and encoding the first 3D point subcloud and the second 3D point subcloud such that the first 3D point subcloud and the second 3D point subcloud are independently decodable.

[0010] A method for decoding 3D data according to one aspect of the present disclosure includes: obtaining first encoded data and second encoded data generated by encoding the first 3D point subcloud and the second 3D point subcloud such that the first 3D point subcloud and the second 3D point subcloud are independently decodable, wherein the first 3D point sub-cloud and the second 3D point sub-cloud are included in the 3D point sub-clouds obtained by separating the 3D points included in the 3D data; obtaining first information indicating the space of the first 3D point subcloud from the header of the first 3D point subcloud; obtaining second information indicating the space of the second 3D point subcloud from the header of the second 3D point subcloud; and recovering the first 3D point subcloud and the second 3D point subcloud by decoding the first encoded data and the second encoded data.

Benefits of the Invention

[0011] The present disclosure provides a 3D data encoding method, a 3D data decoding method, a 3D data encoding apparatus, or a 3D data decoding apparatus that can reduce processing time.

Brief description of the drawings

[0012] FIG. 1 is a diagram showing the structure of encoded 3D data according to Embodiment 1.

Fig. 2 is a diagram showing an example of prediction structures for SPCs that belong to the lowest layer in the GOS according to Embodiment 1.

Fig. 3 is a diagram showing an example of prediction structures for layers according to Embodiment 1.

Fig. 4 is a diagram showing an exemplary GOS encoding order according to Embodiment 1.

Fig. 5 is a diagram showing an exemplary GOS encoding order according to Embodiment 1.

Fig. 6 is a block diagram of a 3D data encoding apparatus according to Embodiment 1.

Fig. 7 is a flowchart for encoding processes according to Embodiment 1.

Fig. 8 is a block diagram of a 3D data decoding apparatus according to Embodiment 1.

Fig. 9 is a flowchart for decoding processes according to Embodiment 1.

Fig. 10 is a diagram showing an example of meta information according to Embodiment 1.

Fig. 11 is a diagram showing an exemplary SWLD structure according to Embodiment 2.

Fig. 12 is a diagram showing exemplary operations performed by the server and the client according to Embodiment 2.

Fig. 13 is a diagram showing exemplary operations performed by the server and the client according to Embodiment 2.

Fig. 14 is a diagram showing exemplary operations performed by the server and clients according to Embodiment 2.

Fig. 15 is a diagram showing exemplary operations performed by the server and clients according to Embodiment 2.

Fig. 16 is a block diagram of a 3D data encoding device according to Embodiment 2.

Fig. 17 is a flowchart for encoding processes according to Embodiment 2.

Fig. 18 is a block diagram of a 3D data decoding apparatus according to Embodiment 2.

Fig. 19 is a flowchart for decoding processes according to Embodiment 2.

Fig. 20 is a diagram showing an exemplary WLD structure according to Embodiment 2.

Fig. 21 is a diagram showing an exemplary WLD octant tree structure according to Embodiment 2.

Fig. 22 is a diagram showing an exemplary SWLD structure according to Embodiment 2.

Fig. 23 is a diagram showing an exemplary SWLD octant tree structure according to Embodiment 2.

Fig. 24 is a block diagram of a 3D data generating apparatus according to Embodiment 3.

Fig. 25 is a block diagram of a 3D data transmission device according to Embodiment 3.

Fig. 26 is a block diagram of a 3D information processing apparatus according to Embodiment 4.

Fig. 27 is a block diagram of a 3D data generating apparatus according to Embodiment 5.

Fig. 28 is a diagram showing the system structure according to Embodiment 6.

Fig. 29 is a block diagram of a client device according to Embodiment 6.

Fig. 30 is a block diagram of a server according to Embodiment 6.

Fig. 31 is a flowchart for a 3D data creation process performed by the client device according to Embodiment 6.

Fig. 32 is a flowchart for a sensor information transmission process performed by the client device according to Embodiment 6.

Fig. 33 is a flowchart for a 3D data creation process executed by the server according to Embodiment 6.

Fig. 34 is a flowchart for a 3D map transmission process performed by the server according to Embodiment 6.

Fig. 35 is a diagram showing the variation structure of the system according to Embodiment 6.

Fig. 36 is a diagram showing the structure of server and client devices according to Embodiment 6.

Fig. 37 is a block diagram of a 3D data encoding apparatus according to Embodiment 7.

Fig. 38 is a diagram showing an example of a prediction residual according to Embodiment 7.

Fig. 39 is a diagram showing an example of a volume according to Embodiment 7.

Fig. 40 is a diagram showing an example of volume octant tree representation according to Embodiment 7.

Fig. 41 is a diagram showing an example of volume bit sequences according to Embodiment 7.

Fig. 42 is a diagram showing an example of volume octant tree representation according to Embodiment 7.

Fig. 43 is a diagram showing an example of a volume according to Embodiment 7.

Fig. 44 is a diagram for describing the intra prediction process according to Embodiment 7.

Fig. 45 is a diagram for describing a rotation and movement process in space according to Embodiment 7.

Fig. 46 is a diagram showing an exemplary syntax of the RT flag and RT information according to Embodiment 7.

Fig. 47 is a diagram for describing the inter-prediction process according to Embodiment 7.

Fig. 48 is a block diagram of a 3D data decoding apparatus according to Embodiment 7.

Fig. 49 is a flowchart for a 3D data encoding process performed by the 3D data encoding device according to Embodiment 7.

Fig. 50 is a flowchart for a 3D data decoding process performed by the 3D data decoding apparatus according to Embodiment 7.

Fig. 51 is a diagram showing the structure of a distribution system according to Embodiment 8.

Fig. 52 is a diagram showing an exemplary bitstream structure of an encoded 3D map according to Embodiment 8.

Fig. 53 is a diagram for describing the coding efficiency advantage according to Embodiment 8.

Fig. 54 is a flowchart for processes executed by the server according to Embodiment 8.

Fig. 55 is a flowchart for processes executed by the client according to Embodiment 8.

Fig. 56 is a diagram showing an exemplary submap syntax according to Embodiment 8.

Fig. 57 is a diagram principally showing the coding type switching process according to Embodiment 8.

Fig. 58 is a diagram showing an exemplary submap syntax according to Embodiment 8.

Fig. 59 is a flowchart for a 3D data encoding process according to Embodiment 8.

Fig. 60 is a flowchart for a 3D data decoding process according to Embodiment 8.

Fig. 61 is a diagram fundamentally showing an operation of varying the coding type switching process according to Embodiment 8.

Fig. 62 is a diagram fundamentally showing an operation of varying the coding type switching process according to Embodiment 8.

Fig. 63 is a diagram fundamentally showing an operation of varying the coding type switching process according to Embodiment 8.

Fig. 64 is a diagram fundamentally showing an operation of varying the differential value calculation process according to Embodiment 8.

Fig. 65 is a diagram fundamentally showing an operation of varying the differential value calculation process according to Embodiment 8.

Fig. 66 is a diagram fundamentally showing an operation of varying the differential value calculation process according to Embodiment 8.

Fig. 67 is a diagram fundamentally showing an operation of varying the differential value calculation process according to Embodiment 8.

Fig. 68 is a diagram showing an exemplary scope syntax according to Embodiment 8.

Fig. 69 is a diagram showing an example of an important area according to Embodiment 9.

Fig. 70 is a diagram showing an example of padding code according to Embodiment 9.

Fig. 71 is a diagram showing an example of a quadtree structure according to Embodiment 9.

Fig. 72 is a diagram showing an example of a fill code and a location code according to Embodiment 9.

Fig. 73 is a diagram showing an example of 3D points obtained through the lidar according to Embodiment 9.

Fig. 74 is a diagram showing an example of an octant tree structure according to Embodiment 9.

Fig. 75 is a diagram showing an example of hybrid coding according to Embodiment 9.

Fig. 76 is a diagram for describing a method for switching between location-based coding and padding-based coding according to Embodiment 9.

Fig. 77 is a diagram showing an example of a location-encoded bitstream according to Embodiment 9.

Fig. 78 is a diagram showing an example of a hybrid coded bitstream according to Embodiment 9.

Fig. 79 is a diagram showing a three-dimensional important point filling code tree structure according to Embodiment 9.

Fig. 80 is a diagram showing a tree structure of filling codes of unimportant 3D points according to Embodiment 9.

Fig. 81 is a diagram showing an example of a hybrid coded bitstream according to Embodiment 9.

Fig. 82 is a diagram showing an example of a bitstream including coding mode information according to Embodiment 9.

Fig. 83 is a diagram showing an exemplary syntax according to Embodiment 9.

Fig. 84 is a flowchart for the encoding process according to Embodiment 9.

Fig. 85 is a flowchart for the node encoding process according to Embodiment 9.

Fig. 86 is a flowchart for a decoding process according to Embodiment 9.

Fig. 87 is a flowchart for the node decoding process according to Embodiment 9.

Fig. 88 is a diagram illustrating an example of a tree structure according to Embodiment 10.

Fig. 89 is a graph showing an example of the number of valid leaves of each branch according to Embodiment 10.

Fig. 90 is a diagram illustrating an example of an application of coding schemes according to Embodiment 10.

Fig. 91 is a diagram illustrating an example of a dense branch zone according to Embodiment 10.

Fig. 92 is a diagram illustrating an example of a dense 3D point cloud according to Embodiment 10.

Fig. 93 is a diagram illustrating an example of a sparse 3D point cloud according to Embodiment 10.

Fig. 94 is a flowchart for the encoding process according to Embodiment 10.

Fig. 95 is a flowchart for a decoding process according to Embodiment 10.

Fig. 96 is a flowchart for the encoding process according to Embodiment 10.

Fig. 97 is a flowchart for a decoding process according to Embodiment 10.

Fig. 98 is a flowchart for the encoding process according to Embodiment 10.

Fig. 99 is a flowchart for a decoding process according to Embodiment 10.

Fig. 100 is a flowchart for a 3D point separation process according to Embodiment 10.

Fig. 101 is a diagram illustrating a syntax example according to Embodiment 10.

Fig. 102 is a diagram illustrating an example of a dense branch according to Embodiment 10.

Fig. 103 is a diagram illustrating an example of a sparse branch according to Embodiment 10.

Fig. 104 is a flowchart for an encoding process according to a variation of Embodiment 10.

Fig. 105 is a flowchart for a decoding process according to a variation of Embodiment 10.

Fig. 106 is a flowchart for a 3D point separation process according to a variation of Embodiment 10.

Fig. 107 is a diagram illustrating a syntax example according to a variation of Embodiment 10.

Fig. 108 is a flowchart for the encoding process according to Embodiment 10.

Fig. 109 is a flowchart for a decoding process according to Embodiment 10.

Fig. 110 is a diagram illustrating an example of a tree structure according to Embodiment 11.

Fig. 111 is a diagram illustrating an example of padding codes according to Embodiment 11.

Fig. 112 is a diagram principally illustrating an operation performed by the 3D data encoding device according to Embodiment 11.

Fig. 113 is a diagram illustrating an example of geometry information according to Embodiment 11.

Fig. 114 is a diagram illustrating an example of codebook selection using geometry information according to Embodiment 11.

Fig. 115 is a diagram illustrating an example of codebook selection using structure information according to Embodiment 11.

Fig. 116 is a diagram illustrating an example of selecting a codebook using attribute information according to Embodiment 11.

Fig. 117 is a diagram illustrating an example of selecting a codebook using attribute information according to Embodiment 11.

Fig. 118 is a diagram illustrating an example of a bitstream structure according to Embodiment 11.

Fig. 119 is a diagram illustrating an example of a codebook according to Embodiment 11.

Fig. 120 is a diagram illustrating an example of a codebook according to Embodiment 11.

Fig. 121 is a diagram illustrating an example of a bitstream structure according to Embodiment 11.

Fig. 122 is a diagram illustrating an example of a codebook according to Embodiment 11.

Fig. 123 is a diagram illustrating an example of a codebook according to Embodiment 11.

Fig. 124 is a diagram illustrating an example of the fill code bit numbers according to Embodiment 11.

Fig. 125 is a flowchart for an encoding process using geometry information according to Embodiment 11.

Fig. 126 is a flowchart for a decoding process using geometry information according to Embodiment 11.

Fig. 127 is a flowchart for an encoding process using structure information according to Embodiment 11.

Fig. 128 is a flowchart for a decoding process using structure information according to Embodiment 11.

Fig. 129 is a flowchart for an encoding process using attribute information according to Embodiment 11.

Fig. 130 is a flowchart for a decoding process using attribute information according to Embodiment 11.

Fig. 131 is a flowchart for a codebook selection process using geometry information according to Embodiment 11.

Fig. 132 is a flowchart for a codebook selection process using structure information according to Embodiment 11.

Fig. 133 is a flowchart for a codebook selection process using attribute information according to Embodiment 11.

Fig. 134 is a block diagram of a 3D data encoding apparatus according to Embodiment 11.

Fig. 135 is a block diagram of a 3D data decoding apparatus according to Embodiment 11.

Fig. 136 is a diagram illustrating a reference relationship in an octant tree structure according to Embodiment 12.

Fig. 137 is a diagram illustrating a reference relationship in a spatial domain according to Embodiment 12.

Fig. 138 is a diagram illustrating an example of neighboring reference nodes according to Embodiment 12.

Fig. 139 is a diagram illustrating the relationship between the parent node and the nodes according to Embodiment 12.

Fig. 140 is a diagram illustrating an example of the parent node filling code according to Embodiment 12.

Fig. 141 is a block diagram of a 3D data encoding apparatus according to Embodiment 12.

Fig. 142 is a block diagram of a 3D data decoding apparatus according to Embodiment 12.

Fig. 143 is a flowchart for a 3D data encoding process according to Embodiment 12.

Fig. 144 is a flowchart for a 3D data decoding process according to Embodiment 12.

Fig. 145 is a diagram illustrating an example of selecting a codebook according to Embodiment 12.

Fig. 146 is a diagram illustrating a reference relationship in the spatial domain according to Variation 1 of Embodiment 12.

Fig. 147 is a diagram illustrating an example of the header information syntax according to Variation 1 of Embodiment 12.

Fig. 148 is a diagram illustrating an example of the header information syntax according to Variation 1 of Embodiment 12.

Fig. 149 is a diagram illustrating an example of adjacent reference nodes according to Variation 2 of Embodiment 12.

Fig. 150 is a diagram illustrating an example of the current node and neighboring nodes according to Variation 2 of Embodiment 12.

Fig. 151 is a diagram illustrating a reference relationship in an octant tree structure according to Variation 3 of Embodiment 12.

Fig. 152 is a diagram illustrating a reference relationship in the spatial domain according to Variation 3 of Embodiment 12.

Fig. 153 is a diagram illustrating an example of the header information syntax according to Embodiment 13.

Fig. 154 is a diagram illustrating an octant tree configuration example when the mode information according to Embodiment 13 indicates 1.

Fig. 155 is a diagram illustrating an octant tree configuration example when mode information according to Embodiment 13 indicates 0.

Fig. 156 is a diagram illustrating an example of the node information syntax according to Embodiment 13.

Fig. 157 is a flowchart for a 3D data encoding process according to Embodiment 13.

Fig. 158 is a flowchart for a 3D data decoding process according to Embodiment 13.

Fig. 159 is a block diagram of a 3D data encoding apparatus according to Embodiment 13.

Fig. 160 is a block diagram of a 3D data decoding apparatus according to Embodiment 13.

Fig. 161 is a flowchart for a 3D data encoding process according to Embodiment 13.

Fig. 162 is a flowchart for a 3D data decoding process according to Embodiment 13.

Fig. 163 is a diagram illustrating examples of a 1-bit occupied position and a remaining bit according to Embodiment 14.

Fig. 164 is a diagram for illustrating a process for determining whether or not to apply encoding based on occupied positions according to Embodiment 14.

Fig. 165 is a diagram illustrating an example of the node information syntax according to Embodiment 14.

Fig. 166 is a flowchart for a 3D data encoding process according to Embodiment 14.

Fig. 167 is a flowchart for the occupied position encoding process according to Embodiment 14.

Fig. 168 is a flowchart for a 3D data decoding process according to Embodiment 14.

Fig. 169 is a flowchart for a decoding process based on occupied positions according to Embodiment 14.

Fig. 170 is a block diagram of a 3D data encoding apparatus according to Embodiment 14.

Fig. 171 is a block diagram of a 3D data decoding apparatus according to Embodiment 14.

Fig. 172 is a flowchart for a 3D data encoding process according to Embodiment 14.

Fig. 173 is a flowchart for a 3D data decoding process according to Embodiment 14.

Fig. 174 is a diagram for illustrating duplicated points according to Embodiment 15.

Fig. 175 is a diagram for illustrating a process performed for duplicated points according to Embodiment 15.

Fig. 176 is a diagram illustrating an example of the header information syntax according to Embodiment 15.

Fig. 177 is a diagram illustrating an example of the node information syntax according to Embodiment 15.

Fig. 178 is a flowchart for a 3D data encoding process according to Embodiment 15.

Fig. 179 is a flowchart for a 3D data encoding process according to Embodiment 15.

Fig. 180 is a flowchart for a 3D data decoding process according to Embodiment 15.

Fig. 181 is a block diagram of a 3D data decoding apparatus according to Embodiment 15.

Fig. 182 is a block diagram of a 3D data decoding apparatus according to Embodiment 15.

Fig. 183 is a flowchart for varying the 3D data encoding process according to Embodiment 15.

Fig. 184 is a diagram for illustrating a process for duplicated points according to Embodiment 15.

Fig. 185 is a diagram illustrating an example of neighboring nodes according to Embodiment 16.

Fig. 186 is a diagram illustrating an example of nodes to be searched according to Embodiment 16.

Fig. 187 is a diagram for illustrating a neighbor search process according to Embodiment 16.

Fig. 188 is a diagram for illustrating an update process for environment information according to Embodiment 16.

Fig. 189 is a diagram for illustrating an update process for environment information according to Embodiment 16.

Fig. 190 is a diagram for illustrating a search process for which a search threshold is provided according to Embodiment 16.

Fig. 191 is a diagram illustrating an example of indexes for which Morton codes are used according to Embodiment 16.

Fig. 192 is a diagram illustrating an example of a queue for which Morton codes are used according to Embodiment 16.

Fig. 193 is a block diagram of a 3D data encoding apparatus according to Embodiment 16.

Fig. 194 is a block diagram of a 3D data decoding apparatus according to Embodiment 16.

Fig. 195 is a flowchart for a 3D data encoding process according to Embodiment 16.

Fig. 196 is a flowchart for a 3D data decoding process according to Embodiment 16.

Fig. 197 is a diagram illustrating an example of the header information syntax according to Embodiment 16.

Fig. 198 is a diagram illustrating an example of the node information syntax according to Embodiment 16.

Fig. 199 is a flowchart for a 3D data encoding process according to Embodiment 16.

Fig. 200 is a flowchart for a 3D data decoding process according to Embodiment 16.

Fig. 201 is a flowchart for a 3D data encoding process according to Embodiment 16.

Fig. 202 is a flowchart for a 3D data decoding process according to Embodiment 16.

Fig. 203 is a diagram illustrating an example of a tree structure according to Embodiment 17.

Fig. 204 is a diagram illustrating an example of a 3D point sub-cloud according to Embodiment 17.

Fig. 205 is a diagram illustrating an example of a 3D point sub-cloud according to Embodiment 17.

Fig. 206 is a diagram illustrating a structural example of a bit stream according to Embodiment 17.

Fig. 207 is a diagram illustrating an example of a tree structure according to Embodiment 17.

Fig. 208 is a diagram for illustrating full parallel coding according to Embodiment 17, and shows an example of a tree structure.

Fig. 209 is a diagram spatially illustrating 3D point subclouds to be processed in parallel according to Embodiment 17.

Fig. 210 is a diagram illustrating a structural example of a bit stream according to Embodiment 17.

Fig. 211 is a diagram for illustrating the parallel decoding process according to Embodiment 17.

Fig. 212 is a diagram principally illustrating a procedure of a full parallel encoding process according to Embodiment 17.

Fig. 213 is a diagram principally illustrating a procedure of a full parallel decoding process according to Embodiment 17.

Fig. 214 is a diagram for illustrating incremental parallel coding according to Embodiment 17, and shows an example of a tree structure.

Fig. 215 is a diagram indicating operations of cores in incremental parallel coding according to Embodiment 17.

Fig. 216 is a diagram principally illustrating a procedure of the incremental parallel encoding process according to Embodiment 17.

Fig. 217 is a flowchart for a 3D data encoding process according to Embodiment 17.

Fig. 218 is a flowchart for a 3D data decoding process according to Embodiment 17.

Detailed description of embodiments

[0013] A method for encoding 3D data according to one aspect of the present disclosure includes: dividing 3D points included in the 3D data into 3D point subclouds including a first 3D point subcloud and a second 3D point subcloud; attaching first information indicating the space of the first 3D point subcloud to a header of the first 3D point subcloud; attaching second information indicating the space of the second 3D point subcloud to a header of the second 3D point subcloud; and encoding the first 3D point subcloud and the second 3D point subcloud such that the first 3D point subcloud and the second 3D point subcloud are independently decodable.

[0014] Accordingly, the 3D data encoding method is capable of generating coded data, which makes it possible to decode the first 3D point sub-cloud and the second 3D point sub-cloud independently of each other. As a result, the 3D data decoding apparatus can process the encoded data in parallel. Alternatively, the 3D data decoding apparatus may decode one of the first 3D point subcloud and the second 3D point subcloud selectively. Thus, it is possible to reduce the processing time in the 3D data decoding apparatus.

[0015] For example, upon partitioning, the N-ary 3D point tree structure may be divided into branches including a first branch corresponding to the first 3D point subcloud and a second branch corresponding to the second 3D point subcloud, where N is an integer greater than or equal to 2.

[0016] For example, the 3D data encoding method may further include encoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs.

[0017] For example, the layer to which the root of the first branch belongs may be identical to the layer to which the root of the second branch belongs.

[0018] For example, the 3D data encoding method may further include entropy encoding each of the first 3D point subcloud and the second 3D point subcloud using a different coding table.

[0019] For example, the method for encoding 3D data may further include initializing a codebook after entropy encoding the first 3D point subcloud and before entropy encoding the second 3D point subcloud.

[0020] For example, when encoding the first 3D sub-cloud, reference to the second 3D sub-cloud may be prohibited, and when encoding the second 3D sub-cloud, reference to the first 3D sub-cloud may be prohibited.

[0021] For example, the 3D data encoding method may further include: encoding pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud such that the pieces the geometric information of the first 3D points and the geometric information pieces of the second 3D points are independently decodable; and encoding the first 3D point attribute information pieces and the second 3D point attribute information pieces such that the first 3D point attribute information pieces and the second 3D point attribute information pieces are independently decodable.

[0022] For example, the 3D data encoding method may further include: encoding one of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud, and (2) the first 3D point attribute information pieces and the second 3D point attribute information pieces such that one of (1) the first 3D point geometry information pieces and the second 3D point geometry information pieces, and (2) the first 3D point attribute information pieces, and the attribute information pieces of the second 3D points are independently decodable; and encoding another of (1) the first 3D point geometry information pieces and the second 3D point geometry information pieces, and (2) the first 3D point attribute information pieces and the second 3D point attribute information pieces, such that the other of the (1) first 3D point geometry information pieces 3D points and pieces of geometric information of the second 3D points; and (2) pieces of attribute information of the first 3D points and pieces of attribute information of the second 3D points have a dependency relationship with each other.

[0023] For example, the 3D data encoding method may further include encoding a flag indicating whether or not the first 3D sub-cloud and the second 3D sub-cloud are encoded such that the first 3D sub-cloud and the second 3D sub-cloud are decodable independently of each other. friend.

[0024] For example, the first information may indicate the maximum space coordinates of the first 3D point subcloud, and the second information may indicate the maximum space coordinates of the second 3D point subcloud.

[0025] A method for decoding 3D data according to one aspect of the present disclosure includes: obtaining first encoded data and second encoded data generated by encoding a first 3D point subcloud and a second 3D point subcloud such that the first 3D point subcloud and the second 3D point subcloud are independently decodable, wherein the first 3D point sub-cloud and the second 3D point sub-cloud are included in the 3D point sub-clouds obtained by separating the 3D points included in the 3D data; obtaining first information indicating the space of the first 3D point subcloud from the header of the first 3D point subcloud; obtaining second information indicating the space of the second 3D point subcloud from the header of the second 3D point subcloud; and recovering the first 3D point subcloud and the second 3D point subcloud by decoding the first encoded data and the second encoded data.

[0026] Accordingly, the 3D data decoding method is capable of processing the first encoded data and the second encoded data in parallel. Alternatively, the 3D data decoding apparatus may decode one of the first 3D point subcloud and the second 3D point subcloud selectively. Thus, it is possible to reduce the processing time in the 3D data decoding apparatus.

[0027] For example, the first encoded data and the second encoded data may be generated by encoding the first branch corresponding to the first 3D point subcloud and the second branch corresponding to the second 3D point subcloud such that the first branch and the second branch are independently decodable, wherein the first branch and the second branch are included in an N-ary tree structure of three-dimensional points, where N is an integer greater than or equal to 2.

[0028] For example, the method for decoding 3D data may further include decoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs.

[0029] For example, the layer to which the root of the first branch belongs may be identical to the layer to which the root of the second branch belongs.

[0030] For example, the method for decoding 3D data may further include entropy decoding each of the first 3D point subcloud and the second 3D point subcloud using a different codebook.

[0031] For example, the method for decoding 3D data may further include initializing a codebook after entropy decoding of the first 3D point subcloud and before entropy decoding of the second 3D point subcloud.

[0032] For example, when decoding the first 3D point subcloud, the second 3D point subcloud may not be referred to, and when decoding the second 3D point subcloud, the first 3D point subcloud may not be referred to.

[0033] For example, the first encoded data may include the first encoded geometric data and the first encoded attribute data, the first encoded geometric data being generated by encoding pieces of geometric information of the first 3D points included in the first 3D point subcloud, the first encoded attribute data being generated by encoding the attribute information pieces of the first 3D points, the second coded data may include the second coded geometry data and the second coded attribute data, the second coded geometry data being generated by encoding the pieces of geometric information of the second 3D points included in the second 3D point subcloud, the second the encoded attribute data is generated by encoding pieces of attribute information of the second 3D points, the first encoded geometry data, and the second encoder The encoded geometry data may be generated such that the first encoded geometry data and the second encoded geometric data are independently decodable, and the first encoded attribute data and the second encoded attribute data may be generated such that the first encoded attribute data and the second encoded attribute data are decodable independently of each other.

[0034] For example, the first encoded data and the second encoded data may be generated by encoding one of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud, and (2) the first 3D point attribute information pieces included in the first 3D point sub-cloud and the second 3D point attribute information pieces included in the second 3D point sub-cloud, such that one of (1) the first 3D point geometry information pieces included in into the first 3D point sub-cloud, and pieces of geometric information of the second 3D points included in the second 3D point sub-cloud, and (2) pieces of first 3D point attribute information included in the first 3D point sub-cloud and pieces of second 3D point attribute information, including of the second 3D point sub-cloud are independently decodable, and one of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud, and (2) the first 3D point attribute information pieces included in the first 3D point subcloud and the second 3D point attribute information pieces included in the second 3D point subcloud can be recovered by decoding the first coded data and the second coded data; and the 3D data decoding method may further include: obtaining third coded data and fourth coded data generated by encoding another of (1) first 3D point geometry information pieces included in the first 3D point sub-cloud and second 3D point geometry information pieces, of the second 3D point sub-cloud, and (2) the pieces of first 3D point attribute information included in the first 3D point sub-cloud and the second 3D point attribute information pieces included in the second 3D point sub-cloud, such that another of the (1) pieces geometry information of the first 3D points included in the first 3D point sub-cloud and pieces of geometric information of the second 3D points included in the second 3D point sub-cloud, and (2) pieces of attribute information of the first 3D points included in the first 3D point sub-cloud ray points, and attribute information pieces of the second 3D points included in the second 3D point sub-cloud have a dependency relationship with each other; and restoring the other of (1) the first 3D point geometry information pieces included in the first 3D point sub-cloud and the second 3D point geometry information pieces included in the second 3D point sub-cloud, and (2) the first 3D point attribute information pieces included in the first a 3D point subcloud, and second 3D point attribute information pieces included in the second 3D point subcloud by decoding the third encoded data and the fourth encoded data.

[0035] For example, the method for decoding 3D data may further include decoding a flag indicating whether or not the first 3D sub-cloud and the second 3D sub-cloud are encoded such that the first 3D sub-cloud and the second 3D sub-cloud are decodable independently of each other. friend.

[0036] For example, the first information may indicate the maximum space coordinates of the first 3D point subcloud, and the second information may indicate the maximum space coordinates of the second 3D point subcloud.

[0037] A 3D data encoding device according to one aspect of the present disclosure includes a processor and a memory. Using the storage device, the processor: separates the 3D points included in the 3D data into 3D point sub-clouds including a first 3D point sub-cloud and a second 3D point sub-cloud; appends first information indicating the space of the first 3D point subcloud to a header of the first 3D point subcloud; appends second information indicating the space of the second 3D point subcloud to a header of the second 3D point subcloud; and encodes the first 3D point subcloud and the second 3D point subcloud such that the first 3D point subcloud and the second 3D point subcloud are independently decodable.

[0038] Accordingly, the 3D data encoding device can generate encoded data, which makes it possible to decode the first 3D point sub-cloud and the second 3D point sub-cloud independently of each other. As a result, the 3D data decoding apparatus can process the encoded data in parallel. Alternatively, the 3D data decoding apparatus may decode one of the first 3D point subcloud and the second 3D point subcloud selectively. Thus, it is possible to reduce the processing time in the 3D data decoding apparatus.

[0039] A 3D data decoding apparatus according to one aspect of the present disclosure includes a processor and a memory. Using the memory, the processor: obtains the first encoded data and the second encoded data generated by encoding the first 3D point sub-cloud and the second 3D point sub-cloud such that the first 3D point sub-cloud and the second 3D point sub-cloud are independently decodable, the first the 3D point subcloud and the second 3D point subcloud are included in the 3D point subclouds obtained by dividing the 3D points included in the 3D data; obtains first information indicating the space of the first 3D point subcloud from the header of the first 3D point subcloud; obtains second information indicating the space of the second 3D point subcloud from the header of the second 3D point subcloud; and recovering the first 3D point sub-cloud and the second 3D point sub-cloud by decoding the first encoded data and the second encoded data.

[0040] Accordingly, the 3D data decoding apparatus can process the first encoded data and the second encoded data in parallel. Alternatively, the 3D data decoding apparatus may decode one of the first 3D point subcloud and the second 3D point subcloud selectively. Thus, it is possible to reduce the processing time in the 3D data decoding apparatus.

[0041] It should be noted that these general or specific aspects may be implemented as a system, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or may be implemented as any combination of a system, method, integrated circuit, computer program. and the recording medium.

[0042] The following describes embodiments with reference to the drawings. It should be noted that the following embodiments show exemplary embodiments of the present disclosure. Numerical values, shapes, materials, structural components, arrangement and connection of structural components, steps, processing order of steps, etc. shown in the following embodiments are merely examples, and thus do not intend to limit the present disclosure. Of the structural components described in the following embodiments, structural components not set forth in any of the independent claims that indicate the most general ideas are described as optional structural components.

[0043] Embodiment 1

First, a data structure for encoded 3D data (hereinafter also referred to as "encoded data") according to the present embodiment is described. Fig. 1 is a diagram showing the structure of encoded 3D data according to the present embodiment.

[0044] In the present embodiment, the 3D space is divided into spaces (SPC) that correspond to pictures when encoding moving pictures, and the 3D data is encoded based on the SPC. Each SPC is further divided into volumes (VLMs), which correspond to macroblocks, and so on. in moving picture coding, both predictions and transformations are performed based on VLM. Each volume includes a plurality of voxels (VXL), each of which represents the minimum unit in which positional coordinates are associated. It should be noted that prediction is a process of generating 3D predictive data similar to the current processing unit by referring to another processing unit and encoding the differential between the 3D predictive data and the current processing unit, as in the case of predictions performed on 2D images. Such prediction includes not only spatial prediction referring to another prediction unit corresponding to the same time, but also temporal prediction referring to a prediction unit corresponding to a different time.

[0045] When encoding a 3D space represented by point group data such as a point cloud, for example, a 3D data encoding device (hereinafter also referred to as an "encoding device") encodes points in a point group or points included in corresponding voxels in a collaborative manner , according to the voxel size. Finer voxels provide a highly accurate representation of the 3D shape of the dotgroup, while larger voxels provide an approximate representation of the 3D shape of the dotgroup.

[0046] It should be noted that the following describes the case where the 3D data is a point cloud, but the 3D data is not limited to a point cloud, and thus 3D data of any format can be used.

[0047] It should also be noted that voxels with a hierarchical structure can be used. In such a case where the hierarchy includes n levels, whether or not the sampling point is included in the (n-1)th level or its lower levels (lower levels of the nth level) may be sequentially indicated. For example, when only the n-th layer is decoded, and the (n-1)-th layer or its lower layers includes a sample point, the n-th layer can be decoded provided that the sample point is included in the center of the voxel at the n-th layer .

[0048] In addition, the encoder acquires point group data using, for example, a distance sensor, a stereo camera, a monocular camera, a gyro sensor, or an inertial sensor.

[0049] As in the case of moving picture coding, each SPC is classified into one of at least three prediction structures, which include: Intra-SPC (I-SPC), which is separately decodable; predictive SPC (P-SPC) allowing only a unidirectional link; and bidirectional SPC (B-SPC) allowing bidirectional links. Each SPC includes two types of time information: a decode time and a display time.

[0050] In addition, as shown in FIG. 1, a processing unit that includes a plurality of SPCs is a Group Space (GOS), which is a random access unit. In addition, a processing unit that includes a plurality of GOS is a world (WLD).

[0051] The spatial area occupied by each world is associated with an absolute position on earth, through the use of, for example, GPS or latitude and longitude information. Such position information is stored as meta-information. It should be noted that meta-information may be included in the encoded data or may be transmitted separately from the encoded data.

[0052] In addition, in the GOS, all SPCs may be three-dimensionally adjacent to each other, or an SPC may be provided that is not three-dimensionally adjacent to another SPC.

[0053] It should be noted that the following also describes processes such as encoding, decoding, and linking to be performed on 3D data included in processing units such as GOS, SPC, and VLM, simply as performing encoding/encoding, decoding/decoding , links, etc. for a handling unit. It should also be noted that the 3D data included in the processing unit includes, for example, at least one pair of a spatial position such as 3D coordinates and an attribute value such as color information.

[0054] Next, prediction structures for SPC in GOS are described. A plurality of SPCs in the same GOS or a plurality of VLMs in the same SPC occupy mutually different spaces when there is identical time information (decode time and display time).

[0055] The SPC in the GOS that comes first in the decoding order is the I-SPC. GOS are of two types: closed GOS and open GOS. A closed GOS is a GOS in which all SPCs in the GOS are decodable when decoding starts from the first I-SPC. Meanwhile, the open GOS is a GOS in which another GOS is referred to in one or more SPCs preceding the first I-SPC in the GOS in display time, and thus cannot be separately decoded.

[0056] It should be noted that in the case of encoded map information data, for example, WLD is sometimes decoded in the reverse direction, which is the opposite of the coding order, and therefore reverse reproduction is difficult when the GOS are interdependent. In such a case, a closed GOS is essentially used.

[0057] Each GOS has a layered structure in the height direction, and the SPCs are sequentially encoded or decoded from the SPCs in the lower layer.

[0058] FIG. 2 is a diagram showing an example of prediction structures for SPC that belong to the lowest layer in the GOS. Fig. 3 is a diagram showing an example of prediction structures for layers.

[0059] The GOS includes at least one I-SPC. Of the objects in 3D space such as a person, an animal, a car, a bicycle, a traffic light, and a building serving as a landmark, an object of small size is particularly effective when encoded as an I-SPC. When decoding the GOS at low bandwidth or high speed, for example, the 3D data decoder (hereinafter also referred to as "decoder") only decodes the I-SPC in the GOS.

[0060] The encoder may also change the coding interval or frequency of occurrence of the I-SPC, depending on the degree of sparseness and continuity of objects in the WLD.

[0061] In the structure shown in FIG. 3, an encoding device or a decoding device encodes or decodes a plurality of layers sequentially from the bottom layer (layer 1). This increases the priority of data on and near the ground, which contains more information when, for example, an unmanned vehicle is considered.

[0062] Regarding the encoded data used for an unmanned vehicle, for example, encoding or decoding may be performed sequentially from the SPC in the upper layer in the GOS in the altitude direction.

[0063] An encoder or decoder can also encode or decode a plurality of layers in such a way that the decoder can have an approximate understanding of the GOS at first, and then the resolution is incrementally increased. An encoding device or a decoding device, for example, may perform encoding or decoding in the order of layers 3, 8, 1, 9, ...,.

[0064] The following describes the processing of static objects and dynamic objects.

[0065] The three-dimensional space includes scenes or stationary objects such as a building and a road (hereinafter collectively referred to as "static objects"), and objects with movement such as a car and a person (hereinafter collectively referred to as "dynamic objects"). Object detection is separately performed, for example, by extracting key points from point cloud data or from a video camera such as a stereo camera. In this description, an exemplary method for encoding a dynamic object will now be described.

[0066] The first method is a method in which a static object and a dynamic object are encoded without discrimination. The second method is a method in which a distinction is made between a static object and a dynamic object based on identification information.

[0067] For example, GOS is used as the identification unit. In such a case, a distinction is made between a GOS that includes SPCs constituting a static object and a GOS that includes SPCs constituting a dynamic object based on the identification information stored in the encoded data or stored separately from the encoded data.

[0068] Alternatively, the SPC may be used as a unit of identification. In such a case, a distinction is made between an SPC that includes VLMs constituting a static entity and an SPC that includes VLMs constituting a dynamic entity based on such described identification information.

[0069] Alternatively, VLM or VXL may be used as the unit of identification. In such a case, a distinction is made between a VLM or VXL that includes a static object and a VLM or VXL that includes a dynamic object based on such described identification information.

[0070] The encoder may also encode the dynamic object as at least one VLM or SPC, and may encode the VLM or SPC including the static object and the SPC including the dynamic object as mutually distinct GOS. When the GOS size is variable depending on the size of the dynamic object, the encoder separately stores the GOS size as metainformation.

[0071] The encoding device may also encode the static object and the dynamic object separately from each other, and may overlay the dynamic object on a world composed of static objects. In such a case, the dynamic object consists of at least one SPC, and each SPC is associated with at least one SPC constituting the static object on which each SPC is to be superimposed. It should be noted that the dynamic object may not be represented by the SPC, but by at least one VLM or VXL.

[0072] The encoder may also encode the static object and the dynamic object as mutually distinct streams.

[0073] An encoder may also generate a GOS that includes at least one SPC constituting a dynamic object. The encoding device may further set the size of the GOS including the dynamic object (GOS_M) and the size of the GOS including the static object corresponding to the GOS_M spatial area to the same size (so that the same spatial area is occupied). This makes it possible to perform GOS-based overlay.

[0074] An SPC included in another encoded GOS may be referred to in a P-SPC or B-SPC constituting a dynamic entity. In the case where the position of the dynamic object is temporarily changed and the identical dynamic object is encoded as an object in a GOS corresponding to a different time, the reference to the SPC by the GOS is efficient in terms of compression ratio.

[0075] The first method and the second method may be selected according to the appropriate use of the encoded data. When the encoded 3D data is used as a map, for example, the dynamic object needs to be separated, and thus the encoder uses the second method. Meanwhile, the encoding device uses the first method when separation of the dynamic object is not required, for example, in the case where 3D event data such as a concert and a sports competition is encoded.

[0076] The decoding time and display time of the GOS or SPC may be stored in the encoded data or as metainformation. All static objects can have identical time information. In such a case, the decoding device can determine the actual decoding time and the display time. Alternatively, a different value may be assigned to each GOS or SPC as the decode time, and the same value may be assigned as the display time. In addition, as in the case of a decoder model, when encoding moving pictures, such as the HEVC-compliant Hypothetical Reference Decoder (HRD), a model can be used that ensures that the decoder can perform decoding smoothly by having a buffer of a predetermined size and by reading the bit stream at a predetermined bit rate according to the decoding times.

[0077] The following describes the topology of the GOS in the world. The coordinates of three-dimensional space in the world are represented by three coordinate axes (X-axis, Y-axis and Z-axis) that are orthogonal to each other. A predefined set of rules for the GOS coding order enables encoding to be performed such that spatially contiguous GOSs are contiguous in the encoded data. In the example shown in FIG. 4, for example, GOS in the X and Z planes are sequentially encoded. After the encoding of all GOSs in the defined X and Z planes is completed, the Y-axis value is updated. In other words, the world expands in the Y-direction as encoding continues. Numeric GOS indices are given according to the encoding order.

[0078] Here, the 3D spaces in the respective worlds are pre-associated one-to-one with absolute geographic coordinates such as GPS coordinates or latitude/longitude coordinates. Alternatively, each 3D space may be represented as a position relative to a predetermined reference position. The directions of the x-axis, y-axis, and z-axis in 3D space are represented by directional vectors, which are determined based on latitudes and longitudes, and so on. Such directional vectors are stored along with the encoded data as meta-information.

[0079] GOS have a fixed size, and the encoder stores this size as metainformation. The GOS size may vary, for example, depending on whether it is an urban area or not, or whether it is indoors or outdoors. In other words, the GOS-size may vary according to the number or attributes of objects with informational values. Alternatively, in the same world, the encoder may adaptively change the GOS size or interval between I-SPCs in the GOS according to object density, and so on. For example, the encoder sets the GOS size to be smaller and the interval between I-SPCs in the GOS to be shorter as the object density becomes higher.

[0080] In the example shown in FIG. 5, in order to provide random access with greater granularity, a GOS with a high object density is segmented into regions of the third to tenth GOS. It should be noted that the seventh-tenth GOS are located after the third-sixth GOS.

[0081] Next, the structure and operation of the 3D data encoding device according to the present embodiment will be described. Fig. 6 is a block diagram of a 3D data encoding apparatus 100 according to the present embodiment. Fig. 7 is a flowchart of an exemplary operation performed by the 3D data encoding apparatus 100.

[0082] The 3D encoding device 100 shown in FIG. 6 encodes 3D data 111, thereby generating encoded 3D data 112. Such a 3D data encoding device 100 includes an acquisition unit 101, an encoding area determination unit 102, a separation unit 103, and an encoder 104.

[0083] As shown in FIG. 7, firstly, the acquisition unit 101 acquires the 3D data 111, which is the dot group data (S101).

[0084] Next, the coding area determining unit 102 determines the current area to be coded from among the spatial areas corresponding to the received point group data (S102). For example, in accordance with the position of the user or the vehicle, the coding region determining unit 102 determines, as the current region, a spatial region around such a position.

[0085] Next, the division unit 103 divides the dot group data included in the current area into processing units. Processing units here mean units such as GOS and SPC as described above. The current area here corresponds, for example, to the world described above. More specifically, the separation unit 103 separates the dot group data into processing units based on a predetermined GOS size or the presence/absence/size of a dynamic object (S103). The splitter 103 further determines the starting position of the SPC that comes first in the coding order in each GOS.

[0086] Next, encoder 104 sequentially encodes a plurality of SPCs in each GOS, thereby generating encoded 3D data 112 (S104).

[0087] It should be noted that although an example is described here in which the current area is divided into GOS and SPC, after which each GOS is encoded, the processing steps are not limited to this order. For example, steps may be used in which the structure of one GO is determined, after which the encoding of that GOS is performed, and then the structure of the subsequent GOS is determined.

[0088] Thus, as described above, the 3D encoding apparatus 100 encodes the 3D data 111, thereby generating encoded 3D data 112. More specifically, the 3D encoding apparatus 100 divides the 3D data into first processing units (GOS), each of which is a random access unit and is associated with three-dimensional coordinates, divides each of the first processing units (GOS) into second processing units (SPC), and divides each of the second processing units (SPC) into third processing units (VLM). Each of the third processing units (VLM) includes at least one voxel (VXL), which is the minimum unit in which position information is associated.

[0089] Next, the 3D encoding apparatus 100 encodes each of the first processing units (GOS), thereby generating encoded 3D data 112. More specifically, the 3D encoding apparatus 100 encodes each of the second processing units (SPC) in each of the first handling units (GOS). The 3D encoding device 100 further encodes each of the third processing units (VLM) in each of the second processing units (SPC).

[0090] When the current first processing unit (GOS) is a closed GOS, for example, the 3D encoding apparatus 100 encodes the current second processing unit (SPC) included in such current first processing unit (GOS) by referring to another second processing unit (SPC) included in the current first handling unit (GOS). In other words, the 3D encoding device 100 does not refer to the second processing unit (SPC) included in the first processing unit (GOS) that is different from the current first processing unit (GOS).

[0091] Meanwhile, when the current first processing unit (GOS) is an open GOS, the 3D encoding device 100 encodes the current second processing unit (SPC) included in such a current first processing unit (GOS) by referring to another second processing unit processing unit (SPC) included in the current first handling unit (GOS), or to a second handling unit (SPC) included in the first handling unit (GOS) that is different from the current first handling unit (GOS).

[0092] Also, the 3D data encoding apparatus 100 selects, as the type of the current second processing unit (SPC), one of the following: a first type (I-SPC) in which no other second processing unit (SPC) is referred to; a second type (P-SPC) in which another one second processing unit (SPC) is referred to; and a third type in which the other two second processing units (SPC) are referred to. The 3D encoding device 100 encodes the current second processing unit (SPC) according to the selected type.

[0093] Next, the structure and operation of the 3D data decoding apparatus according to the present embodiment will be described. Fig. 8 is a block diagram of a 3D data decoding apparatus 200 according to the present embodiment. Fig. 9 is a flowchart of an exemplary operation performed by the 3D data decoding apparatus 200.

[0094] The 3D data decoding apparatus 200 shown in FIG. 8 decodes the encoded 3D data 211, thereby generating decoded 3D data 212. The encoded 3D data 211 here, for example, is encoded 3D data 112 generated by the 3D data encoding apparatus 100. Such a 3D data decoding apparatus 200 includes an acquisition unit 201, a GOS determination unit 202 to start decoding, an SPC determination unit 203 to decode, and a decoder 204.

[0095] First, the receiving unit 201 acquires the encoded 3D data 211 (S201). Then, the decoding start GOS determining unit 202 determines the current decoding GOS (S202). More specifically, the GOS determination module 202, to start decoding, refers to meta-information stored in the encoded 3D data 211 or stored separately from the encoded 3D data to determine, as the current GOS, a GOS that includes an SPC corresponding to a spatial position, an object, or the time at which decoding should start.

[0096] Next, the decoding SPC determination module 203 determines the type(s) (I-, P-, and/or B-) of SPCs to be decoded in the GOS (S203). For example, the decoding SPC determination module 203 determines whether to (1) decode only I-SPCs, (2) decode I-SPCs and P-SPCs, or (3) decode all types of SPCs. It should be noted that this step may not be performed when the type(s) of SPCs to be decoded are predetermined, for example, when all SPCs to be decoded are predetermined.

[0097] Next, the decoder 204 obtains the location of the address in the encoded 3D data 211 from which the SPC starts, which comes first in the GOS in the decoding order (identical to the coding order). The decoder 204 obtains the encoded data of the first SPC from the address location and sequentially decodes the SPC from that first SPC (S204). It should be noted that the location of the address is stored in meta-information, etc.

[0098] The 3D decoder 200 decodes the decoded 3D data 212 in the manner described above. More specifically, the 3D decoder 200 decodes each encoded 3D first processing unit (GOS) data 211, each of which is a random access unit and is associated with 3D coordinates, thereby generating decoded 3D first processing unit (GOS) data 212 . More specifically, the 3D data decoding apparatus 200 decodes each of the second processing units (SPC) in each of the first processing units (GOS). The 3D data decoding apparatus 200 further decodes each of the third processing units (VLMs) in each of the second processing units (SPCs).

[0099] The following describes meta-information for random access. Such metainformation is generated by the 3D encoding device 100 and included in the encoded 3D data 112 (211).

[0100] In traditional random access for a two-dimensional moving picture, decoding starts from the first frame in the random access unit, which is close to the specified time. Meanwhile, in addition to times, random access to spaces (coordinates, objects, etc.) is supposed to be done in the world.

[0101] In order to provide random access to at least three elements of coordinates, objects and times, tables are prepared that associate the corresponding elements with numerical GOS indexes. In addition, the numeric GOS indexes are associated with the addresses of the respective first I-SPCs in the GOS. Fig. 10 is a diagram showing exemplary tables included in meta information. It should be noted that not all tables shown in FIG. 10 must be used, and therefore at least one of the tables is used.

[0102] The following describes an example in which random access is performed from coordinates as a starting point. In order to access the coordinates (x2, y2 and z2), the coordinate table and the GOS are first referenced, which indicates that the point corresponding to the coordinates (x2, y2 and z2) is included in the second GOS. The GOS and address table is then referenced, which indicates that the address of the first I-SPC in the second GOS is addr(2). In this regard, the decoder 204 receives data from this address to start decoding.

[0103] It should be noted that the addresses may either be logical addresses or physical addresses of the HDD or storage device. Alternatively, information that identifies file segments may be used instead of addresses. File segments, for example, are units obtained by segmenting at least one GOS, and so on.

[0104] When an object spans a set of GOS, the table of objects and GOS may show the set of GOS to which such an object belongs. When such a plurality of GOSs are closed GOSs, the encoder and the decoder may perform encoding or decoding in parallel. Meanwhile, when such a plurality of GOS are open GOS, higher compression efficiency is achieved by the plurality of GOS referring to each other.

[0105] Exemplary objects include a person, an animal, a car, a bicycle, a traffic light, and a building serving as a landmark. For example, the 3D encoding device 100 extracts key points specific to an object from a 3D point cloud, etc. when encoding the world, and detects an object based on such key points to set the detected object as a random access point.

[0106] Thus, as described above, the 3D data encoding apparatus 100 generates first information indicative of a plurality of first processing units (GOS) and 3D coordinates associated with respective first processing units (GOS). The encoded 3D data 112 (211) includes such first information. The first information further indicates at least one of the data storage objects, times, and locations that are associated with the respective first processing units (GOS).

[0107] The 3D decoding apparatus 200 obtains first information from the encoded 3D data 211. Using such first information, the 3D decoding apparatus 200 identifies the encoded 3D data 211 of the first processing unit that corresponds to the specified 3D coordinates, object, or time, and decodes the encoded 3D data 211.

[0108] The following describes an example of other metainformation. In addition to the random access meta-information, the 3D data encoding apparatus 100 can also generate and store meta-information as described below, and the 3D data decoding apparatus 200 may use such meta-information during decoding.

[0109] When 3D data is used as mapping information, for example, a profile is set according to appropriate usage, and information indicating such a profile may be included in the meta information. For example, a profile is given for an urban or suburban area, or for a flying object, and a maximum or minimum size, and so on. world, SPC or VLM, etc. set in each profile. For example, more detailed information is required for an urban area than for a suburban area, and thus the minimum VLM size is set small.

[0110] The meta information may include tag values indicating object types. Each of these tag values is associated with the VLM, SPC, or GOS that make up the object. For example, a tag value may be set for each object type such that a tag value of "0" indicates "person", a tag value of "1" indicates "car", and a tag value of "2" indicates "traffic light". Alternatively, when it is difficult to evaluate the type of an object, or such evaluation is not required, a tag value may be used that indicates a size or an attribute indicating, for example, whether the object is a dynamic object or a static object.

[0111] The meta information may also include information indicating the range of the spatial area occupied by the world.

[0112] The meta-information may also store an SPC or VXL size as header information common to an entire encoded data stream or to multiple SPCs, such as the SPC in the GOS.

[0113] The meta-information may also include identification information on a distance sensor or camera that is used to generate the point cloud, or information indicating the positioning accuracy of a point group in the point cloud.

[0114] The meta information may also include information indicating whether the world consists only of static objects or includes a dynamic object.

[0115] Variations of the present embodiment are described next.

[0116] An encoder or a decoder may encode or decode two or more mutually distinct SPCs or GOSs in parallel. The GOS to be encoded or decoded in parallel may be determined based on meta-information, etc., indicating the spatial positions of the GOS.

[0117] When the 3D data is used as a spatial map for use by a car or a flying object, etc. when driving or to create such a spatial map, for example, an encoder or a decoder may encode or decode the GOS or SPC included in the space that is identified based on GPS information, route information, zoom, etc.

[0118] The decoding device may also start decoding sequentially from a space that is close to its own location or travel route. The encoder or decoder may give lower priority to a space remote from one's own location or travel route than the priority of a nearby space in order to encode or decode such a remote space. "Give lower priority" here means, for example, lowering the priority in the processing sequence, lowering the resolution (applying decimation in processing), or lowering the image quality (increasing coding efficiency, for example, by setting the quantization step large).

[0119] When decoding encoded data that is hierarchically encoded in space, the decoding device can only decode the lower level in the hierarchy.

[0120] The decoding device may also start decoding preferentially from a lower level of the hierarchy according to zooming in or proper use of the map.

[0121] For estimating one's own location or recognizing objects, etc., involved in unmanned driving of a car or robot, an encoder or a decoder may encode or decode areas with a lower resolution, except for an area that is lower or at a given height from the ground (the area to be recognized).

[0122] The encoding device can also encode point clouds representing the spatial forms of the interior of the room and the exterior of the room separately. For example, separating the GOS representing the inside of the room (indoor GOS) and the GOS representing the outside of the room (outdoor GOS) allows the decoder to select the GOS to be decoded according to the location of the viewpoint when using encoded data.

[0123] The encoder may also encode the inner GOS and the outer GOS having close coordinates such that such GOSs approach each other in the codestream. For example, the encoder associates the identifiers of such GOS with each other and stores information indicating the associated identifiers in meta-information, which is stored in the codestream or stored separately. This allows the decoder to refer to the information in the meta-information to identify the inner GOS and the outer GOS having close coordinates.

[0124] The encoder may also change the GOS size or SPC size depending on whether the GOS is an inner GOS or an outer GOS. For example, the encoder sets the size of the inner GOS to be less than the size of the outer GOS. The encoder may also change the accuracy of extracting key points from the point cloud or the accuracy of object detection, for example, depending on whether the GOS is an indoor GOS or an outdoor GOS.

[0125] The encoding device may also add, to the encoded data, information by which the decoding device displays objects with a distinction between a dynamic object and a static object. This allows the decoding device to display the dynamic object, for example, along with a red box or letters for explanation. It should be noted that the decoder may only display a red box or letters for explanation instead of a dynamic object. The decoder may also display more specific types of objects. For example, a red box might be used for a car, and a yellow box might be used for a person.

[0126] The encoder or decoder may also determine whether or not to encode or decode the dynamic object and the static object as another SPC or GOS, according to, for example, the frequency of occurrence of dynamic objects or the ratio between static objects and dynamic objects. For example, when the occurrence rate or ratio of dynamic objects exceeds a threshold, an SPC or GOS including a mixture of a dynamic object and a static object is adopted, while when an occurrence frequency or ratio of dynamic objects is below the threshold, an SPC or GOS including a mixture of a dynamic object and a static object is not accepted.

[0127] When detecting a dynamic object not from a point cloud but from 2D camera image information, the encoding device can separately obtain information for identifying the detection result (field or letters) and the position of the object, and encode these pieces of information as part of the encoded 3D data. In such a case, the decoder overlays auxiliary information (field or letters) indicating the dynamic object on the decoding result of the static object to display it.

[0128] The encoder can also change the sparseness and solidity of the VXL or VLM in the SPC according to the complexity of the shape of the static object. For example, the encoder specifies a higher density VXL or VLM as the shape of the static object becomes more complex. The encoder may further determine the quantization step, and so on. to quantize spatial positions or color information according to VXL or VLM sparseness and continuity. For example, the encoder sets the quantization step to be smaller as the VXL or VLM density becomes higher.

[0129] As described above, the encoding device or the decoding device according to the present embodiment encodes or decodes an SPC-based space that includes coordinate information.

[0130] In addition, the encoding device and the decoding device perform encoding or decoding based on the volume in the SPC. Each volume includes a voxel, which is the smallest unit in which position information is associated.

[0131] In addition, using a table that associates the corresponding elements of spatial information, including coordinates, objects and times, with GOS, or using a table that associates these elements with each other, the encoding device and the decoding device associate any elements with each other to perform encoding or decoding. The decoder uses the values of the selected elements to determine the coordinates and identifies the volume, voxel, or SPC from such coordinates to decode the SPC including such volume or voxel, or the identified SPC.

[0132] In addition, the encoding device determines the volume, voxel or SPC, which can be selected in accordance with the elements, through the extraction of key points and object recognition, and encodes the determined volume, voxel or SPC, as the volume, voxel or SPC, for which is possible random access.

[0133] SPCs are classified into three types: I-SPC, which is separately encoded or decodable; P-SPC, which is encoded or decoded by reference to any of the processed SPCs; and B-SPC, which is encoded or decoded by referring to any two of the processed SPCs.

[0134] At least one volume corresponds to a static object or a dynamic object. The SPC including the static object and the SPC including the dynamic object are encoded or decoded as mutually distinct GOS. In other words, an SPC including a static object and an SPC including a dynamic object are assigned to different GOS.

[0135] Dynamic objects are encoded or decoded based on the object and are associated with at least one SPC including the static object. In other words, a plurality of dynamic objects are individually encoded, and the resulting encoded dynamic object data is associated with an SPC including the static object.

[0136] The encoder and the decoder give increased priority to the I-SPC in the GOS to perform encoding or decoding. For example, the encoding device performs encoding in a manner that prevents degradation of I-SPC performance (in a manner that reproduces the original 3D data with higher fidelity after decoding). The decoder only decodes the I-SPC, for example.

[0137] The encoding device may change the frequency of using the I-SPC depending on the sparseness and continuity or the number (number) of objects in the world to perform encoding. In other words, the encoder changes the selection frequency of the I-SPC depending on the number or sparsity and solidity of the objects included in the 3D data. For example, the encoder uses I-SPC at a higher frequency as the density of objects in the world becomes higher.

[0138] The encoder also defines random access points based on the GOS and stores information indicating spatial regions corresponding to the GOS in the header information.

[0139] The encoding device uses, for example, the default value as the spatial size of the GOS. It should be noted that the encoder may change the GOS size depending on the number (amount) or sparsity and solidity of objects or dynamic objects. For example, the encoder sets the spatial size of the GOS to be smaller as the density of objects or dynamic objects becomes higher or the number of objects or dynamic objects becomes larger.

[0140] In addition, each SPC or volume includes a keypoint group that is derived by using information obtained by a sensor such as a depth sensor, a gyro sensor, or a camera sensor. The coordinates of the key points are set in the central positions of the corresponding voxels. In addition, more accurate voxels provide highly accurate position information.

[0141] A group of key points is extracted by using a plurality of images. The plurality of pictures includes at least two types of time information: actual time information and identical time information common to the plurality of pictures that are associated with the SPC (eg, coding time used for rate control, etc.).

[0142] In addition, encoding or decoding is performed based on the GOS, which includes at least one SPC.

[0143] The encoder and the decoder predict the P-SPC or B-SPC in the current GOS by referring to the SPC in the processed GOS.

[0144] Alternatively, the encoder and the decoder predict the P-SPC or B-SPC in the current GOS using the processed SPCs in the current GOS, without reference to another GOS.

[0145] In addition, the encoding device and the decoding device transmit or receive a world-based coded stream that includes at least one GOS.

[0146] In addition, the GOS has a layered structure in one direction at least in the world, and the encoding device and the decoding device start encoding or decoding from the lower layer. For example, the random available GOS belongs to the bottommost layer. The GOS that belongs to the same layer or bottom layer is referred to as the GOS that belongs to the top layer. In other words, the GOS is spatially separated in a predetermined direction in advance to have a plurality of layers including at least one SPC. The encoding device and the decoding device encode or decode each SPC by referring to an SPC included in a layer identical to a layer of each SPC or an SPC included in a layer below each SPC layer.

[0147] In addition, the encoding device and the decoding device sequentially encode or decode the GOS based on the world that includes such GOS. Thus, the encoding device and the decoding device write or read information indicating the encoding or decoding order (direction) as metadata. In other words, the encoded data includes information indicating the encoding order of the GOS.

[0148] An encoder and a decoder also encode or decode mutually distinct two or more SPCs or GOSs in parallel.

[0149] In addition, the encoding device and the decoding device encode or decode spatial information (coordinates, size, etc.) regarding the SPC or GOS.

[0150] An encoder and a decoder encode or decode the SPCs or GOS included in the identified space, which is identified based on external information about its own location and/or size of the area, such as GPS information, route information, or magnification.

[0151] The encoder or decoder gives a lower priority to a space far from its own location than a nearby space to perform encoding or decoding.

[0152] The encoding device specifies a direction in one of the directions in the world, in accordance with the increase or proper use, to encode a GOS having a layered structure in such a direction. In addition, the decoding device decodes the GOS having a layered structure in one of the world directions, which is set in accordance with the increase or appropriate use, preferably from the lower layer.

[0153] The encoding device changes the keypoint extraction accuracy, the object recognition accuracy, or the size of the spatial regions, etc. included in the SPC, depending on whether the object is an indoor object or an outdoor object. It should be noted that the encoder and the decoder encode or decode the inner GOS and the outer GOS having close coordinates in such a way that the GOSs approximate each other in the world, and associate their identifiers with each other for encoding and decoding.

[0154] Embodiment 2

When using encoded point cloud data in an actual device or service, it is desirable if the necessary information is transmitted/received in accordance with the proper usage in order to reduce network bandwidth. However, there is no such functionality in the 3D encoding structure, and no encoding method for it.

[0155] The present embodiment describes a 3D data encoding method and a 3D data encoding apparatus for providing the functionality of transmitting/receiving only necessary information in encoded 3D point cloud data according to appropriate use, as well as a 3D data decoding method and a 3D data decoding apparatus for decoding such encoded data.

[0156] A voxel (VXL) with a feature greater than or equal to a given number is defined as a feature voxel (FVXL), and a world (WLD) consisting of FVXL is defined as a sparse world (SWLD). Fig. 11 is a diagram showing exemplary sparse-world and world structures. SWLD includes: FGOS, each of which is a GOS consisting of FVXL; FSPC, each of which is an SPC consisting of FVXL; and FVLM, each of which is a VLM composed of FVXL. The data structure and prediction structure of FGOS, FSPC, and FVLM may be identical to the data structure and prediction structure of GOS, SPC, and VLM.

[0157] The feature represents 3D position information relative to VXL or visible light range information relative to VXL position. A large number of signs are found, in particular, in the corner, on the edge, etc. three-dimensional object. More specifically, such a feature is a three-dimensional feature or a visible light range feature as described below, but may be any feature that represents position information, luminance information, or color information, etc. regarding VXL.

[0158] Orientation histogram signature features (SHOT), point feature histogram features (PFH), or point pair features (PPF) are used as 3D features.

[0159] SHOT features are obtained by dividing the VXL periphery and calculating the inner product of the reference point and the normal vector of each divided area to represent the result of the calculation as a histogram. SHOT features are characterized by a large number of dimensions and a high-level feature representation.

[0160] PFH features are obtained by selecting a large number of two point pairs around VXL and calculating a normal vector, and so on. of every two point pairs to represent the calculation result as a histogram. PFH features are histogram features and are therefore characterized by resistance to a certain degree of distortion, as well as by high-level feature representation.

[0161] PPF features are obtained by using a normal vector, and so on. for every two VXL points. PPF features, for which all VXLs are used, are resistant to occlusion.

[0162] As visible light range features, scale invariant feature transformation (SIFT), accelerated reliable features (SURF), or histogram oriented gradients (HOG), etc. are used, which use image information such as luminance gradient information .

[0163] The SWLD is generated by calculating the above features of the corresponding VXLs in the WLD in order to extract the FVXL. Here, the SWLD may be updated every time the WLD is updated, or it may be regularly updated after a certain period of time has elapsed, regardless of the time when the WLD is updated.

[0164] SWLD may be generated for each type of features. For example, different SWLDs may be generated for respective feature types, such as SWLD1 based on SHOT features and SWLD2 based on SIFT features, so that the SWLDs are selectively used according to the proper usage. In addition, the calculated feature of each FVXL may be stored in each FVXL as feature information.

[0165] The following describes the use of sparse world (SWLD). SWLD only includes feature voxels (FVXL) and therefore its data size is generally smaller than WLD, which includes all VXLs.

[0166] In an application that uses the features for a specific purpose, using the information on the SWLD instead of the WLD reduces the time required to read data from the hard disk, as well as the bandwidth and time required to transfer data over the network. For example, WLD and SWLD are stored on the server as mapping information such that the mapping information to be sent is selected between WLD and SWLD according to a request from a client. This reduces network bandwidth and the time it takes to transfer data. More specific examples are explained below.

[0167] FIG. 12 and FIG. 13 are diagrams showing examples of using SWLD and WLD. As shown in FIG. 12, when the client 1, which is a vehicle-mounted device, requires map information to use it to determine its own location, the client 1 sends a request for obtaining map data to estimate its own position to the server (S301). The server sends to the client 1 SWLD in response to the receive request (S302). Client 1 uses the received SWLD to determine its own location (S303). Thus, the client 1 obtains VXL information regarding the periphery of the client 1 through various means, including a distance sensor such as a range finder, as well as a stereo camera and a combination of a plurality of monocular cameras. Client 1 then estimates its own location information from the received VXL information and the SWLD. Here, the own location information includes 3D position information, orientation, and so on. client 1.

[0168] As shown in FIG. 13, when the client 2, which is a vehicle-installed device, requires map information to be used for rendering a map such as a 3D map, the client 2 sends a request for obtaining map data for rendering maps to the server (S311). The server sends to the client 2 WLDs in response to the receive request (S312). The client 2 uses the received WLD to render a map (S313). Thus, the client 2 uses, for example, an image that the client 2 captures with a visible light camera, etc., and a WLD obtained from the server to create an image for rendering, and renders such a generated image on car navigation screen, etc.

[0169] As described above, the server sends to the SWLD client when the corresponding VXL features are mainly required, for example, in the case of own location estimation, and sends to the WLD client when detailed VXL information is required, for example, in the case of map rendering . This enables efficient sending/receiving of map data.

[0170] It should be noted that the client can self-evaluate which of the SWLD and WLD is needed and request the server to send the SWLD or WLD. In addition, the server may judge which of the SWLD and WLD should be sent according to the state of the client or network.

[0171] The following describes a method for switching send/receive between sparse world (SWLD) and world (WLD).

[0172] Whether a WLD or SWLD should be received may be switched according to the network bandwidth. Fig. 14 is a diagram showing an exemplary operation in such a case. For example, when a low-speed network is used, which limits the usable network bandwidth, such as in a Long Term Evolution (LTE) environment, the client accesses the server via the low-speed network (S321) and obtains the SWLD from the server as map information (S322) . Meanwhile, when a high-speed network that has an adequately wide network bandwidth is used, such as in a Wi-Fi environment, the client accesses the server via the high-speed network (S323) and acquires the WLD from the server (S324). This allows the client to obtain appropriate mapping information according to the network bandwidth that the client is using.

[0173] More specifically, the client receives the SWLD over an LTE network outdoors, and receives the WLD over a Wi-Fi network indoors, such as in an infrastructure facility. This allows the client to obtain more detailed mapping information regarding the indoor environment.

[0174] As described above, a client may request a WLD or SWLD in accordance with the network bandwidth that such a client uses. Alternatively, the client may send to the server information indicating the network bandwidth that such client is using, and the server may send data (WLD or SWLD) suitable for such client according to the information to the client. Alternatively, the server may identify the network bandwidth the client is using and send to the client data (WLD or SWLD) appropriate for that client.

[0175] In addition, then, should take WLD or SWLD, can be switched in accordance with the speed of movement. Fig. 15 is a diagram showing an exemplary operation in such a case. For example, when driving at high speed (S331), the client receives the SWLD from the server (S332). Meanwhile, when driving at low speed (S333), the client receives the WLD from the server (S334). This allows the client to obtain map information suitable for speed while reducing network bandwidth. More specifically, when driving on an expressway, the client receives a SWLD with a small amount of data, which ensures that the approximate map information is updated at an appropriate rate. Meanwhile, when driving on a public road, the customer adopts WLD, which provides more detailed map information.

[0176] As described above, the client may query the server for a WLD or SWLD in accordance with the movement speed of such a client. Alternatively, the client may send to the server information indicating the running speed of such a client, and the server may send to the client data (WLD or SWLD) suitable for such a client according to the information. Alternatively, the server may identify the movement speed of the client in order to send data (WLD or SWLD) suitable for such a client.

[0177] In addition, the client can receive, from the server, the SWLD first, from which the client can obtain the WLD of the important area. For example, when obtaining map information, the client first obtains the SWLD for the approximate map information, from which the client narrows down to an area in which features such as buildings, traffic lights, or people appear at high frequency, so that the client can later obtain the WLD of such a narrowed area. . This allows the client to get detailed information about the required area, while reducing the amount of data received from the server.

[0178] The server may also generate different SWLDs from the WLDs for the respective entities, and the client may receive the SWLDs according to appropriate usage. This reduces the network bandwidth. For example, the server recognizes people or cars in a WLD in advance and creates a people SWLD and a car SWLD. The client, when desiring to receive information about people near the client, receives SWLDs of people, and, when desiring to receive information regarding cars, receives SWLDs of cars. Such SWLD types can be distinguished by information (flag or type, etc.) added to the header, etc.

[0179] Next, the structure and operation of the 3D encoding apparatus (eg, server) according to the present embodiment will be described. Fig. 16 is a block diagram of a 3D data encoding device 400 according to the present embodiment. Fig. 17 is a flowchart for 3D data encoding processes performed by the 3D data encoding apparatus 400 .

[0180] The 3D encoding device 400 shown in FIG. 16 encodes the input 3D data 411, thereby generating encoded 3D data 413 and encoded 3D data 414, each of which is a coded stream. Here, the encoded 3D data 413 is encoded 3D data corresponding to the WLD, and the encoded 3D data 414 is encoded 3D data corresponding to the SWLD. Such a 3D data encoding device 400 includes an acquisition unit 401, an encoding area determination unit 402, an extraction SWLD unit 403, a WLD encoder 404, and a SWLD encoder 405.

[0181] First, as shown in FIG. 17, the acquisition unit 401 receives input 3D data 411, which is point group data in 3D space (S401).

[0182] Next, the encoding area determination unit 402 determines the current spatial area for encoding based on the spatial area in which point cloud data is present (S402).

[0183] Next, the SWLD extraction module 403 sets the current spatial region as the WLD, and calculates a feature from each VXL included in the WLD. Then, the extraction SWLD module 403 extracts VXLs having a feature number greater than or equal to a predetermined threshold value, sets the extracted VXLs as FVXLs, and adds such FVXLs to the SWLD, thereby generating the extracted 3D data 412 (S403). In other words, the extracted 3D data 412 having the number of features greater than or equal to the threshold value is extracted from the input 3D data 411.

[0184] Next, the WLD encoder 404 encodes the input 3D data 411 corresponding to the WLD, thereby generating encoded 3D data 413 corresponding to the WLD (S404). Thus, the WLD encoder 404 adds information to the header of the encoded 3D data 413 that distinguishes that such encoded 3D data 413 is a stream including the WLD.

[0185] The SWLD encoder 405 encodes the extracted 3D data 412 corresponding to the SWLD, thereby generating encoded 3D data 414 corresponding to the SWLD (S405). Thus, the SWLD encoder 405 adds to the header of the encoded 3D data 414 information that distinguishes that such encoded 3D data 414 is a stream including the SWLD.

[0186] It should be noted that the encoded 3D data generation process 413 and the encoded 3D data generation process 414 may be performed in reverse order. It should also be noted that some or all of these processes may be performed in parallel.

[0187] The parameter "world_type" is set, for example, as information added to each header of the encoded 3D data 413 and the encoded 3D data 414; world_type=0 indicates that the stream includes a WLD, and world_type=1 indicates that the stream includes a SWLD. An increased number of values may optionally be assigned to define more types, eg world_type=2. In addition, one of the encoded 3D data 413 and the encoded 3D data 414 may include the specified flag. For example, encoded 3D data 414 may be assigned a flag indicating that such a stream includes a SWLD. In such a case, the decoding apparatus can distinguish whether such a stream is a stream including a WLD or a stream including a SWLD according to the presence/absence of a flag.

[0188] In addition, the encoding method used by the WLD encoder 404 to encode the WLD may be different from the encoding method used by the SWLD encoder 405 to encode the SWLD.

[0189] For example, SWLD data is decimated and therefore may have a lower correlation with ambient data than WLD correlation. For this reason, from intra prediction and inter prediction, inter prediction may be more preferably performed in the coding method used for SWLD than in the coding method used for WLD.

[0190] In addition, the encoding method used for SWLD and the encoding method used for WLD may represent 3D positions differently. For example, 3D coordinates may be used to represent 3D FVXL positions in SWLD, and the octant tree described below may be used to represent 3D positions in WLD, and vice versa.

[0191] In addition, the SWLD encoder 405 performs encoding in such a way that the encoded SWLD 3D data 414 has a smaller data size than the data size of the encoded WLD 3D data 413. SWLD may have a lower correlation between data, for example, than the correlation for WLD, as described above. This may result in reduced coding efficiency and therefore encoded 3D data 414 having a larger data size than the data size of WLD encoded 3D data 413 . When the data size for the resulting encoded 3D data 414 exceeds the data size for the WLD encoded 3D data 413, the SWLD encoder 405 performs encoding again to regenerate the encoded 3D data 414 having a reduced data size.

[0192] For example, the SWLD extraction module 403 regenerates the extracted 3D data 412 having a reduced number of key points to be extracted, and the SWLD encoder 405 encodes such extracted 3D data 412. Alternatively, the SWLD encoder 405 may perform more approximate quantization . More approximate quantization is achieved, for example, by rounding the data at the lowest level in the octant tree structure described below.

[0193] If the attempt to reduce the data size for the encoded 3D data 414 SWLD to less than the data size for the encoded 3D WLD data 413 fails, the SWLD encoder 405 may not generate the encoded 3D data 414 SWLD. Alternatively, WLD encoded 3D data 413 may be copied as SWLD encoded 3D data 414 . In other words, the WLD encoded 3D data 413 can be used as is as the SWLD encoded 3D data 414 .

[0194] Next, the structure and operation of the 3D data decoding apparatus (eg, client) according to the present embodiment will be described. Fig. 18 is a block diagram of a 3D data decoding apparatus 500 according to the present embodiment. Fig. 19 is a flowchart for 3D data decoding processes performed by the 3D data decoding apparatus 500.

[0195] The 3D data decoding apparatus 500 shown in FIG. 18 decodes encoded 3D data 511, thereby generating decoded 3D data 512 or decoded 3D data 513. Encoded 3D data 511 here, for example, is encoded 3D data 413 or encoded 3D data 414 generated by the 3D encoding device 400.

[0196] Such a 3D data decoding apparatus 500 includes an acquisition unit 501, a header parser 502, a WLD decoder 503, and a SWLD decoder 504.

[0197] First, as shown in FIG. 19, the receiving unit 501 acquires the encoded 3D data 511 (S501). Next, the header parser 502 parses the header of the encoded 3D data 511 to identify whether the encoded 3D data 511 is a stream including a WLD or a stream including a SWLD (S502). For example, the world_type parameter described above is referenced when such identification is performed.

[0198] When the encoded 3D data 511 is a stream including WLD ("Yes" in S503), the WLD decoder 503 decodes the encoded 3D data 511, thereby generating decoded WLD 3D data 512 (S504). Meanwhile, when the encoded 3D data 511 is a stream including SWLD ("No" in S503), the SWLD decoder 504 decodes the encoded 3D data 511, thereby generating decoded 3D SWLD data 513 (S505).

[0199] In addition, as with the encoding device, the decoding method used by the WLD decoder 503 to decode the WLD may be different from the decoding method used by the SWLD decoder 504 to decode the SWLD. For example, from intra prediction and inter prediction, inter prediction may be more preferably performed in the decoding method used for SWLD than in the decoding method used for WLD.

[0200] In addition, the decoding method used for SWLD and the decoding method used for WLD may represent 3D positions differently. For example, 3D coordinates may be used to represent 3D FVXL positions in SWLD, and the octant tree described below may be used to represent 3D positions in WLD, and vice versa.

[0201] Next, octant tree representation, which is a method for representing three-dimensional positions, will be described. The VXL data included in the 3D data is converted into an octant tree structure before encoding. Fig. 20 is a diagram showing exemplary VXLs in a WLD. Fig. 21 is a diagram showing the octant tree structure of the WLD shown in FIG. 20. The example shown in FIG. 20 illustrates three VXLs 1-3 that include point groups (hereinafter referred to as "effective VXLs"). As shown in FIG. 21, the octant tree structure is defined from nodes and leaves. Each node has a maximum of eight nodes or leaves. Each sheet has VXL information. Here, from the leaves shown in Fig. 21, Sheet 1, Sheet 2, and Sheet 3 represent VXL1, VXL2, and VXL3 shown in FIG. 20, respectively.

[0202] More specifically, each node and each leaf corresponds to a three-dimensional position. Node 1 corresponds to the entire block shown in FIG. 20. The block that corresponds to node 1 is divided into eight blocks. Of these eight blocks, blocks including effective VXLs are defined as nodes, while other blocks are defined as leaves. Each block that corresponds to a node is further divided into eight nodes or leaves. These processes are repeated a number of times, which is equal to the number of levels in the structure in the form of an octant tree. All blocks at the lowest level are set as leaves.

[0203] FIG. 22 is a diagram showing an exemplary SWLD formed from the WLD shown in FIG. 20. VXL1 and VXL2 shown in FIG. 20 as FVXL1 and FVXL2 as a result of feature extraction, and thus they are added to the SWLD. Meanwhile, VXL3 is not evaluated as FVXL and thus is not added to SWLD. Fig. 23 is a diagram showing the octant tree structure of the SWLD shown in FIG. 22. In the octant tree structure shown in FIG. 23 sheet 3 corresponding to the VXL3 shown in FIG. 21 is removed. Therefore, node 3 shown in FIG. 21 loses effective VXL and changes to sheet. As described above, the SWLD has fewer leaves in general than the WLD, and therefore the coded 3D data of the SWLD is smaller than the coded 3D data of the WLD.

[0204] Variations of the present embodiment are described next.

[0205] To estimate its own location, for example, a client, which is a device installed in a vehicle, etc., may receive a SWLD from a server to use such SWLD to estimate its own location. Meanwhile, in order to detect interference, the client can detect interference by using 3D peripheral information obtained by such client through various means including a distance sensor such as a rangefinder, as well as a stereo camera and a combination of a plurality of monocular cameras.

[0206] In general, SWLD is less likely to include VXL data relative to a flat area. Accordingly, the server may store the subsample world (subWLD) obtained by subsampling the WLD for static detection and send the SWLD and the subWLD to the client. This allows the client to perform its own location estimation and client-side interference detection while reducing network bandwidth.

[0207] When a client renders 3D map data at a high speed, map information having a mesh structure is more useful in some cases. In this regard, the server may form a mesh network from the WLD so as to store it in advance as a mesh world (MWLD). For example, when wanting to perform approximate 3D rendering, the client receives MWLD, and when wanting to perform detailed 3D rendering, the client receives WLD. This reduces the network bandwidth.

[0208] In the above description, the server sets, as FVXL, VXL having the number of features greater than or equal to the threshold value, but the server may calculate the FVXL through another method. For example, the server may evaluate that the VXL, VLM, SPC, or GOS that constitutes a traffic light or an intersection, etc., is needed to estimate its own location, driving assistance, or self-driving, etc. and includes such VXL, VLM, SPC or GOS in SWLD as FVXL, FVLM, FSPC or FGOS. This assessment can be done manually. In addition, FVXL, etc., which are set based on the number of features, can be added to FVXL, etc. obtained by the above-described method. In other words, the extraction SWLD module 403 can further extract, from the input 3D data 411, data corresponding to an object having a predetermined attribute as the extracted 3D data 412.

[0209] In addition, that a VXL, VLM, SPC, or GOS is required for such intended use may be noted separately from the features. The server can separately store, as the top layer of the SWLD (for example, the world of lanes), the FVXL of a traffic light or an intersection, etc., necessary for estimating its own location, driving assistance or unmanned driving, etc.

[0210] The server may also add an attribute in the VXL to the WLD based on random access or based on a predefined unit. The attribute, for example, includes information indicating whether or not VXLs are necessary for estimating one's own location, or information indicating whether or not VXLs are important as traffic information such as a traffic light or an intersection, etc. The attribute may also include the correspondence between the VXL and the features (intersection or road, etc.) in lane information (geographic data files (GDFs), etc.).

[0211] The method as described below may be used to update the WLD or SWLD.

[0212] Update information indicating changes, etc. person, road works, or treeline (for trucks) is uploaded to the server as pointgroups or metadata. The server updates the WLD based on such downloaded information and then updates the SWLD by using the updated WLD.

[0213] The client, upon detecting a discrepancy between the 3D information that such a client generates during its own location estimation, and the 3D information received from the server, may send the 3D information that such a client generates to the server along with an update notification. In such a case, the server updates the SWLD by using the WLD. When the SWLD is not to be updated, the server judges that the WLD itself is obsolete.

[0214] In the above description, information that distinguishes whether the codestream is a WLD or SWLD codestream is added as codestream header information. However, when a plurality of world types are provided, such as a mesh world and a lane world, information that distinguishes these world types may be added to the header information. In addition, when there are multiple SWLDs with different numbers of features, information that distinguishes the respective SWLDs may be added to the header information.

[0215] In the above description, the SWLD consists of FVXL, but the SWLD may include VXLs that have not been evaluated as FVXL. For example, the SWLD may include an adjacent VXL used to compute the FVXL feature. This allows the client to calculate the FVXL tag when receiving the SWLD, even if the tag information is not added to each FVXL SWLD. In such a case, the SWLD may include information that distinguishes whether each VXL is an FVXL or a VXL.

[0216] As described above, the 3D encoding apparatus 400 extracts, from the input 3D data 411 (first 3D data), the extracted 3D data 412 (second 3D data) having a number of features greater than or equal to a threshold value, and encodes the extracted 3D data 412 to generate encoded 3D data 414 (first encoded 3D data).

[0217] This 3D data encoding device 400 generates encoded 3D data 414, which is obtained by encoding data having a number of features greater than or equal to a threshold value. This reduces the amount of data compared to the case in which the input 3D data 411 is encoded as is. The 3D encoding device 400 thus allows a reduction in the amount of data to be transmitted.

[0218] The 3D encoding device 400 further encodes the input 3D data 411 to generate encoded 3D data 413 (second encoded 3D data).

[0219] This 3D encoding device 400 selectively transmits encoded 3D data 413 and encoded 3D data 414, according to, for example, appropriate usage, etc.

[0220] In addition, the extracted 3D data 412 is encoded by a first encoding method, and the input 3D data 411 is encoded by a second encoding method different from the first encoding method.

[0221] This 3D encoding device 400 ensures that an encoding method suitable for each of the input 3D data 411 and the extracted 3D data 412 is used.

[0222] In addition, from intra prediction and inter prediction, inter prediction is more preferably performed in the first coding method than in the second coding method.

[0223] This 3D encoder 400 enables inter-prediction to be more preferentially performed on extracted 3D data 412 in which adjacent data elements are highly likely to have low correlation.

[0224] In addition, the first encoding method and the second encoding method represent 3D positions differently. For example, the second encoding method represents 3D positions by octant tree, and the first encoding method represents 3D positions by 3D coordinates.

[0225] This 3D data encoding device 400 enables a more suitable method to be used to represent 3D positions of 3D data considering the difference in the number of included data items (VXL or FVXL numbers).

[0226] In addition, at least one of the encoded 3D data 413 and the encoded 3D data 414 includes an identifier indicating whether the encoded 3D data is encoded 3D data obtained by encoding the input 3D data 411, or the encoded 3D data obtained by encoding part of the input 3D data 411. In other words, such an identifier indicates whether the encoded 3D data is WLD encoded 3D data 413 or SWLD encoded 3D data 414 .

[0227] This allows the decoder to easily judge whether the received encoded 3D data is encoded 3D data 413 or encoded 3D data 414.

[0228] In addition, the 3D encoding device 400 encodes the extracted 3D data 412 in such a way that the encoded 3D data 414 has a smaller amount of data than the amount of data for the encoded 3D data 413.

[0229] This 3D encoder 400 allows the encoded 3D data 414 to have a smaller amount of data than the amount of data for the encoded 3D data 413.

[0230] In addition, the 3D encoding device 400 further extracts data corresponding to an object having a predefined attribute from the input 3D data 411 as extracted 3D data 412. The object having a predefined attribute, for example, is an object required for estimation of one's own location, driving assistance or autonomous driving, etc., or more specifically, a traffic light, an intersection, etc.

[0231] This 3D encoding device 400 is capable of generating encoded 3D data 414 that includes data required by the decoding device.

[0232] In addition, the 3D encoding device (server) 400 further sends, to the client, one of the encoded 3D data 413 and the encoded 3D data 414 according to the state of the client.

[0233] This 3D data encoding device 400 is capable of sending corresponding data according to the state of the client.

[0234] In addition, the client state includes one of the communication state (eg, network bandwidth) of the client and the client's motion rate.

[0235] In addition, the 3D encoding apparatus 400 further sends, to the client, one of the encoded 3D data 413 and the encoded 3D data 414 according to a request from the client.

[0236] This 3D encoding device 400 is capable of sending corresponding data according to a request from a client.

[0237] In addition, the 3D data decoding apparatus 500 according to the present embodiment decodes the encoded 3D data 413 or the encoded 3D data 414 generated by the 3D data encoding apparatus 400 described above.

[0238] In other words, the 3D decoding apparatus 500 decodes, by means of the first decoding method, the encoded 3D data 414 obtained by encoding the extracted 3D data 412 having a number of features greater than or equal to a threshold value, the extracted 3D data 412 being extracted from the input 3D data 411. The 3D data decoding apparatus 500 also decodes, by the second decoding method, the encoded 3D data 413 obtained by encoding the input 3D data 411, the second decoding method being different from the first decoding method.

[0239] This 3D data decoding apparatus 500 selectively receives encoded 3D data 414 obtained by encoding data having a number of features greater than or equal to a threshold value and encoded 3D data 413, according to, for example, appropriate use, etc. . The 3D data decoding apparatus 500 thus allows a reduction in the amount of data to be transmitted. Such a 3D decoder 500 further enables the use of a decoding method suitable for each of the input 3D data 411 and the extracted 3D data 412.

[0240] In addition, from intra prediction and inter prediction, inter prediction is more preferably performed in the first decoding method than in the second decoding method.

[0241] This 3D decoder 500 enables inter-prediction to be performed more favorably on extracted 3D data in which adjacent data items are highly likely to have low correlation.

[0242] In addition, the first decoding method and the second decoding method represent 3D positions differently. For example, the second decoding method represents 3D positions by octant tree, and the first decoding method represents 3D positions by 3D coordinates.

[0243] This 3D data decoding apparatus 500 allows a more appropriate method to be used to represent 3D positions of 3D data considering the difference in the number of included data items (VXL or FVXL numbers).

[0244] In addition, at least one of the encoded 3D data 413 and the encoded 3D data 414 includes an identifier indicating whether the encoded 3D data is encoded 3D data obtained by encoding the input 3D data 411 or the encoded 3D data obtained by encoding part of the input 3D data 411. The 3D decoder 500 refers to such an identifier when identifying between the encoded 3D data 413 and the encoded 3D data 414.

[0245] This 3D decoder 500 is capable of easily judging whether the received encoded 3D data is encoded 3D data 413 or encoded 3D data 414.

[0246] The 3D data decoder 500 further notifies the server regarding the status of the client (3D data decoder 500). The 3D decoder 500 receives one of the encoded 3D data 413 and the encoded 3D data 414 from the server, according to the state of the client.

[0247] This 3D data decoding apparatus 500 is capable of receiving corresponding data according to the state of the client.

[0248] In addition, the state of the client includes one of the communication state (eg, network bandwidth) of the client and the speed of the client.

[0249] The 3D decoder 500 further makes a request to the server for one of the encoded 3D data 413 and the encoded 3D data 414, and receives one of the encoded 3D data 413 and the encoded 3D data 414 from the server, according to the request.

[0250] This 3D data decoding apparatus 500 is capable of receiving corresponding data according to proper use.

[0251] Embodiment 3

The present embodiment describes a method for transmitting/receiving 3D data between vehicles. For example, 3D data is transmitted/received between the vehicle in question and a nearby vehicle.

[0252] FIG. 24 is a block diagram of a 3D data generating apparatus 620 according to the present embodiment. Such a 3D data generating apparatus 620, which is included in the vehicle in question, for example, combines the first 3D data 632 generated by the 3D data generating apparatus 620 with the received second 3D data 635, thereby generating a third 3D data 636 having a higher density.

[0253] Such a 3D data creation apparatus 620 includes a 3D data creation unit 621, a request range determination unit 622, a search unit 623, a receiver 624, a decoder 625, and a combine unit 626.

[0254] First, the 3D data generation module 621 creates the first 3D data 632 by using the sensor information 631 detected by the sensor included in the vehicle in question. Next, the query range determination module 622 determines a query range, which is a 3D space range for which there is insufficient data in the generated first 3D data 632.

[0255] Next, the search unit 623 searches for a nearby vehicle having 3D request range data, and sends request range information 633 indicating the request range to the found nearby vehicle 601 (S623). Then, the receiver 624 receives encoded 3D data 634, which is a request range encoded stream, from the nearby vehicle 601 (S624). It should be noted that the search module 623 can indiscriminately send requests to all vehicles included in the specified range to receive encoded 3D data 634 from the vehicle that responds to the request. The search module 623 can send a query not only to vehicles, but also to an object such as a traffic light and a traffic sign, and receive encoded 3D data 634 from the object.

[0256] Next, the decoder 625 decodes the received encoded 3D data 634, thereby obtaining the second 3D data 635. Next, the combiner 626 combines the first 3D data 632 with the second 3D data 635, thereby generating 3D data 636 having a higher density.

[0257] Next, the structure and operations of the 3D data transmission apparatus 640 according to the present embodiment will be described. Fig. 25 is a block diagram of a 3D data transmission apparatus 640.

[0258] The device 640 for transmitting three-dimensional data is included, for example, in the above-described nearby vehicle. The 3D communication device 640 processes the fifth 3D data 652 generated by the nearby vehicle into the sixth 3D data 654 requested by the vehicle in question, encodes the sixth 3D data 654 to generate encoded 3D data 634, and sends the encoded 3D data 634 to the vehicle in question. vehicle.

[0259] The 3D data transmission device 640 includes a 3D data creation module 641, a receiver 642, an extractor 643, an encoder 644, and a transmitter 645.

[0260] First, the 3D data generation module 641 creates the fifth 3D data 652 by using the sensor information 651 detected by the sensor included in the nearby vehicle. Next, receiver 642 receives request range information 633 from the vehicle in question.

[0261] Next, the extractor 643 extracts from the fifth 3D data 652 the 3D data of the request range indicated by the request range information 633, thereby processing the fifth 3D data 652 into the sixth 3D data 654. Next, the encoder 644 encodes the sixth 3D data 654, to generate encoded 3D data 643 that is an encoded stream. The transmitter 645 then sends the encoded 3D data 634 to the vehicle in question.

[0262] It should be noted that although an exemplary case is described here in which the vehicle in question includes the 3D data generation device 620 and the nearby vehicle includes the 3D data communication device 640, each of the vehicles may include functionality as a device 620 for creating 3D data; and a device 640 for transmitting 3D data.

[0263] Embodiment 4

The present embodiment describes operations performed in abnormal cases when own position estimation is performed based on a 3D map.

[0264] The 3D map is expected to find its wide application in unmanned vehicle driving and autonomous movement, etc. a mobile object such as a robot and a flying object (eg, a drone). An exemplary means for enabling such autonomous movement includes a method in which the mobile object moves in accordance with a 3D map when estimating its own position on the map (estimating its own position).

[0265] Self-location estimation is provided by comparing a 3D map with 3D information about the vicinity of the vehicle in question (hereinafter referred to as "self-detected 3D data") obtained by a sensor equipped in the vehicle in question, such as a range finder (e.g., lidar) and a stereo camera to estimate the location of the vehicle in question on a 3D map.

[0266] As with the HD map offered by HERE Technologies, for example, the 3D map may include not only a 3D point cloud, but also 2D map data such as information regarding road shapes and intersections, or information that changes in real time, such as information on traffic congestion and accidents. The 3D map includes a plurality of layers, such as 3D data, 2D data, and metadata layers that change in real time, from which the device can obtain or refer to only the required data.

[0267] The point cloud data may be SWLD as described above, or may include point group data that is different from key points. The transmission/reception of point cloud data is essentially performed in one or more random access units.

[0268] The method described below is used as a method for matching a 3D map with self-discovered 3D data. For example, the device compares the shapes of the point groups in each other's point clouds and determines that parts having a high degree of similarity between key points correspond to an identical position. When a 3D map is generated by the SWLD, the device also performs matching by comparing the key points that form the SWLD with the 3D key points extracted from the self-discovered 3D data.

[0269] Here, in order to provide a highly accurate estimate of one's own location, the following needs must be met: (A) a 3D map and self-located 3D data have already been obtained; and (B) their accuracies satisfy a predetermined requirement. However, one of (A) and (B) cannot be satisfied in abnormal cases such as the cases described below.

[0270] 1. The 3D map is unavailable for communication.

[0271] 2. The 3D map is not present, or the received 3D map is corrupted.

[0272] 3. The sensor of the subject vehicle is problematic, or the accuracy of the generated self-detected 3D data is insufficient due to bad weather.

[0273] The following describes operations in order to overcome such abnormal cases. The following description illustrates the exemplary case of a vehicle, but the method described below is applicable to mobile objects in general that allow autonomous movement, such as a robot and an unmanned vehicle.

[0274] Next, the structure of the 3D information processing apparatus and its operation according to the present embodiment, capable of overcoming abnormal cases regarding the 3D map or self-detected 3D data, will be described. Fig. 26 is a block diagram of an exemplary structure of a 3D information processing apparatus 700 according to the present embodiment.

[0275] The device 700 processing three-dimensional information is equipped, for example, in a mobile object such as a car. As shown in FIG. 26, the 3D information processing apparatus 700 includes a 3D map acquisition unit 701, a self-detected data acquisition unit 702, an abnormal case evaluation unit 703, an overcoming operation determination unit 704, and a function controller 705.

[0276] It should be noted that the 3D information processing apparatus 700 may include an unillustrated 2D or 1D sensor that detects a structural object or a mobile object near the vehicle in question, such as a 2D capable camera, and a sensor for 1D data using ultrasound or laser. The 3D information processing device 700 may also include a non-illustrated communication module that obtains a 3D map over a mobile communication network such as 4G and 5G, or via vehicle-to-vehicle or vehicle-to-road infrastructure communication.

[0277] The module 701 obtain three-dimensional maps receives a three-dimensional map 711 of the vicinity of the route. For example, the 3D map acquisition module 701 acquires the 3D map 711 over a mobile communication network, either through vehicle-to-vehicle or vehicle-to-road infrastructure communications.

[0278] Next, the self-detected data acquisition module 702 obtains the self-detected 3D data 712 based on the sensor information. For example, the self-detected data acquisition module 702 generates self-detected three-dimensional data 712 based on sensor information obtained by a sensor equipped in the vehicle in question.

[0279] Next, the abnormal case evaluation module 703 performs a predefined check on at least one of the received 3D map 711 and the self-detected 3D data 712 to detect an abnormal case. In other words, the abnormal case evaluation module 703 evaluates whether or not at least one of the received 3D map 711 and the self-detected 3D data 712 is abnormal.

[0280] When an abnormal case is detected, the overcoming operation determination module 704 determines an overcoming operation to overcome such an abnormal case. Then, the function controller 705 controls the operation of each of the processing units necessary to perform the override operation.

[0281] Meanwhile, when the abnormal case is not detected, the 3D information processing apparatus 700 terminates the process.

[0282] In addition, the 3D information processing apparatus 700 estimates the location of the vehicle equipped with the 3D information processing apparatus 700 using the 3D map 711 and the self-detected 3D data 712. Then, the 3D information processing apparatus 700 performs automatic operation of the vehicle by using the estimated location vehicle.

[0283] As described above, the 3D information processing apparatus 700 acquires, via a communication channel, map data (3D map 711) that includes first 3D position information. The first 3D position information includes, for example, a plurality of random access units each of which is an assembly of at least one subspace and is separately decodable, wherein at least one subspace has 3D coordinate information and serves as a unit , in which each of the plurality of random access units is encoded. The first 3D position information, for example, is data (SWLD) obtained by encoding key points each having a number of 3D features greater than or equal to a predetermined threshold value.

[0284] The 3D information processing apparatus 700 also generates the second 3D position information (self-detected 3D data 712) from the information detected by the sensor. The 3D information processing apparatus 700 then judges whether or not one of the first 3D position information and the second 3D position information is abnormal by executing, for one of the first 3D position information and the second 3D position information, a process of judging whether or not abnormal.

[0285] The 3D information processing apparatus 700 determines an overcoming operation to overcome an abnormality when one of the first 3D position information and the second 3D position information is evaluated as abnormal. The 3D information processing apparatus 700 then executes the control that is required in order to perform the bridging operation.

[0286] This structure enables the 3D information processing apparatus 700 to detect an abnormality with respect to one of the first 3D position information and the second 3D position information and perform a bridging operation therefor.

[0287] Embodiment 5

The present embodiment describes a method, etc. transmitting 3D data to the vehicle behind.

[0288] FIG. 27 is a block diagram of an exemplary structure of a 3D data generating apparatus 810 according to the present embodiment. Such a 3D data creation device 810 is provided in a vehicle, for example. The 3D data generating device 810 transmits and receives 3D data to/from an external cloud-based traffic monitoring system of a vehicle ahead or a vehicle behind, and creates and stores 3D data.

[0289] The 3D data creation device 810 includes a data receiver 811, a communication module 812, a reception controller 813, a format converter 814, a plurality of sensors 815, a 3D data creation module 816, a 3D data synthesizer 817, a 3D data storage device 818, a module 819 communication controller 820 transfer format converter 821 and transmitting device 822 data.

[0290] The data receiving device 811 receives 3D data 831 from the cloud-based traffic or vehicle ahead monitoring system. The 3D data 831 includes, for example, information regarding an area undetectable by the subject vehicle's sensors 815, such as point cloud, visible light video, depth information, sensor position information, and speed information.

[0291] The communication module 812 communicates with the cloud-based traffic monitoring system or the vehicle in front to transmit a data transmission request, etc. into a cloud-based traffic monitoring system or a vehicle ahead.

[0292] The receive controller 813 exchanges information, such as information regarding supported formats, with the communication partner via the communication module 812 to establish communication with the communication partner.

[0293] The format converter 814 applies format conversion, etc. to the 3D data 831 received by the data receiver 811 to generate the 3D data 832. The format converter 814 also decompresses or decodes the 3D data 831 when the 3D data 831 is compressed or encoded.

[0294] The plurality of sensors 815 is a group of sensors such as visible light cameras and infrared cameras that receive information outside the vehicle and generate sensor information 833. The sensor information 833, for example, is 3D data such as a point cloud (spot array data), while the sensors 815 are laser sensors such as lidars. It should be noted that one sensor may serve as multiple sensors 815.

[0295] Module 816 create three-dimensional data generates three-dimensional data 834 from information 833 sensors. The 3D data 834 includes, for example, information such as point cloud, visible light video, depth information, sensor position information, and speed information.

[0296] The 3D data synthesizer 817 synthesizes the 3D data 834 generated from the sensor information 833 of the vehicle in question with the 3D data 832 generated by the traffic or vehicle ahead monitoring cloud system, etc., thereby generating 3D data 835 space, which includes the space in front of the vehicle in front, not detected by the sensors 815 of the vehicle in question.

[0297] The 3D data storage device 818 stores generated 3D data 835 and so on.

[0298] The communication module 819 communicates with a cloud-based traffic monitoring system or a vehicle following to transmit a data transmission request, etc. into a cloud-based traffic monitoring system or a vehicle driving behind.

[0299] The transmission controller 820 exchanges information, such as information regarding supported formats, with the communication partner via the communication module 819 to establish communication with the communication partner. The transmission controller 820 also determines a transmission area, which is a space of 3D data to be transmitted, based on the 3D data generating information on the 3D data 832 generated by the 3D data synthesizer 817 and the data transmission request from the communication partner.

[0300] More specifically, the transmission controller 820 determines a transmission area that includes the area in front of the vehicle in question, undetectable by the rear vehicle sensor, in response to a data transmission request from the traffic monitoring cloud system or the rear vehicle. The transmission controller 820, for example, judges whether or not a space is transferable, or whether or not an already transmitted space includes an update, based on the 3D data generation information to determine the transmission area. For example, the transmission controller 820 determines, as a transmission area, an area that is: an area indicated by a data transmission request; and an area corresponding to which 3D data 835 is present. The transmission controller 820 then notifies the format converter 821 regarding the format supported by the communication partner and the transmission area.

[0301] From the 3D data 835 stored in the 3D data storage device 818, the format converter 821 converts the 3D transmission area data 836 into a format supported by the receiver end to generate 3D data 837. It should be noted that the format converter 821 can compress or encode 3D data 837 to reduce the amount of data.

[0302] The data transmission device 822 transmits the 3D data 837 to the traffic monitoring cloud system or the vehicle behind. Such 3D data 837 includes, for example, information regarding a dead zone, which is an area hidden from the view of a rearing vehicle, such as a point cloud in front of the vehicle in question, visible light video, depth information, and sensor position information.

[0303] It should be noted that an example has been described in which the format converter 814 and the format converter 821 perform format conversion, etc., but format conversion may not be performed.

[0304] With the above structure, the 3D data generating device 810 receives, from an external device, 3D data 831 of an area undetectable by the sensors 815 of the vehicle in question, and synthesizes the 3D data 831 with the 3D data 834 that is based on the sensor information 833 detected by sensors 815 of the vehicle in question, thereby generating 3D data 835. The 3D data generating apparatus 810 thereby allows generation of 3D data of a range not detected by the sensors 815 of the vehicle in question.

[0305] The 3D data generating apparatus 810 is also capable of transmitting, to the cloud-based traffic monitoring system or a rear vehicle, etc., 3D space data, which includes the space in front of the vehicle in question, undetectable by the rear vehicle sensor, in response to a request to transmit data from a cloud-based traffic monitoring system or a vehicle driving behind.

[0306] Embodiment 6

In Embodiment 5, an example is described in which a vehicle client device or the like. transfers 3D data to another vehicle or to a server, such as a cloud-based traffic monitoring system. In the present embodiment, the client device transmits the sensor information obtained through the sensor to the server or to the client device.

[0307] First, the structure of the system according to the present embodiment will be described. Fig. 28 is a diagram showing the structure of a 3D map and sensor information transmission/reception system according to the present embodiment. This system includes a server 901 and client devices 902A and 902B. It should be noted that client devices 902A and 902B are also referred to as "client device 902" when no specific distinction is made between them.

[0308] The client device 902, for example, is a vehicle-mounted device equipped in a mobile object, such as a vehicle. Server 901, for example, is a cloud-based traffic monitoring system and is capable of communicating with multiple client devices 902.

[0309] The server 901 transmits the 3D map generated by the point cloud to the client device 902. It should be noted that the structure of the 3D map is not limited to the point cloud, and may also be another structure expressing 3D data, such as a mesh structure.

[0310] The client device 902 transmits the sensor information obtained by the client device 902 to the server 901. The sensor information includes, for example, at least one of information obtained by lidar, visible light image, infrared image, image depth, sensor position information, or sensor speed information.

[0311] The data to be transmitted and received between the server 901 and the client device 902 may be compressed to reduce the amount of data, and may also be transmitted decompressed to maintain data accuracy. When compressing data, a 3D point cloud compression method can be used, for example, based on an octant tree structure. A method for compressing 2D images in a visible light image, an infrared image, and a depth image can be used. The 2D image compression method is, for example, MPEG-4 AVC or HEVC standardized by MPEG.

[0312] Server 901 transmits a 3D map managed by server 901 to client device 902 in response to a transmission request for a 3D map from client device 902. It should be noted that server 901 can also transmit a 3D map without waiting for a transmission request to a 3D map item from client device 902. For example, server 901 may broadcast a 3D map to at least one client device 902 located in a predetermined space. The server 901 may also transmit a 3D map suitable for the position of the client device 902 at fixed time intervals to the client device 902, which receives the transmission request once. The server 901 may also transmit the 3D map managed by the server 901 to the client device 902 each time the 3D map is updated.

[0313] The client device 902 sends a 3D map transmission request to the server 901. For example, when the client device 902 wants to perform its own location estimation while driving, the client device 902 sends a 3D map transmission request to the server 901.

[0314] It should be noted that in the following cases, the client device 902 may send a transmission request for a 3D map to the server 901. The client device 902 may send a transmission request for a 3D map to the server 901 when the 3D map stored by the client device 902 is obsolete. For example, client device 902 may send a transmission request for a 3D map to server 901 when a fixed period has elapsed since the 3D map is received by client device 902.

[0315] The client device 902 may also send a transmission request for a 3D map to the server 901 until a fixed time when the client device 902 leaves the space shown on the 3D map stored by the client device 902. For example, the client device 902 may send a request to transmit a 3D map to the server 901 when the client device 902 is located within a predetermined distance from the boundary of the space shown on the 3D map stored by the client device 902. When the travel path and travel speed of the client device 902 are understood, the time when the client device 902 leaving the space shown on the 3D map stored by client device 902 can be predicted based on the travel path and travel speed of client device 902.

[0316] The client device 902 can also send a transmission request for a 3D map to the server 901 when the error during alignment of the 3D data and the 3D map generated from the sensor information by the client device 902 is at least at a fixed level.

[0317] Client device 902 transmits sensor information to server 901 in response to a request to transmit sensor information from server 901. It should be noted that client device 902 may transmit sensor information to server 901 without waiting for a request to transmit sensor information from server 901. server 901. For example, client device 902 may periodically transmit sensor information for a fixed period when client device 902 receives a request to transmit for sensor information from server 901 once. zone of the client device 902, and transmit this information and the sensor information to the server 901 when an error during the alignment of the 3D data generated by the client device 902 based on the sensor information and the 3D map obtained from the server 901 is at least but a fixed level.

[0318] Server 901 sends a transmission request for sensor information to client device 902. For example, server 901 receives position information such as GPS information relative to client device 902 from client device 902. Server 901 sends a transmission request for information sensors to the client device 902 to generate a new 3D map when it is determined that the client device 902 is approaching a space in which the 3D map managed by the server 901 contains little information based on position information relative to the client device 902. The server 901 also can send a transmission request for sensor information, optionally (i) update the 3D map, (ii) check road conditions during snowfall, accident, etc., or (iii) check traffic jam conditions, accident/accident conditions and so on.

[0319] The client device 902 can set the data amount of sensor information to be transmitted to the server 901 in accordance with the communication conditions or bandwidth in the course of receiving a transmission request for sensor information to be received from the server 901. Setting the amount of information data sensors, which should be transmitted to the server 901, for example, is an increase/decrease in the actual data or a proper choice of compression method.

[0320] FIG. 29 is a block diagram showing an exemplary structure of the client device 902. The client device 902 receives a 3D map generated by a point cloud or the like. from the server 901, and estimates the client device 902's own location using a 3D map generated from the client device 902's sensor information. The client device 902 transmits the received sensor information to the server 901.

[0321] The client device 902 includes a data receiver 1011, a communication module 1012, a reception controller 1013, a format converter 1014, sensors 1015, a 3D data generation module 1016, a 3D image processor 1017, a 3D data storage device 1018, a format converter 1019, a communication module 1020, a transmission controller 1021, and a data transmission device 1022.

[0322] The data receiver 1011 receives a 3D map 1031 from the server 901. The 3D map 1031 is data that includes a point cloud such as a WLD or SWLD. The 3D map 1031 may include compressed data or uncompressed data.

[0323] The communication module 1012 communicates with the server 901 and transmits a data transmission request (for example, a transmission request for a 3D map) to the server 901.

[0324] The receiving controller 1013 exchanges information, such as information regarding supported formats, with a communication partner via the communication module 1012 to establish communication with the communication partner.

[0325] The format converter 1014 performs format conversion and the like. for the 3D map 1031 received by the data receiver 1011 to generate the 3D map 1032. The format converter 1014 also performs a decompressing or decoding process when the 3D map 1031 is compressed or encoded. It should be noted that the format converter 1014 does not perform a decompressing or decoding process when the 3D map 1031 is uncompressed data.

[0326] Sensors 815 are a group of sensors, such as lidars, visible light cameras, infrared cameras, or depth sensors, that acquire location information outside of a vehicle equipped with client device 902 and generate information 1033 sensors. The sensor information 1033, for example, is 3D data such as a point cloud (point group data), while the sensors 1015 are laser sensors such as lidars. It should be noted that one sensor can serve as sensors 1015.

[0327] Module 1016 create three-dimensional data generates three-dimensional data 1034 of the surrounding area of the vehicle in question based on information 1033 sensors. For example, the 3D data generation module 1016 generates color information point cloud data about the surrounding area of the vehicle in question using information obtained by lidar and visible light video obtained by a visible light camera.

[0328] The 3D image processor 1017 performs a self-location estimation process, and the like. of the vehicle in question using (i) a received 3D map 1032, such as a point cloud, and (ii) 3D data 1034 of the surrounding area of the vehicle in question generated using the sensor information 1033. It should be noted that the 3D image processor 1017 may generate 3D data 1035 regarding the vicinity of the vehicle in question by combining the 3D map 1032 and 3D data 1034, and may perform a self-location estimation process using the generated 3D data 1035.

[0329] The 3D data storage device 1018 stores the 3D map 1032, the 3D data 1034, the 3D data 1035, and the like.

[0330] The format converter 1019 generates the sensor information 1037 by converting the sensor information 1033 into a format supported at the receiver end. It should be noted that the format converter 1019 may reduce the amount of data by compressing or encoding the sensor information 1037 . The format converter 1019 may omit this process when format conversion is not required. The format converter 1019 can also control the amount of data to be transmitted in accordance with the specified transmission range.

[0331] The communication unit 1020 communicates with the server 901 and receives a data transmission request (transmission request for sensor information) and the like. from server 901.

[0332] The transmission controller 1021 exchanges information, such as information regarding supported formats, with the communication partner via the communication module 1020 to establish communication with the communication partner.

[0333] The transmitting device 1022 transmits the sensor information 1037 to the server 901. The sensor information 1037 includes, for example, information obtained through the sensors 1015, such as information obtained by lidar, a brightness image obtained by a camera for shooting in the visible range light, an infrared image obtained by an infrared camera, a depth image obtained by a depth sensor, sensor position information, and sensor speed information.

[0334] The structure of the server 901 will now be described. FIG. 30 is a block diagram showing an exemplary structure of the server 901. The server 901 transmits sensor information from the client device 902 and generates 3D data based on the received sensor information. The server 901 updates the 3D map managed by the server 901 using the generated 3D data. The server 901 sends the updated 3D map to the client device 902 in response to a send request for a 3D map from the client device 902.

[0335] The server 901 includes a data receiving device 1111, a communication module 1112, a receiving controller 1113, a format converter 1114, a 3D data creation module 1116, a 3D data combining module 1117, a 3D data storage device 1118, a format converter 1119, a communication module 1120 , a transmission controller 1121, and a data transmission device 1122.

[0336] The data receiver 1111 receives sensor information 1037 from the client device 902. The sensor information 1037 includes, for example, information obtained by lidar, a brightness image obtained by a visible light camera, an infrared image obtained by a camera for shooting in the infrared range, the depth image acquired by the depth sensor, the position information of the sensors, the speed information of the sensors, and the like.

[0337] The communication module 1112 communicates with the client device 902 and transmits a data transmission request (eg, a transmission request for sensor information) and the like. to the client device 902.

[0338] The receive controller 1113 exchanges information, such as information regarding supported formats, with a communication partner via a communication module 1112 to establish communication with the communication partner.

[0339] The format converter 1114 generates sensor information 1132 by performing a decompression or decoding process when the received sensor information 1037 is compressed or encoded. It should be noted that the format converter 1114 does not perform the decompressing or decoding process when the sensor information 1037 is uncompressed data.

[0340] Module 1116 create three-dimensional data generates three-dimensional data 1134 of the surrounding area of the client device 902 based on information 1132 sensors. For example, the 3D data generation module 1116 generates point cloud data with color information relative to the surrounding area of the client device 902 using information obtained by lidar and visible light video obtained by a visible light camera.

[0341] The 3D data merging module 1117 updates the 3D map 1135 by combining the 3D data 1134 generated from the sensor information 1132 with the 3D map 1135 managed by the server 901.

[0342] The 3D data storage device 1118 stores the 3D map 1135 or the like.

[0343] The format converter 1119 generates a 3D map 1031 by converting the 3D map 1135 to a format supported at the receiver end. It should be noted that the format converter 1119 may reduce the amount of data by compressing or encoding the 3D map 1135. The format converter 1119 may omit this process when format conversion is not required. The format converter 1119 can also control the amount of data to be transmitted in accordance with the specified transmission range.

[0344] The communication unit 1120 communicates with the client device 902 and receives a data transmission request (transmission request for a 3D map) or the like. from the client device 902.

[0345] The transmission controller 1121 exchanges information, such as information regarding supported formats, with the communication partner via the communication module 1120 to establish communication with the communication partner.

[0346] The data transmission device 1122 transmits the 3D map 1031 to the client device 902. The 3D map 1031 is data that includes a point cloud such as a WLD or SWLD. The 3D map 1031 may include one of compressed data and uncompressed data.

[0347] The following describes the flow of operations of the client device 902. FIG. 31 is a flowchart for an operation when a client device 902 acquires a 3D map.

[0348] The client device 902 first requests the server 901 to transmit a three-dimensional map (point cloud, etc.) (S1001). At this point, by also transmitting position information regarding the client device 902 obtained via GPS or the like, the client device 902 can also request the server 901 to transmit the 3D map associated with this position information.

[0349] The client device 902 then receives the 3D map from the server 901 (S1002). When the received 3D map is compressed data, the client device 902 decodes the received 3D map and generates an uncompressed 3D map (S1003).

[0350] The client device 902 then creates three-dimensional data 1034 of the surrounding area of the client device 902 using the sensor information 1033 obtained by the sensors 1015 (S1004). The client device 902 then estimates the client device 902's own location using the 3D map 1032 received from the server 901 and the 3D data 1034 generated using the sensor information 1033 (S1005).

[0351] FIG. 32 is a flowchart for an operation when the client device 902 transmits sensor information. The client device 902 first receives a transmission request for sensor information from the server 901 (S1011). The client device 902 that receives the transfer request transmits the sensor information 1037 to the server 901 (S1012). It should be noted that the client device 902 can generate sensor information 1037 by compressing each piece of information using a compression method suitable for each piece of information when the sensor information 1033 includes a plurality of pieces of information obtained by the sensors 1015.

[0352] The following describes the sequence of operations of the server 901. FIG. 33 is a flowchart for an operation when the server 901 obtains sensor information. The server 901 first requests the client device 902 to transmit sensor information (S1021). The server 901 then receives the sensor information 1037 transmitted from the client device 902 in accordance with the request (S1022). Server 901 then generates 3D data 1134 using the received sensor information 1037 (S1023). The server 901 then displays the generated 3D data 1134 on the 3D map 1135 (S1024).

[0353] FIG. 34 is a flowchart for an operation when the server 901 transmits a 3D map. The server 901 first receives a transmission request for a 3D map from the client device 902 (S1031). The server 901, which receives the transmission request for the 3D map, transmits the 3D map to the client device 902 (S1032). At this point, the server 901 may extract a 3D map of the vicinity of the client device 902 along with position information relative to the client device 902, and transmit the extracted 3D map. The server 901 may compress the 3D map generated by the point cloud using, for example, an octant tree structure compression method, and transmit the compressed 3D map.

[0354] Hereinafter, variations of the present embodiment are described in this document.

[0355] The server 901 creates 3D data 1134 around the position of the client device 902 using the sensor information 1037 received from the client device 902. The server 901 then calculates the difference between the 3D data 1134 and the 3D map 1135 by comparing the generated 3D data 1134 with the identical 3D map 1135 zone managed by the server 901. The server 901 determines that a certain type of anomaly occurs in the surrounding zone of the client device 902 when the difference is greater than or equal to a predetermined threshold. For example, there is a possibility that a large difference occurs between the 3D map 1135 managed by the server 901 and the 3D data 1134 generated based on the sensor information 1037 when subsidence or the like. occurs as a result of a natural disaster such as an earthquake.

[0356] Sensor information 1037 may include information indicative of at least one of sensor type, sensor performance, and sensor model number. The sensor information 1037 may also be appended with a class ID and the like. according to the performance of the sensor. For example, when sensor information 1037 is obtained by lidar, it is possible to assign identifiers for sensor performance. A sensor capable of receiving information with an accuracy of a few millimeters has a class 1, a sensor capable of receiving information with an accuracy of a few centimeters has a class 2, and a sensor capable of receiving information with an accuracy of a few meters has a class 3. Server 901 may evaluate sensor performance information and the like. from the model number of the client device 902. For example, when the client device 902 is equipped in a vehicle, the server 901 may determine sensor specification information from the vehicle type. In this case, the server 901 may obtain information regarding the vehicle type in advance, and the information may also be included in the sensor information. The server 901 may change the degree of correction with respect to the 3D data 1134 created using the sensor information 1037 using the obtained sensor information 1037 . For example, when the sensor performance is high in accuracy (class 1), the server 901 does not correct the 3D data 1134. When the sensor performance is low in accuracy (class 3), the server 901 corrects the 3D data 1134 according to the accuracy of the sensor. For example, the server 901 increases the degree (intensity) of the correction with decreasing sensor accuracy.

[0357] The server 901 may simultaneously send a transmission request for sensor information to a plurality of client devices 902 in a certain space. The server 901 does not need to use all of the sensor information to generate the 3D data 1134, and for example, may select the sensor information to be used according to sensor performance when receiving multiple pieces of sensor information from multiple client devices 902. For example, when updating the 3D map 1135 , the server 901 may select high-precision sensor information (class 1) from among the received plurality of sensor information pieces and create 3D data 1134 using the selected sensor information.

[0358] The server 901 is not limited to being a server such as a cloud-based traffic monitoring system, and may also be another (vehicle-mounted) client device. Fig. 35 is a system structure diagram in this case.

[0359] For example, client device 902C sends a transmission request for sensor information to client device 902A located nearby and receives sensor information from client device 902A. The client device 902C then creates 3D data using the acquired sensor information of the client device 902A and updates the 3D map of the client device 902C. This allows the client device 902C to generate a three-dimensional map of the space that can be obtained from the client device 902A and fully exploit the performance of the client device 902C. For example, such a case is possible when the client device 902C has high performance.

[0360] In this case, the client device 902A, which provides the sensor information, is given the right to receive a high-precision three-dimensional map generated by the client device 902C. Client device 902A receives a high-precision 3D map from client device 902C in accordance with these rights.

[0361] Server 901 may send a transmission request for sensor information to a plurality of client devices 902 (client device 902A and client device 902B) located in the vicinity of client device 902C. When the sensor of the client device 902A or the client device 902B has a high performance, the client device 902C is capable of generating 3D data using the sensor information acquired by that high performance sensor.

[0362] FIG. 36 is a block diagram showing the functionality structure of the server 901 and the client device 902. The server 901 includes, for example, a 3D map compression/decoding processor 1201 that compresses and decodes a 3D map, and a sensor information compression/decoding processor 1202 that compresses and decodes the sensor information.

[0363] The client device 902 includes a 3D map decoding processor 1211 and a sensor information compression processor 1212. The 3D map decoding processor 1211 receives the compressed 3D map encoded data, decodes the encoded data, and obtains the 3D map. The sensor information compression processor 1212 compresses the sensor information itself instead of the 3D data created using the obtained sensor information, and transmits the compressed sensor information encoded data to the server 901. 3D map data (point clouds, etc.), provided that the client device 902 internally stores a processor that executes a process for decoding the 3D map (point clouds, etc.). This allows you to limit costs, power consumption, etc. client device 902.

[0364] As mentioned above, the client device 902 according to the present embodiment is equipped in the mobile object and generates 3D environment data 1034 of the mobile object using the sensor information 1033 that is obtained through the sensor 1015 equipped in the mobile object and indicates the environment condition of the mobile object . The client device 902 estimates the mobile object's own location using the generated 3D data 1034. The client device 902 transmits the received sensor information 1033 to the server 901 or other mobile object.

[0365] This allows the client device 902 to transmit sensor information 1033 to the server 901 or the like. This makes it possible to further reduce the amount of transmitted data compared to the time of transmission of 3D data. Since it is not necessary for the client device 902 to perform processes such as compressing or encoding 3D data, the amount of processing of the client device 902 can be reduced. In this regard, the client device 902 can reduce the amount of data to be transmitted or simplify the structure of the device.

[0366] Client device 902 further transmits a 3D map transmission request to server 901 and receives 3D map 1031 from server 901. When estimating its own location, client device 902 estimates its own location using 3D data 1034 and 3D map 1032.

[0367] The sensor information 1034 includes at least one of information obtained by the laser sensor, brightness image, infrared image, depth image, sensor position information, or sensor speed information.

[0368] The sensor information 1033 includes information that indicates sensor performance.

[0369] The client device 902 encodes or compresses the sensor information 1033 and, when transmitting the sensor information, transmits the sensor information 1037 that is encoded or compressed to the server 901 or to another mobile entity 902. This allows the client device 902 to reduce the amount of data that is must be transmitted.

[0370] For example, client device 902 includes a processor and a storage device. The processor performs the above processes using the storage device.

[0371] The server 901 according to the present embodiment is capable of communicating with the client device 902 equipped in the mobile object, and receives sensor information 1037 that is obtained through the sensor 1015 equipped in the mobile object and indicates the environment condition of the mobile object. Server 901 creates three-dimensional data 1134 of the surrounding area of the mobile object using the received information 1037 sensors.

[0372] As a consequence, the server 901 generates 3D data 1134 using the sensor information 1037 transmitted from the client device 902. This can further reduce the amount of transmitted data compared to when the client device 902 transmits 3D data. Since it is not necessary for the client device 902 to perform processes such as compressing or encoding 3D data, the amount of processing of the client device 902 can be reduced. In this regard, the server 901 can reduce the amount of data to be transmitted or simplify the structure of the device.

[0373] The server 901 further transmits a transmission request for sensor information to the client device 902.

[0374] The server 901 further updates the 3D map 1135 with the generated 3D data 1134 and transmits the 3D map 1135 to the client device 902 in response to a send request for the 3D map 1135 from the client device 902.

[0375] The sensor information 1037 includes at least one of information obtained by the laser sensor, brightness image, infrared image, depth image, sensor position information, or sensor speed information.

[0376] Sensor information 1037 includes information that indicates sensor performance.

[0377] The server 901 further corrects the 3D data according to sensor performance. This allows the 3D data creation method to improve the quality of the 3D data.

[0378] When receiving sensor information, the server 901 receives a plurality of pieces of sensor information 1037 received from a plurality of client devices 902, and selects the sensor information 1037 to be used in generating the 3D data 1134 based on the plurality of pieces of information each indicating performance a sensor included in a plurality of sensor information pieces 1037 . This allows the server 901 to improve the quality of the 3D data 1134.

[0379] The server 901 decodes or decompresses the received sensor information 1037 and creates three-dimensional data 1134 using the sensor information 1132 that is decoded or decompressed. This allows the server 901 to reduce the amount of data that must be transmitted.

[0380] For example, server 901 includes a processor and a storage device. The processor performs the above processes using the storage device.

[0381] Embodiment 7

In the present embodiment, methods for encoding and decoding 3D data using an inter-prediction process are described below.

[0382] FIG. 37 is a block diagram of a 3D data encoding device 1300 according to the present embodiment. This 3D encoding device 1300 generates an encoded bitstream (hereinafter, also simply referred to as "bitstream"), which is an encoded signal, by encoding the 3D data. As illustrated in FIG. 37, the 3D data encoding device 1300 includes a splitter 1301, a subtractor 1302, a transform 1303, a quantizer 1304, an inverse quantizer 1305, an inverse transform 1306, an adder 1307, a support volume memory 1308, an intra prediction module 1309, a support space memory 1310. , an inter prediction module 1311, a prediction controller 1312, and an entropy encoder 1313.

[0383] The separation module 1301 separates a plurality of volumes (VLMs), which are coding units of each space (SPC) included in the 3D data. The splitter 1301 defines an octant tree representation (setting as an octant tree) of the voxels in each volume. It should be noted that the division module 1301 may define the spaces as an octant tree representation with spaces having the same size as the volumes. The splitter 1301 may also append information (depth information, etc.) necessary to define an octant tree representation to a title or the like. bit stream.

[0384] The subtractor 1302 calculates the difference between the amount (target encoding amount) outputted by the splitter 1301 and the predicted amount generated through intra prediction or inter prediction, which will be described below, and outputs the calculated difference to the converter 1303 as a prediction residual. Fig. 38 is a diagram showing an exemplary prediction residual calculation. It should be noted that the bit sequences of the target coding volume and the predicted volume shown here, for example, are position information indicating the positions of the 3D points included in the volumes.

[0385] Hereinafter, this document describes how to scan a tree representation of octants and voxels. The volume is encoded after transformation into a structure in the form of an octant tree (setting as an octant tree). The octant tree structure includes nodes and leaves. Each node has eight nodes or leaves, and each leaf has voxel (VXL) information. Fig. 39 is a diagram showing an exemplary volume structure including voxels. Fig. 40 is a diagram showing an example of the volume shown in FIG. 39 converted to an octant tree structure. Among the leaves shown in Fig. 40, leaves 1, 2, and 3, respectively, represent VXL 1, VXL 2, and VXL 3, and represent VXLs including a dot group (hereinafter, active VXLs).

[0386] The octant tree is represented, for example, by binary sequences of ones and zeros. For example, by using 1 for nodes or active VXLs and 0 for everything else, each node and leaf is assigned the binary sequence shown in FIG. 40. Thus, this binary sequence is scanned according to the width or depth scan order. For example, when scanning in breadth, the binary sequence shown in A of FIG. 41. When scanning in depth, the binary sequence shown in B in FIG. 41. The binary sequences obtained through this scan are encoded using entropy coding, which reduces the amount of information.

[0387] The following describes depth information in an octant tree representation. The depth in the octant tree view is used to control how much detail the point cloud information included in the volume is retained. After setting a large depth, you can reproduce point cloud information to a more accurate level, but the amount of data to represent nodes and leaves increases. However, after setting a small depth, the amount of data is reduced, but some information, such as point cloud information, originally stored is lost because point cloud information pieces including different positions and different colors are now considered as cloud information pieces. dots that include the same position and the same color.

[0388] For example, FIG. 42 is a diagram showing an example in which the octant tree with a depth of 2 shown in FIG. 40 is represented with a depth of 1. The octant tree shown in FIG. 42 has a lower amount of data than the octant tree shown in FIG. 40. In other words, the binarized octant tree shown in FIG. 42 has fewer bits than the octant tree shown in FIG. 40. Sheet 1 and sheet 2 shown in FIG. 40 are represented by the sheet 1 shown in FIG. 41. In other words, information about sheet 1 and sheet 2 in different positions is lost.

[0389] FIG. 43 is a diagram showing a volume corresponding to the octant tree shown in FIG. 42. VXL 1 and VXL 2 shown in FIG. 39 correspond to the VXL 12 shown in FIG. 43. In this case, the 3D data encoding apparatus 1300 generates the color information of the VXL 12 shown in FIG. 43 using the color information of VXL 1 and VXL 2 shown in FIG. 39. For example, the 3D data encoder 1300 calculates the mean, median, weighted average, and the like. the color information of VXL 1 and VXL 2 as the color information of VXL 12. Thus, the 3D encoder 1300 can control data reduction by changing the depth of the octant tree.

[0390] The 3D data encoding device 1300 may specify octant tree depth information in units of worlds, units of spaces, or units of volumes. In this case, the 3D encoding apparatus 1300 may append depth information to world header information, space header information, or volume header information. In all worlds, spaces and volumes associated with different times, the same value can be used as depth information. In this case, the 3D encoder 1300 may append depth information to header information governing the worlds associated with all times.

[0391] When the color information is included in voxels, the transformer 1303 applies a frequency transform, such as an orthogonal transform, to the prediction residual of the color information of the voxels in the volume. For example, the transformer 1303 creates a one-dimensional array by scanning the prediction residual in a specific scan order. Next, the converter 1303 transforms the one-dimensional array into the frequency domain by applying a one-dimensional orthogonal transform to the created one-dimensional array. Because of this, when the volume prediction residual value is similar, the value of the low frequency component increases and the value of the high frequency component decreases. In this regard, it is possible to more effectively reduce the amount of coding in the quantizer 1304.

[0392] The transformer 1303 need not necessarily use an orthogonal transform in one dimension, but may also use an orthogonal transform in two or more dimensions. For example, the transformer 1303 transforms the prediction residual into a two-dimensional array in a specific scan order and applies a two-dimensional orthogonal transformation to the resulting two-dimensional array. The transformer 1303 may select an orthogonal transform method to be used from a plurality of orthogonal transform methods. In this case, the 3D data encoding apparatus 1300 appends, to the bitstream, information indicating which orthogonal transformation method is used. Transformer 1303 can select an orthogonal transform method to be used from a plurality of orthogonal transform methods in different dimensions. In this case, the 3D data encoding apparatus 1300 appends, to the bitstream, in how many dimensions the orthogonal transform method is used.

[0393] For example, the transformer 1303 matches the scan order of the prediction residual with the scan order (width, depth, etc.) in the octant tree in the volume. This makes it possible to reduce the amount of overhead since the information indicating the scanning order of the prediction residual does not have to be appended to the bit stream. Transformer 1303 may apply a different scan order than the octant tree scan order. In this case, the 3D data encoding apparatus 1300 appends, to the bit stream, information indicating the scanning order of the prediction residual. This enables the 3D encoding device 1300 to efficiently encode the prediction residual. The 3D encoding device 1300 may append, to the bitstream, information (flag, etc.) indicating whether or not to apply the scan order of the octant tree, and may also append, to the bitstream, information indicating the scanning order of the prediction residual. when the octant tree scan order does not apply.

[0394] The transform 1303 not only transforms the color information prediction residual, but can also transform other attribute information included in voxels. For example, the transducer 1303 may transform and encode information such as reflectance information obtained when receiving a point cloud via lidar, and the like.

[0395] Transformer 1303 may skip these processes when the spaces do not include attribute information such as color information. The 3D encoder 1300 may append, to the bitstream, information (flag) indicating whether or not to skip the processes of the converter 1303.

[0396] The quantizer 1304 generates a quantized coefficient by performing quantization using a quantization control parameter for the frequency component of the prediction residual generated by the transformer 1303. As a result, the amount of information is further reduced. The generated quantized coefficient is output to entropy encoder 1313. Quantizer 1304 may control the quantization control parameter in world units, space units, or volume units. In this case, the 3D data encoding apparatus 1300 appends a quantization control parameter to each header information and the like. The quantizer 1304 may perform quantization control by changing the weighting factor based on the frequency component of the prediction residual. For example, quantizer 1304 can accurately quantize the low frequency component and approximately quantize the high frequency component. In this case, the 3D data encoder 1300 may append, to the header, a parameter expressing the weighting factor of each frequency component.

[0397] The quantizer 1304 may skip these processes when the spaces do not include attribute information such as color information. The 3D encoder 1300 may append, to the bit stream, information (flag) indicating whether or not to skip the quantizer 1304 processes.

[0398] The inverse quantizer 1305 generates an inversely quantized prediction residual coefficient by performing inverse quantization on the quantized coefficient generated by the quantizer 1304 using a quantization control parameter, and outputs the generated inversely quantized coefficient to the inverse transform 1306.

[0399] The inverse transform 1306 generates a prediction residual after applying the inverse transform by applying an inverse transform to the inverse quantized coefficient generated by the inverse quantizer 1305. This prediction residual after applying the inverse transform need not be exactly the same as the prediction residual outputted by the transform 1303 because the prediction residual after applying the inverse transform is the prediction residual that is generated after quantization.

[0400] The adder 1307 sums, to form the recovered volume (i), the prediction residual after applying the inverse transform, generated by the inverse transform 1306, with (ii) the predicted volume, which is generated through intra prediction or intra prediction, which is described below and is used for in order to generate a preliminary quantized prediction residual. This reconstructed volume is stored in the reference volume memory 1308 or the reference space memory 1310 .

[0401] The intra prediction unit 1309 generates a predicted volume of the target coding volume using neighboring volume attribute information stored in the reference volume memory 1308 . The attribute information includes color information or voxel reflectance. An intra prediction unit 1309 generates a predicted color information value or a reflection coefficient of the target encoding amount.

[0402] FIG. 44 is a diagram for describing the operation of the intra prediction unit 1309. For example, the intra prediction unit 1309 generates a predicted amount of the target encoding amount (volume idx=3) shown in FIG. 44 using an adjacent volume (volume idx=0). The Idx of the volume here is identifier information that is attached to the volume in the space, and a different value is assigned to each volume. The order in which the volume idx is assigned may be identical to the encoding order, and may also be different from the encoding order. For example, the intra prediction unit 1309 uses the average color information value of the voxels included in the volume idx=0, which is the adjacent volume, as the predicted color information value of the target encoding volume shown in FIG. 44. In this case, the prediction residual is generated by subtracting the predicted color information value from the color information of each voxel included in the target encoding volume. The following processes are performed by converter 1303 and subsequent processors with respect to this prediction residual. In this case, the 3D data encoding apparatus 1300 appends, to the bitstream, neighboring volume information and prediction mode information. The neighboring volume information here is information indicating the neighboring volume used in prediction, and indicates, for example, the volume idx of the neighboring volume used in prediction. The prediction mode information here indicates the mode used to generate the predicted volume. The mode, for example, is an average mode in which a predicted value is generated using the average value of voxels in an adjacent volume, or a median mode in which a predicted value is generated using a median of voxels in an adjacent volume.

[0403] The intra prediction module 1309 may generate a predicted volume using a plurality of adjacent volumes. For example, in the structure shown in FIG. 44, the intra prediction module 1309 generates a predicted volume of 0 using the volume with volume idx=0, and generates a predicted volume of 1 using the volume with volume idx=1. The intra prediction module 1309 then generates the average of the forecast volume 0 and the forecast volume 1 as the final forecast volume. In this case, the 3D data encoder 1300 may append, to the bitstream, a plurality of volumes idx for the plurality of volumes used to generate the predicted volume.

[0404] FIG. 45 is a diagram principally showing an inter-prediction process according to the present embodiment. An inter-prediction unit 1311 encodes (inter-predicts) a space (SPC) associated with a certain time T_Cur using a coded space associated with another time T_LX. In this case, the inter-prediction unit 1311 performs an encoding process by applying a spatial rotation and translation process to a coded space associated with a different time T_LX.

[0405] The 3D data encoder 1300 appends, to the bit stream, RT information associated with a rotation and movement process in a space suitable for a space associated with another time T_LX. Another time T_LX, for example, is a time T_L0 before a certain time T_Cur. At this point, the 3D data encoding apparatus 1300 may append, to the bit stream, RT information RT_L0 associated with a process of rotation and movement in space suitable for the space associated with time T_L0.

[0406] Alternatively, another time T_LX, for example, is a time T_L1 after a certain time T_Cur. At this point, the 3D data encoding apparatus 1300 may append, to the bit stream, RT information RT_L1 associated with a rotation and movement process in a space suitable for the space associated with time T_L1.

[0407] Alternatively, the inter-prediction module 1311 encodes (bidirectional prediction) with reference to spaces associated with time T_L0 and time T_L1 that are different from each other. In this case, the 3D encoding apparatus 1300 may append, to the bit stream, both RT information RT_L0 and RT information RT_L1 associated with a rotation and movement process in a space suitable for their respective spaces.

[0408] It should be noted that T_L0 is described as being before T_Cur and T_L1 as being before, but not necessarily limited to, T_Cur. For example, both of T_L0 and T_L1 may be before T_Cur. Also, both of T_L0 and T_L1 may be after T_Cur.

[0409] The 3D data encoding apparatus 1300 may append, to the bitstream, RT information associated with a rotation and movement process in space suitable for spaces associated with different times, when encoding with reference to each of the spaces. For example, the 3D encoder 1300 manages the set of code spaces to be accessed using two reference lists (list L0 and list L1). When the first reference space in the L0 list is L0R0, the second reference space in the L0 list is L0R1, the first reference space in the L1 list is L1R0, and the second reference space in the L1 list is L1R1, the 3D encoding apparatus 1300 appends, to bitstream, RT information RT_L0R0 L0R0, RT information RT_L0R1 L0R1, RT information RT_L1R0 L1R0 and RT information RT_L1R1 L1R1. For example, the 3D encoder 1300 appends these pieces of RT information to a header and the like. bit stream.

[0410] The 3D data encoding device 1300 determines whether or not to apply rotation and translation in space per reference space when encoding with reference to reference spaces associated with different times. In this case, the 3D encoder 1300 may append to header information or the like. bit stream, information (RT flag, etc.) indicating whether or not rotation and movement in space is applied per reference space. For example, the 3D encoder 1300 calculates RT information and an iterative closest point (ICP) algorithm error value using the ICP algorithm per reference space to be referenced from the encoding target space. The 3D encoder 1300 determines that rotation and movement in space need not be performed, and deactivates the RT flag when the ICP error value is less than or equal to a predetermined fixed value. On the contrary, the 3D data encoder 1300 activates the RT flag and appends RT information to the bit stream when the ICP error value exceeds the above fixed value.

[0411] FIG. 46 is a diagram showing an exemplary syntax to be attached to an RT information header and an RT flag. It should be noted that the number of bits assigned to each syntax may be determined based on the range of that syntax. For example, when eight reference spaces are included in the L0 reference list, 3 bits may be assigned to MaxRefSpc_l0. The number of bits to be assigned may be variable according to the value each syntax may take, and may also be fixed regardless of the value each syntax may take. When the number of bits to be assigned is fixed, the 3D encoder 1300 may append this fixed number of bits to other header information.

[0412] MaxRefSpc_l0 shown in FIG. 46 indicates the number of reference spaces included in the reference list L0. RT_flag_l0[i] is the RT flag of the reference space i in the reference list L0. When RT_flag_l0[i] is equal to 1, rotation and translation in space are applied to reference space i. When RT_flag_l0[i] is 0, rotation and translation in space are not applied to reference space i.

[0413] R_l0[i] and T_l0[i] are the RT information of the reference space i in the reference list L0. R_l0[i] is the rotation information of the reference space i in the reference list L0. The rotation information indicates the content of the applied rotation process and is, for example, a rotation matrix or a quaternion. T_l0[i] is the space movement information of the reference space i in the reference list L0. The space movement information indicates the content of the applied space movement process, and is, for example, a space movement vector.

[0414] MaxRefSpc_l1 indicates the number of reference spaces included in the reference list L1. RT_flag_l1[i] is the RT flag of reference space i in reference list L1. When RT_flag_l1[i] is equal to 1, rotation and translation in space are applied to reference space i. When RT_flag_l1[i] is 0, rotation and translation in space are not applied to reference space i.

[0415] R_l1[i] and T_l1[i] are the RT information of the reference space i in the reference list L1. R_l1[i] is the rotation information of the reference space i in the reference list L1. The rotation information indicates the content of the applied rotation process and is, for example, a rotation matrix or a quaternion. T_l1[i] is space movement information of the reference space i in the reference list L1. The space movement information indicates the content of the applied space movement process, and is, for example, a space movement vector.

[0416] The inter-prediction module 1311 generates a predicted amount of the target encoding amount using information regarding the encoded reference space stored in the reference space memory 1310. As mentioned above, prior to generating a predicted volume for the target encoding volume, inter prediction module 1311 calculates RT information in the encoding target space and the reference space using the ICP algorithm to approximate the full positional relationship between the encoding target space and the reference space. The inter-prediction module 1311 then obtains the B reference space by applying a rotation and spatial movement process to the reference space using the computed RT information. Then, the inter-prediction unit 1311 generates a predicted amount of the target encoding amount in the target encoding space using the information in the reference space B. target coding space, RT information used to obtain reference space B.

[0417] In this way, the inter-prediction module 1311 is capable of improving the accuracy of the predicted volume by generating the predicted volume using the reference space information, after approaching the full positional relationship between the encoding target space and the reference space, by applying the rotation and movement process in space to the reference space . It is possible to reduce the amount of coding since the prediction residual can be limited. It should be noted that an example in which ICP is performed using a target coding space and a reference space has been described, but is not necessarily limited to this. For example, inter prediction module 1311 may calculate RT information by performing ICP using at least one of (i) a target encoding space in which the number of voxels or point clouds are clipped, or (ii) a reference space in which the number of voxels or point clouds are clipped to reduce the amount of processing.

[0418] When the ICP error value obtained as the ICP result is less than the predetermined first threshold value, i. e. when, for example, the positional relationship between the encoding target space and the reference space is similar, the inter-prediction unit 1311 determines that the space rotation and movement process is not required, and the space rotation and movement process need not necessarily be performed. In this case, the 3D encoder 1300 can control the amount of overhead without attaching RT information to the bitstream.

[0419] When the ICP error value exceeds a predetermined second threshold, the inter-prediction module 1311 determines that the shape change between spaces is large, and intra prediction can be applied to all volumes of the encoding target space. Hereinafter in this document, the spaces to which intra prediction is applied are referred to as "internal spaces". The second threshold value exceeds the above first threshold value. The present embodiment is not limited to ICP, and any type of method can be used as long as the method calculates RT information using two voxel sets or two point cloud sets.

[0420] When attribute information, such as shape information or color information, is included in the 3D data, the inter prediction unit 1311 searches, for example, for a volume whose attribute information, for example, shape information or color information, is most similar to the target encoding volume in reference space, as the predicted amount for the target encoding amount in the target encoding space. This reference space, for example, is a reference space on which the above process of rotating and moving in space is performed. The inter-prediction module 1311 generates a predicted volume using the volume (reference volume) obtained through the search. Fig. 47 is a diagram for describing an operation of generating a predicted volume. When encoding the target coding volume (volume idx=0) shown in FIG. 47, using inter prediction, the inter prediction module 1311 searches for the volume with the smallest prediction residual, which is the difference between the encoding target volume and the reference volume, when sequentially scanning the reference volume in the reference space. The inter-prediction module 1311 selects the volume with the smallest prediction residual as the predicted volume. The prediction residuals of the target encoding amount and the predicted amount are encoded by processes performed by converter 1303 and subsequent processors. Here, the prediction residual is the difference between the attribute information of the target encoding amount and the attribute information of the predicted amount. The 3D encoder 1300 attaches, to a header, or the like. bitstream, idx of the volume of the reference volume in the reference space, as the predicted volume.

[0421] In the example shown in FIG. 47, a reference volume with volume idx=4 from the L0R0 reference space is selected as the predicted volume for the encoding target volume. The prediction residuals of the encoding target volume and the reference volume and the reference volume idx=4 are then encoded and appended to the bitstream.

[0422] It should be noted that an example has been described in which the predicted amount of attribute information is generated, but the same process can be applied to the predicted amount of position information.

[0423] The prediction controller 1312 controls whether the target coding amount should be encoded using intra prediction or inter prediction. A mode including intra prediction and inter prediction is referred to herein as a "prediction mode". For example, the prediction controller 1312 calculates a prediction residual when a target coding amount is predicted using intra prediction and a prediction residual when a target coding amount is predicted using inter-prediction as estimation values, and selects a prediction mode whose estimation value is smaller. It should be noted that the prediction controller 1312 can calculate the actual coding amount by applying orthogonal transform, quantization, and entropy coding to the intra prediction residual and the inter prediction prediction residual, and select a prediction mode using the calculated coding amount as an estimation value. Overhead information (reference volume information idx, etc.) other than the prediction residual may be added to the estimation value. The prediction controller 1312 may continuously select intra prediction when it is predetermined to encode the encoding target space using the intra space.

[0424] Entropy encoder 1313 generates an encoded signal (encoded bit stream) by variable length encoding of a quantized coefficient, which is input from quantizer 1304. Specifically, entropy encoder 1313, for example, binarizes the quantized coefficient and arithmetically encodes the resulting binary signal.

[0425] Next, a 3D data decoding apparatus that decodes an encoded signal generated by the 3D data encoding apparatus 1300 will be described. Fig. 48 is a block diagram of a 3D data decoding apparatus 1400 according to the present embodiment. This 3D decoder 1400 includes an entropy decoder 1401, an inverse quantizer 1402, an inverse transform 1403, an adder 1404, a reference volume memory 1405, an intra prediction unit 1406, a reference space memory 1407, an inter prediction unit 1408, and a prediction controller 1409.

[0426] The entropy decoder 1401 decodes the encoded signal (coded bit stream) based on the variable length code. For example, the entropy decoder 1401 generates a binary signal by arithmetic decoding the encoded signal, and generates a quantized coefficient using the generated binary signal.

[0427] The inverse quantizer 1402 generates an inverse quantized coefficient by inverse quantizing the quantized coefficient input from the entropy decoder 1401 using a quantization parameter attached to the bit stream, and the like.

[0428] The inverse transform 1403 generates a prediction residual by inversely transforming the inverse quantized coefficient input from the inverse quantizer 1402. For example, the inverse transform 1403 generates a prediction residual by inversely orthogonal transforming the inversely quantized coefficient, based on information appended to the bit stream.

[0429] The adder 1404 adds, to generate the recovered volume, (i) the prediction residual generated by the inverse transform 1403 with (ii) the predicted volume generated via intra prediction or intra prediction. This reconstructed volume is output as decoded 3D data and stored in the reference volume memory 1405 or the reference space memory 1407 .

[0430] The intra prediction unit 1406 generates a predicted volume through intra prediction using a reference volume in the reference volume memory 1405 and information attached to the bit stream. Specifically, the intra prediction unit 1406 obtains the neighboring volume information (e.g., volume idx) attached to the bit stream and the prediction mode information, and generates a predicted volume through the mode indicated by the prediction mode information using the neighboring volume indicated in the neighboring volume information. It should be noted that the specifics of these processes are identical to the above processes performed by the intra prediction module 1309, except for what the information attached to the bitstream is used for.

[0431] The inter-prediction unit 1408 generates a predicted volume through inter-prediction using the reference space in the reference space memory 1407 and the information attached to the bitstream. Specifically, the inter-prediction module 1408 applies the rotation and movement process in space to the reference space using the RT information per reference space attached to the bit stream, and generates a predicted volume using the reference space after the rotation and movement in space. It should be noted that when the RT flag is present in the bitstream per reference space, the inter prediction module 1408 applies the rotation and movement process in space to the reference space in accordance with the RT flag. It should be noted that the specifics of these processes are identical to the above processes performed by the inter-prediction unit 1311, except for what the information attached to the bit stream is used for.

[0432] The prediction controller 1409 controls whether the target decoding amount should be decoded using intra prediction or inter prediction. For example, the prediction controller 1409 selects intra prediction or inter prediction according to the information that is attached to the bit stream and indicates the prediction mode to be used. It should be noted that the prediction controller 1409 may continuously select intra prediction when it is predetermined to decode the decoding target space using the intra space.

[0433] Later in this document, variations of the present embodiment are described. In the present embodiment, an example is described in which rotation and displacement in space are applied in units of spaces, but rotation and displacement in space can also be applied in smaller units. For example, the 3D encoder 1300 may divide space into subspaces and apply rotation and translation in space in units of subspaces. In this case, the 3D data encoding apparatus 1300 generates RT information per subspace, and attaches the generated RT information to a header or the like. bit stream. The 3D encoding device 1300 may apply rotation and movement in space in terms of volumes, which is a unit of encoding. In this case, the 3D data encoding apparatus 1300 generates RT information in units of encoded volumes, and appends the generated RT information to a header or the like. bit stream. The above can also be combined. In other words, the 3D encoder 1300 may apply rotation and translation in space in large units, and then apply rotation and translation in space in small units. For example, the 3D encoder 1300 may apply rotation and translation in space in units of spaces, and may also apply different rotations and translations in space to each of the plurality of volumes included in the resulting spaces.

[0434] In the present embodiment, an example is described in which rotation and movement in space is applied to the reference space, but is not necessarily limited to this. For example, the 3D data encoder 1300 may apply a scaling process and resize the 3D data. The 3D encoder 1300 may also apply one or two of rotation, translation, and scaling. When applying processes across multiple stages and different units as above, the type of processes applied in each unit may differ. For example, rotation and translation in space may be applied in units of spaces, and translation in space may be applied in units of volumes.

[0435] It should be noted that these variations are also applicable to the device 1400 decoding three-dimensional data.

[0436] As mentioned above, the 3D data encoding device 1300 according to the present embodiment performs the following processes. Fig. 48 is a flowchart for the inter-prediction process performed by the 3D data encoding apparatus 1300.

[0437] The 3D data encoding device 1300 generates predicted position information (eg, predicted volume) using position information about 3D points included in 3D reference data (eg, reference space) associated with a time different from the time associated with the current 3D data (eg, target encoding space) (S1301). Specifically, the 3D data encoding apparatus 1300 generates predictive position information by applying a rotation and movement process in space to position information about the 3D points included in the 3D reference data.

[0438] It should be noted that the 3D data encoding device 1300 may perform a rotation and movement process in space using the first unit (for example, spaces), and may perform position prediction information generation using a second unit (for example, volumes) that is less than the first unit. . For example, the 3D data encoding apparatus 1300 searches for a volume among the plurality of volumes included in the reference space after rotating and moving in the space whose position information is the least different from the position information of the encoding target volume included in the encoding target space. It should be noted that the 3D data encoding apparatus 1300 can perform a process of rotating and moving in space and generating predictive position information in the same unit.

[0439] The 3D data encoding apparatus 1300 can generate predictive position information by applying (i) a first spatial rotation and translation process to position information about the 3D points included in the 3D reference data, and (ii) a second spatial rotation and translation process to position information about 3D points obtained through a first spatial rotation and translation process, wherein the first spatial rotation and translation process uses a first unit (e.g., spaces), and wherein the second spatial rotation and translation process uses a second unit (e.g., volumes) which is less than the first one.

[0440] For example, as illustrated in FIG. 41, position information about 3D points and predicted position information are represented using an octant tree structure. For example, position information about 3D points and predicted position information are expressed in a scan order that prioritizes latitude over depth in an octant tree structure. For example, position information about 3D points and predicted position information are expressed in a scan order that prioritizes depth over latitude in an octant tree structure.

[0441] As illustrated in FIG. 46, the 3D data encoding apparatus 1300 encodes an RT flag that indicates whether or not to apply a rotation and movement process in space to position information regarding the 3D points included in the 3D reference data. In other words, the 3D encoding device 1300 generates an encoded signal (encoded bitstream) including an RT flag. The 3D encoding device 1300 encodes RT information that indicates the content of the rotation and movement process in space. In other words, the 3D data encoding apparatus 1300 generates an encoded signal (encoded bitstream) including RT information. It should be noted that the 3D encoding device 1300 may encode RT information when the RT flag indicates a need to apply a rotation and translation process in space, and need not encode RT information when the RT flag indicates a need not to apply a rotation and translation process. in space.

[0442] The 3D data includes, for example, position information regarding the 3D points and attribute information (color information, etc.) of each 3D point. The 3D data encoding apparatus 1300 generates predictive attribute information using the attribute information of the 3D points included in the 3D reference data (S1302).

[0443] The 3D data encoding device 1300 then encodes position information about the 3D points included in the current 3D data using the predicted position information. For example, as illustrated in FIG. 38, the 3D data encoding apparatus 1300 calculates differential position information, wherein the differential position information is the difference between predicted position information and position information about 3D points included in the current 3D data (S1303).

[0444] The 3D data encoding device 1300 encodes the attribute information of the 3D points included in the current 3D data using the predicted attribute information. For example, the 3D data encoding apparatus 1300 calculates differential attribute information, where the differential attribute information is the difference between predicted attribute information and attribute information about 3D points included in the current 3D data (S1304). The 3D encoding device 1300 then performs transformation and quantization on the calculated differential attribute information (S1305).

[0445] Finally, the 3D data encoding device 1300 encodes (eg, entropy encodes) differential position information and quantized differential attribute information (S1036). In other words, the 3D data encoding apparatus 1300 generates an encoded signal (encoded bitstream) including differential position information and differential attribute information.

[0446] It should be noted that when the attribute information is not included in the 3D data, the 3D data encoding device 1300 need not necessarily perform steps S1302, S1304, and S1305. The 3D encoding device 1300 can also only perform one of encoding the position information regarding the 3D points and encoding the attribute information of the 3D points.

[0447] The order of the processes shown in FIG. 49 is merely an example and is not limited thereto. For example, since the processes regarding the position information (S1301 and S1303) and the processes regarding the attribute information (S1302, S1304 and S1305) are separate from each other, they can be executed in a preferred order, and part of them can also be executed in parallel.

[0448] In view of the above, the 3D data encoding apparatus 1300 according to the present embodiment generates predictive position information using position information about the 3D points included in the 3D reference data associated with a time different from the time associated with the current 3D data; and encodes differential position information, which is a difference between predicted position information and position information about 3D points included in the current 3D data. This allows the coding efficiency to be improved since the data amount of the encoded signal can be reduced.

[0449] The 3D data encoding apparatus 1300 according to the present embodiment generates predictive attribute information using attribute information about the 3D points included in the 3D reference data; and encodes differential attribute information, which is a difference between predicted attribute information and attribute information about 3D points included in the current 3D data. This makes it possible to improve the coding efficiency since the data amount of the encoded signal can be reduced.

[0450] For example, the device 1300 encoding three-dimensional data includes a processor and a storage device. The processor uses the storage device to carry out the above processes.

[0451] FIG. 48 is a flowchart for the inter-prediction process performed by the 3D data decoding apparatus 1400.

[0452] The 3D data decoding apparatus 1400 decodes (eg, entropy decodes) differential position information and differential attribute information from the encoded signal (encoded bitstream) (S1401).

[0453] The 3D data decoding apparatus 1400 decodes, from the encoded signal, an RT flag that indicates whether or not to apply the rotation and movement process in space to position information regarding the 3D points included in the 3D reference data. The 3D data decoder 1400 encodes RT information that indicates the content of the rotation and movement process in space. It should be noted that the 3D decoder 1400 can decode RT information when the RT flag indicates the need to apply the rotation and translation process in space, and need not necessarily decode the RT information when the RT flag indicates the need not to apply the rotation and translation process. in space.

[0454] The 3D decoder 1400 then performs inverse transformation and inverse quantization on the decoded differential attribute information (S1402).

[0455] The 3D data decoding apparatus 1400 then generates predicted position information (eg, predicted volume) using position information about the 3D points included in the 3D reference data (eg, reference space) associated with a time different from the time associated with the current 3D data (eg, decoding target space) (S1403). Specifically, the 3D data decoding apparatus 1400 generates predictive position information by applying a rotation and movement process in space to position information about the 3D points included in the 3D reference data.

[0456] More specifically, when the RT flag indicates a need to apply a rotation and translation process in space, the 3D data decoding apparatus 1400 applies the rotation and translation process in space to position information regarding the 3D points included in the 3D reference data indicated in the RT information. . On the contrary, when the RT flag indicates not to apply the rotation and translation process, the 3D decoder 1400 does not apply the rotation and translation process to the position information with respect to the 3D points included in the 3D reference data.

[0457] It should be noted that the 3D decoding apparatus 1400 may perform a rotation and movement process in space using a first unit (eg, spaces), and may perform position prediction information generation using a second unit (eg, volumes) that is smaller than the first unit. . It should be noted that the 3D data decoding apparatus 1400 may perform a process of rotating and moving in space and generating predictive position information in the same unit.

[0458] The 3D data decoding apparatus 1400 can generate predictive position information by applying (i) a first spatial rotation and translation process to position information about the 3D points included in the 3D reference data, and (ii) a second spatial rotation and translation process to position information about 3D points obtained through a first spatial rotation and translation process, wherein the first spatial rotation and translation process uses a first unit (e.g., spaces), and wherein the second spatial rotation and translation process uses a second unit (e.g., volumes) which is less than the first one.

[0459] For example, as illustrated in FIG. 41, position information about 3D points and predicted position information are represented using an octant tree structure. For example, position information about 3D points and predicted position information are expressed in a scan order that prioritizes latitude over depth in an octant tree structure. For example, position information about 3D points and predicted position information are expressed in a scan order that prioritizes depth over latitude in an octant tree structure.

[0460] The 3D data decoding apparatus 1400 generates predictive attribute information using the attribute information of the 3D points included in the 3D reference data (S1404).

[0461] The 3D data decoding apparatus 1400 then recovers position information about the 3D points included in the current 3D data by decoding the encoded position information included in the encoded signal using the predicted position information. The encoded position information here is differential position information. The 3D data decoding apparatus 1400 recovers position information about the 3D points included in the current 3D data by adding differential position information to the predicted position information (S1405).

[0462] The 3D data decoding apparatus 1400 recovers the attribute information of the 3D points included in the current 3D data by decoding the encoded attribute information included in the encoded signal using the predicted attribute information. The encoded attribute information here is differential position information. The 3D data decoding apparatus 1400 recovers attribute information about the 3D points included in the current 3D data by adding differential attribute information to the predicted attribute information (S1406).

[0463] It should be noted that when the attribute information is not included in the 3D data, the 3D data decoding apparatus 1400 need not necessarily perform steps S1402, S1404, and S1406. The 3D data decoding apparatus 1400 can also only perform one of decoding position information with respect to 3D points and decoding 3D point attribute information.

[0464] The order of the processes shown in FIG. 50 is merely an example and is not limited thereto. For example, since the processes regarding the position information (S1403 and S1405) and the processes regarding the attribute information (S1402, S1404 and S1406) are separate from each other, they can be executed in a preferred order, and part of them can also be executed in parallel.

[0465] Embodiment 8

In the present embodiment, a means for representing 3D points (point clouds) when encoding 3D data is described below.

[0466] FIG. 51 is a block diagram showing the structure of a 3D data distribution system according to the present embodiment. The distribution system shown in FIG. 51 includes a server 1501 and a plurality of clients 1502.

[0467] The server 1501 includes a storage device 1511 and a controller 1512. The storage device 1511 stores an encoded 3D map 1513 that is encoded 3D data.

[0468] FIG. 52 is a diagram showing an exemplary bitstream structure of an encoded 3D map 1513. The 3D map is divided into a plurality of submaps, and each submap is encoded. Attached to each sub-map is a random access (RA) header including sub-coordinate information. The sub-coordinate information is used to improve the coding efficiency of the sub-map. This sub-coordinate information indicates the sub-coordinates of the sub-map. The sub-coordinates are the coordinates of the sub-map having the reference coordinates. It should be noted that a three-dimensional map including a plurality of sub-maps is referred to as a "complete map". Coordinates that are referenced in the complete map (eg, the origin) are called "reference coordinates". In other words, sub-coordinates are sub-map coordinates in the full map coordinate system. In other words, the sub-coordinates indicate the offset between the full map coordinate system and the sub-map coordinate system. Coordinates in the coordinate system of a complete map having reference coordinates are referred to as "complete coordinates". The coordinates in the coordinate system of the sub-map having the sub-coordinates as a reference are called "differential coordinates".

[0469] The client 1502 transmits a message to the server 1501. This message includes position information regarding the client 1502. The controller 1512 included in the server 1501 obtains the bitstream of the submap closest to the client 1502 based on the position information included in the received message. The sub-map bit stream includes sub-coordinate information and is transmitted to the client 1502. The decoder 1521 included in the client 1502 obtains the full coordinates of the sub-map having the reference coordinates using this sub-coordinate information. An application 1522 included in the client 1502 executes the application associated with its own location using the received complete submap coordinates.

[0470] The submap indicates a partial area of the full map. Sub-coordinates are the coordinates at which the sub-map is located in the reference coordinate space of the full map. For example, in a complete map called "A", there is a sub-map A called "AA" and a sub-map B called "AB". When the vehicle wants to access card AA, decoding starts from sub-map A, and when the vehicle wants to access card AB, decoding starts from sub-map B. The sub-map here is a random access point. Specifically, A is Osaka Prefecture, AA is Osaka City, and AB is Takatsuki City.

[0471] Each sub-map is transmitted along with sub-coordinate information to the client. The sub-coordinate information is included in the header information of each sub-map, transmitted packet, and the like.

[0472] The reference coordinates that serve as a reference for the sub-coordinate information of each sub-map may be attached to the space header information at a higher level than the sub-map, for example, to the full map header information.

[0473] A submap may be formed by a single space (SPC). The submap may also be formed by a plurality of SPCs.

[0474] A submap may include a group of spaces (GOS). A sub-map can be formed by means of the world. For example, in the case where a plurality of objects are provided in a submap, the submap is formed by the plurality of SPCs by designating the plurality of objects to separate the SPCs. A sub-map is formed by one SPC by assigning a plurality of objects to one SPC.

[0475] The following describes the advantage in terms of coding efficiency when using sub-coordinate information. Fig. 53 is a diagram to describe this advantage. For example, a high number of bits is needed in order to encode a 3D point A that is located at a large distance from the reference coordinates shown in FIG. 53. The distance between the sub-coordinates and the 3D point A is smaller than the distance between the reference coordinates and the 3D point A. Accordingly, it is possible to improve encoding efficiency by encoding the coordinates of the 3D point A having the sub-coordinates as a reference larger than when encoding the coordinates of the 3D point A A that has the reference coordinates. The sub-map bitstream includes sub-coordinate information. By transmitting the submap bit stream and reference coordinates to the decoding end (client), it is possible to recover the complete submap coordinates at the decoder end.

[0476] FIG. 54 is a flowchart for processes executed by the server 1501, which is the transmitting end of the sub-map.

[0477] The server 1501 first receives a message including position information regarding the client 1502 from the client 1502 (S1501). The controller 1512 obtains the sub-map encoded bitstream based on position information with respect to the client from the storage device 1511 (S1502). The server 1501 then transmits the encoded bitstream of the submap and reference coordinates to the client 1502 (S1503).

[0478] FIG. 55 is a flowchart for processes executed by client 1502, which is the receiving end of the sub-map.

[0479] The client 1502 first receives the encoded bitstream of the submap and reference coordinates transmitted from the server 1501 (S1511). The client 1502 then obtains the sub-coordinate information of the sub-map by decoding the encoded bitstream (S1512). The client 1502 then restores the differential coordinates in the submap to full coordinates using the reference coordinates and subcoordinates (S1513).

[0480] The following describes an exemplary syntax of information associated with a submap. When encoding the sub-map, the 3D data encoding device calculates differential coordinates by subtracting the sub-coordinates from the coordinates of each point cloud (3D points). The 3D encoder then encodes the differential coordinates into a bit stream as the value of each point cloud. An encoder encodes sub-coordinate information indicating sub-coordinates as bitstream header information. This enables the 3D decoder to obtain the full coordinates of each point cloud. For example, a 3D data encoder is included in the server 1501 and a 3D data decoder is included in the client 1502.

[0481] FIG. 56 is a diagram showing an exemplary submap syntax. NumOfPoint shown in Fig. 56 indicates the total number of point clouds included in the submap; sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z are sub-coordinate information; sub_coordinate_x specifies the X coordinate of the subcoordinates; sub_coordinate_y specifies the Y coordinate of the subcoordinates; sub_coordinate_z specifies the Z coordinate of the subcoordinates.

[0482] Diff_x[i], diff_y[i], and diff_z[i] are the differential coordinates of the i-th point cloud in the submap; diff_x[i] is the differential value between the X coordinate of the i-th point cloud and the X coordinate of the sub-coordinates in the sub-map; diff_y[i] is the differential value between the y-coordinate of the i-th point cloud and the y-coordinate of the sub-coordinates in the sub-map; diff_z[i] is the differential value between the z-coordinate of the i-th point cloud and the z-coordinate of the sub-coordinates in the sub-map.

[0483] The 3D data decoding apparatus decodes point_cloud[i] _x, point_cloud[i] _y, and point_cloud[i] _z, which are full coordinates of the i-th point cloud, using the following expression; point_cloud[i] _x is the X coordinate of the full coordinates of the i-th point cloud; point_cloud[i] _y is the Y coordinate of the full coordinates of the i-th point cloud; point_cloud[i] _z is the Z coordinate of the total coordinates of the i-th point cloud.

[0484] point_cloud[i] _x=sub_coordinate_x+diff_x[i]

point_cloud[i] _y=sub_coordinate_y+diff_y[i]

point_cloud[i] _z=sub_coordinate_z+diff_z[i]

[0485] Next, a switching process for applying octant tree-based coding is described. The 3D data encoder chooses, when encoding a submap, whether to encode each point cloud using an octant tree representation (hereinafter referred to as "octant tree coding") or to encode differential values from subcoordinates (hereinafter referred to as document is referred to as "non-octant tree coding"). Fig. 57 is a diagram showing this operation in principle. For example, the 3D data encoding device applies octant tree encoding to a submap when the total number of point clouds in the submap is at least a predetermined threshold value. The 3D encoder applies non-octant tree encoding to the submap when the total number of point clouds in the submap is below a predetermined threshold. This enables the 3D data encoding device to improve encoding efficiency since it can appropriately select whether to use octant-tree based coding or non-octant-tree based coding according to the shape and density of the objects included in the sub-map.

[0486] The 3D data encoder attaches, to a header, or the like. submap, information indicating whether octant tree based coding or non octant tree based coding is applied to the submap (hereinafter referred to as "octant tree based coding application information"). This allows the 3D data decoder to identify whether a bitstream is being obtained by encoding based on the submap octant tree or not based on the submap octant tree.

[0487] The 3D data encoding device can calculate the coding gain when applying octant tree-based coding and the coding gain when applying non-octant tree based coding to the same point cloud, and apply the coding method whose coding gain is better to the submap.

[0488] FIG. 58 is a diagram showing an exemplary submap syntax when performing this switch; coding_type shown in FIG. 58 is information indicating the coding type and the above octant tree-based encoding application information; coding_type=00 indicates that octant tree based coding is applied; coding_type=01 indicates that non-octant tree based coding is applied; coding_type=10 or 11 indicates that a coding method or the like other than the above coding methods is applied.

[0489] When the encoding type is non-octant tree (non_octree) encoding, the submap includes NumOfPoint and subcoordinate information (sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z).

[0490] When the coding type is octant tree based coding (octant tree), the submap includes octree_info; octree_info is information necessary for octtree-based encoding and includes, for example, depth information.

[0491] When the encoding type is non-octtree (non_octree) encoding, the submap includes differential coordinates (diff_x[i], diff_y[i] and diff_z[i]).

[0492] When the coding type is octant tree based coding (octant tree), the submap includes octree_data, which is encoded data associated with octant tree based coding.

[0493] It should be noted that an example is described here in which the XYZ coordinate system is used as the point cloud coordinate system, but the polar coordinate system can also be used.

[0494] FIG. 59 is a flowchart for a 3D data encoding process performed by a 3D data encoding device. The 3D encoding device first calculates the total number of point clouds in the current submap, which is the submap to be processed (S1521). The 3D encoder then determines whether or not the calculated total number of point clouds is at least a predetermined threshold value (S1522).

[0495] When the total number of point clouds is at least a predetermined threshold ("Yes" in S1522), the 3D data encoding apparatus applies octant tree-based encoding to the current submap (S1523). The 3D data encoding apparatus appends, to the bitstream header, octant tree based coding application information indicating that octant tree based coding is applied to the current submap (S1525).

[0496] In contrast, when the total number of point clouds is below a predetermined threshold ("No" in S1522), the 3D data encoding device applies non-octant tree based encoding to the current submap (S1524). The 3D data encoding apparatus appends, to the bitstream header, octant tree-based encoding application information indicating that non-octant tree-based encoding is applied to the current submap (S1525).

[0497] FIG. 60 is a flowchart for a 3D data decoding process performed by a 3D data decoding apparatus. The 3D data decoding apparatus first decodes octant tree-based encoding application information from the header of the bitstream (S1531). The 3D data decoding apparatus then determines whether or not the coding type applied to the current submap is octant tree based coding based on the decoded octant tree based coding application information (S1532).

[0498] When the octant tree based encoding application information indicates that the coding type is octant tree based coding ("Yes" in S1532), the 3D decoding apparatus decodes the current submap through octant tree based decoding (S1533). On the contrary, when the octant tree based encoding application information indicates that the coding type is non-octant tree based coding ("No" in S1532), the 3D data decoding apparatus decodes the current submap through non-octant tree decoding (S1534).

[0499] Hereinafter, variations of the present embodiment are described in this document. Fig. 61 to 63 are diagrams principally showing operations of variations of the coding type switching process.

[0500] As illustrated in FIG. 61, the 3D data encoding device can select whether to apply octant tree based coding or non-octant tree based coding per space. In this case, the 3D data encoder appends octant tree-based encoding application information to the space header. This allows the 3D data decoding apparatus to determine whether or not octant tree based coding per space is applied. In this case, the 3D data encoding device specifies the sub-coordinates per space, and encodes a differential value, which is the value of the sub-coordinates subtracted from the coordinates of each point cloud in space.

[0501] This enables the 3D data encoding device to improve encoding efficiency since it can appropriately select whether or not to apply octant tree based encoding according to the shape of the objects or the total number of point clouds in space.

[0502] As illustrated in FIG. 62, the 3D data coding device can select whether to apply octant tree based coding or non-octant tree based coding per volume. In this case, the 3D data encoder appends octant tree-based encoding application information to the volume header. This allows the 3D data decoding apparatus to determine whether or not octant tree based coding per volume is applied. In this case, the 3D data encoding apparatus specifies the sub-coordinates per volume, and encodes a differential value, which is the value of the sub-coordinates subtracted from the coordinates of each point cloud in the volume.

[0503] This enables the 3D data encoding device to improve encoding efficiency since it can appropriately select whether or not to apply octant tree based encoding according to the shape of the objects or the total number of point clouds in the volume.

[0504] In the above description, an example is shown in which a difference that is the sub-coordinates of each point cloud subtracted from the coordinates of each point cloud is encoded as non-octant tree based coding, but is not limited to this, and any other type of method can be used. coding other than octant tree based coding. For example, as illustrated in FIG. 63, the 3D data encoding apparatus can not only encode the difference from sub-coordinates as a non-octant tree encoding, but also use a method in which a point cloud value in a sub-map, a space, or a volume itself is encoded (hereinafter referred to as "original coordinate coding).

[0505] In this case, the 3D data encoding device stores, in the header, information indicating that the original coordinate encoding is applied to the current space (submap, space, or volume). This allows the 3D decoder to determine whether or not the original coordinate encoding is applied to the current space.

[0506] When applying the original coordinate encoding, the 3D data encoding device can perform encoding without applying quantization and arithmetic coding to the original coordinates. The 3D encoder may encode the original coordinates using a predetermined fixed bit length. This allows the 3D encoder to generate a fixed bit length stream at a specific time.

[0507] In the above description, an example is shown in which a difference that is the sub-coordinates of each point cloud subtracted from the coordinates of each point cloud is encoded as non-octant tree based coding, but is not limited to this.

[0508] For example, a 3D data encoder may sequentially encode a differential value between the coordinates of each point cloud. Fig. 64 is a diagram for describing the operation in this case. For example, in the example shown in FIG. 64, the 3D data encoding device encodes a differential value between the coordinates of the point cloud PA and the predicted coordinates, using the sub-coordinates as the predicted coordinates, when encoding the point cloud PA. The 3D data encoding device encodes a differential value between the point cloud PB and the predicted coordinates, using the coordinates of the point cloud PA as the predicted coordinates, when encoding the point cloud PB. The 3D data encoding device encodes a differential value between the point cloud PC and the predicted coordinates, using the coordinates of the point cloud PB as the predicted coordinates, when encoding the point cloud PC. Thus, the 3D data encoder can specify the scan order for a plurality of point clouds and encode a differential value between the coordinates of the current point cloud to be processed and the coordinates of the point cloud just before the current point cloud in the scan order.

[0509] In the above description, the sub-coordinates are the coordinates in the lower left front corner of the sub-map, but the location of the sub-coordinates is not limited to this. Fig. 65-67 are diagrams showing other examples of sub-coordinate locations. The location of sub-coordinates can be specified as coordinates in the current space (in a sub-map, in a space, or in a volume). In other words, the sub-coordinates, as indicated above, may be the coordinates in the lower left front corner of the current space. As illustrated in FIG. 65, the sub-coordinates may be the coordinates at the center of the current space. As illustrated in FIG. 66, the sub-coordinates may be the coordinates at the top right rear corner of the current space. The sub-coordinates are not limited to being coordinates at the bottom left front corner or the top right back corner of the current space, but may also be coordinates at any corner of the current space.

[0510] The location of the sub-coordinates may be identical to the cloud coordinates of a certain moment in the current space (in a sub-map, in a space, or in a volume). For example, in the example shown in FIG. 67, the coordinates of the sub-coordinates are the same as the coordinates of the point cloud PD.

[0511] In the present embodiment, an example is shown that switches between applying octant tree based coding or non octant tree based coding, but is not necessarily limited to this. For example, the 3D data encoder may switch between using a tree structure other than an octant tree or a non-tree structure other than a tree structure. For example, another tree structure is a k-d tree in which partitioning is performed using perpendicular planes on the same coordinate axis. It should be noted that any other method may be used as another tree structure.

[0512] In the present embodiment, an example is shown in which the coordinate information included in the point cloud is encoded, but is not necessarily limited to this. The 3D data encoding apparatus may encode, for example, color information, the number of 3D features, or the number of visible light features using the same method as for the coordinate information. For example, the 3D data encoding device may define the average value of the color information included in each point cloud in the sub-map as sub-color information, and encode the difference between the color information and the sub-color information of each point cloud.

[0513] In the present embodiment, an example is shown in which a coding method (octant tree based coding or non-octant tree based coding) with good coding efficiency is selected according to the total number of point clouds, etc., but not necessarily limited to this. For example, the 3D data encoding apparatus which is the server end may store a bitstream of a point cloud encoded by octant tree coding, a bitstream of a point cloud encoded by non-octant tree coding, and a bitstream of a point cloud encoded by both methods, and switch the bit stream to be transmitted to the 3D data decoding apparatus according to the transmission environment or the processing power of the 3D data decoding apparatus.

[0514] FIG. 68 is a diagram showing an exemplary scope syntax when octant tree-based encoding is applied. The syntax shown in Fig. 68 is essentially identical to the syntax shown in FIG. 58, but differs in that each piece of information is information in units of volumes. Specifically, NumOfPoint indicates the total number of point clouds included in the volume; sub_coordinate_x, sub_coordinate_y, and sub_coordinate_z are the volume's sub-coordinate information.

[0515] Diff_x[i], diff_y[i], and diff_z[i] are the differential coordinates of the i-th point cloud in the volume; diff_x[i] is the differential value between the X coordinate of the i-th point cloud and the X coordinate of the sub-coordinates in the volume; diff_y[i] is the differential value between the Y-coordinate of the i-th point cloud and the Y-coordinate of the sub-coordinates in the volume; diff_z[i] is the differential value between the z-coordinate of the i-th point cloud and the z-coordinate of the sub-coordinates in the volume.

[0516] It should be noted that when the relative position of the volume in space can be calculated, the 3D data encoding device need not necessarily include sub-coordinate information in the volume header. In other words, the 3D data encoding device can calculate the relative position of the volume in space without including sub-coordinate information in the header, and use the calculated position as the sub-coordinates of each volume.

[0517] As mentioned above, the 3D data encoding device according to the present embodiment determines whether or not to encode, using an octant tree structure, a current space unit from a plurality of space units (e.g., submaps, spaces, or volumes) included in 3D data (eg S1522 in FIG. 59). For example, the 3D encoder determines that the current unit of space is to be encoded using an octant tree structure when the total number of 3D points included in the current unit of space is above a predetermined threshold. The 3D encoding device determines that the current space unit should not be encoded using an octant tree structure when the total number of 3D points included in the current space unit is less than or equal to a predetermined threshold.

[0518] When it is determined that the current space unit is to be encoded using an octant tree structure ("Yes" in S1522), the 3D data encoding device encodes the current space unit using an octant tree structure (S1523). When it is determined that the current unit of space is not to be encoded using an octant tree structure ("No" in S1522), the 3D data encoding device encodes the current unit of space using another method that is not an octant tree structure (S1524 ). For example, in another method, the 3D encoder encodes the coordinates of the 3D points included in the current unit of space. Specifically, in another method, the 3D encoding device encodes the difference between the reference coordinates of the current unit of space and the coordinates of the 3D points included in the current unit of space.

[0519] The 3D encoder then appends, to the bitstream, information that indicates whether or not the current space unit is encoded using an octant tree structure (S1525).

[0520] This enables the 3D data encoding device to improve the encoding efficiency since the data amount of the encoded signal can be reduced.

[0521] For example, the 3D data encoding device includes a processor and a memory, where the processor uses the memory to perform the above processes.

[0522] The 3D data decoding apparatus according to the present embodiment decodes, from the bitstream, information that indicates whether or not to decode, using an octant tree structure, the current space unit from a plurality of space units (e.g., submaps, spaces, or volumes) included in the 3D data (eg S1531 in FIG. 60). When the information indicates that the current unit of space is to be decoded using the octant tree structure ("Yes" in S1532), the 3D data decoding apparatus decodes the current unit of space using the octant tree structure (S1533).

[0523] When the information indicates a need not to decode the current space unit using an octant tree structure ("No" in S1532), the 3D data decoding apparatus decodes the current space unit using another method that is not an octant tree structure ( S1534). For example, in another method, the 3D decoder decodes the coordinates of the 3D points included in the current unit of space. Specifically, in another method, the 3D data decoding apparatus decodes the difference between the reference coordinates of the current space unit and the coordinates of the 3D points included in the current space unit.

[0524] This enables the 3D data decoding apparatus to improve coding efficiency since the data amount of the encoded signal can be reduced.

[0525] For example, the 3D data decoding apparatus includes a processor and a memory. The processor uses the storage device to carry out the above processes.

[0526] Embodiment 9

In the present embodiment, a method for encoding a tree structure such as an octant tree structure is described below.

[0527] It is possible to improve efficiency by identifying the important area and preferably decoding three-dimensional data of the important area.

[0528] FIG. 69 is a diagram showing an example of an important area on a 3D map. The important area includes, for example, at least a fixed number of 3D points, from among the 3D points on the 3D map having a high number of features. The important area may also include, for example, a fixed number of 3D points needed when, for example, a client installed in a vehicle performs its own position estimation. Alternatively, the important area can also be a face in a 3D human model. Such an important area may be set according to the type of application and may be switched accordingly.

[0529] In the present embodiment, padding-based coding and location-based coding are used as a method for representing an octant tree structure and the like. The bit sequence obtained through padding-based coding is called a "padding code". The bit sequence obtained through location-based coding is called a "location code".

[0530] FIG. 70 is a diagram showing an example of a filling code. Fig. 70 shows an example code for populating a quadtree structure. In FIG. 70, a padding code is assigned to each node. Each fill code fragment specifies whether or not the 3D point is included in a child node or leaf node. In the case of a quadtree, for example, information that indicates whether or not the four child nodes or leaves included in each node include 3D points is expressed by a 4-bit fill code. In the case of an octant tree, information that indicates whether or not eight child leaf nodes included in each node include 3D points is expressed by an 8-bit fill code. It should be noted that an example of a quadtree structure is described here in order to simplify the description, but the same is applicable to the octant tree structure. As illustrated in FIG. 70, for example, the fill code is a bit sequence in which nodes and leaves are width-scanned as described in FIG. 40 etc. In the fill code, since a plurality of pieces of 3D point information are decoded in a fixed order, it is not possible to preferentially decode a piece of preferred 3D point information. It should be noted that the fill code can also be a bit sequence in which nodes and leaves are depth-scanned, as described in FIG. 40 etc.

[0531] Hereinafter, this document describes location-based coding. It is possible to directly decode the important parts in the octant tree structure by using the location code. It is also possible to efficiently encode important 3D points at deeper levels.

[0532] FIG. 71 is a diagram for describing location-based coding and shows an example of a quadtree structure. In the example shown in FIG. 71, the 3D points A-I are represented with a quadtree structure. 3D points A and C are important 3D points included in the important area.

[0533] FIG. 72 is a diagram showing fill codes and location codes expressing important three-dimensional points A and C in the quadtree structure shown in FIG. 71.

[0534] In location-based encoding, the index of each node present on the path up to the leaf that owns the current 3D point, which is the target 3D encoding point, and the index of each leaf in the tree structure are encoded. The index here is a numeric value assigned to each node and each leaf. In other words, the index is an identifier for identifying child nodes relative to the current node. In the case of a quadtree, as shown in FIG. 71, indices between 0 and 3 are shown.

[0535] In the quadtree structure shown in FIG. 71, for example, sheet A is represented as 0 → 2 → 1 → 0 → 1 → 2 → 1 when sheet A is the current 3D point. Since the maximum value of each index in the case of FIG. 71 is 4 (represented as a 2-bit value), the number of bits required for the location code of sheet A is 7*2 bits=14 bits. Similarly, the number of bits required when sheet C is the encoding target is 14 bits. It should be noted that in the case of an octant tree, it is possible to calculate the number of bits needed for "3 bits * tile depth" since the maximum value of each index is 8 (represented as a 3-bit value). It should be noted that the 3D data encoding device can reduce the amount of data by entropy encoding after binarization of each index.

[0536] As illustrated in FIG. 72, in the padding code, it is necessary to decode all nodes of the upper levels of leaves A and C in order to decode leaves A and C. On the other hand, it is only possible to decode the data of leaves A and C in the location code. As illustrated in FIG. 72, this allows the number of bits to be reduced more than for the fill code by using the location code.

[0537] As illustrated in FIG. 72, it is possible to further reduce the amount of code by performing dictionary compression such as LZ77 on part or all of the location code.

[0538] The following describes an example in which location-based coding is applied to 3D points (point cloud) acquired via lidar. Fig. 73 is a diagram showing an example of 3D points obtained through lidar. Three-dimensional points obtained through lidar are sparsely located. In other words, when expressing these 3D points with a fill code, the number of null values is high. High 3D accuracy is required for these 3D points. In other words, the hierarchy of the octant tree structure becomes deeper.

[0539] FIG. 74 is a diagram showing an example of such a sparse and deep octant tree structure. The code for filling the octant tree structure shown in FIG. 74 is a 136-bit value (= 8 bits * 17 nodes). Since the octant tree structure has a depth of 6 and six 3D points, the location code is "3 bits * 6 * 6=108 bits". In other words, the location code allows for a 20% code reduction for the padding code. Thus, it is possible to reduce the amount of code by applying location-based coding to the sparse and deep octant tree structure.

[0540] Hereinafter, this document describes the code scopes for the padding code and the location code. When the octant tree structure has a depth of 10, the maximum number of 3D points is 8 ¹⁰ =1073741824. The number L _o bits of the fill code structure in the form of an octant tree is expressed below.

[0541] L _o =8+8 ² +...+8 ¹⁰ =127133512 bits

[0542] In this regard, the number of bits of one three-dimensional point is 1.143 bits. It should be noted that in the padding code, this number of bits does not change even if the total number of 3D points included in the octant tree structure changes.

[0543] On the other hand, in the location code, the number of bits of one 3D point is directly affected by the depth of the octant tree structure. Specifically, the number of bits of the location code of one 3D point is "3 bits * depth 10=30 bits".

[0544] In this regard, the number L _l bits of the location code of the structure in the form of an octant tree is expressed below.

[0545] L _l =30*N

[0546] N here is the total number of three-dimensional points included in the octant tree structure.

[0547] In this regard, in the case of N<L _o /30=40904450.4, i.e. when the total number of 3D points is lower than 40904450, the code amount for the location code is less than the code amount for the fill code (L _l <L _o ).

[0548] Thus, the amount of code for the location code is lower than the amount of code for the fill code in the case of a low number of 3D points, and the amount of code for the location code is higher than the amount of code for the fill code in the case of a high number of 3D points.

[0549] Accordingly, the 3D data encoding apparatus may switch between using location-based coding or population-based coding according to the total number of inputted 3D points. In this case, the 3D data encoder may append to header information or the like. bitstream, information indicating whether location-based encoding or padding-based encoding has been performed.

[0550] Hereinafter, this document describes hybrid coding, which is a combination of location-based coding and padding-based coding. When encoding a dense important area, hybrid coding, which is a combination of location-based coding and padding-based coding, is effective. Fig. 75 is a diagram showing this example. In the example shown in FIG. 75, important three-dimensional points are densely packed. In this case, the 3D data encoding apparatus performs location-based encoding for upper layers at a shallow depth, and uses padding-based encoding for lower layers. Specifically, location-based encoding is used up to the deepest common node, and padding-based encoding is used from the deepest common node up to the deepest level. The deepest common node here is the deepest node among the nodes that are the common ancestors of many important 3D points.

[0551] The following describes hybrid coding that prioritizes compression efficiency. The 3D encoder may switch between location-based encoding or padding-based encoding according to a predetermined rule during octant tree encoding.

[0552] FIG. 76 is a diagram showing an example of this rule. The 3D encoder first checks the percentage of nodes including 3D points at each level (depth). When the percentage is higher than a predetermined threshold value, the filling of the 3D data encoder encodes several upper layer nodes relative to the current layer. For example, a 3D data encoder applies padding-based encoding from the current level to levels up to the deepest common node.

[0553] For example, in the example of FIG. 76, the percentage of nodes including 3D points in the third level is above a predetermined threshold. In this regard, the 3D data encoding apparatus applies padding-based encoding from the third layer up to the second layer including the deepest common node, and applies location-based encoding to other layers, i.e. to the first level and the fourth level.

[0554] The following describes a method for calculating the above threshold value. One level of the octant tree structure includes one root node and eight child nodes. Therefore, in padding-based coding, 8 bits are needed to encode one level of octant tree structure. On the other hand, in location-based coding, 3 bits are needed per child node including a 3D point. Because when the total number of nodes including 3D points is higher than 2, padding-based coding is more efficient than location-based coding. In other words, in this case, the threshold value is 2.

[0555] Hereinafter, this document describes an exemplary structure of a bitstream generated through the above location-based coding, padding-based coding, or hybrid coding.

[0556] FIG. 77 is a diagram showing an example of a bitstream generated through location-based coding. As illustrated in FIG. 77, the bitstream generated via location-based coding includes a header and location code fragments. Each location code fragment corresponds to one 3D point.

[0557] This structure allows the 3D data decoding apparatus to separately decode a plurality of 3D points with high accuracy. It should be noted that FIG. 77 shows an example of a bitstream in the case of a quadtree structure. In the case of an octant tree structure, each index can take on a value between 0 and 7.

[0558] A 3D data coding device can entropy encode an index sequence expressing one 3D point after binarization of the index sequence. For example, when the index sequence is 0121, the 3D data encoder may binarize 0121 to 00011001 and perform arithmetic encoding on that bit sequence.

[0559] FIG. 78 is a diagram showing an example of a bitstream generated through hybrid coding when the bitstream includes important 3D points. As illustrated in FIG. 78, the location code of the upper levels, the filling code of the important 3D points of the lower levels, and the filling code of the unimportant 3D points of the lower levels are arranged in this order. Note that the location code length shown in FIG. 78 expresses a code amount for a subsequent location code. The padding code scope expresses the code scope for the subsequent padding code.

[0560] This structure allows the 3D data decoding device to select a decoding plan according to the type of application.

[0561] Encoded important 3D data is stored near the head of the bit stream data, and encoded data of unimportant 3D points not included in the important area is stored after the encoded important 3D data.

[0562] FIG. 79 is a diagram showing a tree structure expressed by the 3D important point filling code shown in FIG. 78. FIG. 80 is a diagram showing a tree structure expressed by the filling code of the unimportant 3D points shown in FIG. 78. As illustrated in FIG. 79, information related to unimportant 3D points is omitted in the important 3D point filling code. Specifically, since node 0 and node 3 at depth 5 do not include important 3D points, a value of 0 is assigned, which indicates that node 0 and node 3 do not include 3D points.

[0563] On the other hand, as illustrated in FIG. 80, information related to important 3D points is omitted in the non-important 3D point filling code. Specifically, since node 1 at depth 5 does not include unimportant 3D points, a value of 0 is assigned, which indicates that node 1 does not include a 3D point.

[0564] Thus, the 3D data encoding apparatus divides the original tree structure into a first tree structure including important 3D points and a second tree structure including unimportant 3D points, and separately encodes the first tree structure and the second tree structure based on the padding. structure. This allows the 3D data decoder to preferentially decode important 3D points.

[0565] The following describes an exemplary structure of a bitstream generated through efficiency-oriented hybrid coding. Fig. 81 is a diagram showing an exemplary structure of a bitstream generated through efficiency-targeting hybrid coding. As illustrated in FIG. 81, the root location of the subtree, the scope of the padding code, and the padding code are arranged per subtree in this order. The root location of the subtree shown in FIG. 81 is the location code of the subtree root.

[0566] In the above structure, the following is true when only one of location-based coding or padding-based coding is applied to the octant tree structure.

[0567] When the length of the location code of the root of the subtree is identical to the depth of the octant tree structure, the subtree does not include child nodes. In other words, location-based coding is applied to the entire tree structure.

[0568] When the root of the subtree is identical to the root of the octant tree structure, padding-based coding is applied to the entire tree structure.

[0569] For example, the 3D data decoding apparatus is capable of distinguishing whether the bitstream includes a location code or a padding code based on the above rule.

[0570] The bitstream may include coding mode information indicating which of location-based coding, padding-based coding, and hybrid coding is used. Fig. 82 is a diagram showing an example of a bitstream in this case. As illustrated in FIG. 82, for example, 2-bit coding mode information indicating the coding mode is attached to the bit stream.

[0571] (1) "Number of 3D points" in location-based encoding expresses the total number of subsequent 3D points. (2) "Amount of padding code" in padding-based coding expresses the amount of code for a subsequent padding code. (3) "Number of important subtrees" in hybrid coding (important 3D points) expresses the total number of subtrees including important 3D points. (4) The "number of padding subtrees" in hybrid coding (efficiency emphasis) expresses the total number of padding-encoded subtrees.

[0572] The following describes an exemplary syntax used to switch between applying padding-based coding or location-based coding. Fig. 83 is a diagram showing this exemplary syntax.

[0573] The Isleaf shown in FIG. 83 is a flag indicating whether or not the current node is a leaf; isleaf=1 indicates that the current node is a leaf, and isleaf=0 indicates that the current node is not a leaf.

[0574] When the current node is a leaf, point_flag is attached to the bit stream; point_flag is a flag indicating whether or not the current node (leaf) includes a 3D point; point_flag=1 indicates that the current node includes a 3D point, and point_flag=0 indicates that the current node does not include a 3D point.

[0575] When the current node is not a leaf, coding_type is attached to the bit stream; coding_type is coding type information indicating which coding type is applied; coding_type=00 indicates that location-based encoding is applied, coding_type=01 indicates that padding-based encoding is applied, and coding_type=10 or 11 indicates that another encoding method is applied.

[0576] When the encoding type is location-based encoding, numPoint, num_idx[i] and idx[i][j] are attached to the bitstream.

[0577] NumPoint indicates the total number of 3D points that can be coded based on location; num_idx[i] specifies the total number (depth) of indices from the current node up to the 3D point i. When the 3D points to be coded based on location are at the same depth, each num_idx[i] has the same value. In this regard, num_idx may be given as a general value before the for (for (i=0; i<numPoint; i++){) statement shown in FIG. 83.

[0578] Idx[i][j] indicates the value of the j-th index among the indices from the current node up to the 3D point i. In the case of an octant tree, the number of bits in idx[i][j] is 3 bits.

[0579] It should be noted that, as indicated above, the index is an identifier for identifying child nodes relative to the current node. In the case of an octant tree, idx[i][j] indicates a value between 0 and 7. In the case of an octant tree, there are eight child nodes that appropriately correspond to eight sub-blocks obtained by spatially dividing the current block corresponding to the current node into eight. In this regard, idx[i][j] may be information indicating the three-dimensional position of the sub-block corresponding to the child node. For example, idx[i][j] may be 3-bit information in a sum that includes three pieces of 1-bit information, each indicating the position of each of the x, y, and z subblocks.

[0580] When the coding type is occupancy-based coding, occupancy_code is attached to the bitstream; occupancy_code is the occupancy code of the current node. In the case of an octant tree, occupancy_code is an 8-bit bit sequence, such as the bit sequence "00101000".

[0581] When the value of the (i+1)-th bit of occupancy_code is 1, processing of the child node starts. In other words, the child node is set as the next current node, and the bit sequence is recursively generated.

[0582] In the present embodiment, an example is shown in which the ends of the octant tree are expressed by appending leaf information (isleaf, point_flag) to the bitstream, but the present embodiment is not necessarily limited to this. For example, the 3D data encoder may append, to the header portion of the start node (root), the maximum depth from the start node of the fill code up to the ends (leaves) including the 3D points. The 3D encoder can recursively transform information about child nodes as the depth increases from the start node, and can determine how to enter the leaves when the depth becomes the maximum depth. The 3D encoder may also append information indicating the maximum depth to the first node, the encoding type becoming padding-based encoding, and may also append this information to the start node (root) of the octant tree.

[0583] As mentioned above, the 3D data encoder may append, to the bit stream, information for switching between padding-based coding and location-based coding as header information of each node.

[0584] The 3D data coding device can entropy encode the coding_type, numPoint, num_idx, idx, and occupancy_code of each node generated using the method described above. For example, a 3D data encoder arithmetically encodes each value after binarizing each value.

[0585] In the above syntax, an example is shown when a bit depth octant tree pattern is used as a padding code, but the present embodiment is not necessarily limited to this. The 3D data encoder may use a widthwise bit sequence of the octant tree structure as the padding code. The 3D data coding apparatus may append, to the bit stream, information for switching between padding-based coding and location-based coding as header information of each node, also when using a wide bit sequence.

[0586] In the present embodiment, an example of an octant tree structure is shown, but the present embodiment is not necessarily limited to this, and the above method can be applied to an N-ary (N is an integer of 2 or higher) structure such as a quad tree or a hexant tree, or to another tree structure.

[0587] Hereinafter, an exemplary encoding process flow for switching between applying padding-based encoding or location-based encoding is described. Fig. 84 is a flowchart for an encoding process according to the present embodiment.

[0588] The 3D data encoding device first represents a plurality of 3D points included in the 3D data using an octant tree structure (S1601). The 3D encoder then sets the root in the octant tree structure as the current node (S1602). The 3D data encoding device then generates a bit sequence of the octant tree structure by performing a node encoding process for the current node (S1603). The 3D data encoding device then generates a bit sequence by entropy encoding the generated bit sequence (S1604).

[0589] FIG. 85 is a flowchart for a node encoding process (S1603). The 3D data encoding device first determines whether or not the current node is a sheet (S1611). When the current node is not a leaf ("No" in S1611), the 3D encoder sets the leaf flag (isleaf) to 0 and appends the leaf flag to the bit sequence (S1612).

[0590] The 3D data encoder then determines whether or not the total number of child nodes including the 3D points is higher than a predetermined threshold (S1613). It should be noted that the 3D data encoder may append this threshold value to the bit sequence.

[0591] When the total number of child nodes including 3D points is higher than a predetermined threshold ("Yes" in S1613), the 3D data encoding device sets the coding type (coding_type) as padding-based coding and appends the coding type to the bit sequence (S1614).

[0592] The 3D data encoder then configures the padding-based encoding information and appends the padding-based encoding information to the bit sequence. Specifically, the 3D data encoding device generates a fill code for the current node, and appends the fill code to the bit sequence (S1615).

[0593] The 3D encoder then sets the next current node based on the fill code (S1616). Specifically, the 3D encoder specifies the next current node from the raw child node whose fill code is "1".

[0594] The 3D data encoding device performs a node encoding process for a new predetermined current node (S1617). In other words, the process shown in FIG. 85 is executed for the newly specified current node.

[0595] When all child nodes have not yet been processed ("No" in S1618), the processes from step S1616 are executed again. On the other hand, when all child nodes are processed (Yes in S1618), the 3D data encoding device ends the node encoding process.

[0596] In step S1613, when the total number of child nodes including 3D points is lower than or equal to a predetermined threshold value ("No" in S1613), the 3D data encoding apparatus sets the encoding type to location-based encoding and attaches the encoding type to the bit sequence (S1619).

[0597] The 3D encoding device then configures the location-based encoding information and appends the location-based encoding information to the bit sequence. Specifically, the 3D encoder then generates a location code and appends the location code to the bit sequence (S1620). The location code includes numPoint, num_idx and idx.

[0598] In step S1611, when the current node is a leaf ("Yes" in S1611), the 3D data encoding apparatus sets the leaf flag to 1 and appends the leaf flag to the bit sequence (S1621). The 3D data encoding apparatus configures a point flag (point_flag), which is information indicating whether or not a sheet includes a 3D point, and appends the point flag to a bit sequence (S1622).

[0599] The following describes an example of a decoding process flow for switching between applying padding-based coding or location-based coding. Fig. 85 is a flowchart for a decoding process according to the present embodiment.

[0600] The 3D data encoding apparatus generates a bit sequence by entropy decoding of the bitstream (S1631). The 3D data decoding apparatus then reconstructs the octant tree structure by performing a node decoding process on the received bit sequence (S1632). The 3D data decoder then generates 3D points from the reconstruction octant tree structure (S1633).

[0601] FIG. 87 is a flowchart for a node decoding process (S1632). The 3D data decoding apparatus first obtains (decodes) the leaf flag (isleaf) from the bit sequence (S1641). The 3D data decoding apparatus then determines whether or not the current node is a leaf based on the leaf flag (S1642).

[0602] When the current node is not a leaf ("No" in S1642), the 3D data decoding apparatus obtains the coding type (coding_type) from the bit sequence (S1643). The 3D data decoding apparatus determines whether or not the encoding type is padding-based encoding (S1644).

[0603] When the coding type is padding-based coding ("Yes" in S1644), the 3D data decoding apparatus obtains padding-based coding information from the bit sequence. Specifically, the 3D data decoding apparatus obtains a stuffing code from a bit sequence (S1645).

[0604] The 3D data decoder then sets the next current node based on the fill code (S1646). Specifically, the 3D decoder specifies the next current node from the raw child node whose fill code is "1".

[0605] The 3D data decoding apparatus then performs a node decoding process for the new predetermined current node (S1647). In other words, the process shown in FIG. 87 is executed for the new given current node.

[0606] When all child nodes have not yet been processed ("No" in S1648), the processes from step S1646 are executed again. On the other hand, when all child nodes are processed (Yes in S1648), the 3D data decoding apparatus ends the node decoding process.

[0607] In step 1644, when the encoding type is location-based encoding ("No" in S1644), the 3D data decoding apparatus obtains location-based encoding information from the bit sequence. Specifically, the 3D data decoding apparatus obtains a location code from a bit sequence (S1649). The location code includes numPoint, num_idx and idx.

[0608] In step S1642, when the current node is a leaf ("Yes" in S1642), the 3D data decoding apparatus obtains, from the bit sequence, a point flag (point_flag) which is information indicating whether or not the leaf is included in itself a three-dimensional point (S1650).

[0609] It should be noted that in the present embodiment, an example is shown in which the coding type is switched per node, but the present embodiment is not necessarily limited to this. The type of encoding can be fixed per unit of volume, space, or world. In this case, the 3D data encoding apparatus may append encoding type information to volume, space, or world header information.

[0610] As above, the 3D data encoding device according to the present embodiment: generates first information in which an N-ary (N is an integer of 2 or higher) tree structure of a plurality of 3D points included in the 3D data is expressed using the first formulas (location-based coding); and generates a bit stream including the first information. The first information includes pieces of 3D point information (location code), each of which is associated with a respective one of the plurality of 3D points. The 3D point information pieces include indexes (idx), each of which is associated with a respective one of a plurality of levels in an N-ary tree structure. The indexes indicate a sub-block, from among N sub-blocks belonging to the corresponding one of the plurality of levels to which the corresponding one of the plurality of three-dimensional points belongs.

[0611] In other words, the 3D point information pieces indicate a path to a corresponding one of the plurality of 3D points in the N-ary tree structure. The indexes indicate a child node, out of N child nodes belonging to the corresponding layer (node) included in the path.

[0612] This allows the 3D data encoding method to generate a bit stream from which 3D points can be selectively decoded.

[0613] For example, pieces of 3D point information (location code) include information (num_idx) indicating the total number of indexes included in the piece of 3D point information. In other words, the information indicates the depth (number of layers) to the corresponding 3D point in the N-ary tree structure.

[0614] For example, the first information includes information (numPoint) indicating the total number of pieces of 3D point information included in the first information. In other words, the information indicates the total number of 3D points included in the N-ary tree structure.

[0615] For example, N is 8, and the indices are a 3-bit value.

[0616] For example, in a 3D data encoding device, the first encoding mode is used to generate the first information, and the second encoding mode is used to (i) generate the second information (fill code) in which an N-ary tree structure is expressed using the second formula ( padding-based coding), and (ii) generating a bit stream including the second information. The second information includes pieces of 1-bit information, each of which (i) is associated with a respective one of the plurality of sub-blocks belonging to the plurality of levels in the N-ary tree structure, and (ii) indicates whether or not a three-dimensional point is present in the respective one from many subblocks.

[0617] For example, the 3D data encoding device uses the first encoding mode when the total number of the 3D point set is less than or equal to a predetermined threshold, and the second encoding mode may be used when the total number of the 3D point set is above the predetermined threshold. This allows the 3D data encoding method to reduce the code amount of the bitstream.

[0618] For example, the first information and the second information include information (coding mode information) indicating whether an N-ary tree structure is expressed using the first formula or the second formula.

[0619] For example, as illustrated in FIG. 75 or the like, the 3D data encoding apparatus uses the first encoding mode for one part of the N-ary tree structure and the second encoding mode for another part of the N-ary tree structure.

[0620] For example, the 3D data encoding device includes a processor and a memory, where the processor uses the memory to perform the above processes.

[0621] The 3D data decoding apparatus according to the present embodiment obtains, from the bit stream, first information (location code) in which an N-ary (N is an integer of 2 or higher) tree structure of a plurality of 3D points included in the 3D data, expressed using the first formula (location-based coding). The first information includes pieces of 3D point information (location code), each of which is associated with a respective one of the plurality of 3D points. The 3D point information pieces include indexes (idx), each of which is associated with a respective one of a plurality of levels in an N-ary tree structure. The indexes indicate a sub-block, from among N sub-blocks belonging to the corresponding one of the plurality of levels to which the corresponding one of the plurality of three-dimensional points belongs.

[0622] In other words, the 3D point information pieces indicate a path to a corresponding one of the plurality of 3D points in the N-ary tree structure. The indexes indicate a child node, out of N child nodes belonging to the corresponding layer (node) included in the path.

[0623] The 3D data decoding method further recovers, using the respective one of the 3D point information pieces, a 3D point associated with the corresponding one of the 3D point information pieces.

[0624] This allows the 3D decoder to selectively generate 3D points from the bit stream.

[0625] For example, pieces of 3D point information (location code) include information (num_idx) indicating the total number of indexes included in the piece of 3D point information. In other words, the information indicates the depth (number of layers) to the corresponding 3D point in the N-ary tree structure.

[0626] For example, the first information includes information (numPoint) indicating the total number of pieces of 3D point information included in the first information. In other words, the information indicates the total number of 3D points included in the N-ary tree structure.

[0627] For example, N is 8, and the indices are a 3-bit value.

[0628] For example, the 3D data decoding apparatus further obtains, from the bit stream, second information (padding code) in which an N-ary tree structure is expressed using a second formula (padding-based coding). The 3D data decoding apparatus reconstructs a plurality of 3D points using the second information. The second information includes pieces of 1-bit information, each of which (i) is associated with a respective one of the plurality of sub-blocks belonging to the plurality of levels in the N-ary tree structure, and (ii) indicates whether or not a three-dimensional point is present in the respective one from many subblocks.

[0629] For example, the first information and the second information include information (coding mode information) indicating whether an N-ary tree structure is expressed using the first formula or the second formula.

[0630] For example, as illustrated in FIG. 75 and the like, one part of the N-ary tree structure is expressed using the first formula, and the other part of the N-ary tree structure is expressed using the second formula.

[0631] For example, the 3D data decoding apparatus includes a processor and a storage device, wherein the processor uses the storage device to perform the above processes.

[0632] Embodiment 10

In the present embodiment, another example of a method for encoding a tree structure such as an octant tree structure is described below. Fig. 88 is a diagram illustrating an example of a tree structure according to the present embodiment. In particular, FIG. 88 shows an example of a quadtree structure.

[0633] A sheet including the 3D point is called a "valid sheet", and a sheet not including the 3D point is called an "invalid sheet". A branch having a number of valid leaves greater than or equal to a threshold value is called a "dense branch". A branch having a number of valid leaves less than a threshold value is called a "sparse branch".

[0634] The 3D encoder calculates the number of 3D points (ie, the number of valid leaves) included in each branch in the tree structure layer. Fig. 88 shows an example in which the threshold value is 5. In this example, two branches are present in layer 1. Since the left branch includes seven 3D points, the left branch is determined to be a dense branch. Since the right branch includes two 3D points, the right branch is defined as a sparse branch.

[0635] FIG. 89 is a graph showing an example of the number of valid leaves (three-dimensional points) of each branch in layer 5. The horizontal axis of FIG. 89 indicates an index which is a branch identification number in layer 5. As clearly shown in FIG. 89, specific branches include a plurality of 3D points compared to other branches. Padding-based encoding is more efficient for such dense branches than for sparse branches.

[0636] The following describes how padding-based coding and location-based coding are applied. Fig. 90 is a diagram illustrating the relationship between coding schemes to be applied and the number of 3D points (number of valid leaves) included in each branch in layer 5. As illustrated in FIG. 90, the 3D data encoder applies padding-based encoding to dense branches and applies location-based encoding to sparse branches. As a result, coding efficiency can be improved.

[0637] FIG. 91 is a diagram illustrating an example of a dense branch area in lidar data. As illustrated in FIG. 91, the density of 3D points calculated from the number of 3D points included in each branch varies depending on the area.

[0638] The separation of dense 3D points (branches) and sparse 3D points (branches) provides the following advantage. The density of 3D points becomes higher with decreasing distance to the lidar sensor. Therefore, separation of branches according to sparsity and continuity provides separation in the distance direction. This separation is effective for specific applications. Using a method other than padding-based encoding is efficient for sparse branches.

[0639] In the present embodiment, the 3D data encoding apparatus divides the input 3D point cloud into two or more 3D point subclouds and applies a different encoding method to each of the two or more 3D point subclouds.

[0640] For example, the 3D data encoding device separates the inputted 3D point cloud into a 3D point sub-cloud A (dense 3D point cloud: dense cloud) including a dense branch, and a 3D point sub-cloud B (sparse 3D point cloud: sparse cloud). Fig. 92 is a diagram illustrating an example of a 3D point sub-cloud A (dense 3D point cloud) including a dense branch that is separated from the tree structure illustrated in FIG. 88. FIG. 93 is a diagram illustrating an example of a 3D point sub-cloud B (sparse 3D point cloud) including a sparse branch that is separated from the tree structure illustrated in FIG. 88.

[0641] Next, the 3D data encoding device encodes the 3D point sub-cloud A using padding-based coding, and encodes the 3D point sub-cloud B using location-based coding.

[0642] It should be noted that although the above describes an example in which different coding schemes (padding-based coding and location-based coding) are applied as different coding methods, for example, a 3D data encoding device may apply the same coding scheme to 3D subcloud A points and 3D point sub-cloud B, and may use different parameters when encoding the 3D point sub-cloud A and the 3D point sub-cloud B.

[0643] The following describes a procedure for a 3D data encoding process performed by a 3D data encoding device. Fig. 94 is a flowchart for a 3D data encoding process performed by a 3D data encoding device according to the present embodiment.

[0644] First, the 3D data encoding device divides the inputted 3D point cloud into sub-3D point clouds (S1701). The 3D encoder may perform this separation automatically or based on information entered by the user. For example, the user may specify a range of 3D point subclouds. With regard to an example of automatic separation, for example, when the input data is lidar data, the 3D data encoding device performs separation using distance information indicating the distance to each point cloud. In particular, the 3D data encoding apparatus separates point clouds within a certain range from the measurement point and point clouds outside a certain range. In addition, the 3D data encoding device can perform separation using information indicating important areas and unimportant areas, for example.

[0645] Next, the 3D data encoding apparatus generates coded data (encoded bitstream) by encoding the 3D dot sub-cloud A using method A (S1702). In addition, the 3D data encoding device generates coded data by encoding the 3D dot sub-cloud B using method B (S1703). It should be noted that the 3D encoding device can encode the 3D point sub-cloud B using method A. In this case, the 3D encoding device encodes the 3D point sub-cloud B using a parameter different from the encoding parameter used when encoding the 3D point sub-cloud A. For example, this parameter may be a quantization parameter. For example, the 3D encoding device encodes the 3D dot sub-cloud B using a quantization parameter larger than the quantization parameter used when encoding the 3D dot sub-cloud A. In this case, the 3D data encoding apparatus may append information indicating a quantization parameter used in encoding each of the 3D point subclouds to a header of the encoded 3D point subcloud data.

[0646] Thereafter, the 3D data encoding apparatus generates a bitstream by combining the encoded data obtained in step S1702 and the encoded data obtained in step S1703 (S1704).

[0647] Further, the 3D data encoding device may encode, as bitstream header information, information for decoding each 3D point subcloud. For example, a 3D data encoding device may encode information as described below.

[0648] The header information may include information indicating the number of encoded 3D subdots. In this example, this information indicates 2.

[0649] The header information may include information indicating the number of 3D points included in each 3D point subcloud and encoding methods. In this example, this information indicates the number of 3D points included in the 3D point sub-cloud A, the encoding method (method A) applied to the 3D point sub-cloud A, the number of 3D points included in the 3D point sub-cloud B, and the encoding method (method B) applied to subcloud B of 3D points.

[0650] The header information may include information for identifying a starting position or an ending position of the encoded data of each 3D point subcloud.

[0651] In addition, the 3D data encoding apparatus can encode the 3D point sub-cloud A and the 3D point sub-cloud B in parallel. Alternatively, the 3D encoding device may encode the 3D dot sub-cloud A and the 3D dot sub-cloud B sequentially.

[0652] The method for subclouding 3D points is not limited to the above method. For example, the 3D data encoding apparatus changes the division method, performs encoding using each of the division methods, and calculates the encoding efficiency of encoded data obtained using each division method. Then, the 3D data encoding apparatus selects a division method having the highest encoding efficiency among the division methods. For example, the 3D data encoder can (i) separate the 3D point clouds in each of the layers, (ii) calculate the encoding efficiency in each of the cases, (iii) select a separation method (i.e., a layer in which the separation is performed) having the highest encoding efficiency, among the partitioning methods, (iv) generate sub-clouds of 3D points using the selected partitioning method, and (v) perform encoding.

[0653] In addition, when combining the encoded data, the 3D data encoding device can place the coding information of the more important 3D dot subcloud at a position closer to the head of the bitstream. Since this enables the 3D data decoder to obtain important information by decoding only the head of the bit stream, the 3D data decoder can obtain important information quickly.

[0654] The following describes a procedure for a 3D data decoding process performed by a 3D data decoding apparatus. Fig. 95 is a flowchart for a 3D data decoding process performed by a 3D data decoding apparatus according to the present embodiment.

[0655] First, for example, the 3D data decoding apparatus receives the bit stream generated by the above 3D data encoding apparatus. Then, the 3D data decoding apparatus separates, from the received bit stream, the encoded data of the 3D dot sub-cloud A and the encoded data of the 3D dot sub-cloud B (S1711). Specifically, the 3D data decoding apparatus decodes, from the bitstream header information, information for decoding each 3D point subcloud, and separates the encoded data of each 3D point subcloud using the information.

[0656] Then, the 3D data decoding apparatus obtains the 3D point subcloud A by decoding the encoded data of the 3D point subcloud A using method A (S1712). In addition, the 3D data decoding apparatus obtains the 3D point sub-cloud B by decoding the encoded data of the 3D point sub-cloud B using method B (S1713). Thereafter, the 3D data decoding apparatus combines the 3D point sub-cloud A and the 3D point sub-cloud B (S1714).

[0657] It should be noted that the 3D data decoding apparatus can decode the 3D point sub-cloud A and the 3D point sub-cloud B in parallel. Alternatively, the 3D data decoding apparatus may decode the 3D point sub-cloud A and the 3D point sub-cloud B sequentially.

[0658] In addition, the 3D data decoding apparatus can decode the required 3D point sub-cloud. For example, the 3D data decoding apparatus can decode the 3D point sub-cloud A and need not necessarily decode the 3D point sub-cloud B. For example, when the 3D point cloud A is a 3D point cloud included in the important lidar data area, the 3D data decoding apparatus decodes the 3D point cloud included in the important area. Estimation of own location, etc. in a vehicle, etc. is performed using a 3D point cloud included in the important area.

[0659] Next, a specific example of the encoding process according to the present embodiment will be described. Fig. 96 is a flowchart for a 3D data encoding process performed by a 3D data encoding device according to the present embodiment.

[0660] First, the 3D data encoder separates the inputted 3D points into a sparse 3D point cloud and a dense 3D point cloud (S1721). Specifically, the 3D encoder counts the number of valid branch leaves in the octant tree structure layer. The 3D encoder defines each branch as a dense branch or a sparse branch according to the number of valid branch leaves. Then, the 3D data encoding device generates a 3D point sub-cloud (dense 3D point cloud) obtained by collecting dense branches, and a 3D point sub-cloud (sparse 3D point cloud) obtained by collecting sparse branches.

[0661] Next, the 3D data encoding apparatus generates encoded data by encoding the sparse 3D point cloud (S1722). For example, a 3D encoder encodes a sparse 3D point cloud using location-based coding.

[0662] In addition, the 3D data encoding apparatus generates encoded data by encoding a dense 3D point cloud (S1723). For example, a 3D encoder encodes a dense 3D point cloud using fill-based coding.

[0663] Thereafter, the 3D data encoding device generates a bit stream by combining the encoded sparse 3D point cloud data obtained in step S1722 and the encoded dense 3D point cloud data obtained in step S1723 (S1724).

[0664] In addition, the 3D data encoding apparatus can encode, as bitstream header information, information for decoding a sparse 3D point cloud and a dense 3D point cloud. For example, a 3D data encoding device may encode information as described below.

[0665] The header information may include information indicating the number of encoded 3D point subclouds. In this example, this information indicates 2.

[0666] The header information may include information indicating the number of 3D points included in each 3D point subcloud and encoding methods. In this example, this information indicates the number of 3D points included in the sparse 3D point cloud, the encoding method (location-based encoding) applied to the sparse 3D point cloud, the number of 3D points included in the dense 3D point cloud, and the encoding method (location-based encoding) infill-based) applied to a dense 3D point cloud.

[0667] The header information may include information for identifying a starting position or an ending position of the encoded data of each 3D point subcloud. In this example, this information indicates at least one of a start position and an end position of encoded sparse 3D point cloud data or a start position and end position of encoded dense 3D point cloud data.

[0668] In addition, the 3D data encoding apparatus can encode the sparse 3D point cloud and the dense 3D point cloud in parallel. Alternatively, the 3D data encoder may encode the sparse 3D point cloud and the dense 3D point cloud sequentially.

[0669] Next, a specific example of a 3D data decoding process will be described. Fig. 97 is a flowchart for a 3D data decoding process performed by a 3D data decoding apparatus according to the present embodiment.

[0670] First, for example, the 3D data decoding apparatus receives the bit stream generated by the above 3D data encoding apparatus. Then, the 3D data decoding apparatus separates, from the received bitstream, the encoded data of the sparse 3D point cloud and the encoded data of the dense 3D point cloud (S1731). Specifically, the 3D data decoding apparatus decodes, from the bitstream header information, information for decoding each 3D point subcloud, and separates the encoded data of each 3D point subcloud using the information. In this example, the 3D data decoding apparatus separates, from the bit stream, sparse 3D point cloud encoded data and dense 3D point cloud encoded data using header information.

[0671] Then, the 3D data decoding apparatus obtains the sparse 3D point cloud by decoding the encoded data of the sparse 3D point cloud (S1732). For example, the 3D data decoding apparatus decodes the sparse 3D point cloud using location-based decoding to decode encoded data obtained as a result of location-based encoding.

[0672] In addition, the 3D data decoding apparatus obtains a dense 3D point cloud by decoding encoded data of a dense 3D point cloud (S1733). For example, the 3D data decoding apparatus decodes a dense 3D point cloud using padding-based decoding to decode encoded data obtained as a result of padding-based encoding.

[0673] Thereafter, the 3D data decoding apparatus combines the sparse 3D point cloud obtained in step S1732 and the dense 3D point cloud obtained in step S1733 (S1734).

[0674] It should be noted that the 3D data decoding apparatus can decode the sparse 3D point cloud and the dense 3D point cloud in parallel. Alternatively, the 3D data decoding apparatus may decode the sparse 3D point cloud and the dense 3D point cloud sequentially.

[0675] In addition, the 3D data decoding apparatus can decode a portion of the necessary 3D point subclouds. For example, a 3D decoder can decode a dense 3D point cloud and need not decode a sparse 3D point cloud. For example, when the dense 3D point cloud is a 3D point cloud included in the important area of the lidar data, the 3D data decoding apparatus decodes the 3D point cloud included in the important area. Estimation of own location, etc. in a vehicle, etc. is performed using a 3D point cloud included in the important area.

[0676] FIG. 98 is a flowchart for an encoding process according to the present embodiment. First, encoding the 3D data separates the inputted 3D point cloud into a sparse 3D point cloud and a dense 3D point cloud (S1741).

[0677] Next, the 3D data encoding apparatus generates encoded data by encoding a dense 3D point cloud (S1742). Thereafter, the 3D data encoding device generates encoded data by encoding the sparse 3D point cloud (S1743). Finally, the 3D data encoding device generates a bit stream by combining the encoded sparse 3D point cloud data obtained in step S1742 and the encoded dense 3D point cloud data obtained in step S1743 (S1744).

[0678] FIG. 99 is a flowchart for a decoding process according to the present embodiment. First, the 3D data decoding apparatus extracts, from the bit stream, sparse 3D point cloud encoded data and dense 3D point cloud encoded data (S1751). Then, the 3D data decoding apparatus obtains the decoded 3D dense cloud data by decoding the encoded 3D dense cloud data (S1752). Then, the 3D data decoding apparatus obtains the decoded sparse 3D point cloud data by decoding the encoded sparse 3D point cloud data (S1753). Finally, the 3D data decoding apparatus generates a 3D point cloud by combining the decoded dense 3D point cloud data obtained in step S1752 and the decoded sparse 3D point cloud data obtained in step S1753 (S1754).

[0679] It should be noted that the 3D data encoding device and the 3D data decoding device can encode and decode any of the dense 3D point cloud and the sparse 3D point cloud first. In addition, encoding processes or decoding processes may be performed in parallel using processors and so on.

[0680] Furthermore, the 3D data encoding apparatus can encode one of a dense 3D point cloud and a sparse 3D point cloud. For example, when the dense 3D point cloud includes important information, the 3D encoding apparatus extracts the dense 3D point cloud and the sparse 3D point cloud from the input 3D point cloud, and encodes the dense 3D point cloud but does not encode the sparse 3D point cloud. This allows the 3D encoder to attach important information to the stream while reducing the number of bits. For example, when, between the server and the client, the client sends a transmission request to the server for 3D point cloud information about the client's neighborhood, the server encodes the important information about the client's neighborhood as a dense 3D point cloud and transmits the encoded important information to the client. This allows the server to transmit the required information via the client while reducing the network bandwidth.

[0681] Further, the 3D data decoding apparatus can decode one of a dense 3D point cloud and a sparse 3D point cloud. For example, when the dense 3D point cloud includes important information, the 3D data decoding apparatus decodes the dense 3D point cloud, but does not decode the sparse 3D point cloud. This enables the 3D data decoding apparatus to obtain the necessary information while reducing the processing load of the decoding process.

[0682] FIG. 100 is a flowchart for the 3D point separation process (S1741) illustrated in FIG. 98. First, the 3D data encoder sets the layer L and the threshold value TH (S1761). It should be noted that the 3D data encoding device may append information indicating the predetermined layer L and the threshold value TH to the bit stream. In other words, the 3D data encoding device can generate a bit stream including information indicating a predetermined layer L and a threshold value TH.

[0683] Next, the 3D encoder moves the target position from the root of the octant tree to the top branch in layer L. In other words, the 3D encoder selects the top branch in layer L as the current branch (S1762).

[0684] Thereafter, the 3D encoder counts the number of valid leaves of the current branch in layer L (S1763). When the number of valid leaves of the current branch exceeds the threshold value TH ("Yes" in S1764), the 3D encoder registers the current branch as a dense branch with a dense 3D point cloud (S1765). On the contrary, when the number of valid leaves of the current branch is less than the threshold value TH ("No" in S1764), the 3D encoder registers the current branch as a sparse branch with a sparse 3D point cloud (S1766).

[0685] When the processing of all branches in the L layer is not completed ("No" in S1767), the 3D encoder moves the target position to the next branch in the L layer. In other words, the 3D encoder selects the next branch in the L layer as the current branches (S1768). In addition, thereafter, the 3D data encoding apparatus executes step S1763 and the following steps for the selected next current branch.

[0686] The above process is repeated until the processing of all branches in layer L is completed ("Yes" in S1767).

[0687] It should be noted that although the layer L and the threshold value TH are preset in the above description, the present embodiment is not necessarily limited to this. For example, the 3D data encoder sets different combinations of layer L and threshold TH, generates a dense 3D point cloud and a sparse 3D point cloud using each of the combinations, and encodes a dense 3D point cloud and a sparse 3D point cloud. The 3D encoding apparatus finally encodes the dense 3D point cloud and the sparse 3D point cloud using, among the combinations, the combination of layer L and threshold TH having the highest encoding efficiency for the generated encoded data. This makes it possible to improve the coding efficiency. In addition, for example, the 3D data encoder may calculate the layer L and the threshold value TH. For example, the 3D data encoder may set, for layer L, a value that is half the maximum value of the layers included in the tree structure. In addition, the 3D data encoder may set, for the threshold value TH, a value that is half of the total number of 3D points included in the tree structure.

[0688] In the above description, an example is shown in which the inputted 3D point cloud is divided into two types of 3D point cloud, i. into a dense 3D point cloud and a sparse 3D point cloud. However, the 3D data encoder can separate the inputted 3D point cloud into at least three types of 3D point cloud. For example, when the number of valid leaves of the current branch is greater than or equal to the first threshold TH1, the 3D encoder classifies the current branch into the first dense 3D point cloud, and when the number of valid leaves of the current branch is less than the first threshold TH1 and greater than or equal to the second threshold TH2 , the 3D encoder classifies the current branch into a second dense 3D point cloud. When the number of valid leaves of the current branch is less than the second threshold TH2 and greater than or equal to the third threshold TH3, the 3D encoder classifies the current branch into a first sparse 3D point cloud, and when the number of valid leaves of the current branch is less than the third threshold TH3, the 3D encoder data classifies the current branch into a second sparse 3D point cloud.

[0689] Next, an example of the syntax of encoded 3D point cloud data according to the present embodiment will be described. Fig. 101 is a diagram illustrating an example of this syntax; pc_header, for example, is the header information of the entered 3D points.

[0690] Num_sub_pc illustrated in FIG. 101 indicates the number of 3D point subclouds; numPoint[i] indicates the number of 3D points included in the i-th subcloud of 3D points; coding_type[i] is coding type information indicating a coding type (coding scheme) applied to the i-th 3D point subcloud. For example, coding_type=00 indicates that location-based coding is applied; coding_type=01 indicates that padding-based encoding is applied; coding_type=10 or 11 indicates that a different coding scheme is being applied.

[0691] Data_sub_cloud is the encoded data of the i-th subcloud of three-dimensional points; coding_type_00_data, for example, is coded data to which coding type 00 indicated by coding_type is applied, and is coded data to which location-based coding is applied; coding_type_01_data, for example, is encoded data to which coding type 01 indicated by coding_type is applied, and is encoded data to which padding-based encoding is applied.

[0692] End_of_data is end information indicating the end of the encoded data. For example, a constant bit sequence not used for encoded data is assigned to end_of_data. This allows the 3D data decoder to, for example, skip decoding encoded data that does not need to be decoded by performing a search in the bitstream for the end_of_data bit sequence.

[0693] It should be noted that the 3D data encoding device can entropy encode the encoded data generated by the method described above. For example, the 3D data encoder binarizes each value and performs arithmetic encoding on the binarized value.

[0694] Although an example of a quadtree structure or an octant tree structure is shown in the present embodiment, the present embodiment is not necessarily limited thereto. The above method can be applied to an N-ary (N is an integer greater than or equal to 2) tree, such as a binary tree and a hexadecant tree, or other tree structure.

[0695] Variation

In the above example, as illustrated in FIG. 92 and FIG. 93, a tree structure that includes a dense branch and a top layer (tree structure from the root of the full tree structure to the root of the dense branch) is encoded, and a tree structure that includes a sparse branch and the top layer (tree structure from the root of the full tree structure) is encoded. structure to the root of the sparse branch). In the present variation, the 3D encoder separates the dense branch and the sparse branch, and encodes the dense branch and the sparse branch. In other words, the tree structure to be encoded does not include the top layer tree structure. For example, the 3D encoder applies padding-based encoding to the dense branch and applies location-based encoding to the sparse branch.

[0696] FIG. 102 is a diagram illustrating an example of a dense branch separated from the tree structure illustrated in FIG. 88. FIG. 103 is a diagram illustrating an example of a sparse branch separated from the tree structure illustrated in FIG. 88. In the present variation, the tree structures illustrated in FIG. 102 and FIG. 103.

[0697] The 3D data encoding device encodes information indicating the position of a branch, instead of encoding the top layer tree structure. For example, this information indicates the position of the branch root.

[0698] For example, the 3D data encoding apparatus encodes, as dense branch encoded data, layer information indicating the layer in which the dense branch is formed, and branch information indicating which branch number in the layer is the dense branch. This enables the 3D decoder to decode layer information and branch information from the bitstream and understand which 3D point cloud with which branch number in which layer represents the decoded dense branch. Similarly, the 3D data encoding device encodes, as the sparse branch coded data, layer information indicating the layer in which the sparse branch is generated, and branch information indicating which branch number in the layer has the sparse branch, using this layer information, and branch information.

[0699] This enables the 3D data decoding apparatus to decode layer information and branch information from the bitstream and understand which 3D point cloud with which branch number in which layer is the decoded sparse branch using that layer information and branch information. Accordingly, since it is possible to reduce the amount of overhead resulting from encoding the layer information above the dense branch or the sparse branch, it is possible to improve the encoding efficiency.

[0700] It should be noted that the branch information may indicate a value assigned to each branch in the layer indicated by the layer information. In addition, the branch information may indicate a value assigned to each node from the root of the octant tree as a starting point. In this case, the layer information does not need to be encoded. In addition, the 3D data encoding device can generate dense branches and sparse branches.

[0701] FIG. 104 is a flowchart for the encoding process according to the present variation. First, the 3D data encoder generates one or more sparse branches and one or more dense branches from the inputted 3D point cloud (S1771).

[0702] Next, the 3D data encoding apparatus generates encoded data by encoding each of one or more dense branches (S1772). Thereafter, the 3D data encoding apparatus determines whether or not all dense branches generated in S1771 (S1774) are encoded.

[0703] When encoding of all dense branches is not completed ("No" in S1773), the 3D data encoding apparatus selects the next dense branch (S1774) and generates encoded data by encoding the selected dense branch (S1772).

[0704] On the other hand, when the encoding of all dense branches is completed ("Yes" in S1773), the 3D data encoding device generates encoded data by encoding each of the one or more sparse branches (S1775). Then, the 3D data encoding apparatus determines whether or not all sparse branches generated in S1771 (S1776) are encoded.

[0705] When encoding of all sparse branches is not completed ("No" in S1776), the 3D data encoding apparatus selects the next sparse branch (S1777) and generates encoded data by encoding the selected sparse branch (S1775).

[0706] On the other hand, when the encoding of all sparse branches is completed ("Yes" in S1776), the 3D data encoding device combines the encoded data generated in steps S1772 and S1775 to generate a bitstream (S1778).

[0707] FIG. 105 is a flowchart for the decoding process according to the present variation. First, the 3D data decoding apparatus extracts one or more encoded data elements of the respective dense branches and one or more encoded data elements of the respective sparse branches from the bitstream (S1781). Then, the 3D data decoding apparatus obtains the decoded data of each of the dense branches by decoding the encoded dense branch data (S1782).

[0708] Then, the 3D data decoding apparatus determines whether or not decoding of the encoded data items of all the dense branches extracted in step S1781 (S1783) is completed. When the decoding of the coded data elements of all dense branches is not completed ("No" in S1783), the 3D data decoding apparatus selects the next dense branch coded data (S1784) and obtains the decoded dense branch data by decoding the selected dense branch coded data (S1782).

[0709] On the other hand, when the decoding of the encoded data elements of all the dense branches is completed ("Yes" in S1783), the 3D data decoding apparatus obtains the decoded data of each of the sparse branches by decoding the encoded data of the sparse branch (S1785).

[0710] Thereafter, the 3D data decoding apparatus determines whether or not decoding of the encoded data elements of all sparse branches extracted in step S1781 (S1786) is completed. When decoding of the encoded data elements of all sparse branches is not completed ("No" in S1786), the 3D data decoding apparatus selects encoded data of the next sparse branch (S1787) and obtains decoded sparse branch data by decoding the selected sparse branch encoded data (S1785).

[0711] On the other hand, when decoding of the encoded data elements of all sparse branches is completed ("Yes" in S1786), the 3D data decoding apparatus combines the decoded data obtained in steps S1782 and S1785 to form a 3D point cloud (S1788).

[0712] It should be noted that the 3D data encoding device and the 3D data decoding device can encode and decode any of the dense branch and the sparse branch first. In addition, encoding processes or decoding processes may be performed in parallel using processors and so on.

[0713] Further, the 3D data encoding device may encode one of the dense branch and the sparse branch. In addition, the 3D encoding device can encode a portion of the dense branches. For example, when a particular dense branch includes important information, the 3D encoding device extracts dense branches and sparse branches from the input 3D point cloud and encodes a dense branch including important information, but does not encode other dense branches and sparse branches. This allows the 3D encoder to append important information to the stream while reducing the number of bits. For example, when, between the server and the client, the client sends a transmission request to the server for 3D point cloud information about the client's neighborhood, the server encodes the important information about the client's neighborhood as a dense branch and transmits the important information to the client. This allows the server to transmit the required information via the client while reducing the network bandwidth.

[0714] Further, the 3D data decoding apparatus can decode one of the dense branch and the sparse branch. In addition, the 3D data decoding apparatus can decode a portion of the dense branches. For example, when a specific dense branch includes important information, the 3D decoder decodes the specific dense branch, but does not decode other dense branches and sparse branches. This enables the 3D data decoding apparatus to obtain the necessary information while reducing the processing load of the decoding process.

[0715] FIG. 106 is a flowchart for the 3D point separation process (S1771) illustrated in FIG. 104. First, the 3D data encoder sets the layer L and the threshold value TH (S1761). It should be noted that the 3D data encoding apparatus may append information indicating the predetermined layer L and the threshold value TH to the bit stream.

[0716] Next, the 3D data encoder selects the master branch in the L layer as the current branch (S1762). Thereafter, the 3D data encoder counts the number of valid leaves of the current branch in the L layer (S1763). When the number of valid leaves of the current branch exceeds the threshold value TH ("Yes" in S1764), the 3D encoder sets the current branch as a dense branch, and appends layer information and branch information to the bitstream (S1765A). On the other hand, when the number of valid leaves of the current branch is less than the threshold value TH ("No" in S1764), the 3D encoder sets the current branch as a sparse branch, and appends layer information and branch information to the bitstream (S1766A).

[0717] When the processing of all branches in the L layer is not completed ("No" in S1767), the 3D data encoding apparatus selects the next branch in the L layer as the current branch (S1768). In addition, thereafter, the 3D data encoding apparatus executes step S1763 and the following steps for the selected next current branch. The above process is repeated until the processing of all branches in layer L is completed ("Yes" in S1767).

[0718] It should be noted that although the layer L and the threshold value TH are predetermined in the above description, the present disclosure is not necessarily limited thereto. For example, the 3D data coding device sets various combinations of layer L and threshold TH, generates a dense branch and a sparse branch using each of the combinations, and encodes the dense branch and sparse branch. The 3D encoder finally encodes the dense branch and the sparse branch using, among the combinations, the combination of layer L and threshold TH having the highest encoding efficiency for the generated encoded data. This makes it possible to improve the coding efficiency. In addition, for example, the 3D data encoder may calculate the layer L and the threshold value TH. For example, the 3D data encoder may set, for layer L, a value that is half the maximum value of the layers included in the tree structure. In addition, the 3D data encoder may set, for the threshold value TH, a value that is half of the total number of 3D points included in the tree structure.

[0719] The following describes an example of the syntax of encoded 3D point cloud data according to the present variation. Fig. 107 is a diagram illustrating an example of this syntax. An example of the syntax illustrated in FIG. 107 is obtained by concatenating layer_id[i], which is layer information, and branch_id[i], which is branch information, such as the syntax illustrated in FIG. 101.

[0720] Layer_id[i] indicates the layer number of the i-th subcloud of three-dimensional points; branch_id[i] indicates the branch number in layer_id[i] of the i-th 3D point subcloud.

[0721] Layer_id[i] and branch_id[i] are layer information and branch information that indicate, for example, the position of a branch in an octant tree. For example, layer_id[i] = 2 and branch_id[i] = 5 indicate that the i-th branch is the fifth branch in layer 2.

[0722] It should be noted that the 3D data encoding apparatus can entropy encode the encoded data generated by the method described above. For example, the 3D data encoder binarizes each value and performs arithmetic encoding on the binarized value.

[0723] Although an example of a quadtree structure or an octant tree structure is given in this variation, the present disclosure is not necessarily limited thereto. The above method can be applied to an N-ary (N is an integer greater than or equal to 2) tree, such as a binary tree and a hexadecant tree, or other tree structure.

[0724] As mentioned above, the 3D data encoding device according to the present embodiment performs the process illustrated in FIG. 108.

[0725] First, the 3D data encoding device generates an N-ary (N is an integer greater than or equal to 2) tree structure of 3D points included in the 3D data (S1801).

[0726] Then, the 3D data encoding device generates the first encoded data by encoding, using the first encoding process, the first branch having, as a root, the first node included in the first layer, which is one of the layers included in N- ary tree structure (S1802).

[0727] In addition, the 3D data encoding apparatus generates second encoded data by encoding, using a second encoding process different from the first encoding process, a second branch having, as a root, a second node that is included in the first layer and is different from the first node (S1803).

[0728] Thereafter, the 3D data encoding apparatus generates a bit stream including the first encoded data and the second encoded data (S1804).

[0729] Since this enables the 3D data encoding device to apply an encoding process suitable for each branch included in the N-ary tree structure, encoding efficiency can be improved.

[0730] For example, the number of 3D points included in the first branch is less than a predetermined threshold, and the number of 3D points included in the second branch is greater than the threshold. In other words, when the number of 3D points included in the current branch is less than the threshold, the 3D encoder sets the current branch as the first branch, and when the number of 3D points included in the current branch exceeds the threshold, the 3D encoder sets the current branch as the second branch.

[0731] For example, the first encoded data includes first information indicating that the first N-ary tree structure of the first three-dimensional points included in the first branch is expressed using the first formula. The second encoded data includes second information indicating that a second N-ary tree structure of the second 3D points included in the second branch is expressed using the second formula. In other words, the first encoding process and the second encoding process differ in the encoding scheme.

[0732] For example, location-based encoding is used in the first encoding process, and padding-based encoding is used in the second encoding process. In other words, the first information includes pieces of 3D point information, each of which is associated with a respective one of the first 3D points. Each of the 3D point information pieces includes an index associated with each of the layers in the first N-ary tree structure. Each of the indexes indicates, from the number of N sub-blocks belonging to the respective one of the layers, the sub-block to which the corresponding one of the first three-dimensional points belongs. The second information includes pieces of 1-bit information, each of which is associated with a respective one of the sub-blocks belonging to the layers in the second N-ary tree structure, and indicates whether or not a three-dimensional point is present in the corresponding sub-block.

[0733] For example, the quantization parameter used in the second encoding process is different from the quantization parameter used in the first encoding process. In other words, the first encoding process and the second encoding process are identical in the encoding scheme, but differ in the parameter to be used.

[0734] For example, as illustrated in FIG. 92 and FIG. 93, when encoding the first branch, the 3D data encoding device encodes, using the first encoding process, the tree structure including the first branch and the tree structure from the root of the N-ary tree structure to the first node, and when encoding the second branch, the encoding device 3D data encodes, using the second encoding process, a tree structure including a second branch, and a tree structure from the root of the N-ary tree structure to the second node.

[0735] For example, the first encoded data includes encoded data of the first branch and third information indicating the position of the first node in the N-ary tree structure. The second encoded data includes second branch encoded data and fourth information indicating the position of the second node in the N-ary tree structure.

[0736] For example, the third information includes information (layer information) indicating the first layer and information (branch information) indicating which of the nodes included in the first layer is the first node. The fourth information includes information (layer information) indicating the first layer and information (branch information) indicating which of the nodes included in the first layer is the second node.

[0737] For example, the first encoded data includes information (numPoint) indicating the number of 3D points included in the first branch, and the second encoded data includes information (numPoint) indicating the number of 3D points included in the second branch.

[0738] For example, the 3D encoding device includes a processor and a memory, and the processor performs the above process using the memory.

[0739] The 3D data decoding apparatus according to the present embodiment performs the process illustrated in FIG. 109.

[0740] First, the 3D data decoding apparatus obtains, from the bitstream, the first encoded data obtained by encoding the first branch having, as a root, the first node included in the first layer, which is one of the layers included in An N-ary (N is an integer greater than or equal to 2) tree structure of 3D points, and second encoded data obtained by encoding a second branch having, as a root, a second node that is included in the first layer and is different from the first node ( S1811).

[0741] Next, the 3D data decoding apparatus generates the first decoded data of the first branch by decoding the first encoded data using the first decoding process (S1812).

[0742] In addition, the 3D data decoding apparatus generates second decoded data of the second branch by decoding the second encoded data using a second decoding process different from the first decoding process (S1813).

[0743] Then, the 3D data decoding apparatus reconstructs the 3D points using the first decoded data and the second decoded data (S1814). For example, these 3D points include 3D points indicated by the first decoded data and 3D points indicated by the second decoded data.

[0744] This enables the 3D data decoding apparatus to decode a bitstream for which the coding efficiency is improved.

[0745] For example, the number of 3D points included in the first branch is less than a predetermined threshold, and the number of 3D points included in the second branch is greater than the threshold.

[0746] For example, the first encoded data includes first information indicating that the first N-ary tree structure of the first three-dimensional points included in the first branch is expressed using the first formula. The second encoded data includes second information indicating that a second N-ary tree structure of the second 3D points included in the second branch is expressed using the second formula. In other words, the first decoding process and the second decoding process differ in the encoding scheme (decoding scheme).

[0747] For example, location-based coding is used for the first encoded data, and padding-based coding is used for the second encoded data. In other words, the first information includes pieces of 3D point information, each of which is associated with a respective one of the first 3D points. Each of the 3D point information pieces includes an index associated with each of the layers in the first N-ary tree structure. Each of the indexes indicates, from the number of N sub-blocks belonging to the respective one of the layers, the sub-block to which the corresponding one of the first three-dimensional points belongs. The second information includes pieces of 1-bit information, each of which is associated with a respective one of the sub-blocks belonging to the layers in the second N-ary tree structure, and indicates whether or not a three-dimensional point is present in the corresponding sub-block.

[0748] For example, the quantization parameter used in the second decoding process is different from the quantization parameter used in the first decoding process. In other words, the first decoding process and the second decoding process are identical in coding scheme (decoding scheme), but differ in a parameter to be used.

[0749] For example, as illustrated in FIG. 92 and FIG. 93, when decoding the first branch, the 3D data decoding apparatus decodes, using the first decoding process, the tree structure including the first branch and the tree structure from the root of the N-ary tree structure to the first node, and when decoding the second branch, the decoding apparatus 3D data decodes, using the second decoding process, a tree structure including a second branch, and a tree structure from the root of the N-ary tree structure to the second node.

[0750] For example, the first encoded data includes encoded data of the first branch and third information indicating the position of the first node in the N-ary tree structure. The second encoded data includes second branch encoded data and fourth information indicating the position of the second node in the N-ary tree structure.

[0751] For example, the third information includes information (layer information) indicating the first layer, and information (branch information) indicating which of the nodes included in the first layer is the first node. The fourth information includes information (layer information) indicating the first layer and information (branch information) indicating which of the nodes included in the first layer is the second node.

[0752] For example, the first encoded data includes information (numPoint) indicating the number of 3D points included in the first branch, and the second encoded data includes information (numPoint) indicating the number of 3D points included in the second branch.

[0753] For example, the 3D data decoding device includes a processor and a memory, and the processor performs the above process using the memory.

[0754] Embodiment 11

In the present embodiment, adaptive entropy coding (arithmetic coding) performed on octant tree fill codes is described below.

[0755] FIG. 110 is a diagram illustrating an example of a quadtree structure. Fig. 111 is a diagram illustrating the filling codes of the tree structure illustrated in FIG. 110. FIG. 112 is a diagram principally illustrating an operation performed by the 3D data encoding device according to the present embodiment.

[0756] The 3D data encoding apparatus according to the present embodiment entropy encodes an 8-bit stuffing code in an octant tree. The 3D encoder also updates the codebook in the entropy encoding process for the fill code. Further, the 3D data encoder does not use a single codebook, but uses an adaptive codebook so as to use the similarity information of the 3D points. In other words, the 3D data encoder uses codebooks.

[0757] The similarity information, for example, is 3D point geometric information, octant tree structure information, or 3D point attribute information.

[0758] It should be noted that although the quadtree is shown as an example in FIG. 110-112, an identical method can be applied to an N-ary tree such as a binary tree, an octant tree, and a hexadecant tree. For example, a 3D data encoder entropy encodes an 8-bit fill code in the case of an octant tree, a 4-bit fill code in the case of a quad tree, and a 16-bit fill code in the case of a hexadecant tree using an adaptive table (also referred to as a "coding table"). .

[0759] The following describes an adaptive entropy encoding process using 3D point geometry information.

[0760] When the local geometries of two nodes in a tree structure are similar to each other, there is a possibility that the fill states (ie, states indicating whether or not a 3D point is included) of child nodes are similar to each other. As a result, the 3D encoder performs grouping using the local geometry of the parent node. This allows the 3D encoder to group the population states of child nodes and use a different encoding table for each group. Accordingly, it is possible to improve the efficiency of entropy coding.

[0761] FIG. 113 is a diagram illustrating an example of geometry information. The geometry information includes information indicating whether or not each of the neighboring nodes relative to the current node is occupied (ie, whether or not it includes a 3D point). For example, the 3D encoder calculates the local geometry of the current node using information indicating whether or not the neighboring node includes the 3D point (occupied or unoccupied). A neighbor node, for example, is a node spatially located near the current node, or a node located in an identical position at a different time relative to the current node, or spatially located near the position.

[0762] FIG. 113, the shaded cube indicates the current node. The white cube represents a neighbor node and indicates the node that includes the 3D point. In FIG. 113, the geometric pattern indicated in (2) is obtained by rotating the geometric pattern indicated in (1). Accordingly, the 3D data encoding apparatus determines that these geometric patterns have high geometric similarity and entropy encodes the geometric patterns using the same codebook. In addition, the 3D data encoding device determines that the geometric patterns indicated in (3) and (4) have low geometric similarity, and entropy encodes the geometric patterns using other coding tables.

[0763] FIG. 114 is a diagram illustrating an example of filling codes of the current nodes in the geometric patterns (1)-(4) illustrated in FIG. 113 and coding tables used for entropy coding. As illustrated above, the 3D data encoding device determines that the geometry patterns (1) and (2) are included in the same geometry group, and uses the same coding table A for the geometry patterns (1) and (2). The 3D encoding device uses the coding table B and the coding table C for geometric patterns (3) and (4), respectively.

[0764] As illustrated in FIG. 114, a case occurs where the fill codes of the current nodes in the geometry patterns (1) and (2) included in the same geometry group are identical to each other.

[0765] The following describes an adaptive entropy encoding process using tree structure structure information. For example, the structure information includes information indicating the layer to which the current node belongs.

[0766] FIG. 115 is a diagram illustrating an example of a tree structure. Generally speaking, the local form of an object depends on the search criteria. For example, a tree structure tends to be more sparse in the bottom layer than in the top layer. Accordingly, the 3D encoding device uses different codebooks for the upper layers and lower layers, as illustrated in FIG. 115, which makes it possible to increase the efficiency of entropy coding.

[0767] In other words, when the 3D data encoding device encodes the padding code of each layer, the 3D data encoding device may use a different coding table for each layer. For example, when a 3D data encoding device encodes an N (N=0-6) layer fill code, the 3D data encoding device may perform entropy encoding for the tree structure illustrated in FIG. 115 using the codebook for layer N. Since this enables the 3D data encoding device to select a codebook according to the occurrence pattern of the fill code of each layer, the 3D data encoding device can improve encoding efficiency.

[0768] In addition, as illustrated in FIG. 115, the 3D data encoding apparatus may use a coding table A for the padding codes of layers 0-2, and may use a coding table B for the padding codes of layers 3-6. Since this enables the 3D data encoding apparatus to select a coding table according to the occurrence pattern of the fill code for each layer group, the 3D data encoding apparatus can improve encoding efficiency. The 3D data encoder may append codebook information used for each layer to the header of the bitstream. Alternatively, the codebook used for each layer may be predetermined by standards, etc.

[0769] Next, an adaptive entropy encoding process using attribute information (property information) of a three-dimensional point will be described. For example, the attribute information includes information about an object including the current node or information about the normal vector of the current node.

[0770] It is possible to group 3D points having similar geometry using pieces of 3D point attribute information. For example, a normal vector indicating the direction of each of the 3D points can be used as general 3D point attribute information. You can find the geometry associated with the same fill code in the tree structure by using the normal vector.

[0771] In addition, color or reflectance (reflectance) can be used as attribute information. For example, the 3D data encoding device groups 3D points having the same geometry using colors or reflectances of the 3D points, and performs a process such as switching between codebooks for each of the groups.

[0772] FIG. 116 is a diagram for describing switching between codebooks based on a normal vector. As illustrated in FIG. 116, when the normal vector groups to which the normal vectors of the current nodes belong differ, different coding tables are used. For example, a normal vector included in a predetermined range is classified into one normal vector group.

[0773] When objects belong to different categories, there is a high probability that the padding codes are different. Accordingly, the 3D data encoding apparatus can select the encoding table according to the category of the object to which the current node belongs. Fig. 117 is a diagram for describing switching between coding tables based on an object category. As illustrated in FIG. 117, when objects belong to different categories, different coding tables are used.

[0774] Next, an example of a bitstream structure according to the present embodiment will be described. Fig. 118 is a diagram illustrating an example of a structure of a bit stream generated by a 3D data encoding device according to the present embodiment. As illustrated in FIG. 118, the bitstream includes a set of codebooks, table indices, and coded fill codes. The coding table group includes coding tables.

[0775] The table index indicates a coding table used for entropy coding of a subsequent encoded fill code. An encoded padding code is a padding code that is entropy encoded. As illustrated in FIG. 118, the bitstream also includes combinations of a table index and an encoded fill code.

[0776] For example, in the example illustrated in FIG. 118, encoded padding code 0 is data that is entropy encoded using a context model (also referred to as "context") indicated by table index 0. Encoded padding code 1 is data that is entropy encoded using context indicated by table index 1. The context for encoding the encoded 0-stuff code may be predetermined by standards, etc., and the 3D data decoding apparatus may use this context when decoding the encoded 0-stuff code. Since this eliminates the need to attach a table index to the bitstream, the overhead can be reduced.

[0777] In addition, the 3D data encoder may append, in the header, information to reset each context.

[0778] The 3D data encoding device determines a codebook using geometric information, structure information, or attribute information of the current node, and encodes a fill code using the determined codebook. The 3D data encoding apparatus appends the encoding result and information (eg, table index) of the encoding table used for encoding to the bitstream, and transmits the bitstream to the 3D data decoding apparatus. This allows the 3D data decoder to decode the padding code using codebook information attached to the header.

[0779] In addition, the 3D data encoding device does not need to append codebook information used for encoding to the bit stream, and the 3D data decoding device can determine the codebook using geometric information, structure information, or attribute information of the current node that is decoded. , using a method identical to that of the 3D data encoding device, and decode the fill code using the determined codebook. Since this eliminates the need to attach codebook information to the bitstream, the overhead can be reduced.

[0780] FIG. 119 and FIG. 120 are a diagram illustrating an example of a codebook. As illustrated in FIG. 119 and FIG. 120, one codebook shows, for each 8-bit fill code value, the context model and context model type associated with the value.

[0781] As with the codebook illustrated in FIG. 119, the same context model (context) may be applied to padding codes. In addition, a different context model may be assigned to each padding code. Since this allows a context model to be assigned according to the probability of occurrence of a padding code, coding efficiency can be improved.

[0782] The context model type indicates, for example, whether the context model is a context model that updates the probability table according to the frequency of occurrence of the fill code, or is a context model having a fixed probability table.

[0783] The following is another example of a bitstream and a codebook. Fig. 121 is a diagram illustrating variation of the bitstream pattern. As illustrated in FIG. 121, the bitstream includes a codebook group and an encoded fill code. The coding table group includes coding tables.

[0784] FIG. 122 and FIG. 123 are a diagram illustrating an example of a codebook. As illustrated in FIG. 122 and FIG. 123, one codebook shows, for each 1 bit included in the fill code, the context model and context model type associated with the 1 bit.

[0785] FIG. 124 is a diagram illustrating an example of the relationship between the fill code and the bit numbers of the fill code.

[0786] As mentioned above, the 3D data encoder can process the padding code as binary data, assign a different context model to each bit, and entropy encode the padding code. Since this allows the context model to be assigned according to the probability of occurrence of each bit of the stuffing code, coding efficiency can be improved.

[0787] In particular, each bit of the fill code corresponds to a subblock obtained by dividing the spatial block corresponding to the current node. Accordingly, when sub-blocks at the same spatial position in the block have the same trend, coding efficiency can be improved. For example, when a ground or road surface intersects a block, in the octant tree, the bottom four blocks include 3D points and the top four blocks do not include 3D points. Additionally, an identical pattern appears in horizontally stacked boxes. Accordingly, it is possible to improve coding efficiency by switching between contexts for each bit as described above.

[0788] A context model can also be used that updates the probability table according to the frequency of occurrence of each bit of the fill code. In addition, a context model having a fixed probability table can be used.

[0789] The following describes procedures for a 3D data encoding process and a 3D data decoding process according to the present embodiment.

[0790] FIG. 125 is a flowchart for a 3D data encoding process including an adaptive entropy encoding process using geometric information.

[0791] During the decomposition process, the octant tree is formed from the initial bounding box of 3D points. The bounding box splits according to the position of the 3D point in the bounding box. In particular, a non-empty subspace is further split. Then, information indicating whether or not the subspace includes a 3D point is encoded into a fill code. It should be noted that the same process is carried out in the processes illustrated in FIG. 127 and FIG. 129.

[0792] First, the 3D encoder receives inputted 3D points (S1901). Then, the 3D data encoding apparatus determines whether or not the decomposition process per unit length is completed (S1902).

[0793] When the per unit length decomposition process is not completed ("No" in S1902), the 3D data encoding apparatus generates an octant tree by performing the decomposition process for the current node (S1903).

[0794] Thereafter, the 3D data encoding apparatus acquires the geometry information (S1904), and selects a coding table based on the received geometry information (S1905). Here, as mentioned above, the geometry information is information indicating, for example, the geometry of the filling states of neighboring blocks of the current node.

[0795] After that, the 3D data encoding device entropy encodes the padding code of the current node using the selected coding table (S1906).

[0796] Steps S1903-S1906 are repeated until the decomposition process per unit length is completed. When the per-unit length decomposition process is completed ("YES" in S1902), the 3D data encoding device outputs a bit stream including generated information (S1907).

[0797] The 3D data encoding device determines a codebook using geometric information, structure information, or attribute information of the current node, and encodes the fill code bit sequence using the determined codebook. The 3D data encoding apparatus appends the encoding result and information (eg, table index) of the encoding table used for encoding to the bitstream, and transmits the bitstream to the 3D data decoding apparatus. This allows the 3D data decoder to decode the padding code using codebook information attached to the header.

[0798] In addition, the 3D data encoding device does not need to append codebook information used for encoding to the bit stream, and the 3D data decoding device can determine the codebook using geometric information, structure information, or attribute information of the current node that is decoded. , using a method identical to that of the 3D data encoding device, and decode the fill code using the determined codebook. Since this eliminates the need to attach codebook information to the bitstream, the overhead can be reduced.

[0799] FIG. 126 is a flowchart for a 3D data decoding process including an adaptive entropy decoding process using geometric information.

[0800] The decomposition process included in the decoding process is similar to the decomposition process included in the above encoding process, they differ in the following aspect. The 3D decoder separates the initial bounding box using the decoded fill code. When the 3D data decoding apparatus completes the process per unit length, the 3D data decoding apparatus stores the position of the bounding box as the position of the 3D point. It should be noted that the same process is carried out in the processes illustrated in FIG. 128 and FIG. 130.

[0801] First, the 3D data decoding apparatus receives the input bitstream (S1911). Then, the 3D data decoding apparatus determines whether or not the decomposition process per unit length is completed (S1912).

[0802] When the per unit length decomposition process is not completed ("No" in S1912), the 3D data decoding apparatus generates an octant tree by performing the decomposition process for the current node (S1913).

[0803] Thereafter, the 3D data decoding apparatus acquires geometric information (S1914), and selects a codebook based on the obtained geometric information (S1915). Here, as mentioned above, the geometry information is information indicating, for example, the geometry of the filling states of neighboring blocks of the current node.

[0804] Thereafter, the 3D data decoding apparatus entropy decodes the fill code of the current node using the selected codebook (S1916).

[0805] Steps S1913-S1916 are repeated until the decomposition process per unit length is completed. When the unit length decomposition process is completed ("Yes" in S1912), the 3D data decoding apparatus outputs 3D points (S1917).

[0806] FIG. 127 is a flowchart for a 3D data encoding process including an adaptive entropy encoding process using structure information.

[0807] First, the 3D encoder receives inputted 3D points (S1921). Then, the 3D data encoding apparatus determines whether or not the decomposition process per unit length is completed (S1922).

[0808] When the per unit length decomposition process is not completed ("No" in S1922), the 3D data encoding apparatus generates an octant tree by performing the decomposition process for the current node (S1923).

[0809] Thereafter, the 3D data encoding apparatus obtains the structure information (S1924), and selects a coding table based on the received structure information (S1925). Here, as mentioned above, the structure information is information indicating, for example, the layer to which the current node belongs.

[0810] Thereafter, the 3D data encoding device entropy encodes the padding code of the current node using the selected coding table (S1926).

[0811] Steps S1923-S1926 are repeated until the decomposition process per unit length is completed. When the per-unit length decomposition process is completed ("YES" in S1922), the 3D data encoding device outputs a bit stream including generated information (S1927).

[0812] FIG. 128 is a flowchart for a 3D data decoding process including an adaptive entropy decoding process using structure information.

[0813] First, the 3D data decoding apparatus receives the input bitstream (S1931). Then, the 3D data decoding apparatus determines whether or not the decomposition process per unit length is completed (S1932).

[0814] When the per unit length decomposition process is not completed ("No" in S1932), the 3D data decoding apparatus generates an octant tree by performing the decomposition process for the current node (S1933).

[0815] Thereafter, the 3D decoding apparatus obtains the structure information (S1934), and selects a codebook based on the received structure information (S1935). Here, as mentioned above, the structure information is information indicating, for example, the layer to which the current node belongs.

[0816] Thereafter, the 3D data decoding apparatus entropy decodes the padding code of the current node using the selected codebook (S1936).

[0817] Steps S1933-S1936 are repeated until the decomposition process per unit length is completed. When the unit length decomposition process is completed ("Yes" in S1932), the 3D data decoding apparatus outputs 3D points (S1937).

[0818] FIG. 129 is a flowchart for a 3D data encoding process including an adaptive entropy encoding process using attribute information.

[0819] First, the 3D data encoder receives inputted 3D points (S1941). Then, the 3D data encoding apparatus determines whether or not the decomposition process per unit length is completed (S1942).

[0820] When the per unit length decomposition process is not completed ("No" in S1942), the 3D data encoding apparatus generates an octant tree by performing the decomposition process for the current node (S1943).

[0821] Thereafter, the 3D data encoding apparatus acquires the attribute information (S1944), and selects a coding table based on the obtained attribute information (S1945). Here, as mentioned above, the attribute information is information indicating, for example, the normal vector of the current node.

[0822] Thereafter, the 3D encoder entropy encodes the padding code of the current node using the selected coding table (S1946).

[0823] Steps S1943-S1946 are repeated until the decomposition process per unit length is completed. When the per-unit length decomposition process is completed ("YES" in S1942), the 3D data encoding device outputs a bitstream including generated information (S1947).

[0824] FIG. 130 is a flowchart for a 3D data decoding process including an adaptive entropy decoding process using attribute information.

[0825] First, the 3D data decoding apparatus receives the input bit stream (S1951). Then, the 3D data decoding apparatus determines whether or not the decomposition process per unit length is completed (S1952).

[0826] When the per unit length decomposition process is not completed ("No" in S1952), the 3D data decoding apparatus generates an octant tree by performing the decomposition process for the current node (S1953).

[0827] Thereafter, the 3D data encoding apparatus acquires the attribute information (S1954), and selects a coding table based on the obtained attribute information (S1955). Here, as mentioned above, the attribute information is information indicating, for example, the normal vector of the current node.

[0828] Thereafter, the 3D data decoding apparatus entropy decodes the padding code of the current node using the selected codebook (S1956).

[0829] Steps S1953-S1956 are repeated until the decomposition process per unit length is completed. When the unit length decomposition process is completed ("Yes" in S1952), the 3D data decoding apparatus outputs 3D points (S1957).

[0830] FIG. 131 is a flowchart for a codebook selection process using geometry information (S1905).

[0831] The 3D data coding device may select a coding table to be used for entropy coding of the fill code, using, for example, the geometry group information of the tree structure, for example. Here, the geometry group information is information indicating a geometry group including the geometry pattern of the current node.

[0832] As illustrated in FIG. 131, when the geometry group indicated by the geometry information is geometry group 0 (Yes in S1961), the 3D data encoding apparatus selects the coding table 0 (S1962). When the geometry group indicated by the geometry information is geometry group 1 (Yes in S1963), the 3D data encoding device selects the coding table 1 (S1964). In any other case ("No" in S1963), the 3D encoding device selects the encoding table 2 (S1965).

[0833] It should be noted that the coding table selection method is not limited to the above. For example, when the geometry group indicated by the geometry information is geometry group 2, the 3D encoding apparatus can further select a codebook according to the value of the geometry group, for example, using codebook 2.

[0834] For example, a geometric group is determined using padding information indicating whether or not a node adjacent to the current node includes a point cloud. Geometric patterns that take on an identical shape through a transformation, such as applying a rotation, may be included in an identical geometry group. The 3D encoder may select the geometry group using the padding information of a node that is adjacent to the current node or located near the current node and belongs to a layer identical to that of the current node. In addition, the 3D data encoding apparatus may select a geometry group using the padding information of a node that belongs to a layer different from the layer of the current node. For example, the 3D encoder may select a geometry group using the padding information of a parent node, a node adjacent to the parent node, or a node located near the parent node.

[0835] It should be noted that the same applies to the codebook selection process using geometric information (S1915) in the 3D data decoding apparatus.

[0836] FIG. 132 is a flowchart for a codebook selection process using structure information (S1925).

[0837] The 3D data coding device may select a coding table to be used for entropy coding of the fill code using, for example, tree structure layer information as structure information. Here, the layer information indicates, for example, the layer to which the current node belongs.

[0838] As illustrated in FIG. 132, when the current node belongs to layer 0 (Yes in S1971), the 3D data encoding device selects the encoding table 0 (S1972). When the current node belongs to layer 1 (Yes in S1973), the 3D encoding device selects the coding table 1 (S1974). In any other case ("No" in S1973), the 3D encoding device selects the encoding table 2 (S1975).

[0839] It should be noted that the coding table selection method is not limited to the above. For example, when the current node belongs to layer 2, the 3D encoding apparatus may further select a codebook according to the layer to which the current node belongs, for example, using codebook 2.

[0840] The same applies to the codebook selection process using structure information (S1935) in the 3D data decoding apparatus.

[0841] FIG. 133 is a flowchart for a codebook selection process using attribute information (S1945).

[0842] The 3D data coding device may select a coding table to be used for entropy coding of the fill code using, as attribute information, information regarding the object to which the current node belongs, or information regarding the normal vector of the current node.

[0843] As illustrated in FIG. 133, when the normal vector of the current node belongs to the normal vector group 0 ("Yes" in S1981), the 3D data encoding device selects the coding table 0 (S1982). When the normal vector of the current node belongs to the normal vector group 1 ("Yes" in S1983), the 3D data encoding device selects the coding table 1 (S1984). In any other case ("No" in S1983), the 3D encoding device selects the encoding table 2 (S1985).

[0844] It should be noted that the coding table selection method is not limited to the above. For example, when the normal vector of the current node belongs to the normal vector group 2, the 3D encoding device can further select a codebook according to the normal vector group to which the current node's normal vector belongs, for example, using the codebook 2.

[0845] For example, the 3D data encoder selects a group of normal vectors using information about the normal vector of the current node. For example, the 3D data encoder determines, as an identical group of normal vectors, normal vectors having a distance between normal vectors that is less than or equal to a predetermined threshold value.

[0846] The information regarding the object to which the current node belongs may be information regarding, for example, a person, a vehicle, or a building.

[0847] Next, configurations of the 3D data encoding apparatus 1900 and the 3D data decoding apparatus 1910 according to the present embodiment will be described. Fig. 134 is a block diagram of a 3D data encoding device 1900 according to the present embodiment. The 3D encoder 1900 illustrated in FIG. 134 includes an octant tree generator 1901, a similarity information calculation module 1902, a codebook selection module 1903, and an entropy encoder 1904.

[0848] The octant tree builder 1901 generates, for example, an octant tree from the input 3D points and generates a fill code for each node included in the octant tree. The similarity information calculation unit 1902 obtains, for example, similarity information that is geometry information, structure information, or attribute information of the current node. Codebook selection module 1903 selects a context to be used for entropy encoding of the fill code according to the similarity information of the current node. An entropy encoder 1904 generates a bit stream by entropy encoding the stuffing code using the selected context. It should be noted that the entropy encoder 1904 may append, to the bitstream, information indicating the selected context.

[0849] FIG. 135 is a block diagram of a 3D data decoding apparatus 1910 according to the present embodiment. The 3D decoder 1910 illustrated in FIG. 135 includes an octant tree generator 1911, a similarity information calculation module 1912, a codebook selection module 1913, and an entropy decoder 1914.

[0850] The octant tree generator 1911 generates an octant tree, for example, in order from the bottom layer to the top layer, using the information obtained from the entropy decoder 1914. The similarity information calculation module 1912 obtains similarity information, which is geometric information, structure information, or attributes of the current node. Codebook selection module 1913 selects a context to be used for entropy encoding of the fill code according to the similarity information of the current node. Entropy decoder 1914 generates 3D points by entropy decoding the fill code using the selected context. It should be noted that the entropy decoder 1914 can obtain, by performing decoding, selected context information attached to the bit stream and use the context indicated by the information.

[0851] As illustrated in FIG. 122-124 above, the contexts are provided in the corresponding bits of the fill code. In other words, the 3D data encoding device entropy encodes a bit sequence representing an N-ary (N is an integer greater than or equal to 2) tree structure of 3D points included in the 3D data using a coding table selected from the coding tables. The bit sequence includes N-bit information for each node in the N-ary tree structure. The N-bit information includes N pieces of 1-bit information, each of which indicates whether or not a three-dimensional point is present at the corresponding one of the N child nodes relative to the corresponding node. In each of the codebooks, context is provided to each bit of N-bit information. The 3D encoder entropy encodes each bit of the N-bit information using the context provided per bit in the selected codebook.

[0852] This allows the 3D data encoder to improve coding efficiency by per-bit context selection.

[0853] For example, in entropy encoding, the 3D data encoder selects a coding table to be used from the coding tables based on whether or not a 3D point is present at each of the neighboring nodes relative to the current node. This allows the 3D encoding device to improve encoding efficiency by selecting a coding table based on whether or not a 3D point is present at an adjacent node.

[0854] For example, in entropy encoding, the 3D data encoding device (i) selects a coding table based on a layout pattern indicating the placement position of a neighbor node in which the 3D point is present from among the neighboring nodes, and (ii) selects an identical codebook for layout templates that become identical by rotation, from among the layout templates. This allows the 3D data encoder to reduce the increase in the number of codebooks.

[0855] For example, in entropy encoding, the 3D data encoding device selects a codebook to be used from the codebooks based on the layer to which the current node belongs. This allows the 3D data encoder to improve encoding efficiency by selecting a codebook based on the layer to which the current node belongs.

[0856] For example, in entropy encoding, the 3D data encoder selects a codebook to be used from the codebooks based on the normal vector of the current node. This enables the 3D encoding device to improve encoding efficiency by selecting a coding table based on a normal vector.

[0857] For example, the 3D data encoding device includes a processor and a memory, and the processor performs the above process using the memory.

[0858] The 3D data decoding apparatus entropy decodes a bit sequence representing an N-ary (N is an integer greater than or equal to 2) tree structure of 3D points included in the 3D data using a coding table selected from the coding tables. The bit sequence includes N-bit information for each node in the N-ary tree structure. The N-bit information includes N pieces of 1-bit information, each of which indicates whether or not a three-dimensional point is present at the corresponding one of the N child nodes relative to the corresponding node. In each of the codebooks, context is provided to each bit of N-bit information. The 3D data decoder entropy decodes each bit of the N-bit information using the context provided per bit in the selected codebook.

[0859] This allows the 3D data decoder to improve coding efficiency by selecting a context for each bit.

[0860] For example, in entropy decoding, the 3D data decoding apparatus selects a codebook to be used from codebooks based on whether or not a 3D point is present at each of the neighboring nodes relative to the current node. This enables the 3D data decoding apparatus to improve coding efficiency by selecting a coding table based on whether or not a 3D point is present at an adjacent node.

[0861] For example, in entropy decoding, the 3D data decoding apparatus (i) selects a codebook based on a layout pattern indicating the placement position of a neighbor node in which a 3D point is present from among the neighbor nodes, and (ii) selects an identical codebook for layout templates that become identical by rotation, from among the layout templates. This enables the 3D data decoding apparatus to reduce the increase in the number of codebooks.

[0862] For example, in entropy decoding, the 3D data decoding apparatus selects a codebook to be used from the codebooks based on the layer to which the current node belongs. This allows the 3D data decoding apparatus to improve coding efficiency by selecting a coding table based on the layer to which the current node belongs.

[0863] For example, in entropy decoding, the 3D data decoding apparatus selects a codebook to be used from the codebooks based on the normal vector of the current node. This enables the 3D data decoding apparatus to improve coding efficiency by selecting a coding table based on a normal vector.

[0864] For example, the 3D data decoding device includes a processor and a memory, and the processor performs the above process using the memory.

[0865] Embodiment 12

In the present embodiment, the link control method when the stuffing code is encoded is described below. It should be noted that although the following will mainly describe the operation of the 3D data encoding apparatus, the 3D data decoding apparatus may perform the same process.

[0866] FIG. 136 and FIG. 137 are a diagram illustrating a reference relationship according to the present embodiment. In particular, FIG. 136 is a diagram illustrating a reference relationship in an octant tree structure, and FIG. 137 is a diagram illustrating a reference relationship in a spatial domain.

[0867] In the present embodiment, when the 3D encoding device encodes encoding information of the current node to be encoded (hereinafter referred to as "current node"), the 3D encoding device refers to the encoding information of each node in the parent node to which the current node belongs. . However, in this regard, the 3D encoding device does not refer to the encoding information of each node in another node (hereinafter referred to as "parent neighbor") which is in the same layer as the parent node. In other words, the 3D encoder disables or disables the parent neighbor link.

[0868] It should be noted that the 3D data encoding device can resolve the reference to the encoding information of the parent node (hereinafter also referred to as "grandparent node") relative to the parent node. In other words, the 3D encoding apparatus can encode the encoding information of the current node with reference to the encoding information of each ancestor node and parent node to which the current node belongs.

[0869] Here, the encoding information, for example, is a padding code. When the 3D encoder encodes the fill code of the current node, the 3D encoder refers to information (hereinafter referred to as "fill information") indicating whether or not a point cloud is included in each node in the parent node to which the current node belongs. In other words, when the 3D encoder encodes the padding code of the current node, the 3D encoder refers to the padding code of the parent node. On the other hand, the 3D encoder does not refer to the padding information of each node in the parent neighbor. In other words, the 3D encoder does not refer to the padding code of the parent neighbor. In addition, the 3D data encoder can refer to the padding information of each node in the parent node. In other words, the 3D data encoder can refer to the padding information of each of the parent node and the parent neighbor node.

[0870] For example, when the 3D encoder encodes the padding code of the current node, the 3D encoder selects a codebook to be used to entropy encode the padding code of the current node using the padding code of the ancestor node or the parent node to which the current node belongs. It should be noted that the details are described below. At this time, the 3D encoder need not necessarily refer to the padding code of the parent neighbor. Since this enables the 3D data encoder, when encoding the fill code of the current node, to appropriately select the encoding table according to the fill code information of the parent node or the parent node, the 3D data encoder can improve encoding efficiency. In addition, by not referencing the neighbor, the 3D data encoding device can suppress the parent neighbor information checking process and reduce the information storage capacity. In addition, scanning the fill code of each node of the octant tree in depth order simplifies coding.

[0871] The following describes an example of selecting a codebook using a parent node padding code. Fig. 138 is a diagram illustrating an example of the current node and neighboring reference nodes. Fig. 139 is a diagram illustrating the relationship between a parent node and nodes. Fig. 140 is a diagram illustrating an example of the parent node filling code. Here, the adjacent reference node is a node that is referred to when the current node is encoded from among the nodes spatially adjacent to the current node. In the example shown in FIG. 138, neighboring nodes belong to the same layer as the current node. In addition, the X node adjacent to the current node in the X direction, the Y node adjacent to the current block in the Y direction, and the Z node adjacent to the current block in the Z direction are used as reference neighbor nodes. In other words, one neighbor node is set as the reference neighbor node in each of the X, Y, and Z directions.

[0872] It should be noted that the node numbers shown in FIG. 139 are one example, and the relationship between node numbers and node positions is not limited to the relationship shown in FIG. 139. Although node 0 is assigned to the lowest order bit and node 7 is assigned to the highest order bit in FIG. 140, appointments can be made in reverse order. In addition, each node can be assigned to any bit.

[0873] The 3D data encoder determines a codebook to be used when the 3D data encoder entropy encodes the fill code of the current node, for example, using the following equation.

[0874] CodingTable = (FlagX<<2)+(FlagY<<1)+(FlagZ)

[0875] Here, CodingTable indicates a coding table for the fill code of the current node, and indicates one of values ranging from 0 to 7. FlagX represents the padding information of neighbor node X. FlagX indicates 1 when neighbor node X includes a point cloud (is busy), and indicates 0 when it does not turn on. FlagY is the filling information of neighbor Y. FlagY indicates 1 when neighbor Y includes the point cloud (is busy), and indicates 0 when it does not. FlagZ is the filling information of neighbor Z. FlagZ indicates 1 when neighbor Z includes the point cloud (is busy), and indicates 0 when it does not.

[0876] It should be noted that since information indicating whether or not a neighbor is busy is included in the padding code of the parent node, the 3D data encoding device can select a codebook using the value indicated by the padding code of the parent node.

[0877] From the above, the 3D data encoding apparatus can improve encoding efficiency by selecting a coding table using information indicating whether or not a neighboring node relative to the current node includes a point cloud.

[0878] In addition, as illustrated in FIG. 138, the 3D encoder may select an adjacent reference node according to the spatial position of the current node in the parent node. In other words, the 3D encoder can select a neighbor node to be referred to from neighbor nodes according to the spatial position of the current node in the parent node.

[0879] The following describes configuration examples of the 3D data encoding apparatus and the 3D data decoding apparatus. Fig. 141 is a block diagram of a 3D data encoding apparatus 2100 according to the present embodiment. The 3D encoding device 2100 illustrated in FIG. 141 includes an octant tree generator 2101, a geometry information calculation unit 2102, a codebook selection unit 2103, and an entropy encoder 2104.

[0880] The octant tree builder 2101 generates, for example, an octant tree from inputted 3D points (point clouds) and generates a fill code for each node included in the octant tree. The geometric information calculation unit 2102 obtains padding information indicating whether or not the adjacent reference node relative to the current node is occupied. For example, the geometric information calculation module 2102 obtains the padding information of the adjacent reference node from the padding code of the parent node to which the current node belongs. It should be noted that, as illustrated in FIG. 138, the geometric information calculation module 2102 may select an adjacent reference node according to the position of the current node in the parent node. In addition, the geometric information calculation module 2102 does not refer to the padding information of each node in the parent neighbor node.

[0881] Codebook selection module 2103 selects a codebook to be used for entropy encoding the fill code of the current node using the neighbor reference node's fill information calculated by geometric information calculation module 2102. An entropy encoder 2104 generates a bitstream by entropy encoding the stuffing code using the selected codebook. It should be noted that the entropy encoder 2104 may append, to the bitstream, information indicating the selected codebook.

[0882] FIG. 142 is a block diagram of a 3D data decoding apparatus 2110 according to the present embodiment. The 3D data decoding apparatus 2110 illustrated in FIG. 142 includes an octant tree generator 2111, a geometry information calculation unit 2112, a codebook selection unit 2113, and an entropy decoder 2114.

[0883] The octant tree builder 2111 generates an octant tree of space (nodes) using bitstream header information, etc. The octant tree builder 2111 generates an octant tree, for example, by generating a large space (root node) using the size of the space along the X-axis, Z-axis, and Y-axis directions attached to the header information, and forming eight small spaces A (nodes A0 -A7) by dividing space into two along each of the X-axis, Z-axis, and Y-axis directions. Nodes A0-A7 are set as the current node in the sequence.

[0884] The geometry information calculation unit 2112 obtains padding information indicating whether or not a neighboring reference node relative to the current node is busy. For example, the geometric information calculation module 2112 obtains the padding information of the adjacent reference node from the padding code of the parent node to which the current node belongs. It should be noted that, as illustrated in FIG. 138, the geometric information calculation module 2112 may select an adjacent reference node according to the position of the current node in the parent node. In addition, the geometry information calculation module 2112 does not refer to the padding information of each node in the parent neighbor node.

[0885] The codebook selection unit 2113 selects a codebook (decode table) to be used for entropy decoding of the current node's fill code using the neighbor reference node's padding information calculated by the geometric information calculation unit 2112. Entropy decoder 2114 generates 3D points by entropy decoding the fill code using the selected codebook. It should be noted that the codebook selection unit 2113 may obtain, by performing decoding, information of the selected codebook attached to the bit stream, and the entropy decoder 2114 may use the codebook indicated by the acquired information.

[0886] Each bit of the fill code (8 bits) included in the bit stream indicates whether or not the corresponding one of the eight small spaces A (nodes A0-A7) includes a point cloud. In addition, the 3D data decoding apparatus generates an octant tree by dividing the small space node A0 into eight small B spaces (B0 to B7 nodes), and obtains information indicating whether or not each small space B node includes a point cloud by decoding the code filling. Thus, the 3D data decoding apparatus decodes the padding code of each node when forming an octant tree by dividing a large space into small spaces.

[0887] The following describes procedures for processes performed by the 3D data encoding apparatus and the 3D data decoding apparatus. Fig. 143 is a flowchart for a 3D data encoding process in a 3D data encoding apparatus. First, the 3D data encoding device determines (sets) a space (current node) including part or all of the inputted 3D point cloud (S2101). Then, the 3D data encoding apparatus generates eight small spaces (nodes) by dividing the current node into eight (S2102). Thereafter, the 3D data encoding device generates a fill code for the current node according to whether or not each node includes a point cloud (S2103).

[0888] Thereafter, the 3D data encoding apparatus calculates (obtains) padding information of the adjacent reference node with respect to the current node from the padding code of the parent node with respect to the current node (S2104). Then, the 3D data encoding device selects a codebook to be used for entropy encoding based on the calculated neighbor reference node population information with respect to the current node (S2105). Thereafter, the 3D data encoding device entropy encodes the fill code of the current node using the selected codebook (S2106).

[0889] Finally, the 3D data encoding apparatus repeats the process of dividing each node into eight and encoding the node fill code until the node can be separated (S2107). In other words, steps S2102-S2106 are recursively repeated.

[0890] FIG. 144 is a flowchart for a 3D data decoding process in a 3D data decoding apparatus. First, the 3D data decoding apparatus determines (sets) a space (current node) to be decoded using bitstream header information (S2111). Then, the 3D data decoding apparatus generates eight small spaces (nodes) by dividing the current node into eight (S2112). Thereafter, the 3D data decoding apparatus calculates (obtains) padding information of the adjacent reference node with respect to the current node from the padding code of the parent node with respect to the current node (S2113).

[0891] Thereafter, the 3D data decoding apparatus selects a codebook to be used for entropy decoding based on the population information of the adjacent reference node (S2114). Then, the 3D data decoding apparatus entropy decodes the fill code of the current node using the selected codebook (S2115).

[0892] Finally, the 3D data decoding apparatus repeats the process of dividing each node into eight and decoding the node fill code until the node can be separated (S2116). In other words, steps S2112-S2115 are recursively repeated.

[0893] The following describes an example of selecting a coding table. Fig. 145 is a diagram illustrating an example of selecting a coding table. For example, similar to codebook 0 shown in FIG. 145, the same context mode may be applied to padding codes. In addition, a different context model may be assigned to each padding code. Since this allows a context model to be assigned according to the probability of occurrence of a padding code, coding efficiency can be improved. In addition, a context mode can be used that updates the probability table according to the frequency of occurrence of the fill code. Alternatively, a context model having a fixed probability table may be used.

[0894] It should be noted that although the coding tables illustrated in FIG. 119 and FIG. 120 are used in the example shown in FIG. 145, the coding tables illustrated in FIG. 122 and FIG. 123.

[0895] Variation 1 of the present embodiment is described hereinafter. Fig. 146 is a diagram illustrating the reference relationship in the present variation. Although the 3D encoder does not refer to the padding code of the parent neighbor in the above embodiment, the 3D encoder can switch whether or not to refer to the padding code of the parent neighbor according to a particular condition.

[0896] For example, when a 3D encoder encodes an octant tree while scanning the octant tree in breadth, the 3D encoder encodes the padding code of the current node with reference to the padding information of the node at the parent neighbor. On the contrary, when the 3D data encoder encodes the octant tree while scanning the octant tree in depth, the 3D data encoder disallows reference to node padding information at the parent neighbor. By appropriately selecting the link-enabled node according to the scan order (encoding order) of the octant tree nodes in the above manner, it is possible to improve the coding efficiency and reduce the processing load.

[0897] It should be noted that the 3D data encoder may append, to the header of the bitstream, information indicating, for example, whether the octant tree is being encoded in width or in depth. Fig. 147 is a diagram illustrating an example of the header information syntax in this case; octree_scan_order shown in FIG. 147 is coding order information (coding order flag) indicating the coding order for the octant tree. For example, when octree_scan_order is 0, "width" is specified, and when octree_scan_order is 1, "deep" is specified. Since this enables the 3D decoder to determine whether or not the bitstream is width or depth encoded with reference to octree_scan_order, the 3D decoder can properly decode the bitstream.

[0898] In addition, the 3D data encoder may append, to the header information of the bitstream, information indicating whether or not to prohibit the reference to the parent neighbor. Fig. 148 is a diagram illustrating an example of the header information syntax in this case; limit_refer_flag is prohibition switching information (prohibition switching flag) indicating whether or not to prohibit a parent neighbor reference. For example, when limit_refer_flag is 1, parent neighbor reference prohibition is specified, and when limit_refer_flag is 0, reference restriction (parent neighbor reference allowed) is not specified.

[0899] In other words, the 3D encoder determines whether or not to disable the parent neighbor link, and selects whether to disable or enable the parent neighbor link based on the result of the above determination. In addition, the 3D data encoding device generates a bit stream including barring switch information that indicates a determination result and indicates whether or not to forbid the parent neighbor reference.

[0900] The 3D data decoder obtains, from the bit stream, barring switch information indicating whether or not to disable the parent neighbor link, and selects whether or not to disable the parent neighbor link based on the barring switch information.

[0901] This allows the 3D data encoder to control the parent neighbor reference and generate a bitstream. It also allows the 3D decoder to obtain, from the header of the bitstream, information indicating whether or not to disable the parent neighbor reference.

[0902] Although the padding code encoding process is described as an example of an encoding process in which parent neighbor reference is prohibited in the present embodiment, the present disclosure is not necessarily limited thereto. For example, the same method may be applied when other octant tree node information is encoded. For example, the method of the present embodiment may be applied when other attribute information such as color, normal vector, or reflection degree added to a node is encoded. Additionally, the same method may be applied when a codebook or a predicted value is encoded.

[0903] Hereinafter, variation 2 of the present embodiment is described in this document. In the above description, as illustrated in FIG. 138, an example is given in which three reference neighbors are used, but four or more reference neighbors may be used. Fig. 149 is a diagram illustrating an example of the current node and neighboring reference nodes.

[0904] For example, the 3D data encoder calculates a codebook to be used when the 3D data encoder entropy encodes the padding code of the current node shown in FIG. 149 using the following equation.

[0905] CodingTable = (FlagX0 << 3) + (FlagX1 << 2) + (FlagY << 1) + (FlagZ)

[0906] Here, CodingTable indicates a coding table for the fill code of the current node, and indicates one of values ranging from 0 to 15. FlagXN represents the fill information of the adjacent node XN (N=0...1). FlaxXN indicates 1 when neighbor XN includes the point cloud (is busy) and indicates 0 when it does not. FlagY is the filling information of neighbor Y. FlagY indicates 1 when neighbor Y includes the point cloud (is busy), and indicates 0 when it does not. FlagZ is the filling information of neighbor Z. FlagZ indicates 1 when neighbor Z includes the point cloud (is busy), and indicates 0 when it does not.

[0907] At this time, when the neighbor node, for example, the neighbor node X0 in FIG. 149 is non-referenceable (referentially prohibited), the 3D encoder may use a fixed value such as 1 (busy) or 0 (idle) as a substitute value.

[0908] FIG. 150 is a diagram illustrating an example of the current node and neighboring reference nodes. As illustrated in FIG. 150, when the neighbor node is non-referenceable (referentially denied), the neighbor node's padding information may be calculated with reference to the padding code of the parent node with respect to the current node. For example, the 3D data encoder may calculate FlagX0 in the above equation using the padding information of the neighbor node G0 instead of the neighbor node X0 illustrated in FIG. 150, and may determine the value of the encoding table using the calculated FlagX0. Note that the neighbor node G0 illustrated in FIG. 150 is a neighbor node whose fill or fall can be determined using the fill code of the parent node. Neighbor node X1 is a neighbor node that can be filled or unfilled using the parent node's fill code.

[0909] Variation 3 of the present embodiment is described hereinafter. Fig. 151 and FIG. 152 are a diagram illustrating a reference relationship according to the present variation. In particular, FIG. 151 is a diagram illustrating a reference relationship in an octant tree structure, and FIG. 152 is a diagram illustrating a reference relationship in a spatial domain.

[0910] In the present variation, when the 3D encoding device encodes the encoding information of the current node to be encoded (hereinafter referred to as "current node 2"), the 3D encoding device refers to the encoding information of each node in the parent node to which the current node 2 belongs. In other words, the 3D encoder resolves the reference to information (for example, padding information) of the child node with respect to the first node, among neighboring nodes, which has a parent node identical to the parent node with respect to the current node.For example, when the 3D encoder encodes a code filling the current node 2 illustrated in Fig. 151, the 3D data encoding device refers to the node filling code in the parent node to which the current node 2 belongs, for example, the current node illustrated in Fig. 151. As illustrated in Fig. 152, the filling codethe current node illustrated in FIG. 151 indicates, for example, whether or not each node in the current node adjacent to the current node 2 is occupied. Accordingly, since the 3D encoder can select a coding table for the fill code of the current node 2 according to a more specific form of the current node, the 3D data encoding device can improve the encoding efficiency.

[0911] The 3D data encoder can calculate a codebook to be used when the 3D data encoder entropy encodes the fill code of the current node 2, for example, using the following equation.

[0912] CodingTable = (FlagX1<<5)+(FlagX2<<4)+(FlagX3<<3)+(FlagX4<<2)+(FlagY<<1)+(FlagZ)

[0913] Here, CodingTable indicates a coding table for the padding code of the current node 2, and indicates one of values ranging from 0 to 63. FlagXN is the padding information of the neighboring node XN (N=1...4). FlagXN indicates 1 when neighbor XN includes the point cloud (is busy) and indicates 0 when it does not. FlagY is the filling information of neighbor Y. FlagY indicates 1 when neighbor Y includes the point cloud (is busy), and indicates 0 when it does not. FlagZ is the filling information of neighbor Z. FlagZ indicates 1 when neighbor Z includes the point cloud (is busy), and indicates 0 when it does not.

[0914] It should be noted that the 3D data encoder may change the encoding table calculation method according to the node position for the current node 2 in the parent node.

[0915] When the reference to the parent neighbor is not prohibited, the 3D encoding device can refer to the encoding information of each node in the parent neighbor. For example, when the parent neighbor link is not prohibited, the link to the information (eg, padding information) of the child node of the third node has a parent node different from the parent node with respect to the current node. In the example illustrated in FIG. 150, for example, the 3D encoder obtains the padding information of the child node with respect to the neighbor node X0 with reference to the padding code of the neighbor node X0 having a parent node different from the parent node with respect to the current node. The 3D data encoding device selects a codebook to be used for entropy encoding the padding code of the current node based on the received padding information of the child node with respect to the neighboring node X0.

[0916] As mentioned above, the 3D data encoding device according to the present embodiment encodes information (eg, padding code) of the current node included in the N-ary tree structure of 3D points included in the 3D data, where N is an integer greater than or equal to 2. As illustrated in FIG. 136 and FIG. 137, when encoding, the 3D data encoding apparatus enables reference to information (e.g., padding information) of a first node included in neighboring nodes spatially adjacent to the current node, and disallows reference to information of a second node included in neighboring nodes, wherein the first node has a parent node identical to the parent node relative to the current node, wherein the second node has a parent node different from the parent node relative to the current node. In other words, when encoding, the 3D data encoder enables the reference to information (eg, padding code) of the parent node and disallows the reference to information (eg, padding code) of another node (parent neighbor) in the same layer as the parent node.

[0917] Because of this, the 3D data encoding device can improve encoding efficiency with reference to the information of the first node included in neighbor nodes spatially adjacent to the current node, the first node having a parent node identical to the parent node with respect to the current node. In addition, the 3D data encoder can reduce the amount of processing by not referring to information of the second node included in the neighboring nodes, the second node having a parent node different from the parent node with respect to the current node. Thus, the 3D data encoding apparatus can not only improve the encoding efficiency, but also reduce the amount of processing.

[0918] For example, the 3D data encoder further determines whether or not to prohibit the link to the information of the second node. When encoding, the 3D data encoding device selects whether to prohibit or enable the link to the second node information based on the determination result. In addition, the 3D data encoding device generates a bit stream including prohibition switching information (eg, limit_refer_flag shown in FIG. 148) that indicates a determination result and indicates whether or not the reference to the second node information should be prohibited.

[0919] Because of this, the 3D data encoder can choose whether or not to prohibit the link to the information of the second node. In addition, the 3D data decoding apparatus can appropriately perform a decoding process using the inhibit switching information.

[0920] For example, the current node information is information (eg, fill code) that indicates whether or not a three-dimensional point is present in each of the child nodes belonging to the current node. The first node information is information (first node filling information) that indicates whether or not a 3D point is present at the first node. The second node information is information (second node filling information) that indicates whether or not a 3D point is present at the second node.

[0921] For example, when encoding, the 3D data encoding device selects a coding table based on whether or not a 3D point is present at the first node, and entropy encodes the information (eg, padding code) of the current node using the selected coding table.

[0922] For example, as illustrated in FIG. 151 and FIG. 152, when encoding, the 3D data encoding device resolves the reference to information (eg, padding information) of the child node with respect to the first node, the child node being included in neighboring nodes.

[0923] Because of this, since the 3D data encoder provides a link to more detailed neighbor information, the 3D data encoder can improve encoding efficiency.

[0924] For example, as illustrated in FIG. 138, when encoding, the 3D data encoding device selects a neighbor node to be referred to from neighboring nodes according to the spatial position of the current node in the parent node.

[0925] Because of this, the 3D data encoder can refer to the corresponding neighbor node according to the spatial position of the current node in the parent node.

[0926] For example, the 3D data encoding device includes a processor and a memory, and the processor performs the above process using the memory.

[0927] The 3D data decoding apparatus according to the present embodiment decodes the information (eg, padding code) of the current node included in the N-ary tree structure of 3D points included in the 3D data, where N is an integer greater than or equal to 2. As illustrated in fig. 136 and FIG. 137, when decoding, the 3D data decoding apparatus enables reference to information (for example, padding information) of a first node included in neighbor nodes spatially adjacent to the current node, and disallows reference to information of a second node included in neighbor nodes, wherein the first node has a parent node identical to the parent node relative to the current node, wherein the second node has a parent node different from the parent node relative to the current node. In other words, when decoding, the 3D decoder enables the reference to information (eg, padding code) of the parent node and disallows the reference to information (eg, padding code) of another node (parent neighbor) in the same layer as the parent node.

[0928] Because of this, the 3D data decoding apparatus can improve encoding efficiency with reference to the information of the first node included in neighbor nodes spatially adjacent to the current node, the first node having a parent node identical to the parent node with respect to the current node. In addition, the 3D data decoding apparatus can reduce the amount of processing by not referring to the information of the second node included in the neighboring nodes, the second node having a parent node different from the parent node with respect to the current node. Thus, the 3D data decoding apparatus can not only improve the encoding efficiency, but also reduce the amount of processing.

[0929] For example, the 3D data decoding apparatus further obtains, from the bit stream, prohibition switching information (eg, limit_refer_flag shown in FIG. 148) indicating whether or not the second node information reference should be prohibited. When decoding, the 3D data decoding apparatus selects whether to prohibit or enable the link to the second node information based on the inhibit switching information.

[0930] Because of this, the 3D data decoding apparatus can appropriately perform the decoding process using the inhibit switching information.

[0931] For example, the current node information is information (eg, fill code) that indicates whether or not a three-dimensional point is present in each of the child nodes belonging to the current node. The first node information is information (first node filling information) that indicates whether or not a 3D point is present at the first node. The second node information is information (second node filling information) that indicates whether or not a 3D point is present at the second node.

[0932] For example, when decoding, the 3D data encoding device selects a coding table based on whether or not a 3D point is present at the first node, and entropy decodes information (eg, fill code) of the current node using the selected coding table.

[0933] For example, as illustrated in FIG. 151 and FIG. 152, when decoding, the 3D data decoding apparatus resolves the reference to information (eg, padding information) of the child node with respect to the first node, the child node being included in neighboring nodes.

[0934] Because of this, since the 3D data decoding apparatus provides a link to more detailed neighbor information, the 3D data decoding apparatus can improve encoding efficiency.

[0935] For example, as illustrated in FIG. 138, when decoding, the 3D data decoding apparatus selects a neighbor node to be referred to from neighboring nodes according to the spatial position of the current node in the parent node.

[0936] Because of this, the 3D data decoding device can refer to the corresponding neighbor node according to the spatial position of the current node in the parent node.

[0937] For example, the 3D data decoding device includes a processor and a memory, and the processor performs the above process using the memory.

[0938] Embodiment 13

Although the following will mainly describe the operation of the 3D data encoding apparatus, the 3D data decoding apparatus can perform the same process.

[0939] In the present embodiment, if the 3D encoding device encodes the inputted 3D point cloud (point cloud) using an octant tree structure, when the 3D encoding device repeats the division until each leaf included in the tree octants, does not have one 3D point, and performs encoding, the 3D data encoding apparatus appends, to the bit stream, mode information indicating whether each leaf of the octant tree includes one 3D point or one or more 3D points. In addition, when the mode information is true (each leaf of the octant tree includes one 3D point), the 3D data encoding device does not encode leaf information on the sheet, and when the mode information is false (each leaf of the octant tree includes one or more 3D dots), the 3D data encoding device encodes sheet information.

[0940] Here, the sheet information includes, for example, information indicating how many 3D points the sheet includes, information indicating relative coordinates, and so on. a 3D point included in the sheet as illustrated in FIG. 61-67, or both. Accordingly, since it is not necessary to encode leaf information for each leaf of the octant tree, when a leaf includes one 3D point, encoding efficiency can be improved. When each leaf of the octant tree includes one or more 3D points, the 3D encoder appropriately encodes and appends the leaf information to the bitstream. This enables the 3D data decoding apparatus to correctly reconstruct one or more 3D points included in the sheet using the sheet information.

[0941] FIG. 153 is a diagram illustrating an example of the bitstream header information syntax according to the present embodiment. This header information is, for example, WLD, SPC, or VLM; single_point_per_leaf shown in FIG. 153 is information indicating whether each leaf of the octant tree includes one 3D point or one or more 3D points. Here, "each leaf of the octant tree includes one 3D point" means that all leaves included in the octant tree include one 3D point (i.e., there are no leaves including two or more 3D points). It should be noted that the octant tree here, for example, is the unit in which the mode information is attached, and corresponds to, for example, WLD, SPC, or VLM.

[0942] In addition, "each leaf of the octant tree includes one or more 3D points" means that at least one of the leaves included in the octant tree includes two or more 3D points. In other words, some leaves may include two or more 3D points, and the remaining leaves may include one 3D point.

[0943] For example, that the mode information value is 1 indicates that each sheet includes one 3D point, and that the mode information value is 0 indicates that each sheet includes one or more 3D points. points. It should be noted that even when all leaves included in the octant tree include one 3D point, the 3D encoder may set the mode information (single_point_per_leaf) to 0.

[0944] FIG. 154 is a diagram illustrating an octant tree configuration example when the mode information indicates 1. As illustrated in FIG. 154, when the mode information indicates 1, each sheet includes one 3D point. Fig. 155 is a diagram illustrating an octant tree configuration example when mode information indicates 0. As illustrated in FIG. 155, when the mode information indicates 0, each sheet includes one 3D point or two or more 3D points.

[0945] It should be noted that when the sheet includes two or more 3D points, for example, the coordinates at the two or more 3D points included in the sheet are different from each other. Alternatively, the coordinates at two or more 3D points included in a sheet are identical, and pieces of attribute information such as color or reflectance (ie, types of information) are mutually different. Alternatively, both the coordinates of two or more 3D points included in a sheet and the pieces of attribute information may be mutually different.

[0946] The following describes an example of the syntax of the sheet information. It should be noted that although the above is an example in which the mode information is attached to the header of the bit stream, the mode information does not need to be attached to the header, and the standards or profile or level, etc. standards may indicate whether each leaf of the octant tree includes one 3D point or one or more 3D points. In this case, the 3D data decoding apparatus can recover the bitstream correctly, for example, by determining whether each leaf of the octant tree includes one 3D point or one or more 3D points, with reference to the standards information included in the bitstream.

[0947] FIG. 156 is a diagram illustrating an example of the information syntax of each node included in the octant tree; isleaf shown in FIG. 156 is a flag indicating whether or not the node is a leaf. The fact that isleaf is 1 indicates that the node is a leaf. The fact that isleaf is 0 indicates that the node is not a leaf.

[0948] It should be noted that information indicating whether or not a node is a leaf need not necessarily be attached to the header. In this case, the 3D data decoding apparatus determines whether or not the node is a sheet using another method. For example, the 3D decoder may determine whether or not each node of the octant tree is divided into the smallest possible size, and may determine that the node is a leaf when determining that each node is divided into the smallest possible size. This eliminates the need to encode a flag indicating whether or not the node is a leaf, thus reducing the amount of header code.

[0949] Num_point_per_leaf shown in FIG. 156 is sheet information and indicates the number of 3D points included in the sheet; num_point_per_leaf is encoded when single_point_per_leaf == 0 and not encoded when single_point_per_leaf == 1.

[0950] It should be noted that the 3D encoder may entropy encode num_point_per_leaf. At this time, the 3D data encoding apparatus can also perform encoding by switching codebooks. For example, a 3D encoder may perform arithmetic encoding for the first bit of num_point_per_leaf using an encoding table A, and may perform arithmetic encoding for the remaining bits using an encoding table B.

[0951] As mentioned above, the 3D data encoder may append, to the bitstream header, mode information indicating whether each leaf of the octant tree includes one 3D point or one or more 3D points, and may select whether or not to encode sheet information (information indicating the number of 3D points included in a sheet) according to the value of the mode information. In addition, the 3D data encoding apparatus may encode, as the sheet information, the positional information of one or one or more 3D points included in the sheet.

[0952] It should be noted that the 3D data encoding device can entropy encode the single_point_per_leaf, isleaf, and num_point_per_leaf generated by the method described above. For example, the 3D data encoder binarizes each value and performs arithmetic encoding on the binarized value.

[0953] Although the octant tree structure is given as an example in the present embodiment, the present disclosure is not necessarily limited thereto. The above procedure can be applied to an N-ary tree structure such as a quadtree and a hexadecant tree.

[0954] In addition, when the 3D data encoding device encodes, as sheet information, pieces of positional information of two or more 3D points in the same sheet, the 3D data encoding device can also encode pieces of attribute information (color, reflectance, etc.). ) of two or more 3D points. In this case, the position information pieces of two or more 3D points and their attribute information pieces may be associated with each other. For example, when the 3D data encoding device encodes, as sheet information, the position information pieces of points A and B in the same sheet, the 3D data encoding device can encode the point A and B attribute information pieces and append the attribute information pieces to the bit stream. In other words, the leaf information may include point A position information, point A attribute information, point B position information, and point B attribute information. Pieces of point A and B position information may be associated with pieces of attribute information.

[0955] In addition, the 3D data encoding device may round at least M pieces of position information of at least M 3D points in the same sheet, and may encode at least M pieces of position information as N pieces of position information. N 3D points, where N is less than M. In this case, the 3D data encoding device may round at least M pieces of attribute information of at least M 3D points, for example, by averaging, to form N pieces of attribute information, and may encode N pieces of generated attribute information. For example, the 3D data encoding device may round the position information pieces of points A and B in the same sheet to a single point position information piece, and may encode the position information piece. In this case, the 3D data encoding apparatus may round the pieces of attribute information of points A and B, for example, by averaging, to calculate a piece of attribute information of one point, and may encode a piece of the calculated attribute information.

[0956] The following describes the procedures performed by the 3D data encoding apparatus and the 3D data decoding apparatus according to the present embodiment. Fig. 157 is a flowchart for a 3D data encoding process performed by a 3D data encoding device according to the present embodiment. First, the 3D data encoding apparatus determines whether to perform encoding such that each leaf of the octant tree includes one 3D point or perform encoding such that each leaf of the octant tree includes one or more 3D points (S2201 ). For example, the 3D data encoding device may determine whether to perform encoding such that each leaf of the octant tree includes one 3D point, or perform encoding such that each leaf of the octant tree includes one or more 3D points, according to whether or not to losslessly encode the input 3D point cloud. For example, when a lossless 3D data encoder encodes an inputted 3D point cloud, the 3D data encoder determines to perform encoding such that each leaf of the octant tree includes one 3D point. Alternatively, the 3D data encoding device may determine whether to perform encoding such that each leaf of the octant tree includes one 3D point, or perform encoding such that each leaf of the octant tree includes one or more 3D points, according to , whether or not all coordinates of the inputted 3D points are mutually different, and whether or not to perform encoding such that each of the 3D points is included in a different sheet. For example, when the 3D points include 3D points having identical coordinates, the 3D data encoder may determine to perform encoding such that each leaf of the octant tree includes one or more 3D points.

[0957] When the 3D data encoder determines to perform encoding such that each leaf of the octant tree includes one 3D point ("Yes" in S2201), the 3D data encoder sets the mode information as a value indicating that each leaf the octant tree includes one 3D point (single_point_per_leaf=1), and appends mode information to the header (S2202).

[0958] When the 3D encoding device determines to perform encoding such that each leaf of the octant tree includes one or more 3D points ("No" in S2201), the 3D encoding device sets the mode information as a value indicating that each leaf of the octant tree includes one or more 3D points (single_point_per_leaf=0), and appends mode information to a header (S2203).

[0959] Next, the 3D data encoding device generates an octant tree structure by dividing the root node into an octant tree (S2204). At this time, when the mode information indicates that each leaf of the octant tree should include one 3D point, the 3D data encoding apparatus generates an octant tree in which each leaf of the octant tree includes one 3D point, and when the mode information indicates that each leaf of the octant tree should include one or more 3D points, the 3D data encoder generates an octant tree in which each leaf of the octant tree includes one or more 3D points.

[0960] Thereafter, the 3D data encoding device selects the current node to be processed and determines whether or not the current node is a sheet (S2205). When the current node is a leaf ("Yes" in S2205) and the mode information indicates that each leaf of the octant tree should include one or more 3D points (single_point_per_leaf=0) ("Yes" in S2206), the 3D encoder data encodes sheet information indicating, for example, the number of 3D points included in the sheet (S2207).

[0961] In contrast, when the current node is a leaf ("Yes" in S2205) and the mode information indicates that each leaf of the octant tree should include one 3D point (single_point_per_leaf=1) ("No" in S2206), the 3D encoding device does not encode sheet information indicating, for example, the number of 3D points included in a sheet.

[0962] When the current node is a leaf ("No" in S2205), the 3D encoder encodes the fill code of the current node (S2208).

[0963] When the processing of all nodes is not completed ("No" in S2209), the 3D encoder selects the next current node, and executes step S2205 and the following steps for the selected current node. When the processing of all nodes is completed ("Yes" in S2209), the 3D data encoding device terminates the process.

[0964] FIG. 158 is a flowchart for a 3D data decoding process performed by a 3D data decoding apparatus according to the present embodiment. First, the 3D data decoding apparatus decodes the mode information (single_point_per_leaf) in the bitstream header (S2211).

[0965] Next, the 3D data decoding apparatus generates a space (node) octant tree using, for example, header information included in the bitstream (S2212). For example, the 3D data decoding apparatus generates a large space (root node) using the size of the space along the X-axis, Z-axis, and Y-axis directions attached to the header information. Then, the 3D data decoding apparatus generates an octant tree by forming eight small spaces (A0-A7 nodes) by dividing the space into two along each of the X-axis, Z-axis, and Y-axis directions. Similarly, the 3D data decoding apparatus divides each of nodes A0-A7 into eight small spaces. In addition, the 3D data decoding apparatus performs decoding of the fill code of each node and decoding of leaf information sequentially through the process illustrated in FIG. 158.

[0966] Specifically, the 3D data decoding apparatus selects a current node to be processed and determines whether or not the current node is a sheet (S2213). When the current node is a leaf ("Yes" in S2213) and the mode information indicates that each leaf of the octant tree should include one or more 3D points (single_point_per_leaf=0) ("Yes" in S2214), the 3D decoder data decodes sheet information indicating, for example, the number of 3D points included in the sheet (S2215).

[0967] In contrast, when the current node is a leaf ("Yes" in S2213) and the mode information indicates that each leaf of the octant tree should include one 3D point (single_point_per_leaf=1) ("No" in S2214), the 3D data decoding apparatus does not decode sheet information indicating, for example, the number of 3D points included in a sheet.

[0968] When the current node is not a leaf ("No" in S2213), the 3D data decoding apparatus decodes the fill code of the current node (S2216).

[0969] When the processing of all nodes is not completed ("No" in S2217), the 3D data decoding apparatus selects the next current node, and performs step S2213 and the following steps |for the selected current node. When the processing of all nodes is completed ("YES" in S2217), the 3D data decoding apparatus terminates the process.

[0970] Next, the configurations of the 3D data encoding apparatus and the 3D data decoding apparatus according to the present embodiment will be described. Fig. 159 is a block diagram illustrating the configuration of a 3D data encoding device 2200 according to the present embodiment. The 3D encoder 2200 illustrated in FIG. 159 includes an octant tree generator 2201, a mode determination module 2202, and an entropy encoder 2203.

[0971] The octant tree builder 2201 generates, for example, an octant tree from inputted 3D points (point clouds) and generates a corresponding one of the fill code and leaf information for each node included in the octant tree. The mode determining unit 2202 determines whether to perform encoding such that each leaf of the octant tree includes one 3D point or perform encoding such that each leaf of the octant tree includes one or more 3D points, and generates mode information, indicating the result of the determination. In other words, mode determination module 2202 sets the value to single_point_per_leaf.

[0972] The entropy encoder 2203 encodes leaf information according to mode information to generate a bitstream. Additionally, entropy encoder 2203 appends leaf information (single_point_per_leaf) to the bitstream.

[0973] FIG. 160 is a block diagram illustrating the configuration of a 3D data decoding apparatus 2210 according to the present embodiment. The 3D data decoding apparatus 2210 illustrated in FIG. 160 includes an octant tree generator 2211, a mode information decoder 2212, and an entropy decoder 2213.

[0974] The octant tree builder 2211 generates an octant tree of space (nodes) using, for example, bitstream header information. For example, the octant tree generator 2211 generates a large space (root node) using the size of the space along the X-axis, Z-axis, and Y-axis directions attached to the header information, and generates an octant tree by generating eight small spaces (nodes A0-A7 ) by dividing the space into two along each of the X-axis, Z-axis, and Y-axis directions. Similarly, the octant tree generator 2211 divides each of the nodes A0-A7 into eight small spaces. As stated above, the octant tree builder 2211 repeats the generation of the octant tree.

[0975] The mode information decoder 2212 decodes the mode information (single_point_per_leaf) from the bitstream header information. It should be noted that the mode information decoder 2212 may be included in the entropy decoder 2213.

[0976] The entropy decoder 2213 decodes the fill code and the sheet information according to the decoded mode information, and generates 3D points using the fill code and the decoded sheet information.

[0977] As mentioned above, the 3D data encoding device according to the present embodiment performs the process illustrated in FIG. 161. First, the 3D data encoding apparatus appends, to the bit stream, first information (mode information) indicating whether a sheet to be included in the N-ary tree structure of 3D points included in the 3D data should include one 3D point, or two or more 3D points, where N is an integer greater than or equal to 2 (S2221). In other words, the 3D encoding device encodes the first information.

[0978] When the first information indicates that the sheet is to include one 3D point ("Yes" in S2222), the 3D data encoding apparatus generates an N-ary tree structure in which the sheet includes one 3D point (S2223), and encodes an N-ary tree structure (S2224).

[0979] On the contrary, when the first information indicates that the sheet is to include two or more 3D points, the 3D data encoding device generates an N-ary tree structure in which the sheet includes two or more 3D points (S2225), and encodes an N-ary tree structure (S2226).

[0980] Because of this, the 3D data encoding device can selectively use a tree structure in which a sheet includes one 3D point and a tree structure in which a sheet includes two or more 3D points. Accordingly, the 3D data encoding apparatus can improve encoding efficiency.

[0981] For example, when the first information indicates that the sheet is to include two or more 3D dots, the 3D data encoding apparatus appends the second information regarding the sheet (sheet information). When the first information indicates that the sheet is to include one 3D point, the 3D encoder does not append the second information to the bitstream.

[0982] Because of this, the 3D data coding apparatus can improve the coding efficiency by not attaching the second information to the bitstream when the sheet is to include one 3D point.

[0983] For example, the second information indicates the total number of 3D points included in the sheet.

[0984] For example, the first information is usually used through leaves. For example, the first information is typically used by all or part of the leaves to be included in an N-ary tree structure. In other words, the first information indicates whether each of the leaves to be included in the N-ary tree structure should include one 3D point or two or more 3D points. When the first information indicates that each of the leaves should include one 3D point, the 3D data encoder generates an N-ary tree structure in which each of the leaves includes one 3D point, and encodes the N-ary tree structure. When the first information indicates that each of the leaves should include two or more 3D points, the 3D data encoding device generates an N-ary tree structure in which each of the leaves includes two or more 3D points, and encodes the N-ary tree structure.

[0985] Because of this, since leaf formats can be controlled using one first information, the 3D data encoding apparatus can improve encoding efficiency.

[0986] For example, two or more three-dimensional points included in a sheet have mutually different spatial coordinates. In other words, the second information may indicate the coordinates of each of the two or more three-dimensional points included in the sheet.

[0987] For example, two or more 3D points included in a sheet have identical spatial coordinates and mutually different attribute information. In other words, the second information may show attribute information of each of the two or more 3D points included in the sheet.

[0988] For example, each of two or more three-dimensional points included in a sheet has coordinate information and attribute information. In other words, the second information may indicate coordinates and attribute information of each of the two or more 3D points included in the sheet.

[0989] For example, the 3D data encoding device includes a processor and a memory, and the processor performs the above process using the memory.

[0990] The 3D data decoding apparatus according to the present embodiment performs the process illustrated in FIG. 162. First, the 3D data decoding apparatus decodes, from the bit stream, first information (mode information) indicating whether a sheet to be included in the N-ary tree structure of 3D points included in the 3D data should include one 3D point, or two or more 3D points, where N is an integer greater than or equal to 2 (S2231). In other words, the 3D data decoding apparatus obtains the first information from the bitstream.

[0991] When the first information indicates that the sheet is to include one 3D point ("Yes" in S2232), the 3D data decoding apparatus decodes an N-ary tree structure in which the sheet includes one 3D point (S2233). On the contrary, when the first information indicates that the sheet is to include two or more 3D points ("No" in S2232), the 3D data decoding apparatus decodes an N-ary tree structure in which the sheet includes two or more 3D points ( S2234). Here, the phrase "decode an N-ary tree structure in which a leaf includes one 3D point" means to parse information included in a bit stream based on a rule created, for example, based on the fact that a leaf should only include one 3D point. Similarly, the phrase "decode an N-ary tree structure in which a leaf includes two or more 3D points" means to parse the information included in the bitstream based on a rule created, for example, based on what the leaf should include in itself two or more three-dimensional points.

[0992] Because of this, the 3D data decoding apparatus can selectively use a tree structure in which a sheet includes one 3D point and a tree structure in which a sheet includes two or more 3D points. Accordingly, the 3D data decoding apparatus can improve the encoding efficiency.

[0993] For example, when the first information indicates that the sheet is to include two or more 3D points, the 3D data decoding apparatus decodes (obtains), from the bitstream, the second information regarding the sheet (sheet information). When the first information indicates that the sheet is to include one 3D point, the 3D data decoding apparatus does not decode (obtain) the sheet information.

[0994] Because of this, since the second information need not necessarily be attached to the bit stream when the sheet is to include one 3D point, coding efficiency can be improved.

[0995] For example, the second information indicates the total number of 3D points included in the sheet.

[0996] For example, the first information is usually used through leaves. For example, the first information is typically used by all or part of the leaves to be included in an N-ary tree structure. In other words, the first information indicates whether each of the leaves to be included in the N-ary tree structure should include one 3D point or two or more 3D points. When the first information indicates that each of the leaves should include one 3D point, the 3D data decoding apparatus generates an N-ary tree structure in which each of the leaves includes one 3D point, and decodes the N-ary tree structure. When the first information indicates that each of the leaves should include two or more 3D points, the 3D data decoding apparatus forms an N-ary tree structure in which each of the leaves includes two or more 3D points, and decodes the N-ary tree structure.

[0997] Because of this, since leaf formats can be controlled using one first information, the 3D data decoding apparatus can improve encoding efficiency.

[0998] For example, two or more three-dimensional points included in a sheet have mutually different spatial coordinates. In other words, the second information may indicate the coordinates of each of the two or more three-dimensional points included in the sheet.

[0999] For example, two or more 3D points included in a sheet have identical spatial coordinates and mutually different attribute information. In other words, the second information may show attribute information of each of the two or more 3D points included in the sheet.

[1000] For example, each of two or more three-dimensional points included in a sheet has coordinate information and attribute information. In other words, the second information may indicate coordinates and attribute information of each of the two or more 3D points included in the sheet.

[1001] For example, the 3D data decoding apparatus includes a processor and a storage device, and the processor performs the above process using the storage device.

[1002] Embodiment 14

In the present embodiment, the 3D data encoding apparatus expresses the padding code using the 1-bit occupied position and the remaining bit, and encodes each of the 1-bit occupied position and the remaining bit using a different method.

[1003] The 1-bit occupied position indicates the bit position at which 1 appears first when each bit included in the fill code is scanned from the left. For example, a 1-bit occupied position takes a value from 0 to 7 for an octant tree, and takes a value from 0 to 15 for a hexadecant tree.

[1004] The remaining bit indicates the portion of the bit sequence located on the right side of the 1-bit occupied position of the fill code. The remaining bit has a number of bits ranging from 0 to 7 according to the 1-bit occupied position.

[1005] FIG. 163 is a diagram illustrating examples of a 1-bit occupied position and a remaining bit generated from a fill code. In the example shown by (a) in FIG. 163, the fill code is 01000010. Since 1 appears first in the sixth bit when this fill code is scanned from the left, the 1-bit occupied position is 6, and the remaining bit is 000010. In the example shown by (b) in FIG. 163, the fill code is 00000010. Since 1 appears first in the first bit when this fill code is scanned from the left, the 1-bit occupied position is 1, and the remaining bit is 0. In the example shown by (c) in FIG. 163, the fill code is 00000001. Since 1 appears first in bit zero when this fill code is scanned from the left, the 1-bit occupied position is 0, and there is no remaining bit.

[1006] Hereinafter, this document describes an example of a method for encoding a 1-bit occupied position. In the case of an octant tree, the 3D encoder entropy encodes the 1-bit occupied position as a 3-bit value (0 to 7). The 3D encoder performs, for example, arithmetic encoding for a 1-bit occupied position using one codebook. It should be noted that the 3D data encoding device can perform binary encoding (binary arithmetic encoding) one bit at a time for a 1-bit occupied position, which is a 3-bit bit sequence. The 3D data encoder may set the occurrence probability to 0 and the occurrence probability to 1-50% in the codebook in this case. For example, a 3D data encoder may use a bypass mode for binary arithmetic encoding.

[1007] The following describes an example of a method for encoding the remaining bit. For example, with respect to the remaining bit as a binary bit sequence, the 3D data encoding device performs binary encoding for the bits in the order from left to right. For example, when the remaining bit is 000010, the 3D encoder can perform arithmetic encoding for values in 0 and 1 in the order 0 → 0 → 0 → 0 → 1 → 0.

[1008] For example, in the example shown by (a) in FIG. 163, the 3D data encoder performs arithmetic encoding for a value of 6 in the 1-bit occupied position. For example, a 3D data encoder may perform arithmetic encoding for bit sequence 110 indicating a value of 6 using bypass mode. The 3D encoder also performs arithmetic encoding for the 000010 bit sequence of the remaining bit.

[1009] In addition, in the example shown by (b) in FIG. 163, the 3D data encoding device performs arithmetic encoding for a value of 1 in the 1-bit occupied position. For example, the 3D data encoder may perform arithmetic encoding for the bit sequence 1 indicating a value of 1 using the bypass mode. The 3D encoder also performs arithmetic encoding for bit sequence 0 of the remaining bit.

[1010] In addition, in the example shown by (c) in FIG. 163, the 3D data encoder performs arithmetic encoding for the value at 0 in the 1-bit occupied position. For example, the 3D data encoding apparatus may perform arithmetic encoding for the bit sequence 0 indicating a value of 0 using the bypass mode. Since there is no remaining bit, the 3D encoder does not encode the remaining bit.

[1011] It should be noted that the 1-bit occupied position may be defined as the bit position at which 1 appears first when each bit of the fill code is scanned from the right. In this case, the remaining bit indicates the portion of the bit sequence located on the left side of the 1-bit occupied position of the fill code. Further, in this case, the 3D data encoding device can perform binary encoding for the remaining bit in right-to-left order. In addition, the 3D encoder may encode a switch flag for switching between scanning each bit of the fill code on the right and scanning each bit of the fill code on the left. The 3D encoder may encode a toggle flag for each padding code. Accordingly, it is possible to change the scanning order for each fill code. Further, the 3D encoder may add a toggle flag to a world header, a space header, or a volume header, and so on. and change the scan order based on world, space, or volume.

[1012] It should be noted that the scan order may be a predetermined order. In addition, the bit position at which 0 appears first can be used instead of the bit position at which 1 appears first in the scan order.

[1013] In addition, a fill code in which only 1 bit of the number of bits included in the fill code is 1 is set as a 1-bit busy code. For example, in the case of an octant tree, the fill code "0010000" is a 1-bit busy code, but the fill code "0010001" is not a 1-bit busy code.

[1014] The 3D data encoder can determine whether or not the current padding code to be encoded is likely to be a 1-bit busy code. Thereafter, the 3D data encoding apparatus may encode the current padding code in the form of a 1-bit occupied position and the remaining bit (hereinafter referred to as "occupied position coding") when determining that the current padding code is highly likely to be a 1-bit busy code, and may encode the value of the current stuff code (hereinafter referred to as "direct encoding") directly when determining that it is unlikely that the current stuff code should be a 1-bit busy code.

[1015] For example, the 3D data encoder determines whether or not the current padding code is likely to be a 1-bit busy code using the padding code value of the parent node relative to the current node. For example, the 3D encoder determines whether or not the parent node's fill code is a 1-bit busy code; performs encoding based on occupied positions for the fill code of the current node when determining that the fill code of the parent node is a 1-bit busy code; and performs forward encoding on the current node's padding code upon determining that the parent node's padding code is not a 1-bit busy code.

[1016] Here, in the case where coding based on occupied positions is performed when the current fill code of the current node is likely to be a 1-bit occupied code, the value of each bit of the remaining bit tends to be 0. Accordingly, coding efficiency can be improved. by performing binary arithmetic coding on the remaining bit. In addition, the 3D data encoder can effectively determine whether or not the current stuff code is likely to be a 1-bit busy code based on whether or not the stuff code of the parent node is a 1-bit busy code. Additionally, in the case of not a width-encoding order, but a depth-encoding order, an identical determination method can be applied with reference to the padding code of the parent node.

[1017] FIG. 164 is a diagram principally illustrating the above process. As illustrated in FIG. 164, when the parent node's fill code is a 1-bit occupied code, the 3D data encoding device performs encoding based on the occupied positions for the current node. In addition, when the parent node's fill code is a non-1-bit busy code (not a 1-bit busy code), the 3D encoding device performs direct encoding for the current node.

[1018] The following describes an example of the node information syntax. Fig. 165 is a diagram illustrating an example of node information syntax.

[1019] As illustrated in FIG. 165, the node information includes 1bit_occupied_position and remaining_bit. 1bit_occupied_position is the above 1-bit occupied position and indicates the bit position at which 1 appears first when each bit of the fill code is scanned from the left. For example, a 1-bit occupied position takes a value from 0 to 7 for an octant tree, and takes a value from 0 to 15 for a hexadecant tree.

[1020] Remaining_bit is the above remaining bit and indicates the portion of the bit sequence located on the right side of the 1-bit occupied position of the fill code. Since there is no remaining bit when the 1-bit occupied_position is 0, the remaining_bit need not be encoded as illustrated in FIG. 165. In addition, with respect to remaining_bit as a binary bit sequence, the 3D data encoding apparatus may perform binary encoding for the bits in a left-to-right order. In addition, the 3D data encoding apparatus may perform arithmetic encoding on the decimal value calculated from the remaining_bit bit sequence using the encoding table. For example, when the bit sequence of the remaining bit is 000010, the 3D encoder may perform arithmetic encoding for a value of 2.

[1021] When the occupancy code (parent_occupancy_code) of the parent node is not a 1-bit occupancy code, the 3D data encoding device can perform direct encoding for the occupancy code. For example, a 3D data encoder may perform arithmetic encoding for a fill code using one codebook.

[1022] It should be noted that although the octant tree structure is described as an example in the present embodiment, the present disclosure is not necessarily limited thereto. The above procedure can be applied to an N-ary tree, such as a quadtree and a hexadecant tree or other tree structures, where N is an integer greater than or equal to 2. For example, 1bit_occupied_position takes on a value of 0 to 3 for a quadtree, and 1_bit_occupied_position takes on the value 0 to 15 for the hexadecant tree. The 3D encoder may encode, as a 2-bit occupied position (2bit_occupied_position), the position at which 1 appears second when the bit sequence is scanned, in addition to the 1-bit occupied position; and may encode the part of the bit sequence after the 2-bit occupied position as the remaining bit. Additionally, the 3D data encoder may encode, for example, as an N-bit occupied position (Nbit_occupied_position), the position at which 1 appears during the nth time when the bit sequence is scanned; and may encode the part of the bit sequence after the N-bit occupied position as the remaining bit.

[1023] In the present embodiment, when the filling code of the current node is likely to be a 1-bit occupied code, the 3D data encoding device performs encoding based on the occupied positions. In other words, the 3D data encoding apparatus performs arithmetic encoding, as a 1-bit occupied position, for a bit position including values each equal to 1 with a random probability using, for example, encoding table A; and performs arithmetic encoding, as the remaining bit, on the remaining bit sequence including values each tending to be 0 using, for example, the encoding table B. It is possible to improve coding efficiency by representing the padding code in various forms and using different codebooks for arithmetic coding in this manner. For example, the 3D data coding apparatus can perform arithmetic coding for the bit sequence of the remaining bit while keeping the probability of occurrence of values at 0 high using the coding table B. Even when the original bit sequence of the remaining bit is long, the 3D data encoding apparatus can reduce the number of bits in the encoded data as a result.

[1024] Each bit of a 1-bit busy code tends to be a value of 1 with a random probability. As a result, the 3D data encoding apparatus can perform arithmetic encoding for the 1-bit busy code using a bypass mode that skips the occurrence probability calculation by setting the occurrence probability to 0 and the occurrence probability to 1-50%. Accordingly, the amount of processing can be reduced.

[1025] It should be noted that the 3D data decoding apparatus can determine whether encoding based on occupied positions or direct encoding has been performed by a process identical to that of the 3D data encoding apparatus. Alternatively, the 3D data encoding apparatus may generate a bitstream including information indicating whether occupied-position encoding or direct encoding has been performed, and the 3D data decoding apparatus may determine whether occupied-position encoding or direct encoding has been performed based on information included in the bitstream. In this case, the 3D encoder may also refer not to the parent node's padding code but to the current padding code, perform encoding based on occupied positions when the current padding code is a 1-bit occupied code, and perform direct encoding when the current the padding code is not a 1-bit busy code.

[1026] In addition, although the present embodiment describes an example in which coding based on occupied positions and forward coding are switched based on the node, the present disclosure is not limited to this. For example, the 3D encoding device may add, in the world header or the space header, etc., a flag indicating whether or not encoding should be performed based on occupied positions; and can choose whether or not to perform encoding based on occupied positions, based on world or space, and so on. For example, for a world or space including a plurality of sparse 3D point clouds, the 3D data encoder may activate a flag and perform position-based encoding for all 3D point cloud fill codes in the world or space. In addition, for a world or space including a plurality of dense 3D point clouds, the 3D data encoder may deactivate the flag and perform forward encoding for all 3D point cloud data fill codes in the world or space. This eliminates the need to define a selection for each node, and thus the amount of processing can be reduced.

[1027] Since the 3D data decoding apparatus can decode a flag included in a header such as world or space to determine whether encoding based on occupied positions or direct encoding is performed for the fill code in the world or space, the 3D data decoding apparatus can decode the code filling in properly.

[1028] The 3D data encoding device may perform encoding based on occupied positions when the fill code of the current node is likely to be an N-bit occupied code in which the number of bits having a value of 1 included in the fill code is less than or equal to N (any integer), and can perform direct encoding when it is unlikely that the current node's fill code should be an N-bit busy code. For example, the 3D encoding device may perform occupied position encoding when the parent node's fill code is an N-bit occupied code, and may perform direct encoding when the parent node's fill code is not an N-bit occupied code.

[1029] Hereinafter, this document describes the flow of processing in the device encoding three-dimensional data. Fig. 166 is a flowchart for a 3D data encoding process performed by a 3D data encoding device according to the present embodiment.

[1030] First, the 3D data encoder determines whether or not the fill code of the parent node with respect to the current node is a 1-bit busy code (S2301). When the parent node's fill code is a 1-bit occupied code ("Yes" in S2301), the 3D data encoding device performs encoding based on occupied positions for the current fill code of the current node (S2302).

[1031] Specifically, the 3D data encoding apparatus searches for a 1-bit occupied position in the current padding code (S2304), and encodes the obtained 1-bit occupied position (S2305). Thereafter, the 3D data encoding apparatus calculates the remaining bit (S2306) and encodes the obtained remaining bit (S2307).

[1032] On the other hand, when the parent node's fill code is not a 1-bit busy code ("No" in S2301), the 3D data encoding apparatus performs forward encoding for the current fill code (S2303).

[1033] It should be noted that the order of steps included in the occupied-position-based encoding step (S2302) may be re-arranged. For example, the 3D data encoding apparatus may search for the 1-bit occupied position (S2304), calculate the remaining bit (S2306), and then encode the 1-bit occupied position and the remaining bit (S2305, S2307).

[1034] FIG. 167 is a flowchart illustrating a specific example of the occupied-position encoding step (S2302). First, the 3D data encoder calculates a 1-bit occupied position (S2304). Specifically, the 3D data encoder sets the variable a to 7 (S2311). Then, the 3D data encoder sets "fill code - 2 ^a " as a Diff variable (S2312). Thereafter, the 3D data encoding apparatus determines whether or not "Diff>-1" is satisfied (S2313).

[1035] When "Diff>-1" is not satisfied ("No" in S2313), the 3D encoder sets "a-1" as variable a (S2314). When "Diff>-1" is satisfied ("Yes" in S2313), the 3D encoder sets a as the 1-bit occupied position and "Diff" as the remaining bit (S2315).

[1036] Thereafter, the 3D encoder encodes the calculated 1-bit occupied position (S2305).

[1037] Next, the 3D data encoding apparatus calculates the remaining bit (S2306) and performs binary encoding for the remaining bit (S2307). Specifically, the 3D data encoder sets "1-bit occupied position - 1" as variable b (S2321). Thereafter, the 3D encoder sets "remaining bit ^2b " as a Diff variable (S2322). Thereafter, the 3D data encoder determines whether or not "Diff>-1" is satisfied (S2323).

[1038] When "Diff>-1" is satisfied ("Yes" in S2323), the 3D encoder encodes "1" and sets "Diff" as the remaining bit (S2324) and sets "b-1" as variable b (S2326 ). On the other hand, when "Diff>-1" is not satisfied ("No" in S2323), the 3D encoder encodes "0" (S2325) and sets "b-1" as variable b (S2326).

[1039] Finally, the 3D data encoder determines whether or not "b<0" is satisfied (S2327). When "b<0" is not satisfied ("No" in S2327), the 3D data encoding apparatus executes step S2322 and the following steps again. When "b<0" is satisfied ("Yes" in S2327), the 3D encoder terminates the process.

[1040] Hereinafter, this document describes the flow of processing in the device decoding three-dimensional data. Fig. 168 is a flowchart for a 3D data decoding process performed by a 3D data decoding apparatus according to the present embodiment.

[1041] First, the 3D data decoding apparatus determines whether or not the parent node fill code is a 1-bit busy code (S2331). When the parent node fill code is a 1-bit occupied code ("YES" in S2331), the 3D data decoding apparatus performs occupied position decoding to decode encoded data encoded by occupied position coding (S2332).

[1042] Specifically, the 3D data decoding apparatus decodes the 1-bit occupied position from the bitstream (S2334); calculates a part of the fill code (from the left end of the fill code to the 1-bit occupied position), based on the 1-bit occupied position (S2335); and updates the fill code when decoding the remaining bit (S2336).

[1043] On the other hand, when the parent node's stuff code is not a 1-bit busy code ("No" in S2331), the 3D data decoding apparatus decodes the stuff code from the bit stream using forward decoding to decode the encoded stuff code encoded by direct coding (S2333).

[1044] It should be noted that the order of the steps included in the decoding step based on occupied positions (S2332) may be re-arranged. For example, the 3D data decoding apparatus may decode the 1-bit occupied position (S2334), then update the padding code when decoding the remaining bit (S2336), and finally add the padding code calculated from the 1-bit occupied position with the updated padding code (S2335) .

[1045] FIG. 169 is a flowchart illustrating a specific example of the position-occupied decoding step (S2332). First, the 3D data decoding apparatus decodes the 1-bit occupied position from the bitstream (S2334), and calculates the fill code portion using the 1-bit occupied position (S2335).

[1046] Specifically, the 3D data decoding apparatus sets the 1-bit occupied position as variable a (S2341). Then, the 3D data decoding apparatus sets 2 ^a as the fill code (S2342).

[1047] Thereafter, the 3D data decoding apparatus updates the padding code when decoding the remaining bit (S2336). Specifically, the 3D data decoding apparatus sets "a-1" as variable b (S2351). Then, the 3D data decoding apparatus decodes 1 bit and sets the decoded 1 bit as variable c (S2352). Thereafter, the 3D data decoding apparatus determines whether or not "c==1" is satisfied (S2353).

[1048] When "c==1" is satisfied ("Yes" in S2353), the 3D data decoding apparatus adds ^2b to the fill code (S2354), and sets "b-1" as variable b (S2355). When "c==1" is not satisfied ("No" in S2353), the 3D data decoding apparatus sets "b-1" as variable b (S2355).

[1049] Finally, the 3D data decoding apparatus determines whether or not "b<0" is satisfied (S2356). When "b<0" is not satisfied ("No" in S2356), the 3D data decoding apparatus executes step S2352 and the following steps again. When "b<0" is satisfied ("Yes" in S2356), the 3D data decoding apparatus terminates the process.

[1050] Next, a configuration example of a 3D data encoding device will be described. Fig. 170 is a block diagram of a 3D data encoding apparatus 2300 according to the present embodiment. The 3D encoder 2300 includes an octant tree generator 2301 and an entropy encoder 2302.

[1051] The octant tree builder 2301 generates, for example, an octant tree from inputted 3D points (point clouds) and generates a corresponding one of the fill code and leaf information for each node of the octant tree.

[1052] Entropy encoder 2302 encodes the padding code of each node. The entropy encoder 2302 switches between padding code encoding methods according to whether or not the parent node's padding code is a 1-bit busy code. For example, when the parent node's fill code is a 1-bit occupied code, entropy encoder 2302 performs encoding based on occupied positions for the fill code; and when the parent node is not a 1-bit occupied code, the entropy encoder 2302 performs arithmetic encoding for the fill code value using a codebook.

[1053] Next, a configuration example of a 3D data decoding apparatus will be described. Fig. 171 is a block diagram of a 3D data decoding apparatus 2310 according to the present embodiment. The 3D data decoder 2310 includes an octant tree generator 2311 and an entropy decoder 2312.

[1054] The octant tree builder 2311 generates a space (node) octant tree using, for example, bitstream header information. For example, the octant tree builder 2311 generates a large space (root node) using a space size along the X-axis, Z-axis, and Y-axis directions attached to the header information, and generates an octant tree by generating eight small spaces A (nodes A0- A7) by dividing the space into two along each of the X-axis, Z-axis, and Y-axis directions. Similarly, the octant tree generator 2311 further divides each of the nodes A0-A7 into eight small spaces. As stated above, the octant tree builder 2311 repeats the generation of the octant tree.

[1055] The entropy decoder 2312 decodes the padding code of each node. The entropy decoder 2312 switches between methods of decoding the stuff code according to whether or not the decoded stuff code of the parent node is a 1-bit busy code. For example, when the decoded parent node's fill code is a 1-bit occupied code, the entropy decoder 2312 decodes the fill code using occupied position decoding; and when the parent node is not a 1-bit busy code, the entropy decoder 2312 performs arithmetic decoding for the fill code value using a codebook.

[1056] As mentioned above, the 3D data encoding device according to the present embodiment performs the process illustrated in FIG. 172. First, the 3D data encoding device generates a bit sequence (for example, a fill code) including N-bit information that is information of the current node included in the N-ary tree structure of 3D points included in the 3D data, and which indicates whether or not a three-dimensional point is present in each of the child nodes belonging to the current node, where N is an integer greater than or equal to 2 (S2361).

[1057] Next, the 3D data encoding device generates (i) position information indicating the head position (e.g., 1-bit occupied position), which is the position at which the predetermined code appears first in the bit sequence when the bit sequence is scanned. in the predetermined scan order, and (ii) the remaining bit which is the portion of the bit sequence after the position of the head in the predetermined scan order (S2362). Here, for example, the predefined code is 1.

[1058] Thereafter, the 3D encoder encodes the position information and the remaining bit as the current node information (S2363). In other words, the 3D data encoding device generates, as current node information, a bit stream including position information and a remaining bit.

[1059] Accordingly, the 3D data encoding device can improve encoding efficiency.

[1060] For example, the 3D data coding device has a first mode (eg, occupied-position-based coding) for encoding position and remaining bit information and a second mode (eg, direct coding) for encoding a bit sequence, and selects one of the first mode and the second mode based on whether or not the peripheral node, relative to the current node, includes the 3D point.

[1061] Accordingly, the 3D data encoding apparatus can improve the coding efficiency by selecting the mode according to the state of the peripheral 3D point.

[1062] For example, the 3D data encoding apparatus selects the first mode when the total number of peripheral nodes including the 3D point among the peripheral nodes with respect to the current node or the proportion of peripheral nodes including the 3D point with respect to the peripheral nodes with respect to the current node is less than a predetermined threshold value; and the 3D encoding device selects the second mode when the total number of peripheral nodes or the proportion of peripheral nodes exceeds a predetermined threshold value.

[1063] For example, the 3D data encoder selects one of the first mode and the second mode based on the bit sequence of the parent node relative to the current node.

[1064] For example, the 3D data encoding device selects the first mode when the total number of child nodes including the 3D point among the child nodes belonging to the parent node is less than a predetermined threshold, the total number of child nodes being indicated by the bit sequence of the parent node. node; and the 3D encoding device selects the second mode when the total number of child nodes exceeds a predetermined threshold.

[1065] For example, the 3D data encoding device includes a processor and a storage device, and the processor performs the above process using the storage device.

[1066] In addition, the 3D data decoding apparatus according to the present embodiment performs the process illustrated in FIG. 173. First, the 3D data decoding apparatus decodes, from the bitstream, (i) position information indicating the head position (e.g., 1-bit occupied position), which is the position at which the predetermined code appears first in the bit sequence (e.g., fill code) when the bit sequence is scanned in a predetermined scan order, and (ii) a remaining bit that is the portion of the bit sequence after the head position in the predetermined scan order, where the bit sequence includes an N-bit information that is information of the current node included in the N-ary tree structure of 3D points included in the 3D data, and that indicates whether or not a 3D point is present in each of the child nodes belonging to the current node, where N is an integer, greater than or equal to 2 (S2371). In other words, the 3D data decoding apparatus obtains, from the bit stream, the position information and the remaining bit. Additionally, for example, the predetermined code is 1.

[1067] Next, the 3D data decoding apparatus recovers the bit sequence of the current node from the position information and the remaining bit (S2372), and reconstructs the N-ary tree structure using the bit sequence (S2373). In other words, the 3D data decoding apparatus recovers the position information of the 3D points.

[1068] Accordingly, the 3D data decoding apparatus can improve encoding efficiency.

[1069] For example, the 3D data decoding apparatus has a first mode (decoding based on occupied positions) for decoding position information and a remaining bit and a second mode (forward decoding) for decoding a bit sequence, and selects one of the first mode and the second mode based on whether or not the peripheral node, relative to the current node, includes the 3D point.

[1070] Accordingly, the 3D data decoding apparatus can improve coding efficiency by selecting a mode according to the state of the 3D peripheral point.

[1071] For example, the 3D data decoding apparatus selects the first mode when the total number of peripheral nodes including the 3D point among the peripheral nodes with respect to the current node or the proportion of peripheral nodes including the 3D point with respect to the peripheral nodes with respect to the current node is less than a predetermined threshold value; and the 3D data decoding apparatus selects the second mode when the total number of peripheral nodes or the proportion of peripheral nodes exceeds a predetermined threshold value.

[1072] For example, the 3D data decoder selects one of the first mode and the second mode based on the bit sequence of the parent node relative to the current node.

[1073] For example, the 3D data decoding apparatus selects the first mode when the total number of child nodes including the 3D point, among the child nodes belonging to the parent node, is less than a predetermined threshold, the total number of child nodes being indicated by the bit sequence of the parent node. node; and the 3D data decoding apparatus selects the second mode when the total number of child nodes exceeds a predetermined threshold.

[1074] For example, the 3D data decoding apparatus includes a processor and a storage device, and the processor performs the above process using the storage device.

[1075] Embodiment 15

In the present embodiment, the 3D data encoding apparatus performs quantization on the 3D position information of the inputted 3D point cloud, and encodes the 3D position information using an octant tree structure. At this time, 3D dots (hereinafter referred to as "duplicate dots") occur that have the same 3D position but have different attribute information such as color or reflectance due to quantization. The 3D encoder appends, to the header, information for controlling how to encode these duplicated points as leaf information of the octant tree. As a result, the 3D data decoding apparatus can decode the sheet information correctly. Here, the expression "have the same three-dimensional position ... due to quantization" includes a state in which, similarly to point A and point B illustrated in FIG. 174, the original 3D positions are close to each other, and the values of the 3D positions become identical due to the quantization of the 3D position information.

[1076] For example, the 3D data encoder appends, to the header information, a duplicated point merging flag (MergeDuplicatedPointFlag), which is a flag for controlling whether or not duplicated points should be merged. Fig. 175 is a diagram fundamentally illustrating a process according to the duplicated point merge flag.

[1077] When the duplicated point merge flag is 1, the 3D encoder combines the duplicated points into a point and encodes the point. Here, the term "merging" means when, for example, point A and point B are duplicate points, keeping point A and deleting point B, or vice versa. It should be noted that in such a case, the 3D data encoding apparatus can calculate new attribute information from the pieces of attribute information such as color or reflectance, point A and point B; and may assign the computed attribute information to the merged point. For example, the 3D data encoder may assign the average value of the point A and point B attribute information pieces to the combined point.

[1078] In addition, since each sheet, when encoding is performed using an octant tree, includes one dot, when the duplicated dot merge flag is 1, the 3D data encoding device need not necessarily encode, as sheet information, information indicating that , how many 3D points the sheet includes. The 3D data encoding device can also encode the 3D position information of one point in the sheet and information regarding attribute information such as color or reflectance.

[1079] As mentioned above, when the duplicated points are optional after decoding, the 3D encoding device sets the duplicated points merging flag to 1, appends the duplicated points merging flag to the stream, merges the duplicated points, and encodes the merged point. Therefore, it is possible to reduce the amount of data of optional duplicated points, thereby increasing the coding efficiency.

[1080] When the duplicated dot union flag is 0, the 3D data encoding device encodes the duplicated dot information as sheet information. For example, since each sheet may include one or more duplicated points, the 3D encoder encodes information indicating how many 3D points the sheet includes. The 3D encoding device may also encode attribute information of each of the duplicated points. For example, when point A and point B are present as duplicate points in a sheet, the 3D encoding device may encode information indicating that two points are present in a sheet. In addition, the 3D encoding apparatus can encode attribute information of each of point A and point B.

[1081] As mentioned above, when duplicated points are required after decoding, the 3D data encoding device sets the duplicated points merging flag to 0, attaches the duplicated points merging flag to the stream, and encodes the duplicated points. As a result, the 3D data decoding apparatus can decode information regarding duplicated points correctly.

[1082] For example, as an example of quantization of the 3D position, the 3D data encoding device calculates the quantization position (x/qx, y/qy, z/qz) by dividing the 3D position (x, y, z) by the quantization parameter (qx, qy , qz).

[1083] A flag for merging duplicated points may be included in the bitstream header information. For example, a duplicate dot merge flag may be included in the header of a bitstream such as WLD, SPC, or VLM.

[1084] It should be noted that although examples of the attribute information include color or reflectance in the above description, the attribute information is not limited to this. For example, the attribute information may include a normal vector of a point, information indicating the degree of importance of a point, a three-dimensional feature of a point, or position information such as latitude, longitude, and elevation of a point.

[1085] The term "combination" represents the combination of two or more dots into a dot. In addition, the term "union" may represent a combination of M or more points into N points, where M>N.

[1086] As mentioned above, duplicate points occur that have identical coordinates to those of the 3D point cloud, but have different attribute information such as color or reflectance due to quantization. For example, although point A and point B have different 3D positions before quantization, a case arises in which point A and point B end up with the same 3D position but have different attribute information due to quantization. Briefly, point A and point B are duplicate points.

[1087] It should be noted that the above-described case is not limited to quantization, and a case is also envisaged where duplicate points are forced by obtaining, by means of a sensor such as lidar, 3D positions and point cloud attribute information of an identical object at different times or in different directions. .

[1088] The expression "have an identical three-dimensional position" is not limited to the case in which the three-dimensional positions are completely identical. For example, when the difference between the 3D positions of point A and point B is less than or equal to the threshold value α, the 3D data encoding device may consider points A and points B as having the same 3D position and determine that point A and point B are duplicate points. In addition, the 3D encoder may add a threshold α to the stream and notify the 3D decoder that any point less than or equal to the threshold α is treated as a duplicate point.

[1089] In addition, the 3D data encoding apparatus may use the 3D position of point A as the 3D position of the duplicated point. Alternatively, the 3D data encoder may use the 3D position of point B as the 3D position of the duplicated point. Alternatively, the 3D data encoder may use, as the 3D position of the duplicated point, the 3D position calculated from the 3D position of point A and the 3D position of point B. For example, the 3D data encoder may use an average between the 3D position of point A and the 3D position of point b.

[1090] The 3D data encoder may merge, among the duplicated points, points having the same 3D position and identical attribute information, or may delete one of the points regardless of the value of the duplicated points merge flag.

[1091] When the duplicated dot merge flag is 1, the 3D data encoding device may merge the M dots in the sheet into N dots, where M>N. In this case, the 3D data encoding apparatus may encode, as sheet information, each of the 3D position information pieces and the N point attribute information pieces. In addition, the 3D data encoding device can calculate N pieces of attribute information using M pieces of attribute information.

[1092] The 3D encoder may add the number of dots (N) in the sheet after being combined into the header, and notify the number of dots (N) to the 3D decoder. The value of N may be predetermined as a fixed value by standards, etc. This eliminates the need to add information indicating N for each sheet, and thus the generated coding amount can be reduced. Accordingly, the 3D data decoding apparatus can decode N points correctly.

[1093] When the duplicated point merge flag is 1, the duplicated points are merged into a point. For example, the 3D encoder may combine point A and point B into point C having 3D position information identical to the 3D position information of point A and point B. Note that the 3D encoder may assign, to point C, the average value of the pieces of information attributes such as color or reflectance, point A, and point B. Additionally, the 3D encoding device may merge point B with point A, or merge point A with point B.

[1094] The following describes an example of the syntax of the duplicated point merge flag. Fig. 176 is a diagram illustrating an example of the header information syntax. Fig. 177 is a diagram illustrating an example of node information syntax.

[1095] As illustrated in FIG. 176, the header information includes a flag for merging duplicated points (MergeDuplicatedPointFlag). The duplicated point merging flag is information indicating whether or not duplicated points should be merged. For example, a value of 1 for the Duplicate Point Merge Flag indicates that duplicate points should be merged, and a value of 0 for the Duplicate Point Merge Flag indicates that duplicate points should not be merged.

[1096] It should be noted that the 3D encoder may indicate whether or not to merge duplicated points based on standards or profile or level, etc. standards without attaching the flag for concatenating duplicated points to the header. This enables the 3D data decoding apparatus to determine whether or not the stream includes a duplicated point with reference to the standards information and recover the bitstream correctly.

[1097] As illustrated in FIG. 177, the node information includes isleaf and num_point_per_leaf; isleaf is a flag indicating whether or not the current node is a leaf. A value of 1 indicates that the current node is a leaf, and a value of 0 indicates that the current node is not a leaf, but a node. It should be noted that information indicating whether or not a node is a leaf need not necessarily be appended to the header. In this case, the 3D data decoding apparatus determines whether or not the node is a sheet using another method. For example, the 3D data decoding apparatus may determine whether or not each node of the octant tree is divided into the smallest possible size, and may determine that the node is a leaf when determining that each node is divided into the smallest possible size. This eliminates the need to encode a flag indicating whether or not the node is a leaf, thus reducing the amount of header code.

[1098] Num_point_per_leaf is leaf information and indicates the number of 3D points included in the leaf. When the duplicated point merge flag is 0, num_point_per_leaf is encoded. Additionally, since the number of points in a leaf is 1, when the duplicate points merge flag is 1, num_point_per_leaf is not encoded. Accordingly, you can reduce the amount of code.

[1099] It should be noted that although whether or not to encode leaf information is directly selected according to the duplicate dot merge flag in the example described here, whether or not to encode leaf information may be indirectly selected. For example, the 3D data encoder may change the single_point_per_leaf illustrated in FIG. 156 according to the duplicate dot merge flag, and select whether or not to encode the sheet information based on the syntax illustrated in FIG. 156. In other words, when the duplicate point merging flag is 1, the 3D encoder may set single_point_per_leaf to 1; and when the duplicated point merging flag is 0, the 3D encoder may set single_point_per_leaf to 0. In this case, the 3D encoder does not need to append the duplicated points merging flag to the bitstream either.

[1100] A 3D encoder may entropy encode num_point_per_leaf. At this time, the 3D data encoding apparatus can also perform encoding by switching codebooks. For example, the 3D data encoder may perform arithmetic coding for the first bit using coding table A, and may perform arithmetic coding for the remaining bit using coding table B.

[1101] As mentioned above, the 3D data encoder appends, to the header of the bitstream, information indicating whether or not to merge the duplicated points, and selects whether or not to merge the duplicated points according to the value. When merging duplicated points, the 3D data encoding device need not necessarily encode, as sheet information, the number of points included in the sheet. By not merging the duplicated points, the 3D encoder can encode, as sheet information, the number of points included in the sheet.

[1102] The 3D data encoder can also entropy encode the isleaf, MergeDuplicatedPointFlag, and num_point_per_leaf generated by the method described above. For example, a 3D data encoder may binarize each value and perform arithmetic encoding on the value.

[1103] Although the octant tree structure is described as an example in the present embodiment, the present disclosure is not necessarily limited thereto. The above procedure can be applied to an N-ary tree such as a quadtree and a hexadecant tree or other tree structures where N is an integer greater than or equal to 2.

[1104] When encoding is performed with the duplicated point merge flag=1, and the original input 3D point cloud or the quantized 3D point cloud includes duplicated points, lossy coding is used, and thus the amount of code can be reduced. In addition, when the original input 3D point cloud does not include duplicated points, and encoding is performed using lossless coding (encoding is performed in skip quantization), the 3D data encoding apparatus may perform encoding with the duplicated point merge flag=1. Accordingly, it is possible to reduce the amount of code by as much as the amount of code resulting from not encoding num_point_per_leaf while maintaining lossless encoding.

[1105] In addition, when the 3D encoding device encodes, as sheet information, each of two or more duplicate dots in the identical sheet, the 3D encoding device may also encode each of the pieces of attribute information (e.g., color or reflectance) of the respective points. In this case, the attribute information pieces of the respective points may be associated in the order in which the points are coded. For example, when the 3D encoding device encodes each of points A and B in the same sheet as sheet information, the 3D encoding device can encode the attribute information pieces of both points A and B and append the attribute information pieces to the bitstream. Further, the attribute information pieces may be associated in the encoding order of points A and B. For example, when each 3D position is encoded in the order from point A to point B, it is possible that the attribute information pieces are encoded in the order from point A to point B and are associated.

[1106] In addition, when the 3D encoder combines M or more point clouds in an identical sheet and encodes the combined point clouds as N points, where M>N, the 3D encoder may round M or more pieces of M or more attribute information. point clouds, for example, by averaging to form N point attribute information pieces, and may encode the attribute information pieces. For example, when the 3D encoding device combines points A and B in the same sheet into a point and codes the point, the 3D encoding device may round off the pieces of attribute information of the points A and B, for example, by averaging, to calculate the attribute information of the point, and may encode the calculated attribute information.

[1107] In addition, the 3D data encoder may change the method of calculating the attribute information according to the degree of importance or feature of the point. For example, the 3D data encoding apparatus can provide a high weight to the attribute information of a point having an important high degree or a point having a large feature, calculate a weighted average, and use the calculated value as the attribute information after combining. In addition, the 3D data encoding apparatus may change the weighting coefficient according to the difference between the 3D positions before and after quantization. For example, a higher weighting factor may be provided as the difference becomes smaller, a weighted average may be calculated, and the calculated value may be used as attribute information after combining.

[1108] The following describes a procedure for a 3D data encoding process performed by a 3D data encoding apparatus. Fig. 178 and FIG. 179 are a flowchart for a 3D data encoding process performed by a 3D data encoding device.

[1109] First, the 3D data encoding device determines whether or not to combine the duplicated points, and perform encoding (S2401). For example, when the 3D data encoder prioritizes encoding efficiency, the 3D data encoder may determine whether to merge the duplicated points. When the duplicated points are mandatory in the 3D data decoding device, the 3D data encoding device may also determine the need not to merge the duplicated points. In addition, when the input 3D point cloud does not include duplicated points, and lossless coding, i.e. quantization is not performed, the 3D data encoding device can set the duplicated dot merge flag to 1. Since this prevents the number of dots in a sheet from being encoded as sheet information, it is possible to reduce the amount of code.

[1110] When the 3D encoder merges the duplicate dots and performs encoding ("Yes" in S2401), the 3D encoder sets the duplicate dot merge flag to 1 and attaches the duplicate dot merge flag to the header (S2402).

[1111] When the 3D encoder does not merge duplicate dots and does not perform encoding ("No" in S2401), the 3D encoder sets the duplicate dot merge flag to 0 and attaches the duplicate dot merge flag to the header (S2403).

[1112] Next, the 3D data encoding device quantizes the 3D positions of the inputted 3D point cloud (S2404). As an example of 3D position quantization, the 3D data encoding device calculates the quantization position (x/qx, y/qy, z/qz) by dividing the 3D position (x, y, z) by the quantization parameter (qx, qy, qz). Further, the 3D data encoding apparatus may append a quantization parameter to the header, and the 3D data decoding apparatus may perform inverse quantization using the quantization parameter. It should be noted that the 3D data encoder may skip quantization during lossless encoding.

[1113] Thereafter, the 3D data encoder determines whether or not the duplicated point merging flag is equal to 1 (S2405). When the duplicated point merging flag is 1 ("Yes" in S2405), the 3D data encoder combines the duplicated points of the quantized 3D point cloud (S2406). It should be noted that when lossless encoding is performed and the input 3D point cloud does not include duplicated points, the 3D data encoding device may skip this step.

[1114] When the duplicated point merge flag is 0 ("No" in S2405), the 3D encoder does not merge the duplicated points.

[1115] Thereafter, the 3D encoder divides the node into an octant tree (S2411). For example, the 3D data encoding apparatus can calculate the padding code of each octant tree node sequentially when performing octant tree splitting initially for a large space (root node) including the quantized 3D point cloud, and can encode the calculated padding code. In addition, the 3D data encoding device can perform octant tree splitting repeatedly and encode leaf information when octant tree splitting cannot be performed. It should be noted that the 3D data encoding device can calculate the padding codes and pieces of leaf information of all nodes in advance and then encode these pieces of information.

[1116] Next, the 3D data encoding apparatus determines whether or not the next node (current node) is a sheet (S2412). For example, the 3D data encoder may determine whether or not the octant tree is split into the smallest possible size, and may determine that the node is a leaf when determining that the octant tree is split into the smallest possible size.

[1117] When the current node is a leaf ("Yes" in S2412), the 3D encoder determines whether or not the duplicated dot merge flag is 0 (S2413). When the duplicated point merge flag is 0 (Yes in S2413), the 3D data encoding device encodes the number of 3D points included in the sheet (num_point_per_leaf) (S2414). When the duplicated point merging flag is 1 ("No" in S2413), the 3D encoding device does not encode the number of 3D points included in the leaf (num_point_per_leaf).

[1118] In addition, when the current node is not a leaf (No in S2412), the 3D data encoding device encodes the fill code of the current node (S2415).

[1119] Thereafter, the 3D data encoding apparatus determines whether or not the processing of all nodes is completed (S2416). When the processing of all nodes is not completed ("No" in S2416), the 3D data encoding apparatus executes step S2412 and the following steps for the next node.

[1120] When the processing of all nodes is completed ("Yes" in S2416), the 3D encoding device encodes the attribute information regarding the encoded 3D points (S2417).

[1121] It should be noted that the 3D data encoder can adjust the size of the large space (the root node) along the X-axis, the Y-axis, or the Z-axis to a power of two size such that the large space can always be equally divided into two about each axis. In addition, the 3D data encoding apparatus can adjust the size of the large space such that a divided node always becomes a cube. For example, when the 3D positions of the 3D point clouds are 0 to 256 along the X axis, 0 to 120 along the Y axis, and 0 to 62 along the Z axis, firstly, the 3D data encoder compares the minimum value and the maximum value each axis and calculates the minimum value and maximum value of the coordinates of all point clouds. In this case, the minimum value is 0 and the maximum value is 256. Then, the 3D encoder calculates values that include the calculated minimum value and the maximum value and enable the large space to be a power of two. In this case, the size is 512, and the minimum value and maximum value of the coordinates in space are 0 and 511, respectively. As a result, it is possible to include point clouds in the range of 0 to 256. In this case, the 3D encoder also starts splitting the octant tree initially for a large space having a size of 512*512*512.

[1122] The following describes a procedure for a 3D data decoding process performed by a 3D data decoding apparatus. Fig. 180 is a flowchart for a 3D data decoding process performed by a 3D data decoding apparatus. First, the 3D data decoding apparatus decodes the duplicated dot merge flag in the bitstream header (S2421).

[1123] Next, the 3D data decoding apparatus divides the node into an octant tree (S2422). For example, the 3D data decoding apparatus generates a space (node) octant tree using header information, and so on. bit stream. For example, the 3D data decoding apparatus generates a large space (root node) using a space size along the X-axis, Z-axis, and Y-axis directions attached to the header information, and generates an octant tree by forming eight small spaces A (nodes A0- A7) by dividing the space into two along each of the X-axis, Z-axis, and Y-axis directions. Similarly, the 3D data decoding apparatus further divides each of the nodes A0-A7 into eight small spaces. As mentioned above, the 3D data decoding apparatus performs decoding of the fill code of each node and decoding of leaf information sequentially through the above process.

[1124] Thereafter, the 3D data decoding apparatus determines whether or not the next node (current node) is a sheet (S2423). When the current node is a leaf ("YES" in S2423), the 3D data decoding apparatus determines whether or not the duplicate dot merge flag is 0 (S2424). When the duplicated point merge flag is 0 (Yes in S2424), the 3D data decoding apparatus decodes the number of 3D points included in the leaf (num_point_per_leaf) from the bitstream (S2425). On the other hand, when the duplicated point merge flag is 1 ("No" in S2424), the 3D data decoding apparatus does not decode the number of 3D points included in the leaf (num_point_per_leaf) from the bit stream.

[1125] In addition, when the next node is not a leaf ("No" in S2423), the 3D data decoding apparatus decodes the fill code of the current node from the bitstream (S2426).

[1126] Thereafter, the 3D data decoding apparatus calculates 3D leaf positions using the decoded fill code and information regarding the number of times octant tree splitting is performed, and so on. (S2427). For example, when a large space has a size of 8*8*8, performing an octant tree split three times results in a node having a size of 1*1*1. This size (1*1*1) is the smallest separable unit (sheet). Further, the 3D decoder determines whether or not each leaf includes a point based on the decoded fill code of the parent leaf node. Accordingly, the 3D data decoding apparatus can calculate the 3D position of each sheet.

[1127] Next, the 3D data decoding apparatus inversely quantizes the calculated 3D positions (S2428). Specifically, the 3D data decoding apparatus calculates the 3D positions of the point cloud by performing inverse quantization using the quantization parameter decoded from the header. For example, as an example of 3D position inverse quantization, the 3D data decoding apparatus calculates the inverse quantization position (x*qx, y*qy, z*qz) by multiplying the 3D position (x, y, z) to inverse quantization by the quantization parameter (qx , qy, qz). It should be noted that the 3D data decoding apparatus may skip inverse quantization during lossless encoding. In addition, when the scale need not necessarily return to the original scale, the 3D data decoding apparatus may skip inverse quantization even during non-lossy (lossy) encoding. For example, if not an absolute positional relationship between 3D points but a relative positional relationship between 3D points is needed, the 3D decoder may skip inverse quantization.

[1128] Thereafter, the 3D data decoding apparatus determines whether or not the processing of all nodes is completed (S2429). When the processing of all nodes is not completed ("No" in S2429), the 3D data decoding apparatus executes step S2423 and the following steps for the next node.

[1129] When the processing of all nodes is completed ("Yes" in S2429), the 3D data decoding apparatus finally decodes the attribute information regarding the decoded 3D points from the bit stream (S2430). It should be noted that when the duplicated point merge flag is set to 1, attribute information is associated with each point having a different decoded 3D position after decoding. In addition, when the duplicated point merge flag is 0, different pieces of attribute information are decoded and associated with points having the same decoded 3D position.

[1130] Next, a configuration example of a 3D data encoding device will be described. Fig. 181 is a block diagram of a 3D data encoding apparatus 2400 according to the present embodiment. The 3D encoder 2400 includes a quantizer 2401, an octant tree generator 2402, a union determination module 2403, and an entropy encoder 2404.

[1131] The quantizer 2401 quantizes the inputted 3D points (point cloud). It should be noted that in the case of lossless coding, quantization may be skipped.

[1132] The octant tree generator 2402 generates, for example, an octant tree from inputted 3D points (point clouds) and generates a corresponding one of the fill code and leaf information for each node of the octant tree.

[1133] The merge determination unit 2403 determines whether or not to merge duplicated points and perform encoding, and sets the value of a duplicated dot merge flag based on the result of the determination. For example, the merge determination module 2403 determines the value of the duplicated point merge flag using the information of the quantized 3D point cloud. For example, the merge determination module 2403 determines the value of the duplicated point merge flag based on whether or not the quantized 3D point cloud includes duplicated points.

[1134] The entropy encoder 2404 generates a bit stream by encoding the leaf information according to the duplicate dot merge flag. An entropy encoder 2404 may attach a duplicated point merge flag to the bit stream. In addition, the entropy encoder 2404 may encode the padding code. In addition, the entropy encoder 2404 may encode attribute information about the encoded 3D points.

[1135] Next, a configuration example of a 3D data decoding apparatus will be described. Fig. 182 is a block diagram of a 3D data decoding apparatus 2410 according to the present embodiment. The 3D data decoder 2410 includes an octant tree generator 2411, a combination information decoder 2412, an entropy decoder 2413, and an inverse quantizer 2414.

[1136] The octant tree builder 2411 generates a space (node) octant tree using, for example, bitstream header information. For example, the octant tree builder 2411 generates a large space (root node) using the size of the space along the X-axis, Z-axis, and Y-axis directions attached to the header information, and generates an octant tree by generating eight small spaces A (nodes A0- A7) by dividing the space into two along each of the X-axis, Z-axis, and Y-axis directions. Similarly, the octant tree generator 2411 further divides each of the nodes A0-A7 into eight small spaces. As stated above, the octant tree builder 2411 repeats the generation of the octant tree.

[1137] The merge information decoder 2412 decodes the duplicated dot merge flag from the bitstream header information. Note that the merging information decoder 2412 may be included in the entropy decoder 2413.

[1138] The entropy decoder 2413 decodes the leaf information according to the decoded duplicate dot merge flag information and generates a 3D point cloud (3D positions). It should be noted that the entropy decoder 2413 may decode attribute information regarding the decoded 3D points.

[1139] An inverse quantizer 2414 performs inverse quantization on the 3D positions of the decoded point cloud and generates an output 3D point cloud. It should be noted that in the case of lossless coding, inverse quantization may be skipped. In addition, when the scale need not necessarily return to the original scale, the 3D data decoding apparatus may skip inverse quantization even in the case of lossy coding. For example, if not an absolute positional relationship between 3D points but a relative positional relationship between 3D points is needed, the 3D decoder may skip inverse quantization.

[1140] Next, variation of the 3D data encoding process performed by the 3D data encoding device will be described. Fig. 183 is a flowchart for varying a 3D data encoding process.

[1141] First, the 3D data encoder quantizes the 3D positions of the inputted 3D point cloud (S2441). For example, as an example of 3D position quantization, the 3D data encoder calculates the quantization position (x/qx, y/qy, z/qz) by dividing the 3D position (x, y, z) by the quantization parameter (qx, qy, qz) . Further, the 3D data encoding apparatus may append a quantization parameter to the header, and the 3D data decoding apparatus may perform inverse quantization using the quantization parameter. It should be noted that the 3D data encoder may skip quantization during lossless encoding.

[1142] Next, the 3D data encoder determines whether or not the quantized 3D point cloud includes duplicated points (S2442). For example, the 3D data encoder compares the pieces of 3D position information of all 3D point clouds and makes a determination based on whether or not there is an identical value. Alternatively, the 3D data encoder may calculate the difference between all pieces of 3D position information and determine that the quantized 3D point cloud does not include duplicate points when the absolute value of the difference exceeds a predetermined threshold.

[1143] When the 3D point cloud includes duplicate points ("Yes" in S2442), the 3D data encoding device determines whether or not to combine the duplicate points, and perform encoding (S2443). For example, when the 3D data encoder prioritizes encoding efficiency, the 3D data encoder may determine whether to merge the duplicated points. When the duplicated points are mandatory in the 3D data decoding device, the 3D data encoding device may also determine the need not to merge the duplicated points.

[1144] When the 3D point cloud does not include duplicated points ("No" in S2442), or the duplicated points are to be merged ("Yes" in S2443), the 3D data encoding device sets the duplicated point merging flag to 1 and appends the duplicated merging flag dots to the title (S2444). On the contrary, when the duplicated points are not to be merged ("No" in S2443), the 3D data encoding device sets the duplicated points merging flag to 0 and attaches the duplicated points merging flag to the header (S2445).

[1145] Thereafter, the 3D data encoding apparatus determines whether or not the duplicated dot merge flag is equal to 1 (S2446). When the duplicated point merging flag is 1 (Yes in S2446), the 3D data encoder combines the duplicated points of the quantized 3D point cloud (S2447). It should be noted that when lossless encoding is performed and the input 3D point cloud does not include duplicated points, the 3D data encoding device may skip this step. When the duplicated point merging flag is 0 ("No" in S2446), the 3D data encoder does not combine the duplicated points of the quantized 3D point cloud. The following steps are identical to those illustrated in FIG. 179.

[1146] As above, when a leaf included in the N-ary tree structure of 3D points included in 3D data includes two 3D points, and the difference between the 3D positions of the two 3D points is less than a predetermined threshold value, where N is an integer a number greater than or equal to 2, the 3D data encoding device according to the present embodiment appends, to the bit stream, first information (eg, duplicated point merge flag) indicating whether or not to merge two 3D points; when the first information indicates that the two 3D points are to be combined, the 3D data encoding device combines the two 3D points and encodes the combined 3D point; and when the first information indicates that the two 3D points are not to be combined, the 3D encoding device encodes each of the two 3D points.

[1147] According to this configuration, the 3D data encoding apparatus can not only select whether or not to merge 3D points, but also notify the 3D decoding apparatus as to whether or not the 3D points are combined.

[1148] For example, when the first information indicates that two 3D points are not to be combined, the 3D data encoding apparatus appends the second sheet-related information (eg, sheet information) to the bitstream; and when the first information indicates that two 3D points are to be combined, the 3D encoder does not append the sheet information to the bitstream.

[1149] For example, the second information indicates the number of three-dimensional points included in the sheet.

[1150] For example, the 3D data encoder quantizes the 3D positions of the inputted 3D points to generate the 3D positions of the 3D points.

[1151] For example, the 3D data encoding device includes a processor and a memory, and the processor performs the above process using the memory.

[1152] When a leaf included in an N-ary tree structure of 3D points included in 3D data includes two 3D points, and the difference between the 3D positions of the two 3D points is less than a predetermined threshold value, where N is an integer greater than or set to 2, the 3D data decoding apparatus according to the present embodiment decodes, from the bit stream, first information (eg, duplicated point merging flag) indicating whether or not two 3D points are merged; when the first information indicates that the two 3D points are not combined, the 3D data decoding apparatus decodes the second leaf-related information (eg, leaf information) from the bit stream; and when the first information indicates that two 3D points are combined, the 3D data decoding apparatus does not decode the sheet information from the bit stream.

[1153] For example, the second information indicates the number of 3D points included in the sheet.

[1154] For example, the 3D data decoding apparatus recovers the 3D positions of the 3D points using the leaf information and inversely quantizes the recovered 3D positions.

[1155] For example, the 3D data decoding device includes a processor and a memory, and the processor performs the above process using the memory.

[1156] It should be noted that the methods of the present embodiment can be applied even when duplicate dots and non-duplicated dots are present in the quantized sheet. Fig. 184 is a diagram to illustrate this process. For example, as illustrated in FIG. 184, point A, point B, and point D are present in the sheet in a state before quantization. The quantization causes point A and point B to have the same 3D position, and point A and point B become duplicate points. On the contrary, since point D has a different 3D position relative to other points, point D does not become a duplicate point. Such a case occurs when the division of the octant tree is not performed down to the smallest division unit, and the node is encoded as a leaf in processing.

[1157] In such a case, when the MergeDuplicatedPointFlag is set to 0, the 3D encoding device encodes information indicating how many point clouds each sheet includes, position information of each point cloud, and attribute information of each point cloud. Here, the position information may be absolute coordinates relative to the reference position of each point cloud, or may include relative coordinates of points included in the sheet as described above. For example, the 3D data encoding apparatus may encode information indicating that the sheet includes point A, point B, and point D, position information of each of the points, and attribute information of each of the points.

[1158] In addition, when the duplicated dot merge flag is 1, the 3D encoder merges the duplicated dots in the sheet. For example, the 3D data encoding device combines point A and point B, and encodes position information and attribute information after combining. At this time, since the D point is not merged, the 3D encoding device encodes the position information and the attribute information of the D point separately.

[1159] In addition, the 3D data encoding device may encode information indicating the number of dots included in the sheet after merging. In the example illustrated in FIG. 184, since the 3D coder merges point A and point B into a point but does not merge point D, the 3D coder encodes the number of points included in the sheet as 2.

[1160] When the duplicated point merging flag is 1, the 3D data encoding device may combine the M duplicated points in a sheet into N points, where M>N. In this case, the 3D data encoding device can encode, as leaf information, each of the pieces of 3D position information and N point attribute information and each of the pieces of position information and non-merged point attribute information. In addition, the 3D data encoding device can calculate N pieces of attribute information using M pieces of attribute information.

[1161] In addition, the 3D data encoding device may encode information indicating the number of dots included in a sheet. In the example illustrated in FIG. 184, the 3D encoding device may encode information indicating the number of points included in the sheet as 3. Further, the 3D encoding device may encode pieces of position information and attribute information of all points.

[1162] Embodiment 16

In the present embodiment, the 3D data encoding device acquires information of neighbor nodes each having a different parent node by performing a search for coded nodes. Fig. 185 is a diagram illustrating an example of neighboring nodes. In the example illustrated in FIG. 185, three neighboring nodes belong to a parent node identical to the parent node with respect to the current node. The 3D encoder obtains the surrounding information of these three neighboring nodes by checking the padding code of the parent node.

[1163] The three remaining neighboring nodes belong to a parent node that is different from the parent node relative to the current node. The 3D encoder obtains the surrounding information of these three neighboring nodes by checking the information of the encoded nodes. Here, the environment information includes information indicating whether or not the node includes a point cloud (is it busy or not). In addition, the encoded node, for example, is a node belonging to a layer identical to the layer of the current node in the octant tree.

[1164] FIG. 186 is a diagram illustrating an example of nodes to be searched. The 3D data encoding device searches in the search range including the encoded nodes illustrated in FIG. 186 for neighbor information. Fig. 187 is a diagram for illustrating a neighbor search process. As illustrated in FIG. 187, the encoded node information is stored in a queue. The 3D encoder obtains neighbor information by performing a queue search from the head end. For example, the search order for a queue is the encoding order.

[1165] The 3D data encoder calculates the fill code of the current node by calculating information indicating whether or not the child nodes are busy. At this time, the 3D encoder updates the environment information of each child node. For example, the 3D data encoder determines whether or not a neighbor node having a parent node identical to the parent node with respect to the current node is occupied based on the fill code. In addition, the 3D data encoding device searches the queue that stores the encoded node information for information indicating whether or not a neighbor node having a parent node different from the parent node with respect to the current node is occupied, and determines whether whether or not a neighbor node having a parent node different from the parent node relative to the current node is occupied, based on the information sought. In addition, the 3D encoder updates the environment information of each child node, and stores the updated environment information in a queue to calculate the child node's neighbor node for the next node.

[1166] Each time a search is performed, the 3D encoder updates the surrounding information of both the current node and the searched node. Fig. 188 and FIG. 189 are a diagram to illustrate this upgrade process. As illustrated in FIG. 188, each time a search is performed, the 3D encoder updates the surrounding information of both the current node and the searched node. In other words, the surrounding information is transmitted in both directions. That the searched node is a neighbor is added to the current node information, and that the current node is a neighbor is added to the neighbor information.

[1167] As illustrated in FIG. 189, in the search process, the immediately preceding current node may become the searched node. In this case, the environment information of the immediately preceding current node is updated.

[1168] In order to provide the longest processing time for a hardware implementation, the 3D encoder may complete the search process before a neighbor is located. Fig. 190 is a diagram to illustrate this operation.

[1169] As illustrated in FIG. 190, the search threshold is predetermined, i. e. represents the threshold value for stopping the search. This search threshold indicates, for example, the number of searches performed on the queue from the head end.

[1170] In the example illustrated in (1) of FIG. 190, a larger number of search steps than the search threshold is required in order to search the queue for neighbor information. In this example, the 3D encoder searches up to a search threshold and terminates the search process.

[1171] In the example illustrated in (2) of FIG. 190, it is possible to search the queue for a neighbor item with fewer search steps than the search threshold. In this example, the 3D encoder searches for a neighbor node and completes the search process.

[1172] As mentioned above, the 3D data encoder may provide a parameter (search threshold) to limit the number of searches. By limiting the number of searches, it is possible to find a neighbor while maintaining the processing time to perform the search within a certain time. Further, the 3D data encoder may attach, to a title, etc. bitstream, information indicating a limit value (search threshold) for the number of searches. Alternatively, the number of searches may be indicated by standards, etc. Accordingly, since the 3D decoding apparatus can determine the limit value for the number of searches from the header or standards requirements, the 3D decoding apparatus can decode the stream correctly.

[1173] The following describes a specific example of the encoded node queue structure. To identify the environment of the current node, each element of the encoded node queue has an index in three-dimensional space. Examples of this index include the Morton code.

[1174] FIG. 191 is a diagram illustrating an example of indices for which Morton codes are used. Fig. 192 is a diagram illustrating an example of a queue for which Morton codes are used. In the example illustrated in FIG. 191, the current node has an index of 3, the left node has an index of 2, and the bottom node has an index of 1. It is possible to determine the neighbor node using Morton codes as indices in the above manner.

[1175] The use of Morton codes generates the following advantages. The first advantage allows you to speed up the search process. Here, the search process that uses x, y, z coordinates is more complex than the process of finding the Morton code, which is a single integer.

[1176] The second advantage allows you to reduce the amount of data that must be stored through the use of Morton codes. In particular, when x, y, z coordinates are used, three 32-bit data are required. In contrast, a node can be identified by 64-bit data, through the use of Morton codes.

[1177] It should be noted that any method other than Morton codes can be used as a method for converting a three-dimensional position to an integer. For example, a space-filling curve capable of converting a three-dimensional position to an integer, such as a Hilbert curve, may be used.

[1178] Next, a configuration example of a 3D data encoding apparatus according to the present embodiment will be described. Fig. 193 is a block diagram of a 3D data encoding apparatus 2500 according to the present embodiment. The 3D data encoding device 2500 includes an octant tree generator 2501, a parent node information acquisition module 2502, an encoding mode selection module 2503, a search module 2504, a geometric information calculation module 2505, a codebook selection module 2506, and an entropy encoder 2507.

[1179] The octant tree builder 2501 generates, for example, an octant tree from inputted 3D points (point clouds) and generates a fill code for each node of the octant tree.

[1180] The parent node information obtaining module 2502 obtains neighbor node environment information from the parent node's padding code with respect to the current node. In other words, the parent node information obtaining module 2502 acquires, for example, neighbor node neighborhood information that is, among neighbor nodes, neighbor nodes belonging to a parent node identical to the parent node with respect to the current node, and takes into account half of the neighbor nodes.

[1181] The coding mode selector 2503 selects the coding mode (coding mode). For example, this encoding mode includes a mode for performing one of a search process and a process for obtaining environment information from a parent node's padding code, and a mode for performing both processes.

[1182] The searching unit 2504 obtains the neighborhood information of a neighbor node through a searching process using the encoded node information. Although this search process requires some processing time, the search process makes it possible to obtain the surrounding information of all neighboring nodes.

[1183] The geometry information calculation unit 2505 generates ambience information (neighbor padding information) to be used to select a codebook by using one of the ambience information obtained by the parent node information acquisition unit 2502 and the environment information obtained by the unit 2504 search or by integrating both.

[1184] Codebook selection module 2506 selects a codebook to be used for entropy coding using neighbor node padding information generated by geometry information calculation module 2505.

[1185] The entropy encoder 2507 generates a bitstream by entropy encoding the fill code of the current node using the selected codebook. It should be noted that the entropy encoder 2507 may append, to the bitstream, information indicating the selected codebook.

[1186] Next, a configuration example of a 3D data decoding apparatus according to the present embodiment will be described. Fig. 194 is a block diagram of a 3D data decoding apparatus 2510 according to the present embodiment. The 3D data decoding apparatus 2510 includes an octant tree generator 2511, a parent node information acquisition module 2512, a decoding mode selection module 2513, a search module 2514, a geometric information calculation module 2515, a codebook selection module 2516, and an entropy decoder 2517.

[1187] The octant tree builder 2511 generates a space (node) octant tree using header information, etc. bit stream. For example, the octant tree builder 2511 generates a large space (root node) using the size of the space along the X-axis, Z-axis, and Y-axis directions attached to the header information, and generates an octant tree by generating eight small spaces A (nodes A0- A7) by splitting the space into two along each of the X-axis, Z-axis, and Y-axis directions. In addition, nodes A0-A7 are set as the current node in the sequence.

[1188] The parent node information obtaining module 2512 obtains neighbor node environment information from the parent node's padding code with respect to the current node. In other words, the parent node information obtaining module 2512 acquires, for example, neighbor node neighborhood information that is, among neighbor nodes, neighbor nodes belonging to a parent node identical to the parent node with respect to the current node, and takes into account half of the neighbor nodes.

[1189] The decoding mode selector 2513 selects the decoding mode. For example, this decoding mode corresponds to the above encoding mode, and includes a mode for performing one of a search process and a process for obtaining environment information from a parent node's fill code, and a mode for performing both processes.

[1190] The searching module 2514 obtains the neighborhood information of the neighbor node through the search process using the information of the decoded nodes. Although this search process requires some processing time, the search process makes it possible to obtain the surrounding information of all neighboring nodes.

[1191] The geometry information calculation unit 2515 generates the environment information (neighbor node padding information) to be used for selecting a codebook by using one of the environment information obtained through the parent node information acquisition unit 2512 and the environment information obtained through the unit 2514 search or by integrating both.

[1192] Codebook selection module 2516 selects a codebook to be used for entropy decoding using neighbor node padding information generated by geometry information calculation module 2515.

[1193] The entropy decoder 2517 generates 3D points (point cloud) by entropy decoding the fill code using the selected codebook. It should be noted that the entropy decoder 2517 can obtain the selected codebook information from the bitstream and entropy decode the current node's stuff code using the codebook indicated by the information.

[1194] Each bit of the fill code (8 bits) included in the bit stream indicates whether or not the corresponding one of the eight small spaces A (nodes A0-A7) includes the point cloud. Further, the 3D data decoding apparatus generates an octant tree by dividing the small space node A0 into eight small B spaces (B0 to B7 nodes), and calculates information indicating whether or not each small space B node includes a point cloud by decoding the code filling. As mentioned above, the 3D data decoding apparatus decodes the padding code of each node when forming an octant tree by dividing a large space into small spaces.

[1195] Hereinafter, procedures for a 3D data encoding process and a 3D data decoding process according to the present embodiment are described. Fig. 195 is a flowchart for a 3D data encoding process performed by a 3D data encoding apparatus.

[1196] First, the 3D data encoder specifies a space (current node) including part or all of the inputted 3D point cloud (S2501). Then, the 3D data encoding apparatus generates eight small spaces (nodes) by dividing the current node into eight (S2502). Thereafter, the 3D data encoding device generates a fill code of the current node according to whether or not each node includes a point cloud (S2503). Thereafter, the 3D data encoding apparatus calculates the neighborhood information of the neighbor node with respect to the current node from the fill code of the parent node with respect to the current node (S2504).

[1197] Next, the 3D encoding device selects an encoding mode (S2505). For example, a 3D data encoding device selects an encoding mode to perform a search process. Thereafter, the 3D data encoding apparatus obtains the remaining ambient information by searching for encoded node information. In addition, the 3D data encoding device generates ambience information to be used for selecting a coding table by integrating the ambiance information calculated in step S2504 and the ambience information obtained through the search process (S2506).

[1198] Thereafter, the 3D data encoding apparatus selects a codebook to be used for entropy encoding based on the environment information generated in step S2506 (S2507). Then, the 3D data encoding device entropy encodes the fill code of the current node using the selected codebook (S2508). Finally, the 3D data encoding apparatus repeats the process of dividing each node into eight and encoding the fill code of each node until each node can be separated (S2509). In other words, steps S2502-S2508 are recursively repeated.

[1199] FIG. 196 is a flowchart for a 3D data decoding process performed by a 3D data decoding apparatus.

[1200] First, the 3D data decoding apparatus specifies a space (current node) to be decoded using bitstream header information (S2511). Then, the 3D data decoding apparatus generates eight small spaces (nodes) by dividing the current node into eight (S2512). Thereafter, the 3D data decoding apparatus calculates the neighborhood information of the neighbor node with respect to the current node from the fill code of the parent node with respect to the current node (S2513).

[1201] Thereafter, the 3D data decoding apparatus selects a decoding mode corresponding to the above encoding mode (S2514). For example, the 3D data decoding apparatus selects a decoding mode to perform a search process. Then, the 3D data decoding apparatus obtains the remaining ambience information by searching for decoded node information. In addition, the 3D data decoding apparatus generates ambience information to be used for selecting a codebook by integrating the ambience information calculated in step S2513 and the ambiance information obtained through the search process (S2515).

[1202] Thereafter, the 3D data decoding apparatus selects a codebook to be used for entropy decoding based on the environment information generated in step S2515 (S2516). Thereafter, the 3D data decoding apparatus entropy decodes the fill code of the current node using the selected codebook (S2517). Finally, the 3D data decoding apparatus repeats the process of dividing each node into eight and decoding the fill code of each node until each node can be separated (S2518). In other words, steps S2512-S2517 are recursively repeated.

[1203] Next, the encoding mode (decoding mode) will be described. The encoding mode includes at least one of (1) a first mode for skipping a search process, (2) a second mode for performing a search process and terminating the process at the above search threshold, or (3) a third mode for performing a search process and searching for all encoded (decoded) nodes.

[1204] In other words, the 3D data decoding apparatus may select, as the encoding mode, whether to skip neighbor search or perform neighbor search. In addition, when the 3D data encoder performs a neighbor search, the 3D data encoder can limit the number of searches to at most a predetermined threshold value. In addition, the 3D data encoder may append information indicating this threshold value to the header of the bitstream. Alternatively, the threshold value may be specified by standards, etc. Additionally, the 3D data encoder may change the threshold value for each node. For example, the number of possible neighbor nodes increases as the value of the octant tree layer increases (deepens). For this reason, the 3D data encoder may increase the threshold value as the value of the octant tree layer increases (deepens). The linkable range can change for each layer to which the nodes belong. In other words, the range for one or more linkable neighbor nodes from among neighbor nodes spatially adjacent to the current node may vary according to the layer to which the current node belongs in the tree structure. Here, even when the parameter values indicating the linkable range given for the header, etc., are identical, the space represented by the node is reduced with a deeper layer. In other words, the range of space in which nodes are linkable can be much narrower with a deeper layer.

[1205] In addition, the 3D data encoding apparatus may append information indicative of an encoding mode to a bitstream header. The encoding mode may be specified by standards, etc. As a consequence, since the 3D decoding apparatus can determine the decoding mode (encoding mode) from the decoded header or standards requirements, the 3D decoding apparatus can decode the stream correctly.

[1206] In addition, the 3D encoding device can encode the encoding mode for each node and change the encoding mode for each node. For example, the 3D data encoding apparatus performs encoding using all encoding modes once and determines the encoding mode that is most suitable for the 3D data decoding apparatus in terms of encoding efficiency and processing time. Thereafter, the 3D data encoding apparatus may encode, for each node, information indicating the specific encoding mode. As a result, the 3D data decoding apparatus can decode the bitstream correctly by decoding the coding mode encoded for each node.

[1207] In addition, the 3D data encoding device can encode an encoding mode for each set of predetermined nodes and change the encoding mode based on the set. It should be noted that a node set, for example, is a set of nodes included in an identical layer of an octant tree.

[1208] It should be noted that the 3D data decoding apparatus can also determine the decoding mode (coding mode) identically. In other words, the 3D data encoding device and the 3D data decoding device can judge the coding mode for each node identically, and choose whether or not to perform a neighbor node search for each node. As a result, the 3D data encoding apparatus and the 3D data decoding apparatus determine whether or not the current node requires a neighbor search, perform a neighbor search when determining that the current node requires a neighbor search, and skip the neighbor search when determining that the current node does not require searching for neighboring nodes. In addition, it is not necessary to transmit information indicating an encoding mode. Accordingly, it is possible to reduce the amount of processing while increasing the coding efficiency.

[1209] For example, the 3D data encoder and the 3D data decoder determine whether or not the current node needs to search for neighboring nodes, for example, from the parent node's padding code. Here, when all the filling information of the three neighboring nodes calculated from the filling code of the parent node is 1, there is a high probability that other neighboring nodes are occupied. For this reason, in this case, the 3D data encoding device and the 3D data decoding device determine that the current node needs to search for neighboring nodes.

[1210] In addition, the 3D data encoding device and the 3D data decoding device can determine whether or not the current node needs to search for neighbor nodes from the octant tree layer value. For example, when the layer has a small value (is close to the root node), there is a high probability that the octant tree splitting is not done sufficiently. For this reason, the 3D data encoding apparatus and the 3D data decoding apparatus can determine that neighboring nodes are likely to be busy, and determine that the current node needs to search for neighboring nodes. As mentioned above, the 3D data encoding apparatus and the 3D data decoding apparatus can perform encoding and decoding correctly while reducing the amount of code by judging the encoding mode identically.

[1211] In addition, the 3D data encoding apparatus and the 3D data decoding apparatus may evaluate the encoding mode (decoding mode) for each set of predetermined nodes identically and change the encoding mode based on the set. As a result, the 3D data encoding apparatus and the 3D data decoding apparatus determine whether or not the node set requires a neighbor search, perform a neighbor search when determining that the node set requires neighbor search, and skip the neighbor search when determining that a set of nodes does not require searching for neighboring nodes. Accordingly, it is possible to reduce the amount of processing while increasing the coding efficiency.

[1212] It should be noted that a set of nodes, for example, is a set of nodes included in the same layer of the octant tree. Since this allows the 3D data encoder and the 3D data decoder to choose whether or not to perform a neighbor search for each layer, the 3D data encoder and the 3D data decoder can improve encoding efficiency while reducing processing time. For example, when the layer has a small value (is close to the root node), there is a high probability that the octant tree splitting is not done sufficiently. For this reason, the 3D data encoding apparatus and the 3D data decoding apparatus can determine that neighboring nodes are likely to be busy, and determine that the current node needs to search for neighboring nodes.

[1213] The following describes an example of the syntax of information, etc., indicating the encoding mode. Fig. 197 is a diagram illustrating an example of the header information syntax. Fig. 198 is a diagram illustrating an example of node information syntax.

[1214] As illustrated in FIG. 197, the header information includes coding_mode1 and limit_num_of_search; coding_mode1 is information indicating whether or not to perform a neighbor search. For example, a value of 0 indicates that a neighbor search should not be performed, a value of 1 indicates that a neighbor search should be performed for all nodes, and a value of 2 indicates that a neighbor search should be varied for each node.

[1215] It should be noted that the 3D encoder may indicate whether or not to perform a neighbor search based on standards or a profile or level, etc. standards without attaching coding_mode1 to the header. This enables the 3D data decoding apparatus to determine whether or not a neighbor search has been performed with reference to the standards information, and recover the bitstream correctly.

[1216] Limit_num_of_search is information indicating a neighbor threshold, and indicates, for example, a limit for the number of searches (search threshold) when a neighbor search is performed. For example, a value of 0 indicates no limit on the number of searches, and a value of at least 1 indicates a limit on the number of searches.

[1217] Limit_num_of_search is included in the header information when the value of coding_mode1 is at least 1. It should be noted that the 3D encoding device need not necessarily include limit_num_of_search in the header when there is always no need to limit the search. In addition, the 3D encoder may provide limit_num_of_search for each layer of the octant tree and include limit_num_of_search in the header.

[1218] Additionally, the value of limit_num_of_search may be determined before encoding. For example, the value is not set to the limit when the high performance device performs encoding or decoding, and the value is set to the limit when the low performance device performs encoding or decoding.

[1219] As illustrated in FIG. 198, the node information includes coding_mode2 and occupancy_code; coding_mode2 is included in the node information when the value of coding_mode1 is 2; coding_mode2 is information indicating whether or not to perform neighbor node search for each node. For example, a value of 0 indicates that a neighbor search should not be performed, and a value of 1 indicates that a neighbor search should be performed.

[1220] It should be noted that when coding_mode2 is equal to 1, the 3D data encoding device and the 3D data decoding device may set a limit for the number of searches to limit_num_of_search attached to the header. In addition, the 3D data encoding apparatus may encode, for each node, information indicating a limit for the number of searches.

[1221] In addition, the 3D data encoding device does not need to encode the coding_mode2 value, and the 3D data decoding device may evaluate the coding_mode2 value. For example, the 3D data decoder evaluates the coding_mode2 value from the parent node's padding code or octant tree layer information.

[1222] Occupancy_code is the occupancy code of the current node, and is information indicating whether or not the child nodes are occupied with respect to the current node. The 3D data encoding apparatus and the 3D data decoding apparatus calculate neighbor occupancy information according to the value of coding_mode2, and encode or decode the occupancy_code when changing the coding table based on the value.

[1223] Further, the 3D data encoding apparatus can entropy encode coding_mode1, limit_num_of_search, or coding_mode2 generated by the above method. For example, a 3D data encoder binarizes each value and performs arithmetic encoding on the value.

[1224] Although the octant tree structure is described as an example in the present embodiment, the present disclosure is not necessarily limited thereto. The above procedure can be applied to an N-ary tree, such as a binary tree, a quadtree, and a hexadecant tree, or other tree structures, where N is an integer greater than or equal to 2.

[1225] The following describes the details of the encoding process of three-dimensional data. Fig. 199 is a flowchart for a 3D data encoding process according to the present embodiment. First, the 3D data encoding device specifies a space (current node) including part or all of the inputted 3D point cloud (S2521). Then, the 3D data encoding apparatus generates eight small spaces (nodes) by dividing the current node into eight (S2522). Thereafter, the 3D data encoding device generates a fill code of the current node according to whether or not each node includes a point cloud (S2523). Thereafter, the 3D data encoding apparatus calculates neighbor node surrounding information regarding the current node from the parent node's fill code regarding the current node (S2524).

[1226] Next, the 3D data encoding apparatus determines whether or not to execute the search process by checking the encoding mode (S2525). For example, when (1) coding_mode1 is equal to 1 or (2) coding_mode1 is equal to 2, and coding_mode2 is equal to 1, the 3D data encoding apparatus determines to perform the search process ("Yes" in S2525); and in other cases, the 3D encoder does not perform the searching process ("No" in S2525). It should be noted that the 3D data encoding apparatus determines whether or not to perform a neighbor search for all nodes (coding_mode1 value), and whether or not to perform a neighbor search for each node (coding_mode2 value), by the above method, etc. d.

[1227] For example, the 3D data coding device judges whether or not the current node needs to search for neighbor nodes (coding_mode2 value) from the padding code of the parent node. Here, when all the filling information of the three neighboring nodes calculated from the filling code of the parent node is 1, there is a high probability that other neighboring nodes are occupied. For this reason, the 3D data coding device determines that the current node needs to search for neighboring nodes (the value of coding_mode2 is 1). In addition, when the 3D data decoding apparatus evaluates coding_mode2, the 3D data encoding apparatus does not need to encode coding_mode2.

[1228] When the 3D data encoder performs the search process ("Yes" in S2525), the 3D data encoder obtains the remaining ambient information by searching the encoded node information. For example, when the value of limit_num_of_search is not 0 (no limit for the number of searches), the 3D data encoding device searches for a neighbor node while limiting the number of searches according to the value. In addition, the 3D encoder sets the value of limit_num_of_search using the above method, and so on. Further, the 3D data encoding device integrates the environment information calculated from the parent node's fill code and the environment information obtained through the searching process (S2526). Thereafter, the 3D data encoding device selects a codebook to be used for entropy encoding based on the environment information generated in step S2526 (S2527).

[1229] In contrast, when the 3D encoding device does not perform a search process ("No" in S2525), the 3D encoding device selects a codebook to be used for entropy encoding based on the environment information calculated from the padding code of the parent node in step S2524 (S2527).

[1230] Thereafter, the 3D data encoding device entropy encodes the padding code of the current node using the selected codebook (S2528). In addition, the 3D encoding device encodes coding_mode1 and limit_num_of_search as header information. In addition, the 3D encoder encodes coding_mode2 for each node.

[1231] Finally, the 3D data encoding apparatus repeats the process of dividing each node into eight and encoding the fill code of each node until each node can be separated (S2529). In other words, steps S2522-S2528 are recursively repeated.

[1232] The following describes the details of the process of decoding three-dimensional data. Fig. 200 is a flowchart for a 3D data decoding process according to the present embodiment. First, the 3D data decoding apparatus specifies a space (current node) to be decoded using bitstream header information (S2531). At this time, the 3D data decoding apparatus decodes the coding_mode1 and limit_num_of_search header information.

[1233] Next, the 3D data decoding apparatus generates eight small spaces (nodes) by dividing the current node into eight (S2532). Thereafter, the 3D data decoding apparatus calculates the neighborhood information of the neighbor node with respect to the current node from the fill code of the parent node with respect to the current node (S2533).

[1234] Thereafter, the 3D data decoding apparatus determines whether or not to perform the searching process by checking the decoding mode corresponding to the encoding mode (S2534). For example, when (1) coding_mode1 is equal to 1 or (2) coding_mode1 is equal to 2, and coding_mode2 is equal to 1, the 3D data decoding apparatus determines to perform the searching process ("Yes" in S2534); and in other cases, the 3D data decoding apparatus does not perform the searching process ("No" in S2534). In addition, the 3D decoder decodes coding_mode2 for each node, for example.

[1235] It should be noted that the 3D data decoding apparatus can determine whether or not the current neighbor search node (coding_mode2 value) is required using a process identical to the process in the 3D data encoding apparatus. For example, the 3D decoder judges whether or not the current node needs to search for neighbor nodes from the padding code of the parent node. Here, when all the filling information of the three neighboring nodes calculated from the filling code of the parent node is 1, there is a high probability that other neighboring nodes are occupied. For this reason, the 3D data decoding apparatus determines that the current node requires neighbor node search (coding_mode2 value is 1). In addition, when the 3D data decoding apparatus evaluates coding_mode2, the 3D data decoding apparatus need not necessarily decode coding_mode2.

[1236] Then, when the 3D data decoding apparatus performs the search process ("Yes" in S2534), the 3D data decoding apparatus acquires the remaining ambient information by searching for decoded node information. For example, when the value of limit_num_of_search is not equal to 0 (no limit for the number of searches), the 3D data decoding apparatus searches for a neighbor node while limiting the number of searches according to the value. Further, the 3D data decoding apparatus integrates the environment information calculated from the fill code of the parent node and the environment information obtained through the searching process (S2535). Thereafter, the 3D data decoding apparatus selects a codebook to be used for entropy decoding based on the environment information generated in step S2535 (S2536).

[1237] In contrast, when the 3D data decoding apparatus does not perform a search process ("No" in S2534), the 3D data decoding apparatus selects a codebook to be used for entropy decoding based on the environment information calculated from the padding code of the parent node in step S2533, and the environment information obtained through the search process (S2536).

[1238] Thereafter, the 3D data decoding apparatus entropy decodes the padding code of the current node using the selected codebook (S2537). Finally, the 3D data decoding apparatus repeats the process of dividing each node into eight and decoding the fill code of each node until each node can be separated (S2538). In other words, steps S2532-S2537 are recursively repeated.

[1239] It should be noted that the above description shows an example in which the nodes to be searched are encoded nodes, the nodes to be searched are not necessarily limited to this. For example, the 3D data encoding device can obtain neighbor information of all nodes belonging to the same layer by performing a search using the method described in the present embodiment, and then can encode the padding code of each node using the obtained neighbor node information.

[1240] As mentioned above, the 3D data encoding device according to the present embodiment performs the process illustrated in FIG. 201. The 3D encoder encodes information of the current node included in the N-ary tree structure of 3D points included in the 3D data, where N is an integer greater than or equal to 2. When encoding, the 3D encoder encodes the first information (for example, limit_num_of_search) indicating a range for one or more linkable neighbor nodes from among neighbor nodes spatially adjacent to the current node (S2541), and encodes the current node with reference to a neighbor node in the range (S2542).

[1241] Because of this, since the 3D data encoder limits the linkable neighboring nodes, the 3D data encoder reduces the amount of processing.

[1242] For example, when encoding, the 3D data encoder selects a coding table based on whether or not a neighboring node within the range includes a 3D point, and entropy encodes information (e.g., padding code) of the current node using the selected coding table.

[1243] For example, when encoding, the 3D data encoding device searches for information of one or more linkable neighboring nodes from among neighboring nodes spatially adjacent to the current node, and the first information indicates a search range.

[1244] For example, when searching, the 3D data encoding device searches for information of nodes in a predetermined order, and the first information indicates the total number of nodes (eg, search threshold) for which the search is to be performed.

[1245] For example, when searching, indexes of Morton codes are used.

[1246] For example, when encoding, the 3D data coding device encodes second information (coding_mode1) indicating whether or not the range for one or more linkable neighbor nodes should be limited, and encodes the first information when the second information indicates that the range for one or more linkable neighbors must be limited.

[1247] For example, the range for one or more linkable neighboring nodes changes according to the layer to which the current node belongs in the N-ary tree structure.

[1248] For example, the 3D data encoding device includes a processor and a memory, and the processor performs the above process using the memory.

[1249] In addition, the 3D data decoding apparatus according to the present embodiment performs the process illustrated in FIG. 202. The 3D data decoding apparatus decodes information of the current node included in the N-ary tree structure of 3D points included in the 3D data, where N is an integer greater than or equal to 2. When decoding, the 3D data decoding apparatus decodes, from the bit stream, the first information (e.g., limit_num_of_search) indicating a range for one or more linkable neighbor nodes from among the neighbor nodes spatially adjacent to the current node (S2551), and decodes the current node with reference to a neighbor node in the range (S2552).

[1250] Because of this, since the 3D data decoding apparatus limits the linkable neighboring nodes, the 3D data decoding apparatus reduces the amount of processing.

[1251] For example, when decoding, the 3D data decoding apparatus selects a codebook based on whether or not a neighboring node within a range includes a 3D point, and entropy decodes the information (e.g., padding code) of the current node using the selected codebook.

[1252] For example, when decoding, the 3D data decoding apparatus searches for information of one or more linkable neighbor nodes from among neighbor nodes spatially adjacent to the current node, and the first information indicates a search range.

[1253] For example, when searching, the 3D data decoding apparatus searches for node information in a predetermined order, and the first information indicates the total number of nodes (eg, search threshold) for which to search.

[1254] For example, when searching, indexes of Morton codes are used.

[1255] For example, when decoding, the 3D data decoding apparatus decodes second information (coding_mode1) indicating whether or not the range for one or more linkable neighbor nodes should be limited, and decodes the first information when the second information indicates that the range for one or more linkable neighbors must be limited.

[1256] For example, the range for one or more linkable neighboring nodes changes according to the layer to which the current node belongs in the N-ary tree structure.

[1257] For example, the 3D data decoding device includes a processor and a memory, and the processor performs the above process using the memory.

[1258] Embodiment 17

In the present embodiment, the 3D data encoding apparatus divides the inputted 3D point cloud into two or more 3D point subclouds, and encodes each of the 3D point subclouds such that the 3D point subclouds are not dependent on each other. Accordingly, the 3D data encoding device can encode the 3D point subclouds in parallel. For example, the 3D data encoding apparatus divides the inputted 3D point cloud into 3D point sub-cloud A and 3D point sub-cloud B, and encodes 3D point sub-cloud A and 3D point sub-cloud B in parallel.

[1259] It should be noted that, when, for example, the 3D data encoding device performs encoding using the octant tree structure, the 3D data encoding device encodes, in parallel, eight child nodes resulting from the octant tree splitting performed as the splitting method. . For example, a 3D data encoder encodes, in parallel, tree structures each having a corresponding one of the child nodes as a root.

[1260] It should be noted that the 3D data encoding apparatus need not always encode the 3D point subclouds in parallel, and may encode the 3D point subclouds sequentially such that the 3D point subclouds do not have a dependency relationship with each other. In addition, the method of the present embodiment can be applied not only to an octant tree, but also to an N-ary tree such as a quadtree or a hexadecant tree, where N is an integer greater than or equal to 2. In addition, the 3D data coding device may perform separation using attribute information such as color, specularity, or normal point cloud vector. Additionally, as described using FIG. 92, fig. 93 etc. in Embodiment 10, the 3D data encoding apparatus may perform separation based on a difference in point cloud density.

[1261] The 3D data coding device can also combine the coded data of the coded 3D point subclouds with the bit stream. At this time, the 3D data encoding device may include, in a header, etc. a bitstream, a starting position of each encoded data of a corresponding one of the encoded 3D point subclouds. For example, a 3D data encoder may include, in a header, etc., addresses (bit positions or number of bytes, etc.) from the head of the bit stream. As a result, the 3D data decoding apparatus can identify the start position of each encoded data of the corresponding one of the 3D dot subclouds by decoding the head of the bit stream. Further, since the 3D data decoding apparatus can decode encoded 3D point sub-cloud data in parallel, the 3D data decoding apparatus can reduce the processing time.

[1262] It should be noted that the 3D data encoding device may append, to the bitstream header, a flag indicating that the 3D point subclouds are encoded such that the 3D point subclouds do not have a dependency relationship with each other, or such that that subclouds of 3D points are encoded in parallel. As a consequence, the 3D data decoding apparatus can determine whether or not the encoded 3D point cloud data is decodable in parallel by decoding the header.

[1263] Here, that the 3D point subclouds do not have a dependency relationship with each other means, for example, that coding tables (probability tables, etc., to be used for entropy coding) for coding fill codes or sheet information, etc. for nodes of 3D point subclouds are stored independently of 3D point subclouds. For example, the 3D data encoding device uses a different codebook for each of the 3D point sub-cloud A and the 3D point sub-cloud B, such that the 3D point sub-cloud A and the 3D point sub-cloud B have no dependency relationship with each other. Alternatively, when the 3D encoding device processes 3D point sub-cloud A and 3D point sub-cloud B sequentially, the 3D encoding device initializes codebooks after encoding 3D point sub-cloud A and before encoding 3D point sub-cloud B such that 3D point sub-cloud A and sub-cloud B 3D points do not have a dependency relationship with each other. As mentioned above, by storing the codebooks for the 3D point subclouds independently of each other or initializing the codebooks before encoding, the 3D data encoding apparatus can code the 3D point subclouds such that the 3D point subclouds do not have a dependency relationship with each other. In addition, by storing code tables (decoding tables) for 3D point subclouds independently of each other or initializing code tables before decoding the 3D point subclouds, the 3D data decoding apparatus can decode the 3D point subclouds properly in the same way.

[1264] In addition, that the 3D point subclouds do not have a dependency relationship with each other means, for example, that reference between the 3D point subclouds is prohibited when fill codes or sheet information, etc. are encoded. for nodes of subclouds of three-dimensional points. For example, when the 3D data encoder encodes the padding code of the current node to be encoded, the 3D data encoder performs encoding using neighbor node information in the octant tree. In this case, when a neighbor node is included in another 3D point sub-cloud, the 3D encoding device encodes the current node without referring to the neighbor node. In particular, the 3D encoder may perform encoding under the condition that no neighbor is present, or the 3D encoder may encode the current node in a state in which although a neighbor is present, the neighbor is included in a different sub-3D point cloud.

[1265] Similarly, for example, when a 3D data decoding apparatus decodes fill codes or sheet information, etc. for 3D point subcloud nodes, the 3D data decoding apparatus prohibits the link between the 3D point subclouds. For example, when the 3D data decoding apparatus decodes the stuffing code of the current node to be decoded, the 3D data decoding apparatus performs decoding using neighbor node information in the octant tree. In this case, when a neighbor node is included in another sub-3D point cloud, the 3D data decoding apparatus decodes the current node without referring to the neighbor node. In particular, the 3D decoder may perform decoding under the condition that no neighbor is present, or the 3D decoder may decode the current node in a state in which although a neighbor is present, the neighbor is included in another sub-3D point cloud.

[1266] In addition, when the 3D data encoding device encodes the 3D position information pieces and the attribute information pieces (e.g., color, reflectance, or normal vector) of the 3D point subclouds, the 3D data encoding device may encode one of the 3D position information pieces and the information pieces attributes such that one of the 3D position information pieces and the attribute information pieces do not have a dependency relationship with each other, and may encode the other of the 3D position information pieces and the attribute information pieces such that the other of the 3D position information pieces and the pieces Attribute information has a dependency relationship with each other. For example, the 3D data encoding apparatus may encode the pieces of 3D position information such that the pieces of 3D position information have no dependency relationship with each other, and may encode the pieces of attribute information such that the pieces of attribute information have a dependency relationship with each other. friend. Accordingly, the 3D data encoding apparatus reduces processing time by encoding pieces of 3D position information in parallel, and reduces the amount of code by encoding pieces of attribute information in series. It should be noted that the 3D data encoding apparatus can append, to the header, both information indicating whether or not the pieces of 3D position information are encoded such that the pieces of 3D position information do not have a dependency relationship with each other, and information indicating whether or not the attribute information pieces are encoded such that the attribute information pieces do not have a dependency relationship with each other. As a result, by decoding the header, the 3D data decoding apparatus can determine whether or not the pieces of 3D position information are decodable such that the pieces of 3D position information do not have a dependency relationship with each other, and whether or not the pieces of attribute information are decodable such that the attribute information pieces do not have a dependency relationship with each other. For this reason, when there is no dependency relationship, the 3D data decoding apparatus can perform parallel decoding. For example, when the pieces of 3D position information are encoded such that the pieces of 3D position information do not have a dependency relationship with each other, and the pieces of attribute information are encoded such that the pieces of attribute information have a dependency relationship with each other, the 3D decoding apparatus data reduces processing time by decoding pieces of information of the three-dimensional position in parallel and decoding pieces of information of attributes sequentially.

[1267] FIG. 203 is a diagram illustrating an example of a tree structure. It should be noted that although FIG. 203 shows an example of a quadtree, other tree structures such as an octant tree may be used. The 3D encoder divides the tree structure shown in FIG. 203, for example, to the 3D point sub-cloud A shown in FIG. 204 and the 3D point sub-cloud B shown in FIG. 205. It should be noted that in this example, splitting is performed at valid nodes in layer 1. In other words, at most four 3D point subclouds are generated for a quadtree, and at most eight 3D point subclouds are generated for an octant tree. The 3D encoder may also perform separation using attribute information or information such as point cloud density.

[1268] The 3D data encoding device performs encoding such that the 3D point subcloud A and the 3D point subcloud B do not have a dependency relationship with each other. For example, the 3D data encoder selects, for each 3D point subcloud, a codebook to be used for entropy encoding of the fill code. Alternatively, the 3D data encoder initializes codebooks prior to encoding each of the 3D point subclouds. Alternatively, when a neighbor node is included in another 3D point sub-cloud, the 3D encoder disables reference to the neighbor node during calculation of the node's neighborhood information.

[1269] FIG. 206 is a diagram illustrating a structural example of a bit stream according to the present embodiment. As illustrated in FIG. 206, the bitstream includes a header, encoded data of 3D dot sub-cloud A, and encoded data of 3D dot sub-cloud B. The header includes point cloud number information, dependency relationship information, start address information A, and start address information B.

[1270] The point cloud number information indicates the number of 3D point subclouds included in the bit stream. It should be noted that the fill code may indicate, as point cloud number information, the number of 3D point subclouds. For example, in the example shown in FIG. 203, the fill code "1010" in layer 0 is used, and the number "1" included in the fill code indicates the number of 3D point subclouds.

[1271] The dependency relationship information indicates whether or not the 3D point subclouds are encoded such that the 3D point subclouds do not have a dependency relationship with each other. For example, the 3D data decoding apparatus determines whether or not the 3D point subclouds should be decoded in parallel based on the dependency relationship information.

[1272] The start address information A indicates the start address of encoded data of the 3D dot sub-cloud A. The start address information B indicates the start address of encoded data of the 3D dot subcloud B.

[1273] Later in this document, the advantage of parallel coding is described. It is possible to reduce processing time by separating geometric information (3D position information) or attribute information in octant tree data of a 3D point cloud (point cloud) and performing parallel encoding. It is possible to achieve parallel encoding when a node is independent of other nodes in the parent node layer. In other words, there is a need not to refer to an adjacent parent node. There is a need to satisfy this condition for all of the child nodes and grandchild nodes.

[1274] FIG. 207 is a diagram illustrating an example of a tree structure. In the example shown in FIG. 207, when depth-coding is performed, node A is independent of node C in layer 1. Node C is independent of node D in layer 2. Node A is independent of node B in layer 3.

[1275] The 3D data encoding device selects a parallel encoding method from two types of parallel encoding methods, using independent information of each node, based on the type of hardware, user settings, algorithm or data adaptability, and so on.

[1276] The two types are full parallel coding and incremental parallel coding.

[1277] First, full parallel coding is described below. In parallel processing or parallel programming, since it is necessary to process a lot of data at the same time, the processing is very slow.

[1278] The number of nodes processed in parallel is determined using the number of processing units (PUs) included in a graphics processing unit (GPU), the number of cores included in a CPU, or the number of subprocesses in a software implementation.

[1279] Here, the number of nodes included in the octant tree generally exceeds the number of available PUs. The 3D data encoding apparatus determines whether or not the number of nodes included in the layer is an optimal number corresponding to the number of available PUs using information indicating the number of encoded nodes included in the layer; and full parallel coding starts immediately when the number of nodes included in the layer reaches the optimal number. It should be noted that the process in width or in depth can be used in parallel processing.

[1280] The 3D data encoding device may store, in the bitstream header, information indicating the nodes (layer) for which the parallel encoding process has been started. As a result, the 3D data decoding apparatus can perform a parallel decoding process when necessary using the information. It should be noted that the information indicating the nodes for which the parallel encoding process is started may be in any format, and location-based encoding, for example, may be used.

[1281] In addition, the 3D data encoding device prepares a coding table (probability table) for each of the nodes (sub-clouds of 3D points) for which parallel encoding is to be performed. This encoding table is initialized as an initial value or a value that is different for each node. For example, the value that is different for each node is the value based on the fill code of the parent node. Full parallel encoding has the advantage of having to initialize the GPU only once.

[1282] FIG. 208 is a diagram for illustrating full parallel coding and shows an example of a tree structure. Fig. 209 is a diagram spatially illustrating 3D point subclouds to be processed in parallel. The 3D encoder starts parallel processing immediately when the number of nodes correlated with the number of PUs or subprocesses reaches the optimal number.

[1283] In the example shown in FIG. 208, in layer 3, the number of occupied nodes included in the layer is 9, which is more than the optimal number. Accordingly, the 3D data encoding apparatus divides 3D points (nodes) in layers below layer 3 into 3D point subclouds each having a corresponding one of the occupied nodes in layer 3 as a root, and processes the 3D point subclouds in parallel. For example, nine 3D point subclouds are formed in the example shown in FIG. 208.

[1284] The 3D data encoding device may encode layer information indicating the layer in which the parallel processing is started. The 3D encoder may also encode information indicating the number of busy nodes (9 in the example shown in FIG. 208) when parallel processing starts.

[1285] In addition, for example, the 3D data encoding device can perform encoding while prohibiting the sub-clouds of 3D points from referring to each other. In addition, for example, the 3D encoding device initializes coding tables (probability tables, etc.) to be used for entropy encoding before the 3D encoding device encodes the 3D point subclouds.

[1286] FIG. 210 is a diagram illustrating a structural example of a bit stream according to the present embodiment. As illustrated in FIG. 210, the bitstream includes a header, upper layer coded data, a subtitle, 3D point subcloud A coded data, and 3D point subcloud B coded data.

[1287] The header includes space size information and parallel start layer information. The space size information indicates the first 3D space obtained by dividing the 3D point cloud into an octant tree. For example, the space size information indicates the maximum coordinates (x, y, z) of the first three-dimensional space.

[1288] The parallel seed layer information indicates a parallel seed layer, which is a layer where parallel processing can start. Here, the parallel initial layer information indicates layer N, for example.

[1289] The encoded data of the upper layers is the encoded data up to layer N before the parallel processing starts, and is the node information up to layer N. For example, the encoded data of the upper layers includes node fill codes up to layer N.

[1290] The subtitle includes required information to decode any layer after layer N. For example, the subtitle indicates a start address, and so on. encoded data of each subcloud of 3D points. In the example shown in FIG. 210, the subtitle includes start address information A and start address information B. The start address information A indicates the start address of encoded data of the 3D dot sub-cloud A. The start address information B indicates the start address of encoded data of the 3D dot subcloud B.

[1291] It should be noted that the 3D data encoding apparatus may store start address information A and start address information B in a header. As a result, the 3D data decoding apparatus can decode the coded data of the 3D point subclouds in parallel before the coded data of the upper layers. In this case, the subtitle may include information indicating the space of each 3D point subcloud. This information indicates the maximum coordinates (x, y, z) of the subcloud space of three-dimensional points.

[1292] FIG. 211 is a diagram for illustrating a parallel decoding process. As illustrated in FIG. 211, the 3D data decoding apparatus decodes the coded data of the 3D point sub-cloud A and the coded data of the 3D point sub-cloud B in parallel, and generates the decoded data of the 3D point sub-cloud A and the decoded data of the 3D point sub-cloud B. Then, the 3D data decoding apparatus integrates the generated decoded data of the 3D point sub-cloud A and the generated decoded data of the 3D point sub-cloud B, and generates the decoded 3D point cloud data. Thus, the 3D data decoding apparatus integrates pieces of 3D position information and attribute information (color information, reflectance, etc.) included in the decoded 3D point subcloud data. In addition, the 3D data decoding apparatus can output the integrated data as a single file.

[1293] It should be noted that the 3D data decoding apparatus need not necessarily always decode all 3D point subclouds, and may decode necessary 3D point subclouds selectively. For example, when the 3D data decoding device is a mobile device, such as a vehicle-embedded device, the 3D data decoding device can decode, from among the 3D point subclouds, the 3D point subclouds in an area close to the current position obtained by GPS, etc. .d.

[1294] In addition, the 3D data encoding device may store, in the subheader, information indicating the priority order of the 3D point subclouds. In this case, the 3D data decoding apparatus performs parallel decoding while preferentially providing computing resources such as a CPU for the 3D point subclouds having a higher priority according to the priority order indicated by the information included in the subtitle. Accordingly, the 3D data decoding apparatus can efficiently decode 3D point sub-clouds including an area important to the 3D data decoding apparatus.

[1295] FIG. 212 is a diagram principally illustrating a procedure of a full parallel encoding process. First, the 3D encoder determines the number of available PUs (S2601). Then, the 3D encoder processes the octant tree and stores the node positions (S2602). Thereafter, the 3D data encoding apparatus determines whether or not the number of occupied nodes exceeds the number of PUs (S2603).

[1296] When the number of busy nodes is less than the number of PUs ("No" in S2603), the 3D data encoding device executes step S2602 for the next node. When the number of occupied nodes exceeds the number of PUs ("Yes" in S2603), the 3D data encoding device sets the current layer to the parallel start layer, which is the layer where the parallel processing is to start (S2604).

[1297] Thereafter, the 3D data encoding apparatus initializes the encoding tables and starts parallel encoding (S2605). After the parallel encoding is completed, the 3D data encoding apparatus reconstructs the octant tree based on the positions stored in step S2602 (S2606). It should be noted that the 3D encoder need not necessarily reconstruct the octant tree. For example, the 3D data encoding apparatus may append, to the bit stream, the encoded data of each 3D point subcloud obtained by parallel encoding and terminate the process.

[1298] It should be noted that the 3D data encoding apparatus may append, to the bitstream header, parallel start layer information indicating the parallel start layer at which parallel encoding should start. As a result, by decoding the header, the 3D data decoding apparatus can determine from which layer parallel decoding is possible.

[1299] It should be noted that from which layer parallel processing should start can be predetermined. In addition, the parallel start layer information indicating the layer at which the parallel processing should start is not attached to the bitstream, and the layer at which the parallel processing should start may be indicated by standards, etc. For example, a 3D encoder attaches a flag indicating whether or not to perform parallel processing to the bit stream. When the flag is enabled, the 3D decoder can start parallel processing from the first layer; and when the flag is deactivated, the 3D decoder can perform serial processing.

[1300] FIG. 213 is a diagram principally illustrating a procedure of a full parallel decoding process. First, by decoding the header, the 3D data decoding apparatus obtains parallel initial layer information indicating the layer N that is being processed in parallel (S2611). Then, the 3D data decoding apparatus decodes the fill code, divides the current node into eight, and proceeds to fill state sub-node processing (S2612).

[1301] Thereafter, the 3D data decoding apparatus determines whether or not the current layer reaches the parallel start layer N (S2613). When the current layer does not reach the parallel start layer N ("No" in S2613), the 3D data decoding apparatus executes step S2612 for the next node. When the current layer reaches the parallel start layer N ("Yes" in S2613), the 3D data decoding apparatus decodes the subtitle and obtains the start addresses of the 3D point subclouds (S2614).

[1302] Thereafter, the 3D data decoding apparatus initializes codebooks and starts parallel decoding of the 3D point subclouds (S2615). After parallel decoding is completed, the 3D data decoding apparatus integrates the pieces of 3D position information and attribute information of the decoded 3D point subclouds (S2616).

[1303] For example, the 3D data decoder decodes the parallel seed layer information from the header of the bitstream. Accordingly, the 3D data decoding apparatus can determine from which layer parallel decoding is possible.

[1304] It should be noted that the parallel start layer information indicating the layer where parallel processing should start is not attached to the bitstream, and the layer where parallel processing should start may be indicated by standards, etc. For example, a 3D encoder attaches a flag indicating whether or not to perform parallel processing to the bit stream. When the flag is enabled, the 3D decoder can start parallel processing from the first layer; and when the flag is deactivated, the 3D decoder can perform serial processing.

[1305] Next, incremental parallel coding will be described. Fig. 214 and FIG. 215 are a diagram for illustrating incremental parallel coding. Fig. 214 is a diagram illustrating parallel processing in a tree structure, and FIG. 215 is a diagram illustrating time-series-based changes in parallel processing.

[1306] In incremental parallel coding, when there is an available PU, if a parent node is split into one or more child nodes, the number of parallel processes is sequentially increased. In addition, every time a new node requiring an additional PU is found, the coding table is set as a predetermined table.

[1307] In the example shown in FIG. 214, core 1 starts processing at the vertex (root). In layer 1, core 2 is used to process the right node and core 1 processes the left node. In layer 2, core 1 continues processing the left node. In layer 3, kernel 1 processes node A, and kernel 4 processes the newly found base B. In addition, kernel 2 continues to process the left node in layer 2 after processing the right node in layer 1 and the node C processes in layer 3. In addition, the kernel 3 is added and processes the right node and node D in layer 3.

[1308] It should be noted that the 3D data encoder may add a flag indicating whether or not to initialize a codebook for entropy encoding at each node. As a result, the 3D data decoding apparatus can determine that the next child nodes relative to the node for which the flag indicates the need for initialization are processed in parallel.

[1309] FIG. 216 is a diagram principally illustrating a procedure of an incremental parallel encoding process. First, the 3D encoder processes the octant tree and stores the node positions (S2621). Then, the 3D data encoder determines whether or not the current node includes occupied child nodes, which are child nodes in a full state (S2622). When the current node includes occupied child nodes ("Yes" in S2613), the 3D encoder determines whether or not there is a PU available for parallel encoding (S2623).

[1310] When the current node does not include busy child nodes ("No" in S2613), or there is no PU available for parallel encoding ("No" in S2623), the 3D encoder continues to process the octant tree using the PU applied currently (S2624). For example, in the initial state, the 3D encoder continues processing using one PU in the initial state. In addition, the expression "when the current node does not include busy child nodes" includes a case in which the current node does not include busy nodes and a case in which the current node includes one busy child node.

[1311] In contrast, when there is a PU available for parallel encoding ("Yes" in S2623), the 3D encoder adds a new PU to the PUs to be used, initializes the coding tables, and starts the parallel processing (S2625).

[1312] When the processing of all nodes is not completed ("No" in S2626), the 3D data encoding apparatus executes step S2621 for the next node. When the processing of all nodes is completed ("Yes" in S2626), the 3D data encoder reconstructs the octant tree based on the positions stored in step S2602 (S2627). It should be noted that the 3D encoder need not necessarily reconstruct the octant tree. For example, the 3D data encoding apparatus may append, to the bit stream, the encoded data of each 3D point subcloud obtained by parallel encoding and terminate the process.

[1313] With such incremental parallel coding, when the current node includes busy nodes and there is an available PU, the parallel processing starts immediately. Accordingly, since the PU can be assigned to the next processing when the PU completes processing in a short time, ideal processing load balancing is achievable.

[1314] On the other hand, each time parallel processing is requested, initialization is required. Additionally, since PUs do not always complete processing for the next processing in top-down or left-to-right processing order, a mechanism for synchronizing all nodes and child nodes in each layer is required so as to write data back. In other words, since the above full parallel coding does not require such processing, full parallel coding generates the advantage of reducing the amount of processing.

[1315] As mentioned above, in the present embodiment, the original 3D points are divided into branches processed in parallel. In an octant tree, for example, eight branches processed in parallel are formed for nodes. In addition, a new parameter is set, indicating from which layer of the octant tree the branch processed in parallel begins.

[1316] When a transition is made to the processing of the next branch being processed in parallel, the encoding table for entropy encoding is reset. Alternatively, a different coding table is used for each of the branches being processed in parallel.

[1317] For example, reference to nodes included in different branches processed in parallel, such as neighboring node information, is prohibited.

[1318] The modes associated with parallel processing are set. For example, mode 0 is a mode for performing non-parallel processing. Mode 1 is a mode for performing parallel processing of geometric information (structure information). In this mode, with respect to attribute information, reference to other branches processed in parallel is permitted. Mode 2 is a mode for performing parallel processing of geometric information and attribute information. In other words, with respect to both geometry information and attribute information, reference to other branches processed in parallel is prohibited.

[1319] The data start address of each of the branches processed in parallel is encoded into a header, such as a slice header.

[1320] The 3D data decoding apparatus may process, in parallel, all the branches processed in parallel, or may process, in parallel, a part of the branches processed in parallel.

[1321] As mentioned above, the 3D data encoding device according to the present embodiment performs the process shown in FIG. 217. First, the 3D data encoding apparatus divides the 3D points included in the 3D data into 3D point subclouds including the first 3D point subcloud and the second 3D point subcloud (S2631). Then, the 3D data encoding apparatus appends information indicating the space of each of the 3D point subclouds to the subtitle of the 3D point subcloud (S2632). In other words, the 3D data encoding apparatus appends first information indicating the space of the first 3D point sub-cloud to the header of the first 3D point sub-cloud, and second information indicating the space of the second 3D point sub-cloud to the header of the second 3D point sub-cloud. Thereafter, the 3D data encoding apparatus encodes the first 3D point sub-cloud and the second 3D point sub-cloud such that the first 3D point sub-cloud and the second 3D point sub-cloud are independently decodable (S2633). In other words, the 3D data encoding apparatus encodes the first 3D point sub-cloud and the second 3D point sub-cloud such that the first 3D point sub-cloud and the second 3D point sub-cloud do not have a dependency relationship with each other. For example, the 3D data encoding device encodes the first 3D point sub-cloud and the second 3D point sub-cloud in parallel.

[1322] It should be noted that the processing order of steps S2632 and S2633 is an example and can be reversed, and part or all of the steps can be performed in parallel.

[1323] For example, in dividing (S2631), the 3D data encoding apparatus divides the N-ary 3D point tree structure into branches including a first branch corresponding to the first 3D point subcloud and a second branch corresponding to the second 3D point subcloud, where N is an integer greater than or equal to 2. In other words, the 3D data encoding apparatus can divide the N-ary tree structure of 3D points included in the 3D data into a first branch and a second branch (S2631), where N is an integer greater than or equal to 2; and may encode the first branch and the second branch such that the first branch and the second branch are independently decodable (S2633).

[1324] For example, the 3D data encoding device encodes information (parallel seed layer information) indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. In other words, the 3D data encoding apparatus generates a bit stream including information (parallel initial layer information) indicating the first layer. For example, the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs.

[1325] For example, the root of the first branch is the first node included in the first layer of the N-ary tree structure, and the root of the second branch is the second node different from the first node and included in the first layer. In other words, the root of the first branch and the root of the second branch belong to the same layer.

[1326] For example, the 3D data encoding device encodes information (parallel initial layer information) indicating the first layer. In other words, the 3D data encoding apparatus generates a bit stream including information (parallel initial layer information) indicating the first layer.

[1327] For example, the 3D data encoding device entropy encodes each of the first 3D point subcloud (first branch) and the second 3D point subcloud (second branch) using a different coding table.

[1328] For example, the 3D data encoding device initializes the codebook after entropy encoding the first 3D point subcloud (first branch) and before entropy encoding the second 3D point subcloud (second branch).

[1329] For example, the 3D data encoding device prohibits reference to the second 3D point subcloud (second branch) when encoding the first 3D point subcloud, and prohibits reference to the first 3D point subcloud (first branch) when encoding the second 3D point subcloud (second branch).

[1330] For example, the 3D data encoding device encodes pieces of geometric information of the first 3D points included in the first sub-3D point cloud (first branch) and pieces of geometric information of the second 3D points included in the second sub-3D point cloud (second branch) such that pieces of geometric information of the first three-dimensional points and pieces of geometric information of the second three-dimensional points are decodable independently of each other; encodes the first 3D point attribute information pieces and the second 3D point attribute information pieces such that the first 3D point attribute information pieces and the second 3D point attribute information pieces are independently decodable. In other words, the 3D data encoding device encodes both the pieces of geometry information and the pieces of attribute information in such a way that the pieces of geometry information do not have a dependency relationship with each other, and the pieces of attribute information do not have a dependency relationship with each other.

[1331] For example, the 3D data encoding device encodes one of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud (first branch) and pieces of second 3D point geometry information included in the second 3D point sub-cloud (second branch). ), and (2) the first 3D point attribute information pieces and the second 3D point attribute information pieces such that one of (1) the first 3D point geometric information pieces and the second 3D point geometric information pieces, and (2) the first 3D point attribute information pieces points and pieces of attribute information of the second three-dimensional points are decodable independently of each other; and encodes another of (1) the first 3D point geometry information pieces and the second 3D point geometry information pieces, and (2) the first 3D point attribute information pieces and the second 3D point attribute information pieces, such that the other of the (1) first 3D point geometry information pieces 3D points and pieces of geometric information of the second 3D points; and (2) pieces of attribute information of the first 3D points and pieces of attribute information of the second 3D points have a dependency relationship with each other. In other words, the 3D data encoding device encodes one of the geometric information pieces and the attribute information pieces such that one of the geometric information pieces and the attribute information pieces do not have a dependency relationship with each other; and encodes the other of the geometric information pieces and the attribute information pieces such that the other of the geometric information pieces and the attribute information pieces have a dependency relationship with each other. It should be noted that "having a dependency relationship with each other" means, for example, entropy encoding the first 3D point subcloud (first branch) and the second 3D point subcloud (second branch) using the same codebook, uninitializing the codebook after how the first 3D point subcloud (first branch) is entropy encoded, and before the second 3D point subcloud (second branch) is entropy encoded, resolving the reference to the second 3D point subcloud (second branch) when encoding the first 3D point subcloud (first branch) or resolving a reference to the first 3D point subcloud (first branch) when encoding the second 3D point subcloud (second branch).

[1332] For example, the 3D data encoding device encodes a flag indicating whether or not the first 3D point subcloud (first branch) and the second 3D point subcloud (second branch) are encoded such that the first 3D point subcloud and the second 3D point subcloud are decodable. independently of each other. In other words, the 3D data encoding device generates a bitstream including a flag indicating whether or not the first 3D point subcloud (first branch) and the second 3D point subcloud (second branch) are encoded such that the first 3D point subcloud and the second subcloud 3D points are decodable independently of each other.

[1333] For example, the first information indicates the maximum space coordinates of the first 3D point subcloud, and the second information indicates the maximum space coordinates of the second 3D point subcloud.

[1334] For example, the 3D data encoding device includes a processor and a memory, and the processor performs the above process using the memory.

[1335] The 3D data decoding apparatus according to the present embodiment performs the process shown in FIG. 218. First, the 3D data decoding apparatus obtains the first coded data and the second coded data generated by encoding the first 3D point sub-cloud and the second 3D point sub-cloud such that the first 3D point sub-cloud and the second 3D point sub-cloud are independently decodable. , wherein the first 3D point subcloud and the second 3D point subcloud are included in the 3D point subclouds obtained by dividing the 3D points included in the 3D data (S2641). For example, the 3D data decoding apparatus obtains the first encoded data and the second encoded data from the bitstream.

[1336] Next, the 3D data decoding apparatus obtains information indicating the space of each of the 3D point subclouds from the subtitle of the 3D point subcloud included in the bitstream (S2642). In other words, the 3D data decoding apparatus obtains first information indicating the space of the first 3D point sub-cloud from the header of the first 3D point sub-cloud, and obtains second information indicating the space of the second 3D point sub-cloud from the header of the second 3D point sub-cloud.

[1337] Thereafter, the 3D data decoding apparatus recovers the first 3D point sub-cloud and the second 3D point sub-cloud by decoding the first encoded data and the second encoded data (S2643). For example, the 3D data decoding apparatus decodes the first encoded data and the second encoded data in parallel. For example, the 3D data decoding apparatus recovers 3D points using the reconstructed first 3D point subcloud and the second 3D point subcloud, the first information and the second information.

[1338] It should be noted that the processing order of steps S2642 and S2643 is an example and can be reversed, and part or all of the steps can be performed in parallel.

[1339] For example, the first encoded data and the second encoded data are generated by encoding the first branch corresponding to the first 3D point subcloud and the second branch corresponding to the second 3D point subcloud such that the first branch and the second branch are independently decodable, wherein the first branch and the second branch are included in an N-ary 3D dot tree structure, where N is an integer greater than or equal to 2. In other words, the 3D data decoding apparatus obtains the first encoded data and the second encoded data generated by encoding the first branch and the second branch included in an N-ary tree structure of 3D points included in 3D data such that the first branch and the second branch are independently decodable, where N is an integer greater than or equal to 2 (S2641).

[1340] For example, the 3D data decoding apparatus decodes information (parallel start layer information) indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. In other words, the 3D data decoding apparatus obtains information (parallel initial layer information) indicating the first layer from the bit stream. For example, the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs.

[1341] For example, the root of the first branch is the first node included in the first layer of the N-ary tree structure, and the root of the second branch is the second node different from the first node and included in the first layer. In other words, the root of the first branch and the root of the second branch belong to the same layer.

[1342] For example, the 3D data encoding device decodes information (parallel initial layer information) indicating the first layer. In other words, the 3D data decoding apparatus obtains information (parallel initial layer information) indicating the first layer from the bit stream.

[1343] For example, the 3D data decoding apparatus entropy decodes each of the first 3D point subcloud (first branch) and the second 3D point subcloud (second branch) using a different codebook.

[1344] For example, the 3D data decoding apparatus initializes a codebook after entropy decoding of the first 3D point subcloud (first branch) and before entropy decoding of the second 3D point subcloud (second branch).

[1345] For example, the 3D data decoding apparatus does not refer to the second 3D point subcloud (second branch) when decoding the first 3D point subcloud (first branch) and does not refer to the first 3D point subcloud (first branch) when decoding the second 3D point subcloud (second branches).

[1346] For example, the first encoded data includes the first encoded geometric data and the first encoded attribute data, the first encoded geometric data being generated by encoding pieces of geometric information of the first 3D points included in the first 3D point subcloud (first branch), the first encoded attribute data is generated by encoding pieces of attribute information of the first 3D points. The second encoded data includes second encoded geometric data and second encoded attribute data, the second encoded geometric data being generated by encoding pieces of geometric information of the second 3D points included in the second 3D point subcloud (second branch), the second encoded attribute data being generated by encoding pieces of attribute information of the second 3D points. The first encoded geometry data and the second encoded geometry data are generated such that the first encoded geometry data and the second encoded geometric data are independently decodable, and the first encoded attribute data and the second encoded attribute data are generated such that the first encoded attribute data and the second encoded attribute data is independently decodable. In other words, the 3D data encoding apparatus generates the first encoded data and the second encoded data by encoding both the pieces of geometric information and the pieces of attribute information in such a way that the pieces of geometric information do not have a dependency relationship with each other, and the pieces of attribute information do not have interdependence with each other.

[1347] For example, the first encoded data and the second encoded data are generated by encoding one of (1) the first 3D point geometry information pieces included in the first 3D point subcloud (first branch) and the second 3D point geometry information pieces included in the second subcloud 3D points (second branch), and (2) pieces of first 3D point attribute information included in the first 3D point subcloud and pieces of second 3D point attribute information included in the second 3D point subcloud, such that one of the (1) pieces geometry information of the first 3D points included in the first 3D point sub-cloud and pieces of geometric information of the second 3D points included in the second 3D point sub-cloud, and (2) pieces of attribute information of the first 3D points included in the first 3D point sub-cloud and pieces of attribute information second 3D points included in the second 3D point subcloud are independently decodable. The 3D data decoding apparatus recovers, by decoding the first encoded data and the second encoded data, one of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud, and (2) attribute information pieces of first 3D points included in the first 3D point subcloud and attribute information pieces of second 3D points included in the second 3D point subcloud. The 3D data decoding apparatus further obtains third coded data and fourth coded data generated by encoding the other of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud , and (2) the first 3D point attribute information pieces included in the first 3D point sub-cloud and the second 3D point attribute information pieces included in the second 3D point sub-cloud, such that another of the (1) first 3D point geometric information pieces, of the first 3D point sub-cloud, and pieces of geometric information of the second 3D points included in the second 3D point sub-cloud, and (2) pieces of attribute information of the first 3D points included in the first 3D point sub-cloud, and pieces in the attribute information of the second 3D points included in the second 3D point sub-cloud have a dependency relationship with each other. The 3D data decoding apparatus recovers, by decoding the third coded data and the fourth coded data, another of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud, and (2) attribute information pieces of first 3D points included in the first 3D point subcloud and attribute information pieces of second 3D points included in the second 3D point subcloud. In other words, the 3D data encoding device encodes one of the geometric information pieces and the attribute information pieces such that one of the geometric information pieces and the attribute information pieces do not have a dependency relationship with each other; and encodes the other of the geometric information pieces and the attribute information pieces such that the other of the geometric information pieces and the attribute information pieces have a dependency relationship with each other. For example, when two coded data have a dependency relationship with each other, the 3D data decoding apparatus entropy decodes the two coded data using the same codebook, does not initialize the codebook after entropy decoding one of the two coded data and before entropy decoding the other of the two coded data. data or refers to the other of the two encoded data when decoding one of the two encoded data.

[1348] For example, the 3D data decoding apparatus decodes a flag indicating whether or not the first 3D point subcloud (first branch) and the second 3D point subcloud (second branch) are encoded such that the first 3D point subcloud and the second 3D point subcloud are decodable. independently of each other. For example, the 3D data decoding apparatus obtains, from the bit stream, a flag indicating whether or not the first 3D point subcloud (first branch) and the second 3D point subcloud (second branch) are encoded such that the first 3D point subcloud and the second 3D point subcloud are decodable independently of each other. For example, when the flag indicates that the first 3D sub-cloud (first branch) and the second 3D sub-cloud (second branch) are encoded such that the first 3D sub-cloud and the second 3D sub-cloud are independently decodable, the 3D data decoding apparatus decoding the first encoded data and the second encoded data in parallel; and when the flag does not indicate that the first 3D sub-cloud (first branch) and the second 3D sub-cloud (second branch) are encoded such that the first 3D sub-cloud and the second 3D sub-cloud are independently decodable, the 3D data decoding device decodes the first encoded data and the second encoded data sequentially.

[1349] For example, the first information indicates the maximum space coordinates of the first 3D point subcloud, and the second information indicates the maximum space coordinates of the second 3D point subcloud.

[1350] For example, the 3D data decoding apparatus includes a processor and a storage device, and the processor performs the above process using the storage device.

[1351] The above describes a 3D data encoding apparatus, a 3D data decoding apparatus, and the like. according to the embodiments of the present disclosure, but the present disclosure is not limited to these embodiments.

[1352] It should be noted that each of the processors included in the 3D data encoding apparatus, the 3D data decoding apparatus, and the like. according to the above embodiments, is typically implemented as a large scale integrated (LSI) circuit, which is an integrated circuit (IC). They may take the form of individual chips or may be partially or completely packaged on a single chip.

[1353] Such an IC is not limited to the LSI and can therefore be implemented as a dedicated circuit or a general purpose processor. Alternatively, a user-programmable gate array (FPGA) may be used, which allows for post-manufacture programming of the LSI, or a reconfigurable processor, which allows reconfiguration of the connection and tuning of the circuit elements within the LSI.

[1354] In addition, in the above embodiments, the structural components may be implemented as dedicated hardware or may be implemented by executing a software program suitable for such structural components. Alternatively, the structural components may be implemented by a software execution unit such as a CPU or processor that reads and executes a software program recorded on a recording medium such as a hard disk or a semiconductor memory device.

[1355] The present disclosure can also be implemented as a 3D data encoding method, a 3D data decoding method, and the like by a 3D data encoding device, a 3D data decoding device, and the like.

[1356] In addition, the divisions of functional blocks shown in the block diagrams are merely examples, and because of this, a plurality of functional blocks may be implemented as one functional block, or one functional block may be divided into a plurality of functional blocks, or one or more functions can be moved to another function block. In addition, the functions of a plurality of function blocks having similar functions can only be processed by hardware or software in a parallelized or time-separated manner.

[1357] In addition, the order of processing steps shown in the flowcharts is merely illustrative for the specific description of the present disclosure, and may therefore be an order different from the order shown. In addition, one or more steps may be performed simultaneously (in parallel) with another step.

[1358] The above describes a 3D data encoding apparatus, a 3D data decoding apparatus, and the like. according to one or more aspects based on the embodiments, but the present disclosure is not limited to these embodiments. One or more aspects may therefore include forms achievable by making various modifications to the above embodiments, as may be conceived by those skilled in the art, as well as forms achievable by combining structural components in various embodiments, without significant departure. from the spirit of the present disclosure.

Industrial Applicability

[1359] The present disclosure is applicable to a 3D data encoding device and a 3D data decoding device.

References with numbers in drawings

[1360] 100, 400 - 3D encoder

101, 201, 401, 501 - receiving module

102, 402 - module for determining coding areas

103 - separation module

104, 644 - encoder

111 - 3D data

112, 211, 413, 414, 511, 634 - encoded 3D data

200, 500 - 3D decoder

202 - GOS determination module to start decoding

203 - SPC determination module for decoding

204, 625 - decoder

212, 512, 513 - decoded 3D data

403 - SWLD extraction module

404 - WLD encoder

405 - SWLD Encoder

411 - input 3D data

412 - extracted 3D data

502 header parser

503 - WLD decoder

504 - SWLD decoder

620, 620A - 3D data generator

621, 641 - module for creating three-dimensional data

622 - module for determining request ranges

623 - search module

624, 642 - receiver

626 - merge module

631, 651 - sensor information

632 - first 3D data

633 - request range information

635 - second 3D data

636 - third 3D data

640 - 3D data transfer device

643 - extraction module

645 - transmitter

652 - fifth 3D data

654 - sixth 3D data

700 - three-dimensional information processing device

701 - module for obtaining three-dimensional maps

702 - self-discovered data acquisition module

703 - abnormal case evaluation module

704 - overcoming operation definition module

705 - functional controller

711 - 3D map

712 - self-detected 3D data

810 - 3D data generator

811 - data receiver

812, 819 - communication module

813 - receive controller

814, 821 - format converter

815 - sensor

816 - module for creating three-dimensional data

817 - 3D data synthesizer

818 - 3D data storage device

820 - transmission controller

822 - data transmitter

831, 832, 834, 835, 836, 837 - 3D data

833 - sensor information

901 - server

902, 902A, 902B, 902C - client device

1011, 1111 - data receiver

1012, 1020, 1112, 1120 - communication module

1013, 1113 - receive controller

1014, 1019, 1114, 1119 - format converter

1015 - sensor

1016, 1116 - module for creating three-dimensional data

1017 - 3D image processor

1018, 1118 - 3D data storage device

1021, 1121 - transmission controller

1022, 1122 - data transmitter

1031, 1032, 1135 - 3D map

1033, 1037, 1132 - sensor information

1034, 1035, 1134 - 3D data

1117 - module for combining three-dimensional data

1201 - 3D map compression/decoding processor

1202 - processor for compressing / decoding sensor information

1211 - 3D map decoding processor

1212 - sensor information compression processor

1300 - 3D encoder

1301 - separation module

1302 - subtractor

1303 - Converter

1304 - quantizer

1305, 1402 - inverse quantizer

1306, 1403 - inverter

1307, 1404 - adder

1308, 1405 - reference volume memory

1309, 1406 - internal prediction module

1310, 1407 - storage device reference spaces

1311, 1408 - mutual prediction module

1312, 1409 - prediction controller

1313 - entropy encoder

1400 - 3D decoder

1401 - entropy decoder

1501 - server

1502 - client

1511 - storage device

1512 - controller

1513 - coded 3D map

1521 - decoder

1522 - application

1900 - 3D Encoder

1901, 1911 - octant tree shaper

1902, 1912 - similarity information calculation module

1903, 1913 - coding table selection module

1904 - entropy encoder

1910 - 3D data decoder

1914 - entropy decoder

2100 - 3D Encoder

2101, 2111 - octant tree shaper

2102, 2112 - geometric information calculation module

2103, 2113 - coding table selection module

2104 - entropy encoder

2110 - 3D decoder

2114 - entropy decoder

2200 - 3D Encoder

2201, 2211 - octant tree shaper

2202 - mode detection module

2203 - entropy encoder

2210 - 3D decoder

2212 - mode information decoder

2213 - entropy decoder

2300 - 3D Encoder

2301, 2311 - octant tree shaper

2302 - entropy encoder

2310 - 3D decoder

2312 - entropy decoder

2400 - 3D Encoder

2401 - quantizer

2402, 2411 - octant tree shaper

2403 - union definition module

2404 - entropy encoder

2410 - 3D decoder

2412 - combination information decoder

2413 - entropy decoder

2414 - inverse quantizer

2500 - 3D encoder

2501, 2511 - octant tree shaper

2502, 2512 - module for obtaining information of parent nodes

2503 - coding mode selection module

2504, 2514 - search module

2505, 2515 - geometric information calculation module

2506, 2516 - coding table selection module

2507 - entropy encoder

2510 - 3D decoder

2513 - decoding mode selection module

2517 - entropy decoder

Claims (178)

1. A method for encoding three-dimensional data, containing the steps in which: dividing the 3D points included in the 3D data into 3D point sub-clouds including a first 3D point sub-cloud and a second 3D point sub-cloud; attaching first information indicating the space of the first 3D point subcloud to a header of the first 3D point subcloud; attaching second information indicating the space of the second 3D point subcloud to a header of the second 3D point subcloud; And encoding the first sub-cloud of three-dimensional points and the second sub-cloud of three-dimensional points, wherein the encoding comprises the steps of: encoding pieces of geometric information of the first 3D points included in the first subcloud of 3D points and pieces of geometric information of the second 3D points included in the second subcloud of 3D points, so that the pieces of geometric information of the first 3D points and the pieces of geometric information of the second 3D points are decodable sequentially or in parallel ; And encoding the first 3D point attribute information pieces and the second 3D point attribute information pieces such that the first 3D point attribute information pieces and the second 3D point attribute information pieces are decodable sequentially or in parallel. 2. The 3D data encoding method according to claim 1, wherein, with said division, the N-ary tree structure of 3D points is divided into branches including a first branch corresponding to a first 3D point subcloud and a second branch corresponding to a second 3D point subcloud , where N is an integer greater than or equal to 2. 3. The 3D data encoding method according to claim 2, further comprising encoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. 4. The 3D data encoding method according to claim 3, wherein the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs. 5. The method for encoding 3D data according to any one of claims 1 to 4, wherein the encoding further comprises entropy encoding each of the first 3D point subcloud and the second 3D point subcloud using a different codebook. 6. The 3D data encoding method according to claim 5, further comprising initializing the encoding table after entropy encoding of the first 3D point subcloud is performed and before entropy encoding of the second 3D point subcloud is performed. 7. A method for encoding three-dimensional data according to any one of claims 1-4, in which: when encoding the first sub-3D point cloud, reference to the second sub-3D point cloud is prohibited, and when encoding the second 3D point subcloud, reference to the first 3D point subcloud is prohibited. 8. A method for encoding 3D data according to any one of claims 1 to 7, further comprising encoding a flag indicating whether or not the first 3D point subcloud and the second 3D point subcloud are encoded such that the first 3D point subcloud and the second subcloud 3D points are decodable independently of each other. 9. A method for encoding three-dimensional data according to any one of claims 1-7, in which: the first information indicates the maximum space coordinates of the first subcloud of 3D points, and the second information indicates the maximum space coordinates of the second subcloud of three-dimensional points. 10. A method for encoding three-dimensional data, comprising the steps of: dividing the 3D points included in the 3D data into 3D point sub-clouds including a first 3D point sub-cloud and a second 3D point sub-cloud; attaching first information indicating the space of the first 3D point subcloud to a header of the first 3D point subcloud; attaching second information indicating the space of the second 3D point subcloud to a header of the second 3D point subcloud; And encoding the first 3D point subcloud and the second 3D point subcloud so that the first 3D point subcloud and the second 3D point subcloud are decodable independently of each other, the encoding comprising the step of performing entropy encoding of each of the first 3D point subcloud and the second subcloud 3D points using a different coding table. 11. The 3D data encoding method according to claim 10, wherein, with said division, the N-ary 3D point tree structure is divided into branches including a first branch corresponding to a first 3D point subcloud and a second branch corresponding to a second 3D point subcloud , where N is an integer greater than or equal to 2. 12. The 3D data encoding method of claim 11, further comprising encoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. 13. The 3D data encoding method according to claim 12, wherein the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs. 14. The 3D data encoding method according to any one of claims 10 to 13, further comprising initializing the encoding table after entropy encoding of the first 3D point subcloud is performed and before entropy encoding of the second 3D point subcloud is performed. 15. A method for encoding three-dimensional data according to any one of claims 10 to 14, wherein the encoding further comprises the steps of: encoding fragments of geometric information of the first three-dimensional points included in the first sub-cloud of three-dimensional points, and fragments of geometric information of the second three-dimensional points included in the second sub-cloud of three-dimensional points, so that the fragments of geometric information of the first three-dimensional points and the geometric information fragments of the second three-dimensional points are decodable independently of each other friend; And encoding the first 3D point attribute information pieces and the second 3D point attribute information pieces such that the first 3D point attribute information pieces and the second 3D point attribute information pieces are independently decodable. 16. A method for encoding three-dimensional data according to any one of claims 10-15, in which: when encoding the first sub-3D point cloud, reference to the second sub-3D point cloud is prohibited, and when encoding the second 3D point subcloud, reference to the first 3D point subcloud is prohibited. 17. A method for encoding three-dimensional data according to any one of claims 10-15, further comprising the steps of: encoding one of (1) first 3D point geometry information pieces included in the first 3D point subcloud and second 3D point geometry information pieces included in the second 3D point subcloud, and (2) first 3D point attribute information pieces and second 3D point attribute information pieces 3D points such that it is one of (1) the first 3D point geometry information pieces and the second 3D point geometry information pieces, and (2) the first 3D point attribute information pieces and the second 3D point attribute information pieces are independently decodable; And encode another of (1) the first 3D point geometric information pieces and the second 3D point geometric information pieces and (2) the first 3D point attribute information pieces and the second 3D point attribute information pieces, so that it is another of (1) the first 3D point geometric information pieces points and geometric information pieces of the second 3D points; and (2) attribute information pieces of the first 3D points and attribute information pieces of the second 3D points have a dependency relationship with each other. 18. A method for encoding 3D data according to any one of claims 10 to 17, further comprising encoding a flag indicating whether or not the first 3D point subcloud and the second 3D point subcloud are encoded such that the first 3D point subcloud and the second subcloud 3D points are decodable independently of each other. 19. A method for encoding three-dimensional data according to any one of claims 10-17, in which: the first information indicates the maximum space coordinates of the first subcloud of 3D points, and the second information indicates the maximum space coordinates of the second subcloud of three-dimensional points. 20. A method for encoding three-dimensional data, comprising the steps of: dividing the 3D points included in the 3D data into 3D point sub-clouds including a first 3D point sub-cloud and a second 3D point sub-cloud; attaching first information indicating the space of the first 3D point subcloud to a header of the first 3D point subcloud; attaching second information indicating the space of the second 3D point subcloud to a header of the second 3D point subcloud; And encoding the first 3D point subcloud and the second 3D point subcloud such that the first 3D point subcloud and the second 3D point subcloud are independently decodable, wherein when encoding the first sub-3D point cloud, reference to the second sub-3D point cloud is prohibited, and when encoding the second subcloud of 3D points, reference to the first subcloud of 3D points is prohibited 21. The 3D data encoding method according to claim 20, wherein, with said division, the N-ary tree structure of 3D points is divided into branches including a first branch corresponding to a first 3D point subcloud and a second branch corresponding to a second 3D point subcloud , where N is an integer greater than or equal to 2. 22. The 3D data encoding method of claim 21, further comprising encoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. 23. The 3D data encoding method according to claim 22, wherein the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs. 24. The method for encoding 3D data according to any one of claims 20 to 23, wherein the encoding further comprises entropy encoding each of the first 3D point subcloud and the second 3D point subcloud using a different codebook. 25. The 3D data encoding method of claim 24, further comprising initializing the encoding table after entropy encoding of the first 3D point subcloud is performed and before entropy encoding of the second 3D point subcloud is performed. 26. A method for encoding three-dimensional data according to any one of claims 20-25, further comprising the steps of: encoding pieces of geometric information of the first 3D points included in the first sub-cloud of 3D points and pieces of geometric information of the second 3D points included in the second sub-cloud of 3D points, so that the pieces of geometric information of the first 3D points and the pieces of geometric information of the second 3D points are decodable independently of each other. friend; And encoding the first 3D point attribute information pieces and the second 3D point attribute information pieces such that the first 3D point attribute information pieces and the second 3D point attribute information pieces are independently decodable. 27. The method for encoding 3D data according to any one of claims 20 to 26, further comprising encoding a flag indicating whether or not the first 3D point subcloud and the second 3D point subcloud are encoded such that the first 3D point subcloud and the second subcloud 3D points are decodable independently of each other. 28. A method for encoding three-dimensional data according to any one of claims 20-26, in which: the first information indicates the maximum space coordinates of the first subcloud of 3D points, and the second information indicates the maximum space coordinates of the second subcloud of three-dimensional points. 29. A method for encoding three-dimensional data, comprising the steps of: dividing the 3D points included in the 3D data into 3D point sub-clouds including a first 3D point sub-cloud and a second 3D point sub-cloud; attaching first information indicating the space of the first 3D point subcloud to a header of the first 3D point subcloud; attaching second information indicating the space of the second 3D point subcloud to a header of the second 3D point subcloud; And encoding the first 3D point subcloud and the second 3D point subcloud such that the first 3D point subcloud and the second 3D point subcloud are independently decodable, wherein the method further comprises the steps of: encoding pieces of geometric information of the first 3D points included in the first sub-cloud of 3D points and pieces of geometric information of the second 3D points included in the second sub-cloud of 3D points, so that the pieces of geometric information of the first 3D points and the pieces of geometric information of the second 3D points are decodable independently of each other. friend; And encoding the first 3D point attribute information pieces and the second 3D point attribute information pieces such that the first 3D point attribute information pieces and the second 3D point attribute information pieces are independently decodable. 30. The 3D data encoding method according to claim 29, wherein, with said division, the N-ary tree structure of 3D points is divided into branches including a first branch corresponding to a first 3D point subcloud and a second branch corresponding to a second 3D point subcloud , where N is an integer greater than or equal to 2. 31. The 3D data encoding method of claim 30, further comprising encoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. 32. The 3D data encoding method according to claim 31, wherein the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs. 33. The method for encoding 3D data according to any one of claims 29 to 32, wherein the encoding further comprises entropy encoding each of the first 3D point subcloud and the second 3D point subcloud using a different codebook. 34. The 3D data encoding method of claim 33, further comprising initializing the encoding table after entropy encoding of the first 3D point subcloud is performed and before entropy encoding of the second 3D point subcloud is performed. 35. A method for encoding three-dimensional data according to any one of claims 29-32, in which: when encoding the first sub-3D point cloud, reference to the second sub-3D point cloud is prohibited, and when encoding the second 3D point subcloud, reference to the first 3D point subcloud is prohibited. 36. The method for encoding 3D data according to any one of claims 29 to 35, further comprising encoding a flag indicating whether or not the first 3D point subcloud and the second 3D point subcloud are encoded such that the first 3D point subcloud and the second subcloud 3D points are decodable independently of each other. 37. A method for encoding three-dimensional data according to any one of claims 29-35, in which: the first information indicates the maximum space coordinates of the first subcloud of 3D points, and the second information indicates the maximum space coordinates of the second subcloud of three-dimensional points. 38. A method for decoding three-dimensional data, comprising steps in which: obtaining first encoded data and second encoded data generated by encoding the first 3D point sub-cloud and the second 3D point sub-cloud, the first 3D point sub-cloud and the second 3D point sub-cloud being included in the 3D point sub-clouds obtained by separating the 3D points included in the 3D data; obtaining first information indicating the space of the first 3D point subcloud from the header of the first 3D point subcloud; obtaining second information indicating the space of the second 3D point subcloud from the header of the second 3D point subcloud; And restoring the first 3D point sub-cloud and the second 3D point sub-cloud by decoding the first encoded data and the second encoded data, wherein the first encoded data includes the first encoded geometric data and the first encoded attribute data, wherein the first encoded geometric data is generated by encoding pieces of geometric information of the first 3D points included in the first 3D point sub-cloud, wherein the first encoded attribute data is generated by encoding pieces of attribute information the first three-dimensional points, the second encoded data includes the second encoded geometric data and the second encoded attribute data, the second encoded geometric data being generated by encoding pieces of geometric information of the second 3D points included in the second sub-3D point cloud, the second encoded attribute data being generated by encoding the pieces of attribute information second 3D points, the first encoded geometry data and the second encoded geometry data are generated such that the first encoded geometry data and the second encoded geometry data are decodable in series or in parallel, and the first attribute encoded data and the second attribute encoded data are generated such that the first attribute encoded data and the second attribute encoded data are decodable in series or in parallel. 39. The 3D data decoding method of claim 38, wherein the first encoded data and the second encoded data are generated by encoding the first branch corresponding to the first 3D point subcloud and the second branch corresponding to the second 3D point subcloud, such that the first branch and the second branch are decodable independently of each other, with the first branch and the second branch included in an N-ary tree structure of three-dimensional points, where N is an integer greater than or equal to 2. 40. The 3D data decoding method of claim 39, further comprising decoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. 41. The 3D data decoding method of claim 40, wherein the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs. 42. The 3D data decoding method according to any one of claims 38 to 40, wherein the decoding further comprises performing entropy decoding of each of the first 3D point subcloud and the second 3D point subcloud using a different codebook. 43. The 3D data decoding method of claim 42, further comprising initializing the codebook after entropy decoding of the first 3D point subcloud is performed and before entropy decoding of the second 3D point subcloud is performed. 44. The method for decoding three-dimensional data according to any one of claims 38-43, in which: when decoding the first 3D point subcloud, the second 3D point subcloud is not referenced, and when decoding the second 3D point subcloud, the first 3D point subcloud is not referenced. 45. The method for decoding 3D data according to any one of claims 38 to 44, further comprising decoding a flag indicating whether or not the first 3D point subcloud and the second 3D point subcloud are encoded such that the first 3D point subcloud and the second subcloud 3D points are decodable independently of each other. 46. The method for decoding three-dimensional data according to any one of claims 38-45, in which: the first information indicates the maximum space coordinates of the first subcloud of 3D points, and the second information indicates the maximum space coordinates of the second subcloud of three-dimensional points. 47. A method for decoding three-dimensional data, comprising the steps of: obtaining the first encoded data and the second encoded data generated by encoding the first 3D point subcloud and the second 3D point subcloud in such a way that the first 3D point subcloud and the second 3D point subcloud are independently decodable, wherein the first 3D point subcloud and the second 3D subcloud the points are included in sub-clouds of 3D points obtained by dividing the 3D points included in the 3D data; obtaining first information indicating the space of the first 3D point subcloud from the header of the first 3D point subcloud; obtaining second information indicating the space of the second 3D point subcloud from the header of the second 3D point subcloud; And restoring the first 3D point subcloud and the second 3D point subcloud by decoding the first encoded data and the second encoded data, the decoding comprising performing entropy decoding of each of the first 3D point subcloud and the second 3D point subcloud using a different codebook 48. The 3D data decoding method of claim 47, wherein the first encoded data and the second encoded data are generated by encoding the first branch corresponding to the first 3D point subcloud and the second branch corresponding to the second 3D point subcloud, such that the first branch and the second branch are decodable independently of each other, with the first branch and the second branch included in an N-ary tree structure of three-dimensional points, where N is an integer greater than or equal to 2. 49. The 3D data decoding method of claim 48, further comprising decoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. 50. The 3D data decoding method of claim 49, wherein the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs. 51. The 3D data decoding method according to any one of claims 47 to 50, further comprising initializing the codebook after entropy decoding of the first 3D point subcloud is performed and before entropy decoding of the second 3D point subcloud is performed. 52. The 3D data decoding method according to any one of claims 47 to 49, wherein: the first encoded data includes the first encoded geometric data and the first encoded attribute data, wherein the first encoded geometric data is generated by encoding pieces of geometric information of the first 3D points included in the first 3D point sub-cloud, wherein the first encoded attribute data is generated by encoding pieces of attribute information the first three-dimensional points, the second encoded data includes the second encoded geometric data and the second encoded attribute data, the second encoded geometric data being generated by encoding pieces of geometric information of the second 3D points included in the second sub-3D point cloud, the second encoded attribute data being generated by encoding the pieces of attribute information second 3D points, the first encoded geometry data and the second encoded geometry data are generated such that the first encoded geometry data and the second encoded geometry data are independently decodable, and the first attribute encoded data and the second attribute encoded data are generated such that the first attribute encoded data and the second attribute encoded data are independently decodable. 53. The 3D data decoding method according to any one of claims 47 to 52, wherein: when decoding the first 3D point subcloud, the second 3D point subcloud is not referenced, and when decoding the second 3D point subcloud, the first 3D point subcloud is not referenced. 54. The 3D data decoding method according to any one of claims 47 to 53, wherein: the first encoded data and the second encoded data are generated by encoding one of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud, and (2) pieces of information attributes of the first 3D points included in the first 3D point sub-cloud, and pieces of attribute information of the second 3D points included in the second 3D point sub-cloud, so that it is one of (1) pieces of geometric information of the first 3D points included in the first 3D point sub-cloud, and second 3D point geometry information pieces included in the second 3D point sub-cloud, and (2) first 3D point attribute information pieces included in the first 3D point sub-cloud and second 3D point attribute information pieces included in the second 3D point sub-cloud. dimensional points are decodable independently of each other, and said one of (1) the first 3D point geometry information pieces included in the first 3D point sub-cloud and the second 3D point geometry information pieces included in the second 3D point sub-cloud, and (2) the first 3D point attribute information pieces included in the first sub-cloud 3D points, and attribute information pieces of the second 3D points included in the second 3D point sub-cloud are recovered by decoding the first encoded data and the second encoded data, wherein the method for decoding 3D data further comprises the steps of: obtaining third encoded data and fourth encoded data generated by encoding the other of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud, and (2) attribute information pieces of the first 3D points included in the first 3D point sub-cloud, and attribute information pieces of the second 3D points included in the second 3D point sub-cloud, so that it is another of (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud , and pieces of geometric information of the second 3D points included in the second sub-3D point cloud, and (2) pieces of attribute information of the first 3D points included in the first sub-3D point cloud, and pieces of attribute information of the second 3D points included in the W another subcloud of three-dimensional points, have a relationship in the form of dependence with each other; And recovering the other from (1) pieces of first 3D point geometry information included in the first 3D point sub-cloud and pieces of second 3D point geometry information included in the second 3D point sub-cloud, and (2) pieces of first 3D point attribute information included in the first a 3D point subcloud, and second 3D point attribute information pieces included in the second 3D point subcloud by decoding the third encoded data and the fourth encoded data. 55. The method for decoding 3D data according to any one of claims 47 to 54, further comprising decoding a flag indicating whether or not the first 3D point subcloud and the second 3D point subcloud are encoded such that the first 3D point subcloud and the second subcloud 3D points are decodable independently of each other. 56. The 3D data decoding method according to any one of claims 47 to 55, wherein: the first information indicates the maximum space coordinates of the first subcloud of 3D points, and the second information indicates the maximum space coordinates of the second subcloud of three-dimensional points. 57. A method for decoding three-dimensional data, comprising the steps of: obtaining the first encoded data and the second encoded data generated by encoding the first 3D point subcloud and the second 3D point subcloud in such a way that the first 3D point subcloud and the second 3D point subcloud are independently decodable, wherein the first 3D point subcloud and the second 3D subcloud the points are included in sub-clouds of 3D points obtained by dividing the 3D points included in the 3D data; obtaining first information indicating the space of the first 3D point subcloud from the header of the first 3D point subcloud; obtaining second information indicating the space of the second 3D point subcloud from the header of the second 3D point subcloud; And restoring the first 3D point sub-cloud and the second 3D point sub-cloud by decoding the first encoded data and the second encoded data, wherein when decoding the first 3D point subcloud, the second 3D point subcloud is not referenced, and when decoding the second 3D point subcloud, the first 3D point subcloud is not referenced. 58. The 3D data decoding method of claim 57, wherein the first encoded data and the second encoded data are generated by encoding the first branch corresponding to the first 3D point subcloud and the second branch corresponding to the second 3D point subcloud, such that the first branch and the second branch are decodable independently of each other, with the first branch and the second branch included in an N-ary tree structure of three-dimensional points, where N is an integer greater than or equal to 2. 59. The 3D data decoding method of claim 58, further comprising decoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. 60. The 3D data decoding method of claim 59, wherein the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs. 61. The 3D data decoding method according to any one of claims 57 to 60, wherein the decoding further comprises performing entropy decoding of each of the first 3D point subcloud and the second 3D point subcloud using a different codebook. 62. The 3D data decoding method of claim 61, further comprising initializing the codebook after entropy decoding of the first 3D point subcloud is performed and before entropy decoding of the second 3D point subcloud is performed. 63. The 3D data decoding method according to any one of claims 57-62, wherein: the first encoded data includes the first encoded geometric data and the first encoded attribute data, wherein the first encoded geometric data is generated by encoding pieces of geometric information of the first 3D points included in the first 3D point sub-cloud, wherein the first encoded attribute data is generated by encoding pieces of attribute information the first three-dimensional points, the second encoded data includes the second encoded geometric data and the second encoded attribute data, the second encoded geometric data being generated by encoding pieces of geometric information of the second 3D points included in the second sub-3D point cloud, the second encoded attribute data being generated by encoding the pieces of attribute information second 3D points, the first encoded geometry data and the second encoded geometry data are generated such that the first encoded geometry data and the second encoded geometry data are independently decodable, and the first attribute encoded data and the second attribute encoded data are generated such that the first attribute encoded data and the second attribute encoded data are independently decodable. 64. The method for decoding 3D data according to any one of claims 57 to 63, further comprising decoding a flag indicating whether or not the first 3D point subcloud and the second 3D point subcloud are encoded such that the first 3D point subcloud and the second subcloud 3D points are decodable independently of each other. 65. The 3D data decoding method according to any one of claims 57 to 64, wherein: the first information indicates the maximum space coordinates of the first subcloud of 3D points, and the second information indicates the maximum space coordinates of the second subcloud of three-dimensional points. 66. A method for decoding three-dimensional data, comprising the steps of: obtaining the first encoded data and the second encoded data generated by encoding the first 3D point subcloud and the second 3D point subcloud in such a way that the first 3D point subcloud and the second 3D point subcloud are independently decodable, wherein the first 3D point subcloud and the second 3D subcloud the points are included in sub-clouds of 3D points obtained by dividing the 3D points included in the 3D data; obtaining first information indicating the space of the first 3D point subcloud from the header of the first 3D point subcloud; obtaining second information indicating the space of the second 3D point subcloud from the header of the second 3D point subcloud; And restoring the first 3D point sub-cloud and the second 3D point sub-cloud by decoding the first encoded data and the second encoded data, wherein the first encoded data includes the first encoded geometric data and the first encoded attribute data, wherein the first encoded geometric data is generated by encoding pieces of geometric information of the first 3D points included in the first 3D point sub-cloud, wherein the first encoded attribute data is generated by encoding pieces of attribute information the first three-dimensional points, the second encoded data includes the second encoded geometric data and the second encoded attribute data, the second encoded geometric data being generated by encoding pieces of geometric information of the second 3D points included in the second sub-3D point cloud, the second encoded attribute data being generated by encoding the pieces of attribute information second 3D points, the first encoded geometry data and the second encoded geometry data are generated such that the first encoded geometry data and the second encoded geometry data are independently decodable, and the first attribute encoded data and the second attribute encoded data are generated such that the first attribute encoded data and the second attribute encoded data are independently decodable. 67. The 3D data decoding method of claim 66, wherein the first encoded data and the second encoded data are generated by encoding the first branch corresponding to the first 3D point subcloud and the second branch corresponding to the second 3D point subcloud, such that the first branch and the second branch are decodable independently of each other, with the first branch and the second branch included in an N-ary tree structure of three-dimensional points, where N is an integer greater than or equal to 2. 68. The 3D data decoding method of claim 67, further comprising decoding information indicating the layer to which the root of the first branch belongs and the layer to which the root of the second branch belongs. 69. The 3D data decoding method of claim 68, wherein the layer to which the root of the first branch belongs is identical to the layer to which the root of the second branch belongs. 70. The 3D data decoding method according to any one of claims 66 to 69, wherein the decoding further comprises performing entropy decoding of each of the first 3D point subcloud and the second 3D point subcloud using a different codebook. 71. The 3D data decoding method of claim 70, further comprising initializing the codebook after entropy decoding of the first 3D point subcloud is performed and before entropy decoding of the second 3D point subcloud is performed. 72. The 3D data decoding method according to any one of claims 66-69, wherein: when decoding the first 3D point subcloud, the second 3D point subcloud is not referenced, and when decoding the second 3D point subcloud, the first 3D point subcloud is not referenced. 73. The method for decoding 3D data according to any one of claims 66 to 72, further comprising decoding a flag indicating whether or not the first 3D point subcloud and the second 3D point subcloud are encoded such that the first 3D point subcloud and the second subcloud 3D points are decodable independently of each other. 74. The method for decoding three-dimensional data according to any one of claims 66-73, in which: the first information indicates the maximum space coordinates of the first subcloud of 3D points, and the second information indicates the maximum space coordinates of the second subcloud of three-dimensional points. 75. Three-dimensional data encoding device, comprising: CPU; And a storage device in which computer-executable instructions are stored; wherein the computer-executable instructions, when executed by the processor, cause the 3D data encoding device to execute the 3D data encoding method according to any one of claims 1 to 37. 76. Device for decoding three-dimensional data, containing: CPU; And a storage device in which computer-executable instructions are stored; wherein the computer-executable instructions, when executed by the processor, cause the 3D data decoding apparatus to execute the 3D data decoding method according to any one of claims 38 to 74.

Images